Audio recognition method and audio recognition apparatus

Abstract

This application provides an audio recognition method and an audio recognition device, which can improve the accuracy of sound event detection. The method includes: acquiring an audio signal to be tested; determining the acquisition scene of the audio signal to be tested as a first scene based on the audio signal to be tested; and using a sound event recognition model corresponding to the first scene to determine the sound events identified by the audio signal to be tested based on the audio signal to be tested. The sound event recognition model corresponding to the first scene is a neural network model trained using audio signals in the first scene for recognizing sound events in the first scene based on audio signals.

Description

Technical Field

[0001] This application relates to the field of audio, and more particularly to an audio recognition method and an audio recognition device. Background Technology

[0002] Audio signals carry a wealth of information about the everyday environment and the physical events occurring within it, offering unique advantages in situations where visual, tactile, or olfactory senses are inadequate. Audio-based sound event detection technology, applied to terminal devices, can identify sound events such as alarms, smoke detectors, home appliances, car speakers, doorbells, knocking sounds, running water sounds, baby cries, and shouts, serving as supplementary information to help users, especially people with disabilities, understand their surroundings.

[0003] However, due to the complex characteristics of environmental sounds, the average accuracy of sound event detection technology can only reach 40%, resulting in a poor user experience.

[0004] Therefore, there is an urgent need for an audio recognition method that can improve the accuracy of sound event detection. Summary of the Invention

[0005] This application provides an audio recognition method and an audio recognition device, which can improve the accuracy of sound event detection.

[0006] In a first aspect, this application provides an audio recognition method, which includes: acquiring an audio signal to be tested; determining the acquisition scene of the audio signal to be tested as a first scene based on the audio signal to be tested; and using a sound event recognition model corresponding to the first scene to determine the sound event identified by the audio signal to be tested based on the audio signal to be tested, wherein the sound event recognition model corresponding to the first scene is a neural network model trained using the audio signal in the first scene for recognizing sound events in the first scene based on the audio signal.

[0007] There are several possible ways for audio recognition devices to acquire the audio signal to be tested.

[0008] In one possible implementation, the audio recognition device can acquire the audio signal to be tested in real time.

[0009] In another possible implementation, the audio recognition device can receive audio signals from other devices to obtain the audio signal to be tested.

[0010] In another possible implementation, the audio recognition device can use historical audio signals collected as the audio signal to be tested.

[0011] This application can use existing scene detection algorithms to determine the acquisition scene of the audio signal to be tested as the first scene based on the audio signal to be tested. After the audio signal to be tested is initially classified through the first scene, sound event detection is then performed.

[0012] The sound event recognition model corresponding to the first scene is trained using the audio signals in the first scene. The audio signals in the first scene are any one or more audio signals that may appear in the first scene.

[0013] When there is only one audio signal in the first scene, that one audio signal corresponds to one sound event. A sound event recognition model can be trained based on that one audio signal. That is, the audio signal, the sound event, and the sound event recognition model are in one-to-one correspondence. The trained sound event recognition model can be used to determine whether the sound event corresponding to the audio signal under test is the sound event corresponding to the sound event recognition model based on the audio signal under test.

[0014] When there are multiple audio signals in the first scene, each of these multiple audio signals corresponds to a sound event. Multiple sound event recognition models can be trained based on these multiple audio signals. That is, the audio signals, sound events, and sound event recognition models are in one-to-one correspondence. Each of the trained multiple sound event recognition models can be used to determine whether the sound event corresponding to the audio signal under test is the sound event corresponding to the sound event recognition model based on the audio signal under test.

[0015] The audio recognition method provided in this application first classifies the audio signal to be tested according to a scene detection algorithm to determine the acquisition scene of the audio signal to be tested as the first scene. Then, it uses the sound event recognition model corresponding to the first scene to perform sound event detection on the audio signal to be tested. Compared with directly using the sound event recognition model corresponding to different scenes to perform sound event detection on the audio signal to be tested, this can save computation and reduce the interference of sound events in different scenes. It can more accurately identify the sound events corresponding to the audio signal to be tested and improve the accuracy of sound event detection.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the audio signal to be tested includes multiple audio frames; determining the acquisition scene of the audio signal to be tested as a first scene based on the audio signal to be tested includes: inputting each audio frame in the multiple audio frames into a scene recognition model to obtain scene information for each audio frame, wherein the scene recognition model is a neural network model trained using audio frames in multiple scenes and used to determine the acquisition scene of an audio frame based on the audio frame, and the scene information of each audio frame is used to indicate the probability that the acquisition scene of each audio frame is each of the multiple scenes; and determining the acquisition scene of the audio signal to be tested as the first scene among the multiple scenes based on the scene information of each audio frame.

[0017] After acquiring the audio signal to be tested, the audio recognition device can divide the audio signal into multiple audio frames, and input each audio frame into a scene recognition model to obtain scene information for each audio frame. The scene information for each audio frame represents the probability of the acquired scene for each audio frame among multiple scenarios. Based on the scene information of each audio frame, the audio recognition device can determine that the acquired scene of the audio signal to be tested is the first scene among multiple scenarios.

[0018] It should be understood that the sum of the probabilities of each audio frame belonging to each of the multiple scenarios is 1.

[0019] The audio recognition method provided in this application determines the acquisition scene of the audio signal to be tested as the first scene based on a scene recognition model. The scene recognition accuracy of this method is high, which can more accurately identify the sound events corresponding to the audio signal to be tested and improve the accuracy of sound event detection.

[0020] In conjunction with the first aspect, in some implementations of the first aspect, determining the acquisition scene of the audio signal to be tested as the first scene among multiple scenes based on the scene information of each audio frame includes: counting the number of audio frames belonging to each scene among multiple scenes; when the number of audio frames belonging to the first scene among multiple scenes among multiple audio frames satisfies a first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene among multiple audio frames satisfies a second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the scene recognition model is trained based on audio frames from at least one of the following scenes: a street scene, a subway scene, a home scene, or an office scene.

[0022] In conjunction with the first aspect, in some implementations of the first aspect, the audio signal to be tested includes multiple audio frames, the sound event recognition model corresponding to the first scene includes at least one sound event recognition model, the at least one sound event recognition model includes a first sound event recognition model, the first sound event recognition model is trained using audio frames that identify the first sound event in the first scene; determining the sound event identified by the audio signal to be tested based on the audio signal to be tested using the sound event recognition model corresponding to the first scene includes: inputting multiple audio frames into the first sound event recognition model respectively to obtain sound event information identified by multiple audio frames, the sound event information of each audio frame in the sound event information of multiple audio frames is used to indicate the probability of each audio frame identifying the first sound event; when there is sound event information identified by the first audio frame in multiple audio frames that meets a third preset condition, the first sound event is determined as the sound event identified by the audio signal to be tested.

[0023] Generally, commonly used sound event recognition models are neural network models based on the convolutional recurrent neural network (CRNN) architecture, and this application does not limit them.

[0024] The first sound event recognition model can be any one of at least one sound event recognition model, and this application does not limit it. It should be understood that the first sound event recognition model is pre-trained, capable of recognizing the first sound event, and is preset in the audio recognition device.

[0025] The third preset condition can be adjusted according to the actual application scenario, and this application does not limit it.

[0026] The audio recognition device can input multiple audio frames into the first sound event recognition model to obtain sound event information identified by multiple audio frames. If the sound event information identified by the first audio frame in the multiple audio frames meets the third preset condition, the first sound event is identified as the sound event identified by the audio signal to be tested.

[0027] The audio recognition method provided in this application determines the sound events corresponding to the audio signal to be tested based on a sound event recognition model. The method has a high accuracy in recognizing sound events and can improve the accuracy of sound event detection.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, if the sound event information identified by the first audio frame satisfies the third preset condition, and if the sound event information identified by the second audio frame satisfies the fourth preset condition in a preset number of frames prior to the first audio frame, then the time point corresponding to the second audio frame is the start time point of the first sound event.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, the first scenario is a road scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for horn sounds, a sound event recognition model for alarm sounds, a sound event recognition model for impact sounds, or a sound event recognition model for the sound of a vehicle passing by.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, the first scenario is a subway scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for the sound of a train passing by, a sound event recognition model for the sound of a carriage colliding, or a sound event recognition model for the sound of subway station announcements.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, the first scenario is a home scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for vacuum cleaner vacuuming sound, a sound event recognition model for washing machine washing sound, a sound event recognition model for dish collision sound, a sound event recognition model for baby crying sound, or a sound event recognition model for faucet dripping sound.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, the first scenario is an office scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for telephone ringing, a sound event recognition model for keyboard typing, or a sound event recognition model for meeting invitation sounds.

[0033] Secondly, this application provides an audio recognition device, including: an acquisition module and a processing module. The acquisition module is used to: acquire an audio signal to be tested; the processing module is used to: determine, based on the audio signal to be tested, the acquisition scene of the audio signal to be tested as a first scene; and, using a sound event recognition model corresponding to the first scene, determine the sound events identified by the audio signal to be tested based on the audio signal to be tested, wherein the sound event recognition model corresponding to the first scene is a neural network model trained using audio signals in the first scene for recognizing sound events in the first scene based on audio signals.

[0034] In conjunction with the second aspect, in some implementations of the second aspect, the audio signal to be tested includes multiple audio frames; the processing module is further configured to: input each audio frame in the multiple audio frames into a scene recognition model to obtain scene information for each audio frame, wherein the scene recognition model is a neural network model trained using audio frames in multiple scenes and used to determine the acquisition scene of an audio frame based on the audio frame, and the scene information of each audio frame is used to indicate the probability that the acquisition scene of each audio frame is each of the multiple scenes; and determine the acquisition scene of the audio signal to be tested as the first scene among the multiple scenes based on the scene information of each audio frame.

[0035] In conjunction with the second aspect, in some implementations of the second aspect, the processing module is further used to: count the number of audio frames belonging to each of the multiple scenes in the multiple audio frames; when the number of audio frames belonging to the first scene in the multiple scenes in the multiple audio frames meets the first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene in the multiple audio frames meets the second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

[0036] In conjunction with the second aspect, in some implementations of the second aspect, the scene recognition model is trained based on audio frames from at least one of the following scenes: a street scene, a subway scene, a home scene, or an office scene.

[0037] In conjunction with the second aspect, in some implementations of the second aspect, the audio signal to be tested includes multiple audio frames, the sound event recognition model corresponding to the first scene includes at least one sound event recognition model, the at least one sound event recognition model includes a first sound event recognition model, the first sound event recognition model is trained using audio frames that identify the first sound event in the first scene; the processing module is further configured to: input the multiple audio frames into the first sound event recognition model respectively to obtain sound event information identified by the multiple audio frames, the sound event information of each audio frame in the sound event information of the multiple audio frames is used to indicate the probability of each audio frame identifying the first sound event; when there is sound event information identified by the first audio frame in the multiple audio frames that meets the third preset condition, the first sound event is determined as the sound event identified by the audio signal to be tested.

[0038] In conjunction with the second aspect, in some implementations of the second aspect, if the sound event information identified by the first audio frame satisfies the third preset condition, and if the sound event information identified by the second audio frame satisfies the fourth preset condition in a preset number of frames prior to the first audio frame, then the time point corresponding to the second audio frame is the start time point of the first sound event.

[0039] In conjunction with the second aspect, in some implementations of the second aspect, the first scenario is a road scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for horn sounds, a sound event recognition model for alarm sounds, a sound event recognition model for impact sounds, or a sound event recognition model for the sound of a vehicle passing by.

[0040] In conjunction with the second aspect, in some implementations of the second aspect, the first scenario is a subway scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for the sound of a train passing by, a sound event recognition model for the sound of a carriage colliding, or a sound event recognition model for the sound of subway station announcements.

[0041] In conjunction with the second aspect, in some implementations of the second aspect, the first scenario is a home scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for vacuum cleaner vacuuming sound, a sound event recognition model for washing machine washing sound, a sound event recognition model for dish collision sound, a sound event recognition model for baby crying sound, or a sound event recognition model for faucet dripping sound.

[0042] In conjunction with the second aspect, in some implementations of the second aspect, the first scenario is an office scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for telephone ringing, a sound event recognition model for keyboard typing, or a sound event recognition model for meeting invitation sounds.

[0043] Thirdly, this application provides an audio recognition device, including a processor coupled to a memory, which can be used to execute instructions in the memory to implement the method in any of the possible implementations of the first aspect described above. Optionally, the audio recognition device further includes a memory. Optionally, the audio recognition device further includes a communication interface, and the processor is coupled to the communication interface.

[0044] Fourthly, this application provides a processor, including: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive signals through the input circuit and transmit signals through the output circuit, causing the processor to execute the method in any of the possible implementations of the first aspect described above.

[0045] In specific implementation, the processor can be a chip, the input circuit can be input pins, the output circuit can be output pins, and the processing circuit can be transistors, gate circuits, flip-flops, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to and transmitted by a transmitter. Furthermore, the input circuit and the output circuit can be the same circuit, which is used as both the input circuit and the output circuit at different times. This application does not limit the specific implementation of the processor and various circuits.

[0046] Fifthly, this application provides a processing apparatus, including a processor and a memory. The processor is used to read instructions stored in the memory and can receive signals via a receiver and transmit signals via a transmitter to execute the method in any of the possible implementations of the first aspect described above.

[0047] Optionally, there may be one or more processors and one or more memories.

[0048] Alternatively, the memory can be integrated with the processor, or the memory can be set up separately from the processor.

[0049] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0050] It should be understood that the relevant data interaction process, such as sending indication information, can be the process of outputting indication information from the processor, and receiving capability information can be the process of the processor receiving input capability information. Specifically, the processed output data can be output to the transmitter, and the input data received by the processor can come from the receiver. Here, the transmitter and receiver can be collectively referred to as a transceiver.

[0051] The processing device in the fifth aspect above can be a chip. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. The memory can be integrated into the processor or located outside the processor and exist independently.

[0052] In a sixth aspect, this application provides a computer-readable storage medium storing a computer program (also referred to as code or instructions) that, when run on a computer, causes the computer to perform the method in any of the possible implementations of the first aspect described above.

[0053] In a seventh aspect, this application provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the method in any of the possible implementations of the first aspect described above. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a scenario for sound event detection applicable to an embodiment of this application;

[0055] Figure 2 A schematic flowchart illustrating an audio recognition method provided in an embodiment of this application;

[0056] Figure 3 A schematic flowchart illustrating a scene recognition method provided in an embodiment of this application;

[0057] Figure 4 A schematic flowchart illustrating an acoustic event recognition method provided in an embodiment of this application;

[0058] Figure 5 A schematic block diagram of an acoustic event recognition model corresponding to a first scene provided in an embodiment of this application;

[0059] Figure 6 A schematic block diagram of another sound event recognition model corresponding to a first scene provided in an embodiment of this application;

[0060] Figure 7 A schematic block diagram of another sound event recognition model corresponding to a first scene provided in the embodiments of this application;

[0061] Figure 8 A schematic block diagram of another sound event recognition model corresponding to a first scene provided in an embodiment of this application;

[0062] Figure 9 A schematic block diagram of an audio recognition device provided in an embodiment of this application;

[0063] Figure 10 A schematic block diagram of another audio recognition device provided in an embodiment of this application. Detailed Implementation

[0064] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0065] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, the first preset condition and the second preset condition are used to distinguish different preset conditions and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0066] It should be noted that, in this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0067] Furthermore, "at least one" refers to one or more, while "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0068] Audio signals carry a wealth of information about the everyday environment and the physical events occurring within it, offering unique advantages in situations where vision, touch, or smell are inappropriate. Audio signals are generally considered to have a strong correlation with vibration signals, but their non-contact nature avoids the difficulties of vibration signal data acquisition. Computer audition (CA) technology based on general audio / ambient sound is a branch of artificial intelligence (AI) in the audio field, directly impacting all aspects of socio-economic life. It has numerous applications in dozens of fields, including healthcare, manufacturing, transportation, security, warehousing, agriculture, forestry, animal husbandry, fisheries, water conservancy, environmental and public facilities management, construction, mining, daily life, identity recognition, and the military, making it a highly practical technology.

[0069] For example, in the medical field, the human body itself and many diseases produce a variety of sounds. Using CA (Carrier Assistance) for diagnosis and treatment can partially alleviate the burden on doctors and benefit a wide range of consumers, making it an important aspect of smart healthcare.

[0070] For example, in the field of robotics manufacturing, robots need to have auditory perception capabilities of the surrounding environment. Sound event detection (AED) is also a type of CA from a technical perspective and can be applied to various robot application scenarios. For instance, consumer service robots identify everyday sound events in indoor environments; special-purpose robots for disaster response identify certain sound events in noisy environments and perform given operations; and industrial robots for intelligent inspection of valve halls perform intelligent detection and status recognition of equipment.

[0071] The aforementioned sound event detection can also be called "sound event detection" or "audio event detection," and this application embodiment does not limit it to that.

[0072] However, due to the complex characteristics of environmental sounds, acoustic recognition (CA) technology suffers from low accuracy and slow development in practical applications. Currently, sound event detection technology within CA can be applied to terminal devices. For example, these devices can identify sound events such as alarms, smoke detectors, home appliances, car speakers, doorbells, knocking sounds, running water sounds, baby cries, and shouts, serving as supplementary information to help users, especially people with disabilities, understand their surroundings. However, the average accuracy rate is only around 40%, resulting in a poor user experience. Therefore, there is an urgent need for an audio recognition method that can improve the accuracy of sound event detection.

[0073] To better understand the embodiments of this application, the applicable scenarios for the embodiments of this application will be introduced first.

[0074] Figure 1 A scene diagram illustrating sound event detection is shown. For example... Figure 1 As shown, User 101 is riding the subway and holding mobile phone 102. Mobile phone 102 has a sound event detection function, which can provide the user with an auxiliary function through "sound recognition". User 101 has enabled the "sound recognition" function, which can detect various sounds such as alarms, animal sounds, doorbells, knocking, baby crying, and shouting.

[0075] exist Figure 1 In the scenario shown, mobile phone 102 detects the sound of a baby crying and alerts user 101 via a message notification. The notification message can be displayed as "Sound detected, possibly a baby crying".

[0076] Optionally, when displaying a notification message, the mobile phone 102 can also remind the user 101 through vibration or ringtone. This embodiment of the application does not limit this.

[0077] The process of sound event detection by the mobile phone 102 can be as follows: acquire an audio signal, input the audio signal into the sound event recognition model, and obtain the sound event corresponding to the audio signal as the baby crying.

[0078] There are multiple sound event recognition models, each corresponding to a specific sound event. When the mobile phone 102 recognizes a sound event corresponding to an audio signal, the audio signal needs to be input into each sound event recognition model for recognition. This results in low accuracy of sound event detection, high computational load, and increased power consumption of the mobile phone 102, leading to reduced efficiency in sound event detection.

[0079] In view of this, embodiments of this application provide an audio recognition method and an audio recognition device, which can improve the accuracy of sound event detection and reduce the amount of computation.

[0080] Figure 2This illustration shows a schematic diagram of an audio recognition method 200 provided in an embodiment of this application. The method can be executed by an audio recognition device, such as a terminal device. The terminal device can be a wearable device such as a mobile phone, tablet computer, personal computer (PC), or smartwatch; it can also be various teaching aids (e.g., learning machines, early education machines), smart toys, portable robots, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, etc. This method 200 can be applied to any scenario involving the detection of sound events, such as the one described above. Figure 1 The scenario shown in this application is not limited to this embodiment.

[0081] like Figure 2 As shown, the method 200 may include the following steps:

[0082] S201. Obtain the audio signal to be tested.

[0083] The audio signal to be tested can be understood as any audio signal used to detect sound events, and this application embodiment does not limit this. The audio signal to be tested can also be called the sound signal to be tested, the sound signal to be tested, or the audio stream, and this application embodiment does not limit this.

[0084] There are several possible ways for audio recognition devices to acquire the audio signal to be tested.

[0085] In one possible implementation, the audio recognition device can acquire the audio signal to be tested in real time.

[0086] In this implementation, the audio signal to be tested can be any audio signal in the scene where the audio recognition device is located.

[0087] For example, the audio recognition device is a mobile phone. The mobile phone is in the office and can acquire any audio signals occurring in the office in real time, such as telephone ringing and keypad sounds.

[0088] In this implementation, the audio recognition device acquires the audio signal to be tested in real time, and can promptly detect sound events in the scene in which it is located, which is beneficial to improving the user experience.

[0089] In another possible implementation, the audio recognition device can receive audio signals from other devices to obtain the audio signal to be tested.

[0090] Other devices refer to any device, other than audio recognition devices, that has the function of acquiring audio signals.

[0091] Other devices can collect audio signals from their surroundings in real time and send them to an audio recognition device. The audio recognition device receives the audio signal and uses it as the audio signal to be tested for sound event detection. In this scenario, the other devices and the audio recognition device are in the same scene.

[0092] Other devices can also send historical audio signals collected from their current scene to the audio recognition device. Upon receiving these historical audio signals, the audio recognition device uses them as the audio signal to be tested for sound event detection. In this case, the other devices may or may not be in the same scene as the audio recognition device; this embodiment does not limit this.

[0093] Historical audio signals can be understood as audio signals that did not appear at the current time but appeared before the current time.

[0094] In this implementation, the audio recognition device may only have the function of detecting sound events, and may not include the function of collecting audio signals, thus reducing the requirements for the audio recognition device, making it simple and less expensive.

[0095] In another possible implementation, the audio recognition device can use historical audio signals collected as the audio signal to be tested.

[0096] Audio recognition devices can use saved historical audio signals as audio signals to detect sound events.

[0097] In this implementation, the audio recognition device does not need to collect audio signals in real time, which can save the power consumption of the audio recognition device.

[0098] S202. Based on the audio signal to be tested, determine the acquisition scenario of the audio signal to be tested as the first scenario.

[0099] The first scenario can be the environment in which the audio signal to be tested is collected. Examples of the first scenario include a street, subway, home, or office.

[0100] For example, in the scenario of a road, the audio signal to be tested could be a horn, an alarm, an impact, or a vehicle passing by, etc. In the scenario of a subway, the audio signal to be tested could be a train passing by or a carriage colliding, etc. In the scenario of a home, the audio signal to be tested could be the sound of a vacuum cleaner, a washing machine, dishes clattering, a baby crying, or a dripping faucet, etc. In the scenario of an office, the audio signal to be tested could be a telephone ringing, a keyboard clicking, or a meeting invitation sound, etc.

[0101] The method for determining the acquisition scene of the audio signal under test based on the audio signal under test can be called a "scene detection algorithm", or "sound scene recognition" or "sound scene recognition". This application embodiment does not limit this.

[0102] Existing scene detection algorithms have a high accuracy rate. This application embodiment can use existing scene detection algorithms to determine the acquisition scene of the audio signal to be tested as the first scene based on the audio signal to be tested. After the audio signal to be tested is initially classified through the first scene, sound event detection is then performed.

[0103] The preliminary classification can also be called pre-classification or initial classification, and this application does not limit this to the embodiments.

[0104] S203. Using the sound event recognition model corresponding to the first scene, determine the sound event identified by the audio signal to be tested based on the audio signal to be tested. The sound event recognition model corresponding to the first scene is a neural network model trained using the audio signal in the first scene for recognizing sound events in the first scene based on the audio signal.

[0105] The sound event recognition model corresponding to the first scene is trained using audio signals from the first scene. The audio signals in the first scene refer to any one or more audio signals that may appear in the first scene. The sound event recognition model can also be called an audio recognition model; this application embodiment does not limit this terminology.

[0106] When there is only one audio signal in the first scene, that one audio signal corresponds to one sound event. A sound event recognition model can be trained based on that one audio signal. That is, the audio signal, the sound event, and the sound event recognition model are in one-to-one correspondence. The trained sound event recognition model can be used to determine whether the sound event corresponding to the audio signal under test is the sound event corresponding to the sound event recognition model based on the audio signal under test.

[0107] When there are multiple audio signals in the first scene, each of these multiple audio signals corresponds to a sound event. Multiple sound event recognition models can be trained based on these multiple audio signals. That is, the audio signals, sound events, and sound event recognition models are in one-to-one correspondence. Each of the trained multiple sound event recognition models can be used to determine whether the sound event corresponding to the audio signal under test is the sound event corresponding to the sound event recognition model based on the audio signal under test.

[0108] The audio recognition method provided in this application first classifies the audio signal to be tested according to a scene detection algorithm, determining the acquisition scene of the audio signal to be tested as a first scene. Then, it uses the sound event recognition model corresponding to the first scene to perform sound event detection on the audio signal to be tested. Compared with directly using sound event recognition models corresponding to different scenes to perform sound event detection on the audio signal to be tested, this can save computational load and reduce interference from sound events in different scenes, enabling more accurate identification of the sound events corresponding to the audio signal to be tested and improving the accuracy of sound event detection. In addition, if the audio recognition method provided in this application is executed by a terminal device, the saved computational load can reduce the power consumption of the terminal device, further improving the efficiency of sound event detection.

[0109] As an optional embodiment, the audio signal to be tested may include multiple audio frames; S202, determining the acquisition scene of the audio signal to be tested as a first scene based on the audio signal to be tested may include: inputting each audio frame in the multiple audio frames into a scene recognition model to obtain scene information for each audio frame, wherein the scene recognition model is a neural network model trained using audio frames in multiple scenes and used to determine the acquisition scene of an audio frame based on the audio frame, and the scene information of each audio frame is used to indicate the probability that the acquisition scene of each audio frame is each of the multiple scenes; and determining the acquisition scene of the audio signal to be tested as the first scene among the multiple scenes based on the scene information of each audio frame.

[0110] The scene recognition model is trained using audio frames from various scenes. However, this application embodiment does not limit the number of scenes. Furthermore, the scene recognition model can also be called a scene classification model or a scene classifier; this application embodiment does not limit the name.

[0111] For example, a scene recognition model can be trained based on audio frames from at least one of the following scenes: street scene, subway scene, home scene, and office scene.

[0112] Scene recognition models can determine the scene in which an audio frame was acquired based on the audio frame itself. For example, a scene recognition model can determine which of several scenes an audio frame belongs to based on the audio frame in the audio signal being tested.

[0113] After acquiring the audio signal to be tested, the audio recognition device can divide the audio signal into multiple audio frames, and input each audio frame into a scene recognition model to obtain scene information for each audio frame. The scene information for each audio frame represents the probability of the acquired scene for each audio frame among multiple scenarios. Based on the scene information of each audio frame, the audio recognition device can determine that the acquired scene of the audio signal to be tested is the first scene among multiple scenarios.

[0114] It should be understood that the sum of the probabilities of each audio frame belonging to each of the multiple scenarios is 1.

[0115] For example, the audio signal to be tested may include five audio frames: audio frame 1, audio frame 2, audio frame 3, audio frame 4, and audio frame 5. The audio recognition device can input audio frames 1, 2, 3, 4, and 5 into a scene recognition model to obtain scene information for audio frames 1, 2, 3, 4, and 5, respectively. If the scene recognition model is trained based on audio frames from street scenes, subway scenes, home scenes, and office scenes, then the scene information for audio frame 1 may include the probability that the acquisition scene of audio frame 1 is a street scene, a subway scene, a home scene, and a home scene. The scene information for other audio frames is similar and will not be elaborated here. The audio recognition device can determine the acquisition scene of the audio signal to be tested based on the scene information of audio frames 1, 2, 3, 4, and 5.

[0116] Optionally, determining the acquisition scene of the audio signal to be tested as the first scene among multiple scenes based on the scene information of each audio frame may include: counting the number of audio frames belonging to each scene among multiple scenes; when the number of audio frames belonging to the first scene among multiple scenes among multiple audio frames meets a first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene among multiple audio frames meets a second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

[0117] The first and second preset conditions can be adjusted according to the actual application scenario, and the embodiments of this application do not limit them.

[0118] For example, if the number of multiple audio frames is N, the first preset condition can be greater than or equal to N / 2. The second preset condition can be greater than or equal to 70% or 0.7. In the above example, the audio signal to be tested includes 5 audio frames, namely audio frame 1, audio frame 2, audio frame 3, audio frame 4, and audio frame 5. If the probability indicated by the scene information of 3 of the 5 audio frames being in the first scene is greater than or equal to 0.7, then the audio recognition device can determine that the acquisition scene of the audio signal to be tested is the first scene.

[0119] The audio recognition method provided in this application determines the acquisition scene of the audio signal to be tested as the first scene based on the scene recognition model. The scene recognition accuracy of this method is high, which can more accurately identify the sound events corresponding to the audio signal to be tested and improve the accuracy of sound event detection.

[0120] Optionally, after dividing the audio signal to be tested into multiple audio frames, the audio recognition device can extract the first acoustic features of these multiple audio frames, transforming the audio signal from the time domain to the transform domain representation. For example, the first acoustic features of these multiple audio frames can be long-short-time wavelet features, which can take into account both long-term background information and short-term transient information, thus improving the accuracy of scene recognition.

[0121] Scene recognition models are generally neural network models based on deep convolutional neural networks (DCNN) architecture, which can be trained using the first acoustic features corresponding to audio frames in various scenes.

[0122] It should be understood that the scene recognition model is pre-trained, capable of recognizing multiple scenes, and is preset in the audio recognition device.

[0123] For example, Figure 3 A schematic flowchart of a scene recognition method 300 is shown, such as... Figure 3 As shown, the method 300 may include the following steps:

[0124] S301. Acquire the audio signal to be tested.

[0125] The specific implementation of this step can be found in S201 above, and will not be repeated here.

[0126] S302. Perform frame segmentation on the audio signal to be tested to obtain multiple audio frames.

[0127] Audio recognition devices can use existing audio signal framing methods to process the audio signal under test into frames, which will not be described in detail here.

[0128] S303. Extract the first acoustic features of the multiple audio frames to obtain multiple first acoustic features, and the first acoustic features correspond one-to-one with the audio frames.

[0129] The first acoustic feature can be a long-time or short-time wavelet feature.

[0130] S304. Multiple first acoustic features are input into a neural network model based on a DCNN architecture. Alternatively, multiple first acoustic features can be input into a neural network model based on a DCNN architecture sequentially. This application embodiment does not limit this approach.

[0131] The audio recognition device can input multiple primary acoustic features into a neural network model based on the DCNN architecture to obtain the output results, and use a logistic regression model (softmax) to normalize the output results to obtain the scene information corresponding to the multiple primary acoustic features.

[0132] S305. Determine the acquisition scene of the audio signal to be tested based on the scene information corresponding to multiple first acoustic features.

[0133] The method for determining the acquisition scene of the audio signal to be tested by the audio recognition device can be based on the first preset condition and the second preset condition, and is determined according to the method mentioned above, which will not be repeated here.

[0134] The audio recognition method provided in this application first extracts the first acoustic features of the audio signal to be tested, and then determines the acquisition scene of the audio signal to be tested based on the scene recognition model. The first acoustic features can better reflect the characteristics of the audio signal to be tested, and can improve the accuracy of the first scene recognition.

[0135] As an optional embodiment, the audio signal to be tested includes multiple audio frames, and the sound event recognition model corresponding to the first scene includes at least one sound event recognition model, which includes a first sound event recognition model. The first sound event recognition model is trained using audio frames that identify the first sound event in the first scene. In S203 above, using the sound event recognition model corresponding to the first scene to determine the sound event identified by the audio signal to be tested based on the audio signal to be tested includes: inputting multiple audio frames into the first sound event recognition model respectively to obtain sound event information identified by multiple audio frames, wherein the sound event information of each audio frame in the sound event information of multiple audio frames is used to indicate the probability that each audio frame identifies the first sound event; when there is sound event information identified by the first audio frame in multiple audio frames that meets a third preset condition, the first sound event is determined as the sound event identified by the audio signal to be tested.

[0136] The sound event recognition model can also be called the event detection network model, and this application does not limit it to this.

[0137] Generally, commonly used sound event recognition models are neural network models based on the convolutional recurrent neural network (CRNN) architecture, and this application does not limit this.

[0138] The first sound event recognition model can be any one of at least one sound event recognition model, and this application embodiment does not limit it. It should be understood that the first sound event recognition model is pre-trained, can recognize the first sound event, and is preset in the audio recognition device.

[0139] The third preset condition can be adjusted according to the actual application scenario, and this application embodiment does not limit it.

[0140] The audio recognition device can input multiple audio frames into the first sound event recognition model to obtain sound event information identified by multiple audio frames. If the sound event information identified by the first audio frame in the multiple audio frames meets the third preset condition, the first sound event is identified as the sound event identified by the audio signal to be tested.

[0141] For example, if the audio signal to be tested includes five audio frames, namely audio frame 1, audio frame 2, audio frame 3, audio frame 4, and audio frame 5, the audio recognition device can input audio frames 1, 2, 3, 4, and 5 into the first sound event recognition model to obtain the probabilities of audio frame 1, 2, 3, 4, and 5 identifying the first sound event. If the third preset condition is greater than or equal to 80% or 0.8, and the probability that one of the five audio frames identifies the first sound event is greater than or equal to 0.8, then the audio recognition device can determine that the sound event identified by the audio signal to be tested is the first sound event.

[0142] Optionally, if the sound event information identified by the first audio frame satisfies the third preset condition, and the sound event information identified by the second audio frame satisfies the fourth preset condition in the audio frames that are a preset number of frames before the first audio frame, then the time point corresponding to the second audio frame is the start time point of the first sound event.

[0143] The preset frame rate can be an integer greater than 0, such as 50 frames. This application embodiment does not limit the specific value of the preset frame rate. Different sound events can correspond to different preset frame rates, and this application embodiment does not limit this.

[0144] The fourth preset condition can be adjusted according to the actual application scenario, and this application embodiment does not limit it. For example, the fourth preset condition can be greater than or equal to 50% or 0.5.

[0145] For example, the sound event information identified by the first audio frame satisfies the third preset condition, and the first audio frame is 200 frames out of multiple audio frames. The audio recognition device can push the 200 frames forward by 50 frames. That is, if there is a second audio frame identified by the second audio frame that satisfies the fourth preset condition between frames 150 and 200, then the time point corresponding to the second audio frame is the start time point of the first sound event.

[0146] After the audio recognition device determines the start time of the first sound event, it can be used to display or notify the user, and can also be used in conjunction with other applications. This application embodiment does not limit this.

[0147] The audio recognition method provided in this application determines the sound event corresponding to the audio signal to be tested based on the sound event recognition model. The method has a high accuracy in recognizing sound events and can improve the accuracy of sound event detection.

[0148] Optionally, after dividing the audio signal to be tested into multiple audio frames, the audio recognition device can extract second acoustic features from these multiple audio frames, such as Mel-frequency cepstrum coefficient (MFCC) or logarithmic spectral features, and input these second acoustic features into the first sound event recognition model for sound event detection. The Mel-frequency cepstrum coefficient takes into account the human hearing curve, compresses high-frequency components, and focuses more on the low-frequency components of the audio signal to be tested, which can make the sound event detection results more accurate.

[0149] For example, Figure 4 A schematic flowchart of an acoustic event recognition method 400 is shown, such as... Figure 4 As shown, the method 400 may include the following steps:

[0150] S401. Acquire multiple audio frames from the audio signal to be tested.

[0151] The audio recognition device can first acquire the audio signal to be tested, and then perform frame segmentation processing on the audio signal to obtain multiple audio frames. For details, please refer to S302 above, which will not be repeated here.

[0152] S402. Extract the second acoustic features of the multiple audio frames to obtain multiple second acoustic features, and the second acoustic features correspond one-to-one with the audio frames.

[0153] It should be noted that the second acoustic feature is different from the first acoustic feature in S303 above. The first acoustic feature is used to identify the acquisition scene of the audio frame, while the second acoustic feature here is used to identify the sound events of the audio frame.

[0154] S403. Input multiple second acoustic features into at least one sound event recognition model corresponding to the first scene to determine the sound event corresponding to the audio signal to be tested.

[0155] For example, at least one sound event recognition model corresponding to the first scene may include three sound event recognition models, namely sound event recognition model 1, sound event recognition model 2, and sound event recognition model 3. The audio recognition device may input multiple second acoustic features into sound event recognition model 1, sound event recognition model 2, and sound event recognition model 3 respectively to determine whether the sound event corresponding to the audio signal to be tested is the sound event corresponding to sound event recognition model 1, sound event recognition model 2, or sound event recognition model 3. The specific judgment method can refer to the method of the third preset condition mentioned above, which will not be repeated here.

[0156] The audio recognition method provided in this application first extracts the second acoustic features of the audio signal to be tested, and then determines the sound event corresponding to the audio signal to be tested based on the sound event recognition model. The second acoustic features can better reflect the characteristics of the audio signal to be tested and can improve the accuracy of sound event detection.

[0157] As an optional embodiment, the first scenario described above can be a road scenario. The audio signals that may appear in the first scenario can include at least one of the following: horn sound audio signal, alarm sound audio signal, impact sound audio signal, or sound signal of a vehicle passing by. Accordingly, the sound event recognition model corresponding to the first scenario can include at least one of the following: horn sound sound audio event recognition model, alarm sound sound audio event recognition model, impact sound sound audio event recognition model, or sound event recognition model of a vehicle passing by.

[0158] Among them, the horn sound sound event recognition model is used to identify whether the sound event of the audio signal is a horn sound based on the audio signal; the alarm sound sound event recognition model is used to identify whether the sound event of the audio signal is an alarm sound based on the audio signal; the impact sound sound event recognition model is used to identify whether the sound event of the audio signal is an impact sound based on the audio signal; and the vehicle passing sound sound event recognition model is used to identify whether the sound event of the audio signal is the sound of a vehicle passing by based on the audio signal.

[0159] For example, Figure 5 A schematic block diagram of the sound event recognition model corresponding to the first scene is shown. Figure 5 As shown. The first scenario can be a road scene. The corresponding sound event recognition models for the first scenario include: a sound event recognition model for horn sounds, a sound event recognition model for alarm sounds, a sound event recognition model for impact sounds, and a sound event recognition model for the sound of a car passing by, totaling four sound event recognition models. The audio recognition device can input the audio signal to be tested into these four sound event recognition models respectively to determine which of the four sound event recognition models corresponds to the sound event of the audio signal to be tested.

[0160] As an optional embodiment, the first scenario described above can be a subway scenario. The audio signals that may appear in the first scenario can include at least one of the following: the sound of a train passing by, the sound of a carriage colliding, or the sound of a subway station announcement. Accordingly, the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for the sound of a train passing by, a sound event recognition model for the sound of a carriage colliding, or a sound event recognition model for the sound of a subway station announcement.

[0161] Among them, the sound event recognition model for the sound of a train passing by is used to identify whether the sound event of the audio signal is the sound of a train passing by; the sound event recognition model for the sound of a carriage colliding is used to identify whether the sound event of the audio signal is the sound of a carriage colliding; and the sound event recognition model for the sound of subway station announcements is used to identify whether the sound event of the audio signal is the sound of subway station announcements.

[0162] For example, Figure 6 A schematic block diagram of the sound event recognition model corresponding to the first scene is shown. Figure 3 As shown. The first scenario can be a subway scenario, and the corresponding sound event recognition models for the first scenario include: a sound event recognition model for the sound of a train passing by, a sound event recognition model for the sound of carriages colliding, and a sound event recognition model for the sound of subway station announcements, totaling three sound event recognition models. The audio recognition device can input the audio signal to be tested into these three sound event recognition models respectively to determine which of the three sound event recognition models corresponds to the sound event of the audio signal to be tested.

[0163] As an optional embodiment, the first scenario can be a home scenario, and the audio signals that may occur in the first scenario may include at least one of the following: the sound of a vacuum cleaner, the sound of a washing machine, the sound of dishes colliding, the sound of a baby crying, or the sound of a faucet dripping. Accordingly, the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for the sound of a vacuum cleaner, a sound event recognition model for the sound of a washing machine, a sound event recognition model for the sound of dishes colliding, a sound event recognition model for the sound of a baby crying, or a sound event recognition model for the sound of a faucet dripping.

[0164] For example, Figure 7 A schematic block diagram of the sound event recognition model corresponding to the first scene is shown. Figure 4As shown. The first scenario can be a home scene, and the corresponding sound event recognition models for the first scenario include: a vacuum cleaner vacuuming sound model, a washing machine washing sound model, a dish clattering sound model, a baby crying sound model, and a dripping faucet sound model, totaling five sound event recognition models. The audio recognition device can input the audio signal to be tested into each of these five sound event recognition models to determine which of the five sound event recognition models corresponds to the sound event of the audio signal to be tested.

[0165] Optionally, the sound event recognition model corresponding to the first scene may also include at least one of the sound event recognition model for the sound of breaking glass or the sound event recognition model for the sound of knocking.

[0166] As an optional embodiment, the first scenario can be an office scenario. The audio signals that may appear in the first scenario may include at least one of telephone ringing, keyboard tapping, or meeting invitation sounds. Accordingly, the sound event recognition model corresponding to the first scenario includes at least one of the following: a telephone ringing sound event recognition model, a keyboard tapping sound event recognition model, or a meeting invitation sound event recognition model.

[0167] For example, Figure 8 A schematic block diagram of the sound event recognition model corresponding to the first scene is shown. Figure 5 As shown. The first scenario can be an office scenario, and the corresponding sound event recognition models for the first scenario include three sound event recognition models: a telephone ringing sound sound recognition model, a keyboard sound sound recognition model, and a meeting invitation sound sound recognition model. The audio recognition device can input the audio signal to be tested into these three sound event recognition models respectively to determine which of the three sound event recognition models corresponds to the sound event of the audio signal to be tested.

[0168] Optionally, the first scenario can be a park scenario, and the sound event recognition model corresponding to the first scenario can include at least one of the following: a sound event recognition model for dog barking, a sound event recognition model for cat meowing, or a sound event recognition model for broadcasting.

[0169] It should be noted that the example of the first scenario above is merely a possible implementation method, and the embodiments of this application do not limit it.

[0170] The sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0171] The above text combines Figures 1 to 8 The present application describes in detail the audio recognition method provided in the embodiments. The following will be combined with... Figure 9 and Figure 10 This application provides a detailed description of the audio recognition device provided in the embodiments of this application.

[0172] Figure 9 An audio recognition device 900 according to an embodiment of this application is shown. The audio recognition device 900 includes an acquisition module 910 and a processing module 920. The acquisition module 910 is used to: acquire an audio signal to be tested; the processing module 920 is used to: determine, based on the audio signal to be tested, that the acquisition scenario of the audio signal to be tested is a first scenario; and, using a sound event recognition model corresponding to the first scenario, determine the sound events identified by the audio signal to be tested based on the audio signal to be tested. The sound event recognition model corresponding to the first scenario is a neural network model trained using audio signals in the first scenario for recognizing sound events in the first scenario based on audio signals.

[0173] Optionally, the audio signal to be tested includes multiple audio frames; the processing module 920 is further configured to: input each audio frame in the multiple audio frames into the scene recognition model to obtain scene information of each audio frame, wherein the scene recognition model is a neural network model trained using audio frames in multiple scenes and used to determine the acquisition scene of the audio frame based on the audio frame, and the scene information of each audio frame is used to indicate the probability of each audio frame being acquired in each of the multiple scenes; and determine the acquisition scene of the audio signal to be tested as the first scene among the multiple scenes based on the scene information of each audio frame.

[0174] Optionally, the processing module 920 is further configured to: count the number of audio frames belonging to each of the multiple scenes in the multiple audio frames; when the number of audio frames belonging to the first scene in the multiple scenes in the multiple audio frames meets a first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene in the multiple audio frames meets a second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

[0175] Optionally, the scene recognition model is trained based on audio frames from at least one of the following scenes: street scene, subway scene, home scene, or office scene.

[0176] Optionally, the audio signal to be tested includes multiple audio frames, and the sound event recognition model corresponding to the first scene includes at least one sound event recognition model, which includes a first sound event recognition model. The first sound event recognition model is trained using audio frames that identify the first sound event in the first scene. The processing module 920 is further configured to: input the multiple audio frames into the first sound event recognition model respectively to obtain sound event information identified by the multiple audio frames, wherein the sound event information of each audio frame in the sound event information of the multiple audio frames is used to indicate the probability of each audio frame identifying the first sound event; when the sound event information identified by the first audio frame in the multiple audio frames meets the third preset condition, the first sound event is determined as the sound event identified by the audio signal to be tested.

[0177] Optionally, if the sound event information identified by the first audio frame satisfies the third preset condition, and the sound event information identified by the second audio frame satisfies the fourth preset condition in a preset number of frames prior to the first audio frame, then the time point corresponding to the second audio frame is the start time point of the first sound event.

[0178] Optionally, the first scenario is a road scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for horn sounds, a sound event recognition model for alarm sounds, a sound event recognition model for impact sounds, or a sound event recognition model for the sound of a vehicle passing by.

[0179] Optionally, the first scenario is a subway scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for the sound of a train passing by, a sound event recognition model for the sound of a carriage colliding, or a sound event recognition model for the sound of subway station announcements.

[0180] Optionally, the first scenario is a home scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for vacuum cleaner vacuuming sound, a sound event recognition model for washing machine washing sound, a sound event recognition model for dish collision sound, a sound event recognition model for baby crying sound, or a sound event recognition model for faucet dripping sound.

[0181] Optionally, the first scenario is an office scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: a sound event recognition model for telephone ringing, a sound event recognition model for keyboard typing, or a sound event recognition model for meeting invitation sounds.

[0182] It should be understood that the audio recognition device 900 here is embodied in the form of a functional module. The term "module" here can refer to application-specific integrated circuits (ASICs), electronic circuits, processors (e.g., shared processors, proprietary processors, or group processors, etc.) and memories for executing one or more software or firmware programs, integrated logic circuits, and / or other suitable components supporting the described functions. In an alternative example, those skilled in the art will understand that the audio recognition device 900 can be specifically the audio recognition device in the above method embodiments, or the functions of the audio recognition device in the above method embodiments can be integrated into the audio recognition device 900. The audio recognition device 900 can be used to execute the various processes and / or steps corresponding to the audio recognition device in the above method embodiments; to avoid repetition, these will not be described further here.

[0183] The audio recognition device 900 described above has the function of implementing the corresponding steps performed by the audio recognition device in the above method embodiments; the above functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0184] In the embodiments of this application, Figure 9 The audio recognition device 900 in the text can also be a chip or a chip system, such as a system on a chip (SoC).

[0185] Figure 10 This is a schematic block diagram of another audio recognition device 1000 provided in this application embodiment. The audio recognition device 1000 includes a processor 1010, a communication interface 1020, and a memory 1030. The processor 1010, communication interface 1020, and memory 1030 communicate with each other via internal connection paths. The memory 1030 is used to store instructions, and the processor 1010 is used to execute the instructions stored in the memory 1030 to control the communication interface 1020 to send and / or receive signals.

[0186] It should be understood that the audio recognition device 1000 may specifically be the audio recognition device in the above method embodiments, or the functions of the audio recognition device in the above method embodiments may be integrated into the audio recognition device 1000. The audio recognition device 1000 may be used to execute the various steps and / or processes corresponding to the audio recognition device in the above method embodiments. Optionally, the memory 1030 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information. The processor 1010 may be used to execute instructions stored in the memory, and when the processor executes the instructions, the processor may execute the various steps and / or processes corresponding to the audio recognition device in the above method embodiments.

[0187] It should be understood that, in the embodiments of this application, the processor 1010 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0188] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or as a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor executes the instructions in the memory, combining them with its hardware to complete the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0189] This application also provides a computer-readable storage medium for storing a computer program for implementing the method corresponding to the audio recognition device in the above method embodiments.

[0190] This application also provides a chip system for supporting the audio recognition device in the above method embodiments to implement the functions shown in the embodiments of this application.

[0191] This application also provides a computer program product, which includes a computer program (also referred to as code or instructions), which, when run on a computer, enables the computer to execute the method corresponding to the audio recognition device shown in the above method embodiments.

[0192] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0197] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio recognition method, characterized by, include: Acquire the audio signal to be tested; Based on the audio signal to be tested, the acquisition scenario of the audio signal to be tested is determined to be the first scenario among multiple scenarios; After determining that the acquisition scenario of the audio signal to be tested is the first scenario, the sound event recognition model corresponding to the first scenario is used to determine the sound event identified by the audio signal to be tested based on the audio signal to be tested. The sound event recognition model corresponding to the first scenario is a neural network model trained using the audio signal that identifies the corresponding sound event in the first scenario, which is used to identify the sound event in the first scenario based on the audio signal.

2. The method of claim 1, wherein, The audio signal to be tested includes multiple audio frames; The step of determining the acquisition scenario of the audio signal to be tested as a first scenario among multiple scenarios based on the audio signal to be tested includes: Each audio frame in the plurality of audio frames is input into the scene recognition model to obtain scene information for each audio frame. The scene recognition model is a neural network model trained using audio frames from multiple scenes to determine the acquisition scene of an audio frame based on the audio frame. The scene information of each audio frame is used to indicate the probability that the acquisition scene of each audio frame is each of the plurality of scenes. Based on the scene information of each audio frame, the acquisition scene of the audio signal to be tested is determined to be the first scene among the multiple scenes.

3. The method of claim 2, wherein, The step of determining the acquisition scene of the audio signal under test as the first scene among the multiple scenes based on the scene information of each audio frame includes: Count the number of audio frames belonging to each of the multiple scenes among the multiple audio frames; When the number of audio frames belonging to the first scene among the plurality of audio frames meets a first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene among the plurality of audio frames meets a second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

4. The method according to any one of claims 1 to 3, characterized in that, The scene recognition model is trained based on audio frames from at least one of the following scenes: street scene, subway scene, home scene, or office scene.

5. The method according to any one of claims 1 to 3, characterized in that, The audio signal to be tested includes multiple audio frames, and the sound event recognition model corresponding to the first scene includes at least one sound event recognition model. The at least one sound event recognition model includes a first sound event recognition model, which is trained using audio frames that identify the first sound event in the first scene. The step of using the sound event recognition model corresponding to the first scene to determine the sound event identified by the audio signal under test based on the audio signal under test includes: The plurality of audio frames are respectively input into the first sound event recognition model to obtain sound event information identified by the plurality of audio frames. The sound event information of each audio frame in the sound event information of the plurality of audio frames is used to indicate the probability of each audio frame identifying the first sound event. When the sound event information identified by the first audio frame in the plurality of audio frames meets the third preset condition, the first sound event is identified as the sound event identified by the audio signal to be tested.

6. The method according to claim 5, characterized in that, If the sound event information identified by the first audio frame satisfies the third preset condition, and if the sound event information identified by the second audio frame exists in a preset number of frames prior to the first audio frame and satisfies the fourth preset condition, then the time point corresponding to the second audio frame is the start time point of the first sound event.

7. The method according to any one of claims 1 to 3 and 6, characterized in that, The first scenario is a road scene, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for horn sounds, alarm sounds, impact sounds, or the sound of a car passing by.

8. The method according to any one of claims 1 to 3 and 6, characterized in that, The first scenario is a subway scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for the sound of a train passing by, the sound of a carriage colliding, or the sound of subway station announcements.

9. The method according to any one of claims 1 to 3 and 6, characterized in that, The first scenario is a home scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for vacuum cleaner suction sounds, washing machine washing sounds, dish collision sounds, baby crying sounds, or dripping faucets.

10. The method according to any one of claims 1 to 3 and 6, characterized in that, The first scenario is an office scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for telephone ringing, keyboard typing, or meeting invitation sounds.

11. An audio recognition device, characterized in that, include: The acquisition module is used to acquire the audio signal to be tested; The processing module is used to determine, based on the audio signal to be tested, that the acquisition scenario of the audio signal to be tested is the first scenario among multiple scenarios; as well as, After determining that the acquisition scenario of the audio signal to be tested is the first scenario, the sound event recognition model corresponding to the first scenario is used to determine the sound event identified by the audio signal to be tested based on the audio signal to be tested. The sound event recognition model corresponding to the first scenario is a neural network model trained using the audio signal that identifies the corresponding sound event in the first scenario, which is used to identify the sound event in the first scenario based on the audio signal.

12. The apparatus according to claim 11, characterized in that, The audio signal to be tested includes multiple audio frames; The processing module is also used for: Each audio frame in the plurality of audio frames is input into the scene recognition model to obtain scene information for each audio frame. The scene recognition model is a neural network model trained using audio frames from multiple scenes to determine the acquisition scene of an audio frame based on the audio frame. The scene information of each audio frame is used to indicate the probability that the acquisition scene of each audio frame is each of the plurality of scenes. Based on the scene information of each audio frame, the acquisition scene of the audio signal to be tested is determined to be the first scene among the multiple scenes.

13. The apparatus according to claim 12, characterized in that, The processing module is also used for: Count the number of audio frames belonging to each of the multiple scenes among the multiple audio frames; When the number of audio frames belonging to the first scene among the plurality of audio frames meets a first preset condition, and the probability indicated by the scene information corresponding to the audio frames belonging to the first scene among the plurality of audio frames meets a second preset condition, the first scene is determined as the acquisition scene of the audio signal to be tested.

14. The apparatus according to any one of claims 11 to 13, characterized in that, The scene recognition model is trained based on audio frames from at least one of the following scenes: street scene, subway scene, home scene, or office scene.

15. The apparatus according to any one of claims 11 to 13, characterized in that, The audio signal to be tested includes multiple audio frames, and the sound event recognition model corresponding to the first scene includes at least one sound event recognition model. The at least one sound event recognition model includes a first sound event recognition model, which is trained using audio frames that identify the first sound event in the first scene. The processing module is also used for: The plurality of audio frames are respectively input into the first sound event recognition model to obtain sound event information identified by the plurality of audio frames. The sound event information of each audio frame in the sound event information of the plurality of audio frames is used to indicate the probability of each audio frame identifying the first sound event. When the sound event information identified by the first audio frame in the plurality of audio frames meets the third preset condition, the first sound event is identified as the sound event identified by the audio signal to be tested.

16. The apparatus according to claim 15, characterized in that, If the sound event information identified by the first audio frame satisfies the third preset condition, and if the sound event information identified by the second audio frame exists in a preset number of frames prior to the first audio frame and satisfies the fourth preset condition, then the time point corresponding to the second audio frame is the start time point of the first sound event.

17. The apparatus according to any one of claims 11 to 13, 16, characterized in that, The first scenario is a road scene, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for horn sounds, alarm sounds, impact sounds, or the sound of a car passing by.

18. The apparatus according to any one of claims 11 to 13, 16, characterized in that, The first scenario is a subway scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for the sound of a train passing by, the sound of a carriage colliding, or the sound of subway station announcements.

19. The apparatus according to any one of claims 11 to 13, 16, characterized in that, The first scenario is a home scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for vacuum cleaner suction sounds, washing machine washing sounds, dish collision sounds, baby crying sounds, or dripping faucets.

20. The apparatus according to any one of claims 11 to 13, 16, characterized in that, The first scenario is an office scenario, and the sound event recognition model corresponding to the first scenario includes at least one of the following: Sound event recognition models for telephone ringing, keyboard typing, or meeting invitation sounds.

21. An audio recognition device, characterized in that, include: A processor coupled to a memory for storing a computer program, which, when invoked by the processor, causes the apparatus to perform the method as described in any one of claims 1 to 10.

22. A chip system, characterized in that, Includes: a processor for retrieving and running a computer program from memory, causing a device having the chip system installed to implement the method as described in any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that, Used to store a computer program, the computer program including instructions for implementing the method as described in any one of claims 1 to 10.

24. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to implement the method as described in any one of claims 1 to 10.

Images