Audio detection method and device, electronic equipment and storage medium

Abstract

The invention provides an audio detection method and device, electronic equipment and a storage medium, and the method comprises the steps: adjusting a to-be-updated sliding window of audio data based on the energy information of the collected audio data, and obtaining an adjusted target sliding window, the energy information comprises the audio energy values of a plurality of audio frames in the audio data, and the audio energy values of the plurality of audio frames in the audio data comprise the audio energy values of the plurality of audio frames in the audio data; the to-be-updated sliding window is determined based on the generation sequence of the existing sliding windows of the audio data; according to the target sliding window and a target step length corresponding to the target sliding window, performing sliding window processing on the audio data to obtain an audio sliding window segment; extracting time-frequency characteristics of the audio sliding window segment; detecting whether a voice wake-up word exists in the audio sliding window segment according to the time-frequency feature; and under the condition that the voice wake-up word exists in the audio sliding window segment, executing a voice wake-up task corresponding to the audio data. According to the embodiment of the invention, the accuracy of voice wake-up word detection can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an audio detection method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of voice technology, voice wake-up, as one of the core technologies of smart devices, has been widely used in various fields such as smart home, smart assistant, wearable devices, etc. In current practical applications, voice recognition technology is usually used to detect user voice commands. However, in the case of high environmental noise, diverse user accents, hardware limitations of acquisition equipment, etc., it is difficult to accurately detect the wake-up word of voice wake-up from the user's voice command, thereby reducing the accuracy of voice wake-up of the voice wake-up system. Summary of the invention

[0003] The present disclosure provides an audio detection method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, the present disclosure provides an audio detection method, comprising:

[0005] Based on the energy information of the collected audio data, adjusting the sliding window to be updated of the audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of multiple audio frames in the audio data, and the sliding window to be updated is determined based on the generation order of the existing sliding windows of the audio data;

[0006] According to the target sliding window and the target step size corresponding to the target sliding window, the audio data is subjected to sliding window processing to obtain an audio sliding window segment;

[0007] Extracting time-frequency features of the audio sliding window segment;

[0008] Detecting whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature;

[0009] When the voice wake-up word exists in the audio sliding window segment, a voice wake-up task corresponding to the audio data is executed.

[0010] In a second aspect, the present disclosure provides an audio detection device, an adjustment module, configured to adjust a to-be-updated sliding window of the audio data based on energy information of the collected audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of multiple audio frames in the audio data, and the to-be-updated sliding window is determined based on a generation order of existing sliding windows of the audio data;

[0011] A sliding window module is configured to perform sliding window processing on the audio data according to the target sliding window and a target step size corresponding to the target sliding window to obtain an audio sliding window segment;

[0012] An extraction module, configured to extract the time-frequency features of the audio sliding window segment;

[0013] A detection module is configured to detect whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature;

[0014] The execution module is configured to execute the voice wake-up task corresponding to the audio data when the voice wake-up word exists in the audio sliding window segment.

[0015] In a third aspect, the present disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can perform the above-mentioned audio detection method.

[0016] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned audio detection method when executed by a processor.

[0017] In a fifth aspect, the present disclosure provides a computer program product, which includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned audio detection method.

[0018] The audio detection method provided by the embodiment of the present disclosure adjusts the sliding window to be updated of the audio data through the energy information of the collected audio data to obtain the adjusted target sliding window. Since the energy information of the audio data is constantly changing in practical applications, the sliding window to be updated can be adjusted based on the energy information to achieve dynamic adjustment of the sliding window to be updated, that is, the adjusted target sliding window is constantly changing. According to the adjusted target sliding window and the target step size corresponding to the target sliding window, the audio data is processed by sliding window, and different audio features in the audio data can be more flexibly collected, so that the audio features in the audio sliding window segment are richer, and in the subsequent process of extracting time-frequency features based on the audio sliding window segment, the accuracy of extracting time-frequency features is improved. Detecting the voice wake-up word based on the time-frequency features with higher accuracy can further improve the accuracy of voice wake-up word detection, as well as the accuracy of waking up the voice wake-up system and performing the corresponding voice wake-up task.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0021] Figure 1 An application scenario diagram of the audio detection method and device provided in an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of an audio detection method provided by an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of determining a target energy value provided by an embodiment of the present disclosure;

[0024] Figure 4 Another schematic diagram for determining a target energy value provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of the structure of an audio detection model provided in an embodiment of the present disclosure;

[0026] Figure 6 A block diagram of an audio detection device provided in an embodiment of the present disclosure;

[0027] Figure 7 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0029] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0030] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0031] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0032] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0033] In the technical solution disclosed in this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (for example, the "Information Security Technology Personal Information Security Specification", etc.). For example: corresponding prescribed measures are taken for personal information access control; prescribed restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; when using personal information, clear identity orientation is eliminated to avoid precise positioning of specific individuals.

[0034] With the development of voice technology, voice wake-up, as one of the core technologies of smart devices, has been widely used in various fields such as smart home, smart assistant, wearable devices, etc. In current practical applications, voice recognition technology is usually used to detect user voice commands. However, in the case of high environmental noise, diverse user accents, hardware limitations of acquisition equipment, etc., it is difficult to accurately detect the wake-up word of voice wake-up from the user's voice command, thereby reducing the accuracy of voice wake-up of the voice wake-up system.

[0035] Audio signals are continuous and variable in time, which makes it difficult to fully capture the time domain features and frequency domain features in audio data when the audio signals change rapidly. Time domain features and frequency domain features are important features for detecting whether there are voice wake-up words in audio data. Therefore, how to detect whether there are voice wake-up words in audio data based on the captured time domain features and frequency domain features is also crucial.

[0036] Based on this, the embodiments of the present disclosure provide an audio detection method, an audio detection device, an electronic device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0037] Figure 1 A diagram of an application scenario of the audio detection method and device provided in an embodiment of the present disclosure.

[0038] like Figure 1 As shown, the application scenario of the embodiment of the present disclosure may include a terminal device 101, a network 103 and a server 102. The network 103 is used to provide a medium for a communication link between the terminal device 101 and the server 102. The network 103 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0039] The user can use the terminal device 101 to interact with the server 102 through the network 103 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0040] The terminal device 101 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0041] The server 102 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal device 101. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0042] It should be noted that the audio detection method and apparatus provided in the embodiments of the present disclosure can be executed by the server 102. Accordingly, the audio detection method and apparatus provided in the embodiments of the present disclosure can be set in the server 102. The audio detection method and apparatus provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102. Accordingly, the audio detection method and apparatus provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 102 and can communicate with the terminal device 101 and / or the server 102.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0044] Reference Figure 2 , Figure 2 A flow chart of an audio detection method provided according to an embodiment of the present disclosure is shown. The method specifically comprises the following steps:

[0045] Step 202: Based on the energy information of the collected audio data, the to-be-updated sliding window of the audio data is adjusted to obtain an adjusted target sliding window.

[0046] Since the audio signal of audio data has continuity and variability in time, in order to facilitate the analysis and processing of audio data, the audio data is usually segmented to obtain multiple audio segments of the audio data, and then the audio segments are analyzed and processed separately. In practical applications, a sliding window is often used to segment audio data, and the audio data is processed by sliding the window with a fixed window length and step size to obtain the corresponding audio segments. However, a sliding window with a fixed window length often cannot fully capture the audio characteristics of audio data at different time scales, and it is difficult to better adapt to the continuity and variability of audio signals.

[0047] Based on this, the embodiment of the present disclosure adaptively captures the audio features of the audio data by dynamically adjusting the size of the sliding window. In order to obtain the intensity or amplitude changes of the collected audio data over time, the energy information of the audio data is usually calculated. The audio detection method provided in the embodiment of the present disclosure dynamically adjusts the sliding window of the audio data based on the energy information of the audio data, and performs sliding window processing on the audio data based on the adjusted sliding window to obtain an audio sliding window segment of the audio data.

[0048] The energy information includes audio energy values ​​of multiple audio frames in the audio data, and the audio frame is specifically an audio frame obtained by framing the audio data in the process of calculating the energy information of the audio data; the sliding window to be updated is determined based on the generation order of the existing sliding windows of the audio data, and is specifically the last sliding window in the existing sliding windows of the audio data; the target sliding window refers to the sliding window after the sliding window to be updated is adjusted.

[0049] Specifically, after collecting the audio data, the energy information of the audio data is obtained, and the sliding window to be updated of the audio data is adjusted based on the energy information of the audio data to obtain the target sliding window. The specific implementation method of adjusting the sliding window to be updated based on the energy information of the audio data is as follows:

[0050] In a specific implementation provided by the present disclosure, based on the energy information of the collected audio data, adjusting the to-be-updated sliding window of the audio data to obtain the adjusted target sliding window includes:

[0051] Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time;

[0052] According to the target energy value and the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is adjusted to obtain an adjusted target sliding window.

[0053] The end time refers to the time when the sliding window to be updated ends sliding within the time range of the audio data. The target energy value refers to the audio energy value used to adjust the sliding window to be updated. The target energy value can be the audio energy value of a certain audio frame in the audio data, or it can be the average audio energy value between multiple audio frames.

[0054] Specifically, an end time corresponding to the sliding window to be updated is determined within the time range of the audio data, and a target energy value for adjusting the sliding window to be updated is obtained from energy information of the audio data (i.e., audio energy values ​​of multiple audio frames) according to the end time of the sliding window to be updated. The sliding window to be updated is adjusted according to the target energy value and the audio energy value corresponding to the sliding window to be updated to obtain an adjusted target sliding window.

[0055] As described above, the target energy value can be the audio energy value of an audio frame or the average audio energy value between multiple audio frames. Therefore, the corresponding audio frame can be determined in the audio data according to the end time of the sliding window to be updated, and the target energy value can be obtained based on the audio frame.

[0056] Based on this, in a specific implementation provided by the present disclosure, obtaining a target energy value from a plurality of audio energy values ​​according to the end time includes:

[0057] Determining a target audio frame based on the end time;

[0058] A target energy value of the target audio frame is obtained from the multiple audio energy values.

[0059] The target audio frame refers to the audio frame obtained from the end time; the target energy value refers to the audio energy value of the target audio frame.

[0060] Specifically, after determining the end time corresponding to the sliding window to be updated, the target audio frame is obtained in the audio data according to the end time, and the audio energy value of the target audio frame is obtained from the audio energy values ​​of multiple audio frames, that is, the target energy value, so that the sliding window to be updated is adjusted according to the target energy value in the subsequent process to obtain the target sliding window.

[0061] Further, refer to Figure 3 , Figure 3 FIG. 2 shows a schematic diagram of determining a target energy value according to an embodiment of the present disclosure. Figure 3 As shown, the end time of the sliding window to be updated is 0.1 seconds of the audio data ( Figure 3 (not shown in the figure), the target audio frame is determined within the time range of the audio data starting from 0.1 seconds, that is, the first audio frame starting from 0.1 seconds is the target audio frame. In the energy information of the audio data, the audio energy value of the target audio frame is obtained as the target energy value.

[0062] Furthermore, in actual applications, adjusting the sliding window to be updated based on the audio energy frame of each audio frame will increase the computing resources of the terminal device. Therefore, in order to reduce the consumption of computing resources of the terminal device and reduce the number of adjustments to the sliding window to be updated, it is also possible to obtain multiple audio frames within a preset time interval through a pre-set time interval, and adjust the sliding window to be updated according to the audio energy values ​​corresponding to the multiple audio frames to obtain the target sliding window.

[0063] In a specific implementation provided by the present disclosure, obtaining a target energy value from a plurality of audio energy values ​​according to the end time includes:

[0064] Determine a target time interval based on the end time and the preset time interval;

[0065] Determining a target audio frame set within the target time interval;

[0066] Acquire a plurality of audio energy values ​​corresponding to the target audio frame set from the plurality of audio energy values;

[0067] A target energy value is determined based on the plurality of audio energy values.

[0068] Among them, the preset time interval refers to a pre-set time interval for determining a target audio frame set, for example, 0.05 seconds, 0.1 seconds, etc. The preset time interval can be set according to the actual application situation, and the present disclosure does not limit it here. The target audio frame set includes multiple audio frames. The target time interval refers to the time interval corresponding to the preset time interval within the time range of the audio data. The target time interval is determined based on the end time of the sliding window to be updated and the preset time interval. For example, if the end time of the sliding window to be updated is 0.1 seconds and the preset time interval is 0.05 seconds, the target time interval can be determined to be 0.1 seconds to 0.15 seconds of the audio data.

[0069] Specifically, after determining the end time of the sliding window to be updated, the target time interval can be determined within the time range of the audio data according to the end time and the preset time interval, and multiple audio frames within the target time interval are determined as target audio frames (i.e., a target audio frame set). The audio energy value of each target audio frame in the target audio frame set is obtained from the energy information of the audio data, and the average audio energy value between the audio energy values ​​of each audio frame is determined as the target energy value, or the sum of the audio energy values ​​between the audio energy values ​​of each audio frame is determined as the target energy value. For the target audio frame set, the method for determining the corresponding target energy value can be determined according to the actual application situation, and the present disclosure is not limited here.

[0070] Reference Figure 4 , Figure 4 FIG. 2 shows another schematic diagram of determining a target energy value according to an embodiment of the present disclosure. Figure 4 As shown, the end time of the sliding window to be updated is 0.1 seconds of the audio data ( Figure 4 The audio signal change of the audio data is not shown in the figure), the preset time interval is 0.1 seconds, then according to the end time of the sliding window to be updated being 0.1 seconds, and the preset time interval being 0.1 seconds, the target time interval can be determined to be 0.1 seconds to 0.2 seconds of the audio data, and the audio frames between 0.1 seconds and 0.2 seconds of the audio data are all determined as target audio frames to obtain a target audio frame set. In the energy information of the audio data, the audio energy value of each target audio frame in the target audio frame set is obtained, and the target energy value is determined according to the audio energy value of each target audio frame.

[0071] The disclosed embodiment determines the target audio frame by the end time of the sliding window to be updated, and adjusts the sliding window to be updated according to the target energy value of the target audio frame, so that the sliding window to be updated can be adjusted for each audio frame obtained, thereby improving the accuracy of dynamically adjusting the sliding window; it is also possible to obtain a target audio frame set within the target time interval by setting a preset time interval, and determine the target energy value according to the audio energy value corresponding to each audio frame in the target audio frame set, thereby realizing the number of times the sliding window to be updated is adjusted and reducing the consumption of computing resources.

[0072] After determining the target energy value, the fluctuation of the audio signal of the audio data can be determined according to the target energy value and the audio energy value corresponding to the sliding window to be updated to determine whether the sliding window to be updated needs to be adjusted.

[0073] In a specific implementation provided by the present disclosure, adjusting the sliding window to be updated according to the target energy value and the audio energy value corresponding to the sliding window to be updated to obtain an adjusted target sliding window includes:

[0074] When the target energy value is greater than the audio energy value corresponding to the sliding window to be updated, reducing the sliding window to be updated by a preset size to obtain a target sliding window corresponding to the target energy value;

[0075] When the target energy value is less than the audio energy value corresponding to the sliding window to be updated, increasing the sliding window to be updated by the preset size to obtain a target sliding window corresponding to the target energy value;

[0076] When the target energy value is equal to the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is kept unchanged, and the sliding window to be updated is determined as the target sliding window corresponding to the target energy value.

[0077] The preset size refers to a preset size for adjusting the sliding window to be updated, for example, the preset size is 0.01 seconds, 0.02 seconds, etc.

[0078] Specifically, after determining the target energy value, the target energy value is compared with the audio energy value corresponding to the sliding window to be updated. If the target energy value is greater than the audio energy value corresponding to the sliding window to be updated, it means that the energy of the audio signal of the audio data increases. At this time, the sliding window to be updated can be reduced. Specifically, the sliding window to be updated can be reduced by a preset size to obtain the target sliding window. For example, the original size of the sliding window to be updated (that is, the window length of the sliding window to be updated is 0.05 seconds), the preset size is 0.01 seconds, and the size of the adjusted target sliding window is 0.04 seconds.

[0079] If the target energy value is less than the audio energy value corresponding to the sliding window to be updated, it means that the energy of the audio signal of the audio data is reduced. At this time, the sliding window to be updated can be increased. Specifically, the sliding window to be updated can be increased by a preset size to obtain a target sliding window. Still taking the original size of the sliding window to be updated (that is, the window length of the sliding window to be updated is 0.05 seconds) and the preset size of 0.01 seconds as an example, a target sliding window with a window length of 0.06 seconds can be obtained.

[0080] If the target energy value is equal to the audio energy value corresponding to the sliding window to be updated, it means that the energy of the audio signal of the audio data is stable without obvious fluctuations. At this time, the sliding window to be updated can be kept unchanged and determined as the target sliding window.

[0081] Comparing the target energy value with the audio energy value corresponding to the sliding window to be updated, and adjusting the sliding window to be updated based on a preset size, can make the target sliding window dynamically adapt to the energy changes of the audio signal of the audio data, and improve the accuracy of dividing the audio sliding window segments in the subsequent process.

[0082] In order to further improve the accuracy of dynamically adjusting the sliding window to be updated, the size of the sliding window to be updated to be adjusted may be determined according to the difference between the target energy value and the audio energy value corresponding to the sliding window to be updated, and then the sliding window to be updated may be adjusted based on the determined size. The specific implementation method is as follows:

[0083] In a specific implementation provided by the present disclosure, adjusting the sliding window to be updated according to the target energy value and the audio energy value corresponding to the sliding window to be updated to obtain an adjusted target sliding window includes:

[0084] Determine an energy difference between the target energy value and the audio energy value corresponding to the sliding window to be updated;

[0085] Determining an adjustment size according to the energy difference;

[0086] The to-be-updated sliding window is adjusted based on the adjusted size to obtain an adjusted target sliding window.

[0087] Here, adjusting the size refers to adjusting the size of the sliding window to be updated.

[0088] Specifically, after determining the target energy value, the target energy value is compared with the audio energy value corresponding to the sliding window to be updated, and the energy difference between the target energy value and the audio energy value corresponding to the sliding window to be updated is determined. The adjustment size that needs to be adjusted for the sliding window to be updated is determined based on the energy difference. For example, if the energy difference is small, the adjustment size will also be small, and if the energy difference is large, the adjustment size will also be large. The sliding window to be updated is adjusted based on the determined adjustment size to obtain an adjusted target sliding window. The implementation method of adjusting the sliding window to be updated based on the adjustment size is the same as the implementation method of adjusting the sliding window to be updated based on the preset size mentioned above. The implementation method of adjusting the sliding window to be updated based on the adjustment size can refer to the implementation method of adjusting the sliding window to be updated based on the preset size mentioned above, and the present disclosure will not repeat it here.

[0089] By determining the energy difference between the target energy value and the audio energy value corresponding to the sliding window to be updated, an adjustment size for adjusting the sliding window to be updated is determined, and the sliding window to be updated is adjusted according to the adjustment size, thereby further improving the accuracy of adjusting the sliding window to be updated.

[0090] In practical applications, the corresponding relationship between energy information and sliding window size may be preset to adjust the sliding window to be updated directly according to the determined target energy value, thereby simplifying the process of adjusting the sliding window to be updated.

[0091] Based on this, in a specific implementation provided by the present disclosure, based on the energy information of the collected audio data, adjusting the to-be-updated sliding window of the audio data to obtain the adjusted target sliding window includes:

[0092] Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time;

[0093] Determine an energy value interval to which the target energy value belongs;

[0094] A sliding window size corresponding to the energy value interval is obtained, and the size of the sliding window to be updated is adjusted to the sliding window size to obtain a target sliding window corresponding to the target energy value.

[0095] The energy value interval refers to the energy value interval after the energy information of the audio data is divided according to the size of the audio energy value. The sliding window size refers to the sliding window size corresponding to the energy value interval, specifically the size of the adjusted target sliding window.

[0096] Specifically, the same implementation method as the above-mentioned acquisition of the target energy value is used to determine the end time corresponding to the sliding window to be updated, and to acquire the target energy value from the multiple audio energy values ​​of the energy information according to the end time. After acquiring the target energy value, the energy value interval to which the target energy value belongs is determined, and according to the correspondence between the energy value interval and the sliding window size, the sliding window size corresponding to the energy value interval to which the target energy value belongs is acquired. The size of the sliding window to be updated is adjusted to the determined sliding window size, and the adjusted target sliding window can be obtained.

[0097] By pre-dividing the energy information of the audio data into multiple energy value intervals and setting the correspondence between the energy value intervals and the sliding window size, after obtaining the target energy value, by determining the energy value interval to which the target energy value belongs, based on the correspondence between the energy value intervals and the sliding window size, the size of the adjusted target sliding window is determined, and the size of the sliding window to be updated is adjusted to the sliding window size, so as to obtain the target sliding window, which greatly simplifies the process of adjusting the sliding window to be updated and improves processing efficiency.

[0098] The audio detection method provided by the embodiment of the present disclosure adjusts the sliding window to be updated according to the energy information of the audio data, obtains the adjusted target sliding window, and realizes the dynamic adjustment of the sliding window to be updated.

[0099] Step 204: Perform sliding window processing on the audio data according to the target sliding window and the target step size corresponding to the target sliding window to obtain an audio sliding window segment.

[0100] After the sliding window to be updated is adjusted to obtain the target sliding window, the step size of the target sliding window can be determined, so as to perform sliding window processing on the audio data based on the target sliding window and the step size of the target sliding window to obtain an audio sliding window segment.

[0101] The target step length is the moving step length corresponding to the target sliding window, and the target step length can be set to a length smaller than the target sliding window length, such as half or one third of the target sliding window length. The audio sliding window segment is an audio segment obtained based on the target sliding window.

[0102] In practical applications, the target step size corresponding to the target sliding window is determined by the window length of the target sliding window, and the audio data is subjected to sliding window processing according to the target sliding window and the target step size to obtain the audio sliding window segment corresponding to the target sliding window.

[0103] By setting the target step length to be smaller than the window length of the target sliding window, data loss is avoided when the sliding window processing is performed on the audio data.

[0104] Step 206: extracting the time-frequency features of the audio sliding window segment.

[0105] After obtaining the audio sliding window segment, the time-frequency features of the audio sliding window segment are extracted, so as to detect whether there is a voice wake-up word in the audio sliding window segment according to the extracted time-frequency features in the subsequent process. The time-frequency features include time features and frequency features.

[0106] In a specific implementation provided by the present disclosure, extracting the time-frequency features of the audio sliding window segment includes:

[0107] Performing Fourier transform on the audio sliding window segment to obtain a time feature and a first frequency feature of the audio sliding window segment;

[0108] Mapping the first frequency feature to a Mel scale based on the Mel spectrum to obtain a second frequency feature of the audio sliding window segment;

[0109] The time feature and the second frequency feature are determined as the time-frequency feature of the audio sliding window segment.

[0110] Among them, the Fourier transform can specifically be a short-time Fourier transform (STFT) or a wavelet transform, etc., and the short-time Fourier transform is preferred in the present embodiment. The time feature refers to the characteristic distribution of the audio signal in the audio sliding window segment in the time dimension; the first frequency feature refers to the characteristic distribution of the audio signal in the audio sliding window segment in the frequency dimension obtained based on the Fourier transform; the second frequency feature refers to the characteristic distribution of the audio signal in the audio sliding window segment in the frequency dimension obtained based on the Mel-frequency spectrum. The time-frequency feature can be specifically represented by a two-dimensional time-frequency diagram.

[0111] Specifically, the audio sliding window segment is subjected to Fourier transform (short-time Fourier transform or wavelet transform) to obtain the characteristic distribution of the audio signal in the audio sliding window segment in the time dimension, i.e., the time feature, and the characteristic distribution of the audio signal in the audio sliding window segment in the frequency dimension, i.e., the first frequency feature. The process of performing short-time Fourier transform on the audio sliding window segment can be specifically referred to the following formula 1:

[0112]

[0113] Where X(t, f) is the short-time Fourier transform result at time t and frequency f; x(τ) is the input audio signal; w(t-τ) is the window function used to limit the scope of the short-time Fourier transform; e -j2πfτ is a complex exponential function used to transform the audio signal from the time domain to the frequency domain.

[0114] The short-time Fourier transform of the audio signal in the audio sliding window segment is implemented by the above formula 1. After obtaining the time feature and the first frequency feature of the audio sliding window segment, the first frequency feature is mapped to the Mel scale based on the Mel spectrum to obtain the second frequency feature of the audio sliding window segment. For details, refer to the following formula 2:

[0115]

[0116] Among them, f mel is the second frequency feature, i.e., the frequency feature under the Mel scale; f is the first frequency feature. Formula 2 can be used to map the first frequency feature after Fourier transformation to the Mel scale to obtain the second frequency feature of the audio sliding window segment. The obtained time feature and the second frequency feature are the time-frequency features of the audio sliding window segment. A two-dimensional time-frequency diagram of the audio sliding window segment can also be further generated based on the time-frequency features, so as to intuitively understand the characteristic distribution of the audio signal in the audio sliding window segment.

[0117] In practical applications, in order to improve the stability of time-frequency features and reduce the impact of noise, the time-frequency features can be normalized to obtain normalized time-frequency features. For details, please refer to the following formula 3:

[0118]

[0119] Among them, X norm is the normalized time-frequency feature; X is the unnormalized time-frequency feature, specifically including the time feature and the second frequency feature; μ and σ are normalization parameters. The time-frequency feature of the audio sliding window segment can be normalized by the above formula 3 to scale the time-frequency feature to the standard range and obtain the normalized time-frequency feature.

[0120] Furthermore, if the noise in the environment where the audio data is collected is strong, the time-frequency features may be subjected to noise reduction processing to reduce the noise in the audio sliding window segment. For details, see the following formula 4:

[0121]

[0122] Wherein, Y(f) is the spectrum of the audio sliding window segment after noise reduction; S(f) is the original spectrum of the audio sliding window segment; X(f) is the spectrum of the second frequency feature of the audio sliding window segment; N(f) is the noise spectrum of the noise in the audio sliding window segment. The above formula 4 is used to implement noise reduction processing of the time-frequency features in the audio sliding window segment.

[0123] In practical applications, other noise reduction methods may also be used to implement noise reduction processing on the audio sliding window segment, and are not limited to the above noise reduction methods.

[0124] The audio detection method provided by the embodiment of the present disclosure, after obtaining the audio sliding window segment, combines Fourier transform and Mel spectrum to extract the time-frequency features of the audio sliding window segment, thereby improving the accuracy of the obtained time-frequency features.

[0125] Step 208: Detect whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency features.

[0126] After extracting the time-frequency features of the audio sliding window segment, the audio sliding window segment can be detected based on the time-frequency features to determine whether there is a voice wake-up word in the audio sliding window segment. The voice wake-up word refers to a keyword used to wake up the voice wake-up system.

[0127] In a specific implementation provided by the present disclosure, detecting whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature includes:

[0128] Inputting the time-frequency features into an audio detection model to obtain a probability value of the presence of a voice wake-up word in the audio sliding window segment;

[0129] When the probability value reaches a preset probability value, determining that a voice wake-up word exists in the audio sliding window segment;

[0130] When the probability value does not reach the preset probability value, it is determined that there is no voice wake-up word in the audio sliding window segment.

[0131] The audio detection model is used to detect whether there is a voice wake-up word in the audio sliding window segment. The audio detection model can be a convolutional neural network model, a residual convolutional network model, etc. The probability value is used to characterize the probability of the presence of a voice wake-up word in the audio sliding window segment. The preset probability value refers to a pre-set probability threshold used to measure whether there is a voice wake-up word in the audio sliding window segment.

[0132] Specifically, the obtained time-frequency features are input into the audio detection model to obtain the probability value output by the audio detection model. If the probability value reaches the preset probability value, it is determined that there is a voice wake-up word in the audio sliding window segment. If the probability value does not reach the preset probability value, it is determined that there is no voice wake-up word in the audio sliding window segment.

[0133] In order to improve the accuracy of identifying whether there is a voice wake-up word in the audio sliding window segment, the time-frequency features are classified through the audio detection model, and the time-frequency features are processed based on the multi-scale convolutional layer to adapt to the continuity and variability of the audio signal in time. Therefore, the audio detection model includes the first convolutional layer, the second convolutional layer and the fully connected layer. The time-frequency features are processed through convolutional layers of different scales, and the time-frequency features are classified based on the fully connected layer and the probability value is output to determine whether there is a voice wake-up word in the audio sliding window segment.

[0134] Based on this, in a specific implementation provided by the present disclosure, the time-frequency feature is input into an audio detection model to obtain a probability value of the presence of a voice wake-up word in the audio sliding window segment, including:

[0135] Inputting the time-frequency feature into the first convolution layer to obtain first convolution information of the time-frequency feature;

[0136] Inputting the first convolution information into the second convolution layer, performing dilated convolution processing on the first convolution information, and obtaining second convolution information;

[0137] The second convolution information is input into the fully connected layer to obtain a probability value corresponding to the audio sliding window segment.

[0138] Among them, the first convolution layer is specifically a two-dimensional convolution layer without a dilated convolution function, such as a 1×1 convolution layer; the second convolution layer specifically includes a dilated convolution layer with a dilated convolution function, for example, a 3×3 convolution layer with a dilation rate of 2. The first convolution information refers to the information obtained after the time-frequency feature is convolved; the second convolution information refers to the information obtained after the first convolution information is dilated. Convolution processing. Specifically, the time-frequency feature is input into the first convolution layer of the audio detection model to obtain the first convolution information, the first convolution information is input into the second convolution layer of the audio detection model, and the first convolution information is dilated to obtain the second convolution information. The second convolution information is input into the fully connected layer, and the second convolution information is processed to obtain the probability value corresponding to the audio sliding window segment.

[0139] Furthermore, in order to alleviate the problem of gradient vanishing in the convolutional network and avoid information loss in the convolutional network, the audio detection model provided in the present formula embodiment uses residual connections between each convolutional layer to improve the stability of the audio detection model.

[0140] Reference Figure 5 , Figure 5 FIG. 2 shows a schematic diagram of the structure of an audio detection model provided according to an embodiment of the present disclosure. Figure 5 As shown in FIG, the audio detection model includes an input layer, a first convolutional layer, an average pooling layer, a second convolutional layer, a fully connected layer, and an output layer. The second convolutional layer includes multiple dilated convolutional layers and a normalization layer ( Figure 5Take three dilated convolution layers and normalization layers as an example). In actual applications, the second convolution layer may include multiple dilated convolution layers and normalization layers, and the number of convolution kernels and the corresponding dilation rate of the multiple dilated convolution layers can be determined according to the actual application situation. The information of time-frequency features at different time scales can be obtained through multiple dilated convolution layers, which is beneficial to improving the accuracy of speech wake-up word recognition. Furthermore, in order to enable the audio detection model to run on low-power devices with limited computing resources, an average pooling layer can also be deployed in the audio detection model to reduce the spatial size of the first convolution information, thereby reducing the consumption of computing resources and reducing the occupancy of memory space. The average pooling layer can determine whether it needs to be deployed based on the computing resources of the device in the actual application.

[0141] Before using the audio detection model to detect the voice wake-up word based on the time-frequency features, the audio detection model needs to be trained first, so as to perform voice wake-up word detection based on the trained audio detection model. Based on this, in a specific implementation provided by the present disclosure, the audio detection model is trained according to the following method:

[0142] Acquire sample training data, wherein the sample training data includes sample time-frequency features and sample labels corresponding to the sample time-frequency features;

[0143] Inputting the sample time-frequency features into the audio detection model to obtain a predicted probability value;

[0144] Calculating a loss value of the audio detection model according to the predicted probability value and the sample label;

[0145] The audio detection model is trained according to the loss value.

[0146] Among them, sample training data refers to the training data used to train the audio detection model, including sample time-frequency features and sample labels corresponding to the sample time-frequency features; sample time-frequency features refer to the time-frequency features used to train the audio detection model; sample labels refer to the actual labels of whether the audio sliding window segments corresponding to the sample time-frequency features contain the voice wake-up words; predicted probability values ​​refer to the probability values ​​obtained by inputting the sample time-frequency features into the audio detection model; and the loss value is used to measure the difference between the predicted probability values ​​of the sample time-frequency features and the actual labels.

[0147] Specifically, audio data covering different background noises, accents, speaking speeds, etc. are collected from multiple environments, audio data containing voice wake-up words are marked as positive samples, audio data not containing voice wake-up words are marked as negative samples, and corresponding sample labels are generated. Based on the same method as the above method for extracting the time-frequency features of the audio sliding window segment, the sample time-frequency features are obtained. The sample time-frequency features are input into the audio detection model to obtain the predicted probability value output by the audio detection model. Since the audio detection model at this time is an untrained model, the accuracy of the predicted probability value is low, and the model parameters of the audio detection model (such as model learning rate, other hyperparameters, etc.) need to be adjusted accordingly. Specifically, the loss value of the audio detection model is calculated according to the predicted probability value and the sample label. The loss function used to calculate the loss value of the audio detection model can specifically be a cross-entropy loss value, a square loss function, etc. In the embodiment provided in the present disclosure, the cross-entropy loss function is preferably used as the loss function for calculating the loss value of the audio detection model. The model parameters of the audio detection model are adjusted according to the calculated loss value, and the audio detection model is continued to be trained based on the adjusted model parameters for the next batch of sample training data until the stop condition of the model training is reached.

[0148] To ensure that the model does not overfit during training, a validation dataset can be obtained to evaluate the audio detection model regularly. To ensure the accuracy of the trained model, a test set can be obtained to evaluate the model performance of the audio detection model. Specifically, the audio detection model can be evaluated by calculating the accuracy, recall rate, F1 score, etc. of the audio detection model, and the model structure of the audio detection model can be adjusted according to the evaluation results, such as increasing or decreasing the depth of the convolutional layer, adjusting the residual connection method, etc.

[0149] The audio detection method provided in the embodiment of the present disclosure can determine the probability of the presence of a voice wake-up word in an audio sliding window segment based on the time-frequency characteristics of the audio sliding window segment by training an audio detection model, and detect whether there is a voice wake-up word in the audio sliding window segment based on a preset probability value, thereby improving the accuracy of voice wake-up word recognition.

[0150] Step 210: When the voice wake-up word exists in the audio sliding window segment, execute the voice wake-up task corresponding to the audio data.

[0151] After detecting whether there is a voice wake-up word in the audio sliding window segment, it can be determined whether to execute the voice wake-up task corresponding to the audio data based on the detection result. Among them, the voice wake-up task refers to a task that wakes up or starts the voice wake-up system based on audio data. If it is determined that there is a voice wake-up word in the audio sliding window segment, it means that there is a voice wake-up word in the audio data to which the audio sliding window segment belongs. At this time, the voice wake-up task corresponding to the audio data can be executed.

[0152] By detecting the audio sliding window segment, it is determined whether there is a voice wake-up word in the audio sliding window segment. If so, the voice wake-up task corresponding to the audio data can be executed. If not, the voice wake-up task corresponding to the audio data will not be executed, thereby improving the accuracy of the voice wake-up task execution.

[0153] The audio detection method provided by the present disclosure includes: adjusting a sliding window to be updated of the audio data based on energy information of the collected audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of multiple audio frames in the audio data, and the sliding window to be updated is determined based on the generation order of the existing sliding windows of the audio data; performing sliding window processing on the audio data according to the target sliding window and a target step size corresponding to the target sliding window to obtain an audio sliding window segment; extracting time-frequency features of the audio sliding window segment; detecting whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency features; and executing a voice wake-up task corresponding to the audio data when the voice wake-up word exists in the audio sliding window segment.

[0154] The disclosed embodiment realizes that the sliding window to be updated of the audio data is adjusted by the energy information of the collected audio data to obtain the adjusted target sliding window. By adjusting the sliding window to be updated based on the energy information, the dynamic adjustment of the sliding window to be updated can be realized. According to the adjusted target sliding window and the target step size corresponding to the target sliding window, the audio data is subjected to sliding window processing, and different audio features in the audio data can be collected more flexibly, so that the audio features in the audio sliding window segment are richer, and in the subsequent process of extracting time-frequency features based on the audio sliding window segment, the accuracy of extracting time-frequency features is improved. The accuracy of voice wake-up word detection can be improved by detecting the voice wake-up word based on the time-frequency features with higher accuracy. The probability of the presence of the voice wake-up word in the audio sliding window segment is determined by training the audio detection model based on the time-frequency features of the audio sliding window segment, and the presence of the voice wake-up word in the audio sliding window segment is detected in combination with the preset probability value, so as to further improve the accuracy of voice wake-up word recognition, thereby improving the accuracy of waking up the voice wake-up system and performing the corresponding voice wake-up task.

[0155] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0156] In addition, the present disclosure also provides an audio detection device, an electronic device, a computer-readable storage medium and a computer program product, all of which can be used to implement any audio detection method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0157] Figure 6 FIG. 1 is a block diagram of an audio detection device provided according to an embodiment of the present disclosure. Figure 6 , an embodiment of the present disclosure provides an audio detection device, the audio detection device comprising:

[0158] An adjustment module 602 is configured to adjust the to-be-updated sliding window of the audio data based on energy information of the collected audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of multiple audio frames in the audio data, and the to-be-updated sliding window is determined based on a generation order of existing sliding windows of the audio data;

[0159] A sliding window module 604 is configured to perform sliding window processing on the audio data according to the target sliding window and a target step size corresponding to the target sliding window to obtain an audio sliding window segment;

[0160] An extraction module 606 is configured to extract the time-frequency features of the audio sliding window segment;

[0161] A detection module 608 is configured to detect whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature;

[0162] The execution module 610 is configured to execute the voice wake-up task corresponding to the audio data when the voice wake-up word exists in the audio sliding window segment.

[0163] Optionally, the adjustment module 602 is further configured to:

[0164] Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time;

[0165] According to the target energy value and the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is adjusted to obtain an adjusted target sliding window.

[0166] Optionally, the adjustment module 602 is further configured to:

[0167] Determining a target audio frame based on the end time;

[0168] A target energy value of the target audio frame is obtained from the multiple audio energy values.

[0169] Optionally, the adjustment module 602 is further configured to:

[0170] Determine a target time interval based on the end time and the preset time interval;

[0171] Determining a target audio frame set within the target time interval;

[0172] Acquire a plurality of audio energy values ​​corresponding to the target audio frame set from the plurality of audio energy values;

[0173] A target energy value is determined based on the plurality of audio energy values.

[0174] Optionally, the adjustment module 602 is further configured to:

[0175] When the target energy value is greater than the audio energy value corresponding to the sliding window to be updated, reducing the sliding window to be updated by a preset size to obtain a target sliding window corresponding to the target energy value;

[0176] When the target energy value is less than the audio energy value corresponding to the sliding window to be updated, increasing the sliding window to be updated by the preset size to obtain a target sliding window corresponding to the target energy value;

[0177] When the target energy value is equal to the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is kept unchanged, and the sliding window to be updated is determined as the target sliding window corresponding to the target energy value.

[0178] Optionally, the adjustment module 602 is further configured to:

[0179] Determine an energy difference between the target energy value and the audio energy value corresponding to the sliding window to be updated;

[0180] Determining an adjustment size according to the energy difference;

[0181] The to-be-updated sliding window is adjusted based on the adjusted size to obtain an adjusted target sliding window.

[0182] Optionally, the adjustment module 602 is further configured to:

[0183] Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time;

[0184] Determine an energy value interval to which the target energy value belongs;

[0185] A sliding window size corresponding to the energy value interval is obtained, and the size of the sliding window to be updated is adjusted to the sliding window size to obtain a target sliding window corresponding to the target energy value.

[0186] Optionally, the extraction module 606 is further configured to:

[0187] Performing Fourier transform on the audio sliding window segment to obtain a time feature and a first frequency feature of the audio sliding window segment;

[0188] Mapping the first frequency feature to a Mel scale based on the Mel spectrum to obtain a second frequency feature of the audio sliding window segment;

[0189] The time feature and the second frequency feature are determined as the time-frequency feature of the audio sliding window segment.

[0190] Optionally, the detection module 608 is further configured to:

[0191] Inputting the time-frequency features into an audio detection model to obtain a probability value of the presence of a voice wake-up word in the audio sliding window segment;

[0192] When the probability value reaches a preset probability value, determining that a voice wake-up word exists in the audio sliding window segment;

[0193] When the probability value does not reach the preset probability value, it is determined that there is no voice wake-up word in the audio sliding window segment.

[0194] Optionally, the audio detection model includes a first convolutional layer, a second convolutional layer and a fully connected layer;

[0195] The detection module 608 is further configured to:

[0196] Inputting the time-frequency feature into the first convolution layer to obtain first convolution information of the time-frequency feature;

[0197] Inputting the first convolution information into the second convolution layer, performing dilated convolution processing on the first convolution information, and obtaining second convolution information;

[0198] The second convolution information is input into the fully connected layer to obtain a probability value corresponding to the audio sliding window segment.

[0199] Optionally, the audio detection device further includes a training module configured to:

[0200] Acquire sample training data, wherein the sample training data includes sample time-frequency features and sample labels corresponding to the sample time-frequency features;

[0201] Inputting the sample time-frequency features into the audio detection model to obtain a predicted probability value;

[0202] Calculating a loss value of the audio detection model according to the predicted probability value and the sample label;

[0203] The audio detection model is trained according to the loss value.

[0204] The audio detection device provided by the present disclosure includes: an adjustment module, configured to adjust the to-be-updated sliding window of the audio data based on energy information of the collected audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of multiple audio frames in the audio data, and the to-be-updated sliding window is determined based on the generation order of the existing sliding windows of the audio data; a sliding window module, configured to perform sliding window processing on the audio data according to the target sliding window and a target step size corresponding to the target sliding window to obtain an audio sliding window segment; an extraction module, configured to extract time-frequency features of the audio sliding window segment; a detection module, configured to detect whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency features; and an execution module, configured to execute a voice wake-up task corresponding to the audio data when the voice wake-up word exists in the audio sliding window segment.

[0205] The disclosed embodiment realizes that the sliding window to be updated of the audio data is adjusted by the energy information of the collected audio data to obtain the adjusted target sliding window. By adjusting the sliding window to be updated based on the energy information, the dynamic adjustment of the sliding window to be updated can be realized. According to the adjusted target sliding window and the target step size corresponding to the target sliding window, the audio data is subjected to sliding window processing, and different audio features in the audio data can be collected more flexibly, so that the audio features in the audio sliding window segment are richer, and in the subsequent process of extracting time-frequency features based on the audio sliding window segment, the accuracy of extracting time-frequency features is improved. The accuracy of voice wake-up word detection can be improved by detecting the voice wake-up word based on the time-frequency features with higher accuracy. The probability of the presence of the voice wake-up word in the audio sliding window segment is determined by training the audio detection model based on the time-frequency features of the audio sliding window segment, and the presence of the voice wake-up word in the audio sliding window segment is detected in combination with the preset probability value, so as to further improve the accuracy of voice wake-up word recognition, thereby improving the accuracy of waking up the voice wake-up system and performing the corresponding voice wake-up task.

[0206] Each module in the above-mentioned audio detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0207] Figure 7 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0208] Reference Figure 7An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702, and one or more I / O interfaces 703, connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 so that the at least one processor 701 can execute the above-mentioned audio detection method.

[0209] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0210] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned audio detection method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0211] The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned audio detection method.

[0212] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0213] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0214] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0215] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0216] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0217] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0218] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0219] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0220] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0221] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. An audio detection method, characterized in that: include: Based on the energy information of the collected audio data, adjusting the to-be-updated sliding window of the audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of a plurality of audio frames in the audio data, and the to-be-updated sliding window is determined based on the generation order of the existing sliding windows of the audio data; According to the target sliding window and the target step size corresponding to the target sliding window, the audio data is subjected to sliding window processing to obtain an audio sliding window segment; Extracting time-frequency features of the audio sliding window segment; Detecting whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature; When the voice wake-up word exists in the audio sliding window segment, a voice wake-up task corresponding to the audio data is executed.

2. The method according to claim 1, characterized in that Adjusting the to-be-updated sliding window of the audio data based on the energy information of the collected audio data to obtain an adjusted target sliding window includes: Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time; According to the target energy value and the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is adjusted to obtain an adjusted target sliding window.

3. The method according to claim 2, characterized in that Obtaining a target energy value from a plurality of audio energy values ​​according to the end time includes: Determining a target audio frame based on the end time; A target energy value of the target audio frame is obtained from the multiple audio energy values.

4. The method according to claim 2, characterized in that Obtaining a target energy value from a plurality of audio energy values ​​according to the end time includes: Determine a target time interval based on the end time and the preset time interval; Determining a target audio frame set within the target time interval; Acquire a plurality of audio energy values ​​corresponding to the target audio frame set from the plurality of audio energy values; A target energy value is determined based on the plurality of audio energy values.

5. The method according to claim 2, characterized in that According to the target energy value and the audio energy value corresponding to the sliding window to be updated, adjusting the sliding window to be updated to obtain an adjusted target sliding window includes: When the target energy value is greater than the audio energy value corresponding to the sliding window to be updated, reducing the sliding window to be updated by a preset size to obtain a target sliding window corresponding to the target energy value; When the target energy value is less than the audio energy value corresponding to the sliding window to be updated, increasing the sliding window to be updated by the preset size to obtain a target sliding window corresponding to the target energy value; When the target energy value is equal to the audio energy value corresponding to the sliding window to be updated, the sliding window to be updated is kept unchanged, and the sliding window to be updated is determined as the target sliding window corresponding to the target energy value.

6. The method according to claim 2, characterized in that According to the target energy value and the audio energy value corresponding to the sliding window to be updated, adjusting the sliding window to be updated to obtain an adjusted target sliding window includes: Determine an energy difference between the target energy value and the audio energy value corresponding to the sliding window to be updated; Determining an adjustment size according to the energy difference; The to-be-updated sliding window is adjusted based on the adjusted size to obtain an adjusted target sliding window.

7. The method according to claim 1, characterized in that Adjusting the to-be-updated sliding window of the audio data based on the energy information of the collected audio data to obtain an adjusted target sliding window includes: Determine an end time corresponding to the sliding window to be updated, and obtain a target energy value from a plurality of audio energy values ​​according to the end time; Determine an energy value interval to which the target energy value belongs; A sliding window size corresponding to the energy value interval is obtained, and the size of the sliding window to be updated is adjusted to the sliding window size to obtain a target sliding window corresponding to the target energy value.

8. The method according to claim 1, characterized in that Extracting the time-frequency features of the audio sliding window segment includes: Performing Fourier transform on the audio sliding window segment to obtain a time feature and a first frequency feature of the audio sliding window segment; Mapping the first frequency feature to a Mel scale based on the Mel spectrum to obtain a second frequency feature of the audio sliding window segment; The time feature and the second frequency feature are determined as the time-frequency feature of the audio sliding window segment.

9. The method according to claim 1, characterized in that Detecting whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature includes: Inputting the time-frequency features into an audio detection model to obtain a probability value of the presence of a voice wake-up word in the audio sliding window segment; When the probability value reaches a preset probability value, determining that a voice wake-up word exists in the audio sliding window segment; When the probability value does not reach the preset probability value, it is determined that there is no voice wake-up word in the audio sliding window segment.

10. The method according to claim 9, characterized in that The audio detection model includes a first convolutional layer, a second convolutional layer and a fully connected layer; Inputting the time-frequency feature into an audio detection model to obtain a probability value of the presence of a voice wake-up word in the audio sliding window segment includes: Inputting the time-frequency feature into the first convolution layer to obtain first convolution information of the time-frequency feature; Inputting the first convolution information into the second convolution layer, performing dilated convolution processing on the first convolution information, and obtaining second convolution information; The second convolution information is input into the fully connected layer to obtain a probability value corresponding to the audio sliding window segment.

11. The method according to claim 9, characterized in that The audio detection model is trained according to the following method: Acquire sample training data, wherein the sample training data includes sample time-frequency features and sample labels corresponding to the sample time-frequency features; Inputting the sample time-frequency features into the audio detection model to obtain a predicted probability value; Calculating a loss value of the audio detection model according to the predicted probability value and the sample label; The audio detection model is trained according to the loss value.

12. An audio detection device, characterized in that: include: an adjustment module, configured to adjust the to-be-updated sliding window of the audio data based on energy information of the collected audio data to obtain an adjusted target sliding window, wherein the energy information includes audio energy values ​​of a plurality of audio frames in the audio data, and the to-be-updated sliding window is determined based on a generation order of existing sliding windows of the audio data; A sliding window module is configured to perform sliding window processing on the audio data according to the target sliding window and a target step size corresponding to the target sliding window to obtain an audio sliding window segment; An extraction module, configured to extract the time-frequency features of the audio sliding window segment; A detection module is configured to detect whether there is a voice wake-up word in the audio sliding window segment according to the time-frequency feature; The execution module is configured to execute the voice wake-up task corresponding to the audio data when the voice wake-up word exists in the audio sliding window segment.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the audio detection method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the audio detection method according to any one of claims 1 to 11.

Images