Voice endpoint detection method, device, electronic device, storage medium and product

Abstract

The present application provides a method, device, electronic device, storage medium and product for detecting a voice endpoint, which belongs to the field of audio signal processing technology. The method includes: determining a first audio signal, the first audio signal is an audio signal after a wake-up word in an audio signal; dividing the first audio signal into a plurality of first sub-signals; determining first amplitude values ​​corresponding to the plurality of first sub-signals; based on the first amplitude values ​​corresponding to the plurality of first sub-signals, confirming first amplitude change information of the plurality of first sub-signals, the first amplitude change information is used to indicate the change trend of the first amplitude values ​​corresponding to the plurality of first sub-signals; based on the first amplitude change information, determining the tail endpoint corresponding to the wake-up word in the audio signal. This method improves the accuracy of the determined tail endpoint.

Description

Technical Field

[0001] The present application relates to the technical field of audio signal processing, and in particular to a method, device, electronic device, storage medium and product for detecting a voice endpoint. Background Art

[0002] At present, electronic devices generally have voice interaction functions; electronic devices wake up the electronic devices by recognizing the wake-up word in the audio signal, and then perform voice detection on the audio signal after the wake-up word, and execute the detected instructions, thereby realizing voice interaction with the user. Therefore, in order to perform voice detection on the audio signal after the wake-up word, it is necessary to determine the end point of the wake-up word in the audio signal.

[0003] In related technologies, the audio signal after the wake-up word often contains the tail sound signal of the wake-up word. Therefore, after the electronic device recognizes the wake-up word in the audio signal, it generally postpones the signal by a fixed number of frames and uses the end point corresponding to the postponed number of frames as the tail end point of the wake-up word.

[0004] However, the length of the tail signal is not fixed. When the tail signal of the wake-up word is too long, since the number of delayed signal frames is fixed, the tail signal of the wake-up word still exists in the audio signal after the determined tail end point, thereby reducing the accuracy of the determined tail end point. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device, storage medium and product for detecting a voice endpoint, which can improve the accuracy of the determined tail endpoint. The technical solution is as follows:

[0006] In one aspect, a method for detecting a voice endpoint is provided, the method comprising:

[0007] Determine a first audio signal, where the first audio signal is an audio signal after the wake-up word in the audio signal;

[0008] dividing the first audio signal into a plurality of first sub-signals;

[0009] Determine first amplitude values ​​respectively corresponding to the plurality of first sub-signals;

[0010] Based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, confirming first amplitude change information of the plurality of first sub-signals, wherein the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals;

[0011] Based on the first amplitude change information, a tail endpoint corresponding to the wake-up word in the audio signal is determined.

[0012] In some embodiments, determining the tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information includes:

[0013] If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a decreasing trend, the end point of the first audio signal is used as the tail end point.

[0014] In some embodiments, the method further includes any of the following implementations:

[0015] The first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and if the variance is less than or equal to a first threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are in a decreasing trend;

[0016] The first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. If the fitting parameters are less than a second threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the multiple first sub-signals show a decreasing trend, and the fitting parameters are used to indicate the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals.

[0017] In some embodiments, the first amplitude change information includes a variance determined based on first amplitude values ​​respectively corresponding to the plurality of first sub-signals; and confirming the first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes:

[0018] Determine a difference between first amplitude values ​​corresponding to any two adjacent first sub-signals in the plurality of first sub-signals to obtain a plurality of difference values;

[0019] Based on the plurality of difference values, the variance is determined.

[0020] In some embodiments, the first amplitude change information includes fitting parameters of first amplitude values ​​respectively corresponding to the plurality of first sub-signals, the fitting parameters are used to represent the change rates of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and confirming the first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes:

[0021] Linear fitting is performed on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals to obtain the fitting parameters.

[0022] In some embodiments, determining the tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information includes:

[0023] If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a non-decreasing trend, a second audio signal is determined, where the second audio signal is an audio signal subsequent to the first audio signal;

[0024] dividing the second audio signal into a plurality of second sub-signals;

[0025] Determine second amplitude values ​​respectively corresponding to the plurality of second sub-signals;

[0026] Based on the second amplitude values ​​respectively corresponding to the plurality of second sub-signals, obtaining second amplitude change information of the plurality of second sub-signals, wherein the second amplitude change information is used to indicate a change trend of the second amplitude values ​​respectively corresponding to the plurality of second sub-signals;

[0027] If the second amplitude change information indicates that the second amplitude values ​​respectively corresponding to the plurality of second sub-signals are in a decreasing trend, taking the end point of the second audio signal as the tail end point;

[0028] If the second amplitude change information indicates that the second amplitude values ​​respectively corresponding to the plurality of second sub-signals show a non-decreasing trend, the second audio signal is re-determined until the tail endpoint is determined.

[0029] In some embodiments, the first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to represent a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals;

[0030] The determining, based on the first amplitude change information, a tail endpoint corresponding to the wake-up word in the audio signal includes:

[0031] Based on the at least one sub-amplitude change information, target sub-amplitude change information is determined, wherein the target sub-amplitude change information indicates that first amplitude values ​​corresponding to the plurality of first sub-signals corresponding thereto are in a decreasing trend;

[0032] The end point of the first sub-signal corresponding to the target sub-amplitude change information is used as the tail end point.

[0033] In some embodiments, the plurality of first sub-signals respectively include a plurality of sampling points, and determining the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes:

[0034] Determine, based on the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals, an average value of the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals;

[0035] The average value is used as the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0036] On the other hand, a device for detecting a voice endpoint is provided, the device comprising:

[0037] A first determining module, configured to determine a first audio signal, where the first audio signal is an audio signal following a wake-up word in the audio signal;

[0038] A division module, used for dividing the first audio signal into a plurality of first sub-signals;

[0039] A second determining module, configured to determine first amplitude values ​​respectively corresponding to the plurality of first sub-signals;

[0040] a confirmation module, configured to confirm first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, wherein the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals;

[0041] A third determination module is used to determine a tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information.

[0042] In some embodiments, the third determining module is configured to use an end point of the first audio signal as the tail end point if the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals are decreasing.

[0043] In some embodiments, the apparatus further comprises:

[0044] A fourth determination module, configured to determine that the first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and if the variance is less than or equal to a first threshold, determine that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are in a decreasing trend;

[0045] A fifth determination module is used for determining that the first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. If the fitting parameters are less than a second threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the multiple first sub-signals show a decreasing trend, and the fitting parameters are used to indicate the rate of change of the first amplitude values ​​respectively corresponding to the multiple first sub-signals.

[0046] In some embodiments, the first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the multiple first sub-signals; the confirmation module is used to determine the difference between the first amplitude values ​​corresponding to any two adjacent first sub-signals in the multiple first sub-signals to obtain multiple differences; and determine the variance based on the multiple differences.

[0047] In some embodiments, the first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals, and the fitting parameters are used to represent the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. The confirmation module is used to perform linear fitting on the first amplitude values ​​respectively corresponding to the multiple first sub-signals to obtain the fitting parameters.

[0048] In some embodiments, the third determination module is used to determine a second audio signal if the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals are not decreasing, the second audio signal being an audio signal subsequent to the first audio signal; divide the second audio signal into a plurality of second sub-signals; determine the second amplitude values ​​corresponding to the plurality of second sub-signals; obtain second amplitude change information of the plurality of second sub-signals based on the second amplitude values ​​corresponding to the plurality of second sub-signals, the second amplitude change information being used to indicate the change trend of the second amplitude values ​​corresponding to the plurality of second sub-signals; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are decreasing, use the end point of the second audio signal as the tail end point; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are not decreasing, re-determine the second audio signal until the tail end point is determined.

[0049] In some embodiments, the first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to represent a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals;

[0050] The third determination module is used to determine target sub-amplitude change information based on the at least one sub-amplitude change information, wherein the target sub-amplitude change information indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals show a decreasing trend; and take the end point of the first sub-signal corresponding to the target sub-amplitude change information as the tail end point.

[0051] In some embodiments, the multiple first sub-signals respectively include multiple sampling points, and the second determination module is used to determine the average of the absolute values ​​of the second amplitude values ​​of the multiple sampling points respectively included in the multiple first sub-signals based on the absolute values ​​of the second amplitude values ​​of the multiple sampling points respectively included in the multiple first sub-signals; and use the average as the first amplitude values ​​corresponding to the multiple first sub-signals respectively.

[0052] On the other hand, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the voice endpoint detection method described in any of the above-mentioned implementation methods.

[0053] On the other hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the voice endpoint detection method described in any of the above implementations.

[0054] On the other hand, a computer program product is provided, which includes a computer program code, the computer program code is stored in a computer-readable storage medium, a processor of an electronic device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the electronic device performs the voice endpoint detection method described in any of the above-mentioned implementation methods.

[0055] An embodiment of the present application provides a method for detecting a voice endpoint, which is achieved by determining the amplitude change information of an audio signal after a wake-up word, and then determining the tail endpoint corresponding to the wake-up word based on the amplitude change information; since the amplitude value of the tail sound signal of the wake-up word gradually decreases, and the amplitude change information represents the change trend of the first amplitude values ​​of multiple first sub-signals in the first audio signal, the tail endpoint determined based on the amplitude change information is in line with the change law of the tail sound signal of the wake-up word, thereby improving the accuracy of the determined tail endpoint. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0058] Figure 2 It is a flow chart of a method for detecting a voice endpoint provided in an embodiment of the present application;

[0059] Figure 3 is a flow chart of another voice endpoint detection method provided in an embodiment of the present application;

[0060] Figure 4is a flow chart of another voice endpoint detection method provided in an embodiment of the present application;

[0061] Figure 5 It is a block diagram of a voice endpoint detection device provided in an embodiment of the present application;

[0062] Figure 6 It is a block diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0064] The terms "first", "second", "third" and "fourth" in the specification and claims of this application and the drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0065] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the audio signals involved in this application are all obtained with full authorization.

[0066] The voice endpoint detection method provided in the embodiment of the present application can be performed by an electronic device. In some embodiments, the electronic device is a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a human-computer interaction device, a vehicle-mounted terminal, etc., but is not limited thereto. Among them, the electronic device is a device with a voice interaction function. Those skilled in the art will know that the number of the above-mentioned electronic devices may be more or less. For example, the above-mentioned electronic device may be one, or the above-mentioned electronic device may be dozens or hundreds, or a larger number. The embodiment of the present application does not limit the number and type of electronic devices.

[0067] In some embodiments, the voice endpoint detection method is applied in a human-computer interaction scenario. Accordingly, the electronic device involved in the implementation environment of the voice endpoint detection method is a human-computer interaction device 10, such as a smart phone, a tablet computer, an intelligent robot, etc.; accordingly, the human-computer interaction device 10 wakes up the human-computer interaction device 10 by identifying the wake-up word in the audio signal, and then determines the tail endpoint of the wake-up word in the audio signal, performs voice detection on the audio signal after the tail endpoint, and executes the detected instructions, thereby realizing voice interaction with the user.

[0068] In some embodiments, the voice endpoint detection method is applied in a vehicle-mounted scenario. Accordingly, the electronic device is a vehicle-mounted terminal. The vehicle-mounted terminal wakes up the vehicle-mounted terminal by identifying the wake-up word in the audio signal, and then determines the tail endpoint of the wake-up word in the audio signal. It performs voice detection on the audio signal after the tail endpoint and executes the detected instructions, such as turning on navigation, turning on music, etc., thereby realizing voice interaction with the user.

[0069] In some embodiments, the voice endpoint detection method is applied in an elevator scenario. Accordingly, the electronic device is an elevator calling device in the elevator. The elevator calling device wakes up the elevator calling device by identifying the wake-up word in the audio signal, and then determines the tail endpoint of the wake-up word in the audio signal, performs voice detection on the audio signal after the tail endpoint, and executes the detected instructions, such as going up to a certain floor or down to a certain floor, thereby realizing voice interaction with the user.

[0070] The present application embodiment provides a method for detecting a voice endpoint, the execution subject is an electronic device, see Figure 2 , methods include:

[0071] 201. Determine a first audio signal, where the first audio signal is an audio signal after a wake-up word in the audio signal.

[0072] 202. Divide a first audio signal into multiple first sub-signals.

[0073] 203. Determine first amplitude values ​​corresponding to the plurality of first sub-signals respectively.

[0074] 204. Confirm first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, where the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0075] 205. Determine a tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information.

[0076] In some embodiments, determining the tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information includes:

[0077] If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a decreasing trend, the end point of the first audio signal is taken as the tail end point.

[0078] In some embodiments, the method further includes any of the following implementations:

[0079] The first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals. If the variance is less than or equal to the first threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are decreasing.

[0080] The first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. If the fitting parameters are less than the second threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the multiple first sub-signals show a decreasing trend, and the fitting parameters are used to indicate the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals.

[0081] In some embodiments, the first amplitude change information includes a variance determined based on first amplitude values ​​respectively corresponding to the plurality of first sub-signals; based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, confirming the first amplitude change information of the plurality of first sub-signals includes:

[0082] Determine a difference between first amplitude values ​​corresponding to any two adjacent first sub-signals in the plurality of first sub-signals to obtain a plurality of difference values;

[0083] Based on the plurality of differences, a variance is determined.

[0084] In some embodiments, the first amplitude change information includes fitting parameters of first amplitude values ​​respectively corresponding to the plurality of first sub-signals, the fitting parameters are used to represent the change rates of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, confirming the first amplitude change information of the plurality of first sub-signals includes:

[0085] Linear fitting is performed on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals to obtain fitting parameters.

[0086] In some embodiments, determining the tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information includes:

[0087] If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a non-decreasing trend, a second audio signal is determined, and the second audio signal is an audio signal subsequent to the first audio signal;

[0088] dividing the second audio signal into a plurality of second sub-signals;

[0089] Determine second amplitude values ​​respectively corresponding to the plurality of second sub-signals;

[0090] Based on the second amplitude values ​​respectively corresponding to the plurality of second sub-signals, second amplitude change information of the plurality of second sub-signals is obtained, where the second amplitude change information is used to indicate a change trend of the second amplitude values ​​respectively corresponding to the plurality of second sub-signals;

[0091] If the second amplitude change information indicates that the second amplitude values ​​respectively corresponding to the plurality of second sub-signals are in a decreasing trend, the end point of the second audio signal is used as the tail end point;

[0092] If the second amplitude variation information indicates that the second amplitude values ​​respectively corresponding to the plurality of second sub-signals show a non-decreasing trend, the second audio signal is re-determined until the tail endpoint is determined.

[0093] In some embodiments, the first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to represent a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals;

[0094] Determining a tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information includes:

[0095] Based on at least one sub-amplitude change information, target sub-amplitude change information is determined, where the target sub-amplitude change information indicates that first amplitude values ​​corresponding to the plurality of first sub-signals corresponding thereto are in a decreasing trend;

[0096] The end point of the first sub-signal corresponding to the target sub-amplitude change information is used as the tail end point.

[0097] In some embodiments, the plurality of first sub-signals respectively include a plurality of sampling points, and determining the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes:

[0098] Determine, based on the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals, an average value of the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals;

[0099] The average value is used as the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0100] An embodiment of the present application provides a method for detecting a voice endpoint, which is achieved by determining the amplitude change information of an audio signal after a wake-up word, and then determining the tail endpoint corresponding to the wake-up word based on the amplitude change information; since the amplitude value of the tail sound signal of the wake-up word gradually decreases, and the amplitude change information represents the change trend of the first amplitude values ​​of multiple first sub-signals in the first audio signal, the tail endpoint determined based on the amplitude change information is in line with the change law of the tail sound signal of the wake-up word, thereby improving the accuracy of the determined tail endpoint.

[0101] This application embodiment provides another method for detecting a voice endpoint. Figure 3 , methods include:

[0102] 301. The electronic device determines a first audio signal.

[0103] Among them, the first audio signal is the audio signal after the wake-up word in the audio signal. The audio signal includes amplitude values ​​of multiple sampling points, and the amplitude value is used to represent the strength of the audio signal at the sampling point. Optionally, the audio signal is an audio signal output by the target object for voice interaction with the electronic device. The electronic device has an audio acquisition component, such as a microphone, etc., for acquiring the audio signal.

[0104] In some embodiments, a wake-up model is deployed on the electronic device, and the wake-up model is a model that has been trained based on a deep learning algorithm and is used to identify the wake-up word in the audio signal; accordingly, the electronic device inputs the audio signal into the wake-up model, identifies the wake-up word in the audio signal through the wake-up model, and uses the audio signal after the wake-up word as the first audio signal. Among them, the audio signal includes multiple frame signals, and the first audio signal can be one frame or multiple frame signals in the audio signal, and its specific length can be set and changed as needed, and is not specifically limited here.

[0105] It should be noted that before the electronic device inputs the audio signal into the wake-up model, it also determines the length of the audio signal input into the wake-up model each time based on the configuration information of the wake-up model, and then sequentially inputs the audio signal of this length into the wake-up model for wake-up word recognition; the length is one frame or multiple frames, which is not specifically limited here. Among them, if the length of the audio signal input into the wake-up model each time is one frame, the wake-up model will perform phoneme recognition on the multiple frames of signals input multiple times, and then combine the phonemes corresponding to the multiple frames of signals to obtain the text information corresponding to the multiple frames of signals, and perform wake-up word recognition based on the text information.

[0106] It should be noted that the wake-up model recognizes the wake-up word based on the audio features of the audio signal. In one implementation, the electronic device determines the audio features of the audio signal before inputting the audio signal into the wake-up model, inputs the audio features into the wake-up model, and the electronic device then recognizes the wake-up word based on the audio features through the wake-up model. In another implementation, after the electronic device inputs the audio signal into the wake-up model, the audio features of the audio signal are determined by the wake-up model, and the electronic device then recognizes the wake-up word based on the audio features through the wake-up model. Among them, the audio features can be set and changed as needed; optionally, the audio features are mfcc (Mel Frequency Cepstrum Coefficient, Mel frequency cepstrum coefficient) features, which are not specifically limited here.

[0107] In some embodiments, the electronic device performs speech recognition on the audio signal, and after recognizing the wake-up word in the audio signal, caches the audio signal corresponding to the wake-up word, and then performs wake-up word recognition on the cached audio signal based on the wake-up model. If the wake-up word recognized based on the wake-up model is the wake-up word of the electronic device, it is confirmed that the wake-up word has been recognized, and then the electronic device uses the audio signal after the wake-up word as the first audio signal; this further improves the accuracy of wake-up word recognition, thereby improving the accuracy of the determined first audio signal.

[0108] In some embodiments, after acquiring the audio signal, the electronic device pre-processes the audio signal, and then determines the first audio signal from the pre-processed audio signal.

[0109] Among them, the process of preprocessing the audio signal by the electronic device includes at least one of the pre-emphasis process and the framing process. Pre-emphasis is mainly used to increase the high-frequency components in the audio signal to compensate for the problem of low energy of the high-frequency band signal in the audio signal, thereby improving the effect of subsequent wake-up word recognition or speech recognition of the audio signal. Framing is mainly used to divide the continuous audio signal into multiple shorter signals, so as to facilitate the subsequent determination of the tail end point of the wake-up word in the audio signal based on the multi-frame signal obtained by framing. Among them, the framing process includes: the electronic device determines the frame length and frame shift of the framing, and frames the audio signal based on the frame length and frame shift to obtain a multi-frame signal; the frame length and frame shift can be set and changed as needed, and are not specifically limited here. Optionally, the frame length is 256 and the frame shift is 160, which are not specifically limited here. Among them, the frame length of 256 refers to the length of 256 sampling points, and the frame shift of 160 refers to moving 160 sampling points per frame. For example, if the sampling frequency of an audio signal is 16k, that is, the signal value of 16,000 sampling points is collected in 1 second, then the signal value of every 256 sampling points is used as a frame signal, and then the signal value of 160 sampling points out of the 256 sampling points is updated each time to obtain the next frame signal, that is, if the signal value of the 1st sampling point to the 256th sampling point corresponds to the first frame signal, then the signal value of the 161st sampling point to the 256+160th sampling point corresponds to the second frame signal, and the processing is carried out in sequence to realize the framing of the audio signal and obtain a multi-frame signal of the audio signal. It should be noted that if the preprocessing process includes a pre-emphasis process and a framing process, the electronic device performs pre-emphasis on the audio signal and then frames the pre-emphasized audio signal.

[0110] In this embodiment, by preprocessing the audio signal, the influence of aliasing, high-order harmonic distortion, high frequency and other factors on the audio signal quality caused by the device for collecting the audio signal can be eliminated, and the preprocessed signal can be ensured to be more uniform and smooth as much as possible, thereby improving the subsequent wake-up word recognition and speech recognition effect of the audio signal.

[0111] 302. The electronic device divides a first audio signal into multiple first sub-signals.

[0112] Among them, the number of first sub-signals can be set and changed as needed; optionally, the lengths of multiple first sub-signals are the same, that is, the number of sampling points included is the same. In some embodiments, the electronic device determines the number of first sub-signals based on the number of sampling points in the first audio signal; optionally, the electronic device uses the multiple of the number of sampling points that can be divided by an integer as the number of first sub-signals, so that the multiple sampling points included in the first audio signal are evenly distributed in the multiple first sub-signals, and the number of sampling points in the multiple first sub-signals is an integer. If the first audio signal includes 256 sampling points, the multiple of 8 that can be divided by an integer can be used as the number of first sub-signals, so that the number of sampling points in the multiple first sub-signals is 32.

[0113] Optionally, if the length of the first audio signal is L, the electronic device divides the first audio signal into K segments evenly, and the lengths of multiple first sub-signals are N=L / K; wherein, if L represents the number of sampling points, then N represents the number of sampling points respectively included in the multiple first sub-signals.

[0114] In some embodiments, the electronic device determines the lengths of multiple first sub-signals based on the distances of the multiple first sub-signals from the wake-up word; optionally, the length of the first sub-signal is proportional to the length of the first sub-signal from the wake-up word, that is, the closer the distance to the wake-up word, the longer the length of the first sub-signal, thereby ensuring the influence of the first sub-signal close to the wake-up word on the wake-up word, so that the actual changes in the amplitude values ​​of the divided multiple first sub-signals and the tail sound signal of the wake-up word are more matched.

[0115] 303. The electronic device determines first amplitude values ​​corresponding to the plurality of first sub-signals respectively.

[0116] The multiple first sub-signals respectively include multiple sampling points, and the multiple sampling points respectively correspond to a second amplitude value.

[0117] In some embodiments, the electronic device determines the first amplitude values ​​corresponding to the plurality of first sub-signals respectively, comprising the following steps: the electronic device determines the average of the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals based on the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals; the electronic device uses the average as the first amplitude value corresponding to the plurality of first sub-signals respectively. For any first sub-signal, the electronic device uses the average of the absolute values ​​of the second amplitude values ​​of the plurality of sampling points included in the first sub-signal as the first amplitude value corresponding to the first sub-signal.

[0118] It should be noted that the electronic device may obtain the absolute values ​​of the second amplitude values ​​of the plurality of sampling points before executing step 302, or may obtain the absolute values ​​of the second amplitude values ​​of the plurality of sampling points in step 303, which is not specifically limited here.

[0119] In this embodiment, the electronic device uses the average of the absolute values ​​of the amplitude values ​​of multiple sampling points included in the first sub-signal as the first amplitude value corresponding to the first sub-signal, so that the first amplitude value can represent the overall change of the amplitude values ​​of multiple sampling points in the first sub-signal, thereby improving the accuracy of the determined first amplitude value.

[0120] In some embodiments, for any sub-signal among the multiple first sub-signals, the electronic device weights and sums the absolute values ​​of the second amplitude values ​​of the multiple sampling points included in the first sub-signal to obtain the first amplitude value corresponding to the first sub-signal. Optionally, the electronic device assigns different weights to the multiple sampling points based on the distance between the multiple sampling points and the wake-up word, that is, the closer the distance to the wake-up word, the greater the weight, thereby ensuring that the sampling points close to the wake-up word have an influence on the wake-up word, so that the first amplitude value determined subsequently is more accurate.

[0121] In some embodiments, the electronic device pre-screens multiple sampling points included in any sub-signal of multiple first sub-signals, and uses the average of the absolute values ​​of the second amplitude values ​​of the multiple sampling points after pre-screening as the first amplitude value corresponding to the first sub-signal.

[0122] The electronic device pre-screens the multiple sampling points, including at least one of the following implementations: In one implementation, the electronic device samples the multiple sampling points at equal intervals to reduce the number of sampling points and improve the efficiency of determining the first amplitude value corresponding to the first sub-signal. In another implementation, the electronic device pre-screens the multiple sampling points based on the absolute values ​​of the second amplitude values ​​of the multiple sampling points; the electronic device removes the sampling points whose absolute values ​​of the second amplitude values ​​are less than a preset threshold and greater than a preset threshold, so that the sampling points whose amplitude values ​​are too large or too small due to errors are removed, thereby ensuring the accuracy of the amplitude values ​​of the remaining sampling points, and thus ensuring the accuracy of the first amplitude value determined subsequently.

[0123] 304. The electronic device confirms first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0124] The first amplitude change information is used to indicate the change trends of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0125] In some embodiments, the first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the multiple first sub-signals; accordingly, the electronic device confirms the first amplitude change information of the multiple first sub-signals based on the first amplitude values ​​respectively corresponding to the multiple first sub-signals, including the following steps: the electronic device determines the difference between the first amplitude values ​​corresponding to any two adjacent first sub-signals in the multiple first sub-signals to obtain multiple differences; the electronic device determines the variance based on the multiple differences.

[0126] It should be noted that, the larger the variance, the larger the fluctuation of the multiple differences; the smaller the variance, the smaller the fluctuation of the multiple differences; since the multiple differences are the differences between the first amplitude values ​​corresponding to any two adjacent first sub-signals, accordingly, the smaller the variance, the more balanced the differences between the multiple adjacent first sub-signals are, that is, the first amplitude values ​​corresponding to the multiple first sub-signals are gradually reduced, and then the variance is used as the first amplitude change information, which is convenient for the subsequent accurate determination of the change trend of the first amplitude values ​​corresponding to the multiple first sub-signals based on the variance.

[0127] Optionally, if the number of the plurality of first sub-signals is K, the first amplitude values ​​of the plurality of first sub-signals are respectively expressed as M1, M2, ..., Mk, and the plurality of differences are respectively expressed as Dm0, Dm1, ..., Dmk. For example, if K is 10 and the number of the plurality of differences is 9, the first difference Dm0 = M1-M2, the second difference Dm1 = M2-M3, and so on until the ninth difference Dm9 = M10-M9.

[0128] In some embodiments, the first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals, and the fitting parameters are used to represent the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals; accordingly, the electronic device confirms the first amplitude change information of the multiple first sub-signals based on the first amplitude values ​​respectively corresponding to the multiple first sub-signals, including the following steps: the electronic device performs linear fitting on the first amplitude values ​​respectively corresponding to the multiple first sub-signals to obtain the fitting parameters.

[0129] It should be noted that since the amplitude value of the tail sound signal of the wake-up word gradually decreases, in this embodiment, the fitting parameters are obtained by performing linear fitting on the first amplitude values ​​corresponding to the multiple first sub-signals. Since the fitting parameters can represent the change rate of the first amplitude values ​​corresponding to the multiple first sub-signals, the fitting parameters are used as the first amplitude change information, which facilitates the subsequent accurate determination of the change trend of the first amplitude values ​​corresponding to the multiple first sub-signals based on the fitting parameters.

[0130] Optionally, if the number of the plurality of first sub-signals is K, the first amplitude values ​​of the plurality of first sub-signals are respectively expressed as M1, M2, ..., Mk, and then the electronic device performs linear fitting on M1, M2, ..., Mk and uses the slope obtained by the fitting as a fitting parameter.

[0131] 305. If the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals are in a decreasing trend, the electronic device uses the end point of the first audio signal as the tail end point.

[0132] It should be noted that the overall change trend of the amplitude value in the tail sound signal of the wake-up word is a decreasing trend. In the embodiment of the present application, by determining the change trend of the first amplitude values ​​corresponding to the multiple first sub-signals in the first audio signal, and then when the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing, it is indicated that the first audio signal is the tail sound signal of the wake-up word, and then the end point of the first audio signal is used as the tail end point of the wake-up word, which can improve the accuracy of the determined tail end point.

[0133] In some embodiments, the electronic device determines that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are decreasing, including any of the following implementations:

[0134] In one implementation, the first amplitude change information includes a variance determined based on first amplitude values ​​corresponding to the plurality of first sub-signals. If the variance is less than or equal to a first threshold, the electronic device determines that the first amplitude change information indicates that first amplitude values ​​corresponding to the plurality of first sub-signals are decreasing.

[0135] Among them, the closer the variance is to 0, the smaller the fluctuation between the multiple differences is, which further indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are gradually decreasing; therefore, the first threshold is set to a value near 0, and the specific value of the first threshold can be set as needed; optionally, the first threshold is 1.

[0136] It should be noted that if the variance is greater than the first threshold, the electronic device determines that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals present a non-decreasing trend.

[0137] In this implementation, since the variance represents the fluctuation between multiple differences, the smaller the variance, the smaller the fluctuation between the multiple differences, which means that the differences between the multiple first sub-signals are more balanced, that is, based on the variance, the change trend of the first amplitude values ​​corresponding to the multiple first sub-signals can be accurately determined, and then when the variance is less than or equal to the first threshold, the result of determining that the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing has a high accuracy.

[0138] In another implementation, the first amplitude change information includes fitting parameters of the first amplitude values ​​corresponding to the multiple first sub-signals respectively. If the fitting parameters are less than the second threshold, the electronic device determines that the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals respectively show a decreasing trend.

[0139] The fitting parameter is the slope obtained by linearly fitting the first amplitude values ​​corresponding to the multiple first sub-signals. Accordingly, if the first amplitude values ​​corresponding to the multiple first sub-signals show a decreasing trend, the slope is a negative number; therefore, the second threshold is set to a negative number, and the specific value of the second threshold can be set as needed; optionally, the second threshold is -0.58, that is, the tangent value corresponding to the inclination angle of -30 degrees.

[0140] It should be noted that, if the fitting parameter is greater than the second threshold, the electronic device determines that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals present a non-decreasing trend.

[0141] In this implementation, since the amplitude value of the tail sound signal of the wake-up word gradually decreases, and the fitting parameter represents the change rate of the first amplitude values ​​corresponding to the multiple first sub-signals, that is, based on the fitting parameter, the change trend of the first amplitude values ​​corresponding to the multiple first sub-signals can be accurately determined, and then when the fitting parameter is less than the second threshold, the result of determining that the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing has a high accuracy.

[0142] In some embodiments, if the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals show a non-decreasing trend, the electronic device further determines the tail endpoint through the following steps (1)-(5).

[0143] (1) The electronic device determines a second audio signal, where the second audio signal is an audio signal following the first audio signal. (2) The electronic device divides the second audio signal into a plurality of second sub-signals. (3) The electronic device determines the second amplitude values ​​corresponding to the plurality of second sub-signals. (4) The electronic device obtains second amplitude change information of the plurality of second sub-signals based on the second amplitude values ​​corresponding to the plurality of second sub-signals, where the second amplitude change information is used to indicate the change trend of the second amplitude values ​​corresponding to the plurality of second sub-signals. (5) If the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are decreasing, the electronic device uses the end point of the second audio signal as the tail end point; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are not decreasing, the electronic device re-determines the second audio signal until the tail end point is determined. The specific implementation of steps (1)-(5) is the same as steps 301-305 and will not be described in detail here.

[0144] It should be noted that the second audio signal re-determined by the electronic device is the audio signal after the previous second audio signal; in an embodiment of the present application, the tail endpoint detection is performed on the audio signal after the wake-up word in sequence until the tail endpoint is determined, so that the tail sound signal of the wake-up word is included in the audio signal before the tail endpoint, thereby improving the accuracy of the determined tail endpoint.

[0145] In some embodiments, the first amplitude change information includes the variance and fitting parameters of the first amplitude values ​​corresponding to the plurality of sampling points. If the variance is less than or equal to the first threshold value, and the fitting parameter is less than the second threshold value, the electronic device determines that the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals are decreasing, thereby ensuring the accuracy of the subsequent determination of the tail endpoint. If the variance is greater than the first threshold value and the fitting parameter is less than the second threshold value, or the variance is less than or equal to the first threshold value and the fitting parameter is greater than the second threshold value, the electronic device re-executes steps 301-304, and re-determines the first amplitude change information by updating the length of the first audio signal, updating the division method of the plurality of first sub-signals, or updating the method of determining the first amplitude values ​​corresponding to the plurality of first sub-signals. If the variance and fitting parameters included in the first amplitude change information obtained after the update still do not satisfy that the variance is less than or equal to the first threshold value, and the fitting parameter is less than the second threshold value, the electronic device executes the above steps (1)-(5) to determine the second audio signal until the tail endpoint is determined.

[0146] In this embodiment, the parameters of the two dimensions of variance and fitting parameters are used to jointly determine the changing trends of the first amplitude values ​​corresponding to the multiple first sub-signals, thereby ensuring that the determined first amplitude change information represents that the changing trends of the first amplitude values ​​corresponding to the multiple first sub-signals are accurate, thereby ensuring the accuracy of the determined tail endpoint.

[0147] In some embodiments, after the electronic device identifies the wake-up word in the audio signal through the wake-up model, the third audio signal corresponding to the wake-up word is cached and the wake-up is not performed temporarily; after the electronic device determines the tail end point of the wake-up word, the audio signal between the third audio signal and the tail end point is combined with the cached third audio signal to obtain the target audio signal corresponding to the wake-up word; then the electronic device removes the target audio signal from the audio signal to obtain the remaining audio signal, and then wakes up the electronic device based on the wake-up word, and performs other voice detection activities based on the remaining audio signal, such as voice recognition.

[0148] In an embodiment of the present application, the electronic device first recognizes the wake-up word of the audio signal through a wake-up model. After recognizing the wake-up word, the electronic device does not immediately segment the audio signal, but accurately determines the tail end point of the wake-up word through the changing trend of the amplitude value of the audio signal after the wake-up word, and then segments the audio signal based on the tail end point. In this way, the tail sound signal of the wake-up word can be divided before the tail end point, thereby achieving accurate segmentation of the audio signal and reducing the problems caused by the division of the tail sound signal of the wake-up word into the audio signal after the tail end point, such as reducing the false triggering of VAD (Voice Activity Detection) or false recognition of speech in subsequent speech recognition.

[0149] An embodiment of the present application provides a method for detecting a voice endpoint, which is achieved by determining the amplitude change information of an audio signal after a wake-up word, and then determining the tail endpoint corresponding to the wake-up word based on the amplitude change information; since the amplitude value of the tail sound signal of the wake-up word gradually decreases, and the amplitude change information represents the change trend of the first amplitude values ​​of multiple first sub-signals in the first audio signal, the tail endpoint determined based on the amplitude change information is in line with the change law of the tail sound signal of the wake-up word, thereby improving the accuracy of the determined tail endpoint.

[0150] This application embodiment provides another method for detecting a voice endpoint. Figure 4 , methods include:

[0151] 401. The electronic device determines a first audio signal.

[0152] This step is the same as step 301 and will not be described again here.

[0153] 402. The electronic device divides a first audio signal into multiple first sub-signals.

[0154] This step is the same as step 302 and will not be described again here.

[0155] 403. The electronic device determines first amplitude values ​​corresponding to the plurality of first sub-signals respectively.

[0156] This step is the same as step 303 and will not be described again here.

[0157] 404. The electronic device confirms first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

[0158] The first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to indicate a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals.

[0159] It should be noted that the number of multiple first sub-signals corresponding to at least one sub-amplitude change information is less than the number of multiple first sub-signals included in the first audio signal; the multiple first sub-signals corresponding to at least one sub-amplitude change information are obtained by sequentially iterating the multiple first sub-signals in the first audio signal; wherein the electronic device iterates one first sub-signal each time to obtain multiple first sub-signals corresponding to the next sub-amplitude change information.

[0160] For example, if the number of the multiple first sub-signals corresponding to the sub-amplitude change information is a preset number, then the first sub-amplitude change information corresponds to the first sub-signals of the first preset number in the first audio signal; when iterating, the electronic device removes the first sub-signal of the first preset number of first sub-signals, and adds the first sub-signal after the first preset number of first sub-signals, so as to obtain the preset number of first sub-signals corresponding to the second sub-amplitude change information. Similarly, by iterating sequentially, multiple first sub-signals corresponding to at least one sub-amplitude change information can be obtained; and then, based on the first amplitude values ​​of the multiple first sub-signals corresponding to at least one sub-amplitude change information, at least one sub-amplitude change information is obtained. Among them, the specific implementation method of the electronic device determining the sub-amplitude change information is the same as the implementation method of the electronic device determining the first amplitude change information in step 304, which will not be repeated here.

[0161] 405. The electronic device determines target sub-amplitude change information based on at least one sub-amplitude change information.

[0162] The target sub-amplitude change information indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals are decreasing. The electronic device determines whether the sub-amplitude change information indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals are decreasing in the same manner as the electronic device determines whether the first amplitude change information indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals are decreasing in step 305, and will not be repeated here.

[0163] In one implementation, the electronic device obtains at least one sub-amplitude change information by sequentially iterating multiple first sub-signals in the first audio signal. When any sub-amplitude change information obtained by the electronic device indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals show a decreasing trend, the electronic device determines the sub-amplitude change information as the target sub-amplitude change information, and no longer determines other subsequent sub-amplitude change information, thereby saving time and avoiding waste of computing resources.

[0164] It should be noted that if at least one sub-amplitude change information does not include the target sub-amplitude change information, that is, the tail endpoint of the wake-up word and exists in the first audio signal, then the electronic device sequentially determines at least one sub-amplitude change information in the first audio signal, which will cause a waste of computing resources and time; and in order to avoid such waste, in some embodiments, the electronic device confirms the first amplitude change information of multiple first sub-signals based on the first amplitude values ​​corresponding to the multiple first sub-signals, and only determines the at least one sub-amplitude change information if the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing. When the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing, it indicates that the tail endpoint of the wake-up word exists in the first audio signal, and then based on at least one sub-amplitude change information, the position of the tail endpoint in the first audio signal can be further accurately determined, thereby avoiding the waste of computing resources and time.

[0165] It should be noted that if the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are not decreasing, the electronic device obtains the second audio signal after the first audio signal, and does not perform the step of determining at least one sub-amplitude change information until the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing. The implementation method of the electronic device determining whether the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing is the same as the implementation method of the electronic device determining whether the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are decreasing in step 305, and will not be repeated here.

[0166] 406. The electronic device uses the end point of the first sub-signal corresponding to the target sub-amplitude change information as the tail end point.

[0167] The end point of the first sub-signal corresponding to the target sub-amplitude change information is the end point of the last sub-signal among the multiple first sub-signals corresponding to the target sub-amplitude change information.

[0168] In some embodiments, the sub-amplitude variation information includes the variance and fitting parameters of the first amplitude values ​​corresponding to the plurality of sampling points, and the electronic device determines the target sub-amplitude variation information based on the variance and fitting parameters; if the first sub-signals corresponding to the target sub-amplitude variation information determined by the electronic device based on the variance and fitting parameters are the same, the electronic device uses the end point of the first sub-signal corresponding to any target sub-amplitude variation information as the tail end point. If the first sub-signals corresponding to the target sub-amplitude variation information determined by the electronic device based on the variance and fitting parameters are different, the electronic device uses the end point of the first sub-signal that is sorted later among the plurality of first sub-signals corresponding to the two target sub-amplitude variation information determined based on the variance and fitting parameters as the tail end point; if among the plurality of first sub-signals corresponding to the two target sub-amplitude variation information, the plurality of first sub-signals corresponding to the first target sub-amplitude variation information are the first first sub-signal to the fifth first sub-signal, and the plurality of first sub-signals corresponding to the second target sub-amplitude variation information are the second first sub-signal to the sixth first sub-signal, the electronic device uses the end point of the sixth first sub-signal as the tail end point.

[0169] In this embodiment, the target sub-amplitude change information is jointly determined by parameters of two dimensions, namely, variance and fitting parameters, thereby ensuring the accuracy of the determined target sub-amplitude change information, thereby ensuring the accuracy of the determined tail endpoint; and the end point of the first sub-signal sorted later in the multiple first sub-signals corresponding to the two target sub-amplitude change information determined based on the variance and the fitting parameters is taken as the tail endpoint, so that the tail sound signal of the wake-up word is included in the voice signal before the tail endpoint as much as possible, thereby further improving the accuracy of the determined tail endpoint.

[0170] In an embodiment of the present application, at least one sub-amplitude change information is obtained by iterating the multiple first sub-signals in the first audio signal in sequence. Since the sub-amplitude change information represents the change trend of the first amplitude values ​​corresponding to the multiple adjacent first sub-signals, the exact position of the wake-up word in the first audio signal can be determined based on the sub-amplitude change information, thereby further improving the accuracy of the determined tail endpoint.

[0171] The present application also provides a voice endpoint detection device, see Figure 5 , the device comprises:

[0172] A first determination module 501 is used to determine a first audio signal, where the first audio signal is an audio signal after the wake-up word in the audio signal;

[0173] A division module 502, configured to divide the first audio signal into a plurality of first sub-signals;

[0174] A second determination module 503 is used to determine first amplitude values ​​corresponding to the plurality of first sub-signals respectively;

[0175] A confirmation module 504 is used to confirm first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, where the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals;

[0176] The third determination module 505 is used to determine the tail endpoint corresponding to the wake-up word in the audio signal based on the first amplitude change information.

[0177] In some embodiments, the third determining module 505 is configured to use an end point of the first audio signal as a tail end point if the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are decreasing.

[0178] In some embodiments, the apparatus further comprises:

[0179] A fourth determination module, for determining that the first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and if the variance is less than or equal to a first threshold, determining that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are decreasing;

[0180] The fifth determination module is used for the first amplitude change information including fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. If the fitting parameters are less than the second threshold value, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the multiple first sub-signals show a decreasing trend, and the fitting parameters are used to represent the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals.

[0181] In some embodiments, the first amplitude change information includes a variance determined based on first amplitude values ​​corresponding to a plurality of first sub-signals respectively; a confirmation module 504 is used to determine a difference between first amplitude values ​​corresponding to any two adjacent first sub-signals in the plurality of first sub-signals to obtain a plurality of differences; and a variance is determined based on the plurality of differences.

[0182] In some embodiments, the first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals, and the fitting parameters are used to represent the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. The confirmation module 504 is used to perform linear fitting on the first amplitude values ​​respectively corresponding to the multiple first sub-signals to obtain the fitting parameters.

[0183] In some embodiments, the third determination module 505 is used to determine the second audio signal if the first amplitude change information indicates that the first amplitude values ​​corresponding to the plurality of first sub-signals are not decreasing, the second audio signal being the audio signal after the first audio signal; divide the second audio signal into a plurality of second sub-signals; determine the second amplitude values ​​corresponding to the plurality of second sub-signals; obtain second amplitude change information of the plurality of second sub-signals based on the second amplitude values ​​corresponding to the plurality of second sub-signals, the second amplitude change information being used to indicate the change trend of the second amplitude values ​​corresponding to the plurality of second sub-signals; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are decreasing, take the end point of the second audio signal as the tail end point; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are not decreasing, re-determine the second audio signal until the tail end point is determined.

[0184] In some embodiments, the first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to represent a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals;

[0185] The third determination module 505 is used to determine target sub-amplitude change information based on at least one sub-amplitude change information, where the target sub-amplitude change information indicates that the first amplitude values ​​corresponding to the corresponding multiple first sub-signals show a decreasing trend; and take the end point of the first sub-signal corresponding to the target sub-amplitude change information as the tail end point.

[0186] In some embodiments, the multiple first sub-signals respectively include multiple sampling points, and the second determination module 503 is used to determine the average of the absolute values ​​of the second amplitude values ​​of the multiple sampling points respectively included in the multiple first sub-signals based on the absolute values ​​of the second amplitude values ​​of the multiple sampling points respectively included in the multiple first sub-signals; and use the average as the first amplitude values ​​corresponding to the multiple first sub-signals respectively.

[0187] An embodiment of the present application provides a device for detecting a voice endpoint, which determines the amplitude change information of an audio signal after a wake-up word, and then determines the tail endpoint corresponding to the wake-up word based on the amplitude change information; since the amplitude value of the tail sound signal of the wake-up word gradually decreases, and the amplitude change information represents the change trend of the first amplitude values ​​of multiple first sub-signals in the first audio signal, the tail endpoint determined based on the amplitude change information is in line with the change law of the tail sound signal of the wake-up word, thereby improving the accuracy of the determined tail endpoint.

[0188] In some embodiments, the electronic device is provided as a terminal. Figure 6The structure block diagram of a terminal 600 provided by an exemplary embodiment of the present application is shown. The terminal 600 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 600 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0189] Typically, the terminal 600 includes a processor 601 and a memory 602 .

[0190] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0191] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one program code, which is used to be executed by the processor 601 to implement the voice endpoint detection method provided in the method embodiment of the present application.

[0192] In some embodiments, the terminal 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608 and a power supply 609.

[0193] The peripheral device interface 603 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0194] The radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The radio frequency circuit 604 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0195] The display screen 605 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to collect touch signals on the surface or above the surface of the display screen 605. The touch signal can be input to the processor 601 as a control signal for processing. At this time, the display screen 605 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 605 can be one, set on the front panel of the terminal 600; in other embodiments, the display screen 605 can be at least two, respectively set on different surfaces of the terminal 600 or in a folding design; in other embodiments, the display screen 605 can be a flexible display screen, set on the curved surface or folding surface of the terminal 600. Even, the display screen 605 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0196] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0197] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 601 for processing, or input them into the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 600. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 607 may also include a headphone jack.

[0198] The positioning component 608 is used to locate the current geographical location of the terminal 600 to implement navigation or LBS (Location Based Service). The positioning component 608 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0199] The power supply 609 is used to power various components in the terminal 600. The power supply 609 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0200] In some embodiments, the terminal 600 further includes one or more sensors 610 , including but not limited to: an acceleration sensor 611 , a gyroscope sensor 612 , a pressure sensor 613 , a fingerprint sensor 614 , an optical sensor 615 , and a proximity sensor 616 .

[0201] The acceleration sensor 611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 600. For example, the acceleration sensor 611 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for collecting game or user motion data.

[0202] The gyro sensor 612 can detect the body direction and rotation angle of the terminal 600, and the gyro sensor 612 can cooperate with the acceleration sensor 611 to collect the user's 3D actions on the terminal 600. The processor 601 can implement the following functions based on the data collected by the gyro sensor 612: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0203] The pressure sensor 613 can be set on the side frame of the terminal 600 and / or the lower layer of the display screen 605. When the pressure sensor 613 is set on the side frame of the terminal 600, it can detect the user's holding signal of the terminal 600, and the processor 601 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is set on the lower layer of the display screen 605, the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 605. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0204] The fingerprint sensor 614 is used to collect the user's fingerprint, and the processor 601 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 614, or the fingerprint sensor 614 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 601 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 614 can be set on the front, back, or side of the terminal 600. When a physical button or a manufacturer logo is set on the terminal 600, the fingerprint sensor 614 can be integrated with the physical button or the manufacturer logo.

[0205] The optical sensor 615 is used to collect the ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the display screen 605 according to the ambient light intensity collected by the optical sensor 615. Specifically, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is reduced. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 615.

[0206] The proximity sensor 616, also called a distance sensor, is usually arranged on the front panel of the terminal 600. The proximity sensor 616 is used to collect the distance between the user and the front of the terminal 600. In one embodiment, when the proximity sensor 616 detects that the distance between the user and the front of the terminal 600 is gradually decreasing, the processor 601 controls the display screen 605 to switch from the screen-on state to the screen-off state; when the proximity sensor 616 detects that the distance between the user and the front of the terminal 600 is gradually increasing, the processor 601 controls the display screen 605 to switch from the screen-off state to the screen-on state.

[0207] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the terminal 600, and the terminal 600 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0208] An embodiment of the present application also provides a computer-readable storage medium, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement the voice endpoint detection method of any of the above-mentioned implementation methods.

[0209] An embodiment of the present application also provides a computer program product, which includes a computer program code. The computer program code is stored in a computer-readable storage medium. A processor of an electronic device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the electronic device executes the voice endpoint detection method of any of the above-mentioned implementation methods.

[0210] In some embodiments, the computer program product involved in the embodiments of the present application may be deployed for execution on an electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network. Multiple electronic devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0211] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting a voice endpoint, characterized in that: The method comprises: Determine a first audio signal, where the first audio signal is an audio signal after the wake-up word in the audio signal; dividing the first audio signal into a plurality of first sub-signals; Determine first amplitude values ​​respectively corresponding to the plurality of first sub-signals; Based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, confirming first amplitude change information of the plurality of first sub-signals, wherein the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals; If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a non-decreasing trend, a second audio signal is determined, where the second audio signal is an audio signal subsequent to the first audio signal; dividing the second audio signal into a plurality of second sub-signals; Determine second amplitude values ​​respectively corresponding to the plurality of second sub-signals; Based on the second amplitude values ​​respectively corresponding to the plurality of second sub-signals, obtaining second amplitude change information of the plurality of second sub-signals, wherein the second amplitude change information is used to indicate a change trend of the second amplitude values ​​respectively corresponding to the plurality of second sub-signals; If the second amplitude change information indicates that the second amplitude values ​​corresponding to the plurality of second sub-signals are decreasing, the end point of the second audio signal is used as the tail end point corresponding to the wake-up word in the audio signal; If the second amplitude change information indicates that the second amplitude values ​​respectively corresponding to the plurality of second sub-signals show a non-decreasing trend, the second audio signal is re-determined until the tail endpoint is determined.

2. The method according to claim 1, characterized in that The method further comprises: If the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals show a decreasing trend, the end point of the first audio signal is used as the tail end point.

3. The method according to claim 2, characterized in that The method also includes any of the following implementations: The first amplitude change information includes a variance determined based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and if the variance is less than or equal to a first threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the plurality of first sub-signals are in a decreasing trend; The first amplitude change information includes fitting parameters of the first amplitude values ​​respectively corresponding to the multiple first sub-signals. If the fitting parameters are less than a second threshold, it is determined that the first amplitude change information indicates that the first amplitude values ​​respectively corresponding to the multiple first sub-signals show a decreasing trend, and the fitting parameters are used to indicate the change rate of the first amplitude values ​​respectively corresponding to the multiple first sub-signals.

4. The method according to claim 1 or 3, characterized in that: The first amplitude change information includes a variance determined based on first amplitude values ​​respectively corresponding to the plurality of first sub-signals; and confirming the first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes: Determine a difference between first amplitude values ​​corresponding to any two adjacent first sub-signals in the plurality of first sub-signals to obtain a plurality of difference values; Based on the plurality of difference values, the variance is determined.

5. The method according to claim 1 or 3, characterized in that: The first amplitude change information includes fitting parameters of first amplitude values ​​respectively corresponding to the plurality of first sub-signals, the fitting parameters are used to represent the change rates of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, and confirming the first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes: Linear fitting is performed on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals to obtain the fitting parameters.

6. The method according to claim 1, characterized in that The first amplitude change information includes at least one sub-amplitude change information, and the sub-amplitude change information is used to represent a change trend determined based on first amplitude values ​​corresponding to a plurality of adjacent first sub-signals; The method further comprises: Based on the at least one sub-amplitude change information, target sub-amplitude change information is determined, wherein the target sub-amplitude change information indicates that first amplitude values ​​corresponding to the plurality of first sub-signals corresponding thereto are in a decreasing trend; The end point of the first sub-signal corresponding to the target sub-amplitude change information is used as the tail end point.

7. The method according to claim 1, characterized in that The plurality of first sub-signals respectively include a plurality of sampling points, and determining the first amplitude values ​​respectively corresponding to the plurality of first sub-signals includes: Determine, based on the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals, an average value of the absolute values ​​of the second amplitude values ​​of the plurality of sampling points respectively included in the plurality of first sub-signals; The average value is used as the first amplitude values ​​respectively corresponding to the plurality of first sub-signals.

8. A voice endpoint detection device, characterized in that: The device comprises: A first determining module, configured to determine a first audio signal, where the first audio signal is an audio signal following a wake-up word in the audio signal; A division module, used for dividing the first audio signal into a plurality of first sub-signals; A second determining module, configured to determine first amplitude values ​​respectively corresponding to the plurality of first sub-signals; a confirmation module, configured to confirm first amplitude change information of the plurality of first sub-signals based on the first amplitude values ​​respectively corresponding to the plurality of first sub-signals, wherein the first amplitude change information is used to indicate a change trend of the first amplitude values ​​respectively corresponding to the plurality of first sub-signals; A third determination module is used to determine a second audio signal if the first amplitude change information indicates that the first amplitude values ​​corresponding to the multiple first sub-signals are not decreasing, and the second audio signal is an audio signal after the first audio signal; divide the second audio signal into multiple second sub-signals; determine the second amplitude values ​​corresponding to the multiple second sub-signals; based on the second amplitude values ​​corresponding to the multiple second sub-signals, obtain second amplitude change information of the multiple second sub-signals, the second amplitude change information is used to indicate the change trend of the second amplitude values ​​corresponding to the multiple second sub-signals; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the multiple second sub-signals are decreasing, use the end point of the second audio signal as the tail end point corresponding to the wake-up word in the audio signal; if the second amplitude change information indicates that the second amplitude values ​​corresponding to the multiple second sub-signals are not decreasing, re-determine the second audio signal until the tail end point is determined.

9. An electronic device, characterized in that: The electronic device includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the voice endpoint detection method as described in any one of claims 1 to claim 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the voice endpoint detection method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product includes a computer program code, which is stored in a computer-readable storage medium. The processor of the electronic device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the electronic device performs the voice endpoint detection method as described in any one of claims 1 to claim 7.

Images