Endpoint detection method and related device

Abstract

The application provides an endpoint detection method and related equipment applied to an intelligent device, and the method comprises the following steps: collecting initial speech of a user; obtaining first voiceprint information and a first interval duration of the initial speech; establishing a first corresponding relationship between the first voiceprint information and the first interval duration; collecting detection speech of the user; obtaining second voiceprint information and a second interval duration of the detection speech; establishing a second corresponding relationship between the second voiceprint information and the second interval duration; determining matched second voiceprint information and first voiceprint information; determining a first interval duration corresponding to the second interval duration; and updating the corresponding first interval duration according to the second interval duration to obtain an actual interval duration. The application updates the actual interval duration of the user by detecting the interval duration of the detection speech, so as to ensure the timeliness of the interval duration when the user interacts with the intelligent device by voice, and further improves the accuracy of endpoint detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer communication, and in particular to an endpoint detection method and related equipment. BACKGROUND

[0002] In recent years, with the rapid development of artificial intelligence technology, artificial intelligence devices are increasingly common in user life. Users can interact with artificial intelligence devices through voice, including weather queries, chatting, etc.

[0003] Voice activity detection (VAD) is an important technology for users to interact with artificial intelligence devices through voice. Voice activity detection is used to distinguish between voice and non-voice regions. Voice activity detection is used to accurately locate the beginning and end of the voice from the voice with noise, remove the silent part and noise part of the voice, and determine the truly effective content of the voice. The beginning point is the time when the artificial intelligence device detects human voice using voice activity detection technology. After detecting human voice, the artificial intelligence device does not detect human voice for a certain duration. When the duration meets the time threshold, the artificial intelligence device determines the time point when the duration ends as the end point. The time threshold is the interval duration, that is, the artificial intelligence device determines that the user does not output sound within the interval duration, and determines that the detection is complete, thereby determining the end point.

[0004] Because different users speak at different speeds, for example, children speak more slowly and adults speak more quickly. Existing artificial intelligence devices can determine the corresponding interval duration according to different users. However, the speaking speed of the same user is not fixed and will change accordingly with the user's age, experience, and scene. However, the interval duration of a specific user in existing artificial intelligence devices is fixed. When the user's speaking speed changes, if the artificial intelligence device uses voice activity detection to collect voice for a short interval duration, the user's input sentence will be mistakenly truncated. If the artificial intelligence device uses voice activity detection to collect voice for a long interval duration, the recognition speed will be slow and there will be additional invalid voice calculation, increasing the power consumption of the system. SUMMARY

[0005] Embodiments of the present application disclose an endpoint detection method and related equipment, which can adjust the interval duration according to the user's actual speaking speed during use, so that the artificial intelligence device can collect the user's voice according to the user's actual speaking speed.

[0006] A first aspect of the present application discloses an endpoint detection method applied to an artificial intelligence device, the method comprising:

[0007] Collecting initial voice of a user interacting with the artificial intelligence device;

[0008] extracting first voiceprint information of the initial voice;

[0009] obtaining a first interval duration of the initial voice;

[0010] establishing a first correspondence relationship between the first voiceprint information and the first interval duration;

[0011] collecting a detection voice of the user interacting with the smart device;

[0012] extracting second voiceprint information of the detection voice;

[0013] obtaining a second interval duration of the detection voice;

[0014] establishing a second correspondence relationship between the second voiceprint information and the second interval duration;

[0015] determining first voiceprint information in the first correspondence relationship that matches the second voiceprint information;

[0016] based on the first voiceprint information matching the second voiceprint information, determining a first interval duration corresponding to the second interval duration according to the first correspondence relationship and the second correspondence relationship;

[0017] updating the corresponding first interval duration according to the second interval duration to obtain an actual interval duration.

[0018] In this way, the interval duration stored in the smart device is updated according to the interval duration of the voice of the user interacting with the smart device for the first time, so as to ensure that the interval duration stored in the smart device is close to the actual interval duration of the user, and the accuracy of endpoint detection is improved.

[0019] In some embodiments, the method further comprises:

[0020] extracting an emotion type of the detection voice;

[0021] determining a corresponding interval adjustment amount according to the emotion type and the preset adjustment correspondence relationship, wherein the preset adjustment correspondence relationship comprises each emotion type and a corresponding interval adjustment amount;

[0022] adjusting the actual interval duration according to the interval adjustment amount.

[0023] In this way, the actual interval duration of the user is adjusted according to the emotion of the user, so that the interval duration of the endpoint detection of the smart device is close to the actual interval duration of the user, and the accuracy of endpoint detection is improved.

[0024] In some embodiments, the method further comprises:

[0025] obtaining an indication signal of a user input;

[0026] adjusting the actual interval duration according to the indication signal.

[0027] In this way, the actual interval duration is adjusted according to the actual situation of the user, so that the interval duration in the smart device is close to the actual situation.

[0028] In some embodiments, the determining the first voiceprint information in the first correspondence relationship that matches the second voiceprint information comprises:

[0029] determining a similarity between the second voiceprint information and the first voiceprint information in the correspondence relationship;

[0030] determining that the first voiceprint information and the second voiceprint information match when the similarity is greater than a voiceprint threshold.

[0031] In some embodiments, the updating the first interval duration according to the second interval duration to obtain an actual interval duration comprises:

[0032] determining a difference value between the first interval duration and the second interval duration;

[0033] determining whether the difference value is less than a difference threshold;

[0034] determining that the first interval duration is the actual interval duration when the difference value is greater than or equal to the difference threshold;

[0035] obtaining an average value of the first interval duration and the second interval duration when the difference value is less than the difference threshold, the average value being the actual interval duration.

[0036] In some embodiments, the method further comprises:

[0037] obtaining text information of the detected voice;

[0038] determining that a keyword exists in the text information;

[0039] obtaining an adjustment increment corresponding to the keyword;

[0040] adjusting the actual interval duration according to the adjustment increment.

[0041] In some embodiments, the obtaining a first interval duration of the initial voice comprises:

[0042] recognizing text information of the initial voice;

[0043] determining a number of texts in the text information;

[0044] determine speech speed information according to the number of characters and a preset interval length;

[0045] determine a first interval length of the initial speech according to the speech speed information.

[0046] The method further comprises:

[0047] extract a first emotion type of the initial speech, wherein the first correspondence further comprises a correspondence between the first emotion type and the first voiceprint information and the first interval length;

[0048] extract a second emotion type of the detected speech, wherein the second correspondence further comprises a correspondence between the second emotion type and the second voiceprint information and the second interval length;

[0049] determine, according to the first correspondence and the second correspondence, a first interval length corresponding to the second interval length based on the first voiceprint information matching the second voiceprint information, comprises:

[0050] determine, according to the first correspondence and the second correspondence, the first emotion type corresponding to the second emotion type based on the first voiceprint information matching the second voiceprint information;

[0051] determine the second interval length and the first interval length corresponding to the second emotion type and the first emotion type.

[0052] The second aspect of the present application discloses a computer readable storage medium, comprising computer instructions, when the computer instructions run on a smart device, make the smart device execute the endpoint detection method as described in the first aspect.

[0053] The third aspect of the present application discloses a smart device, comprising a processor and a memory, the memory is used to store instructions, the processor is used to call the instructions in the memory, make the smart device execute the endpoint detection method as described in the first aspect.

[0054] The fourth aspect of the present application discloses a chip system, which is applied to a smart device; the chip system comprises an interface circuit and a processor; the interface circuit and the processor are interconnected through a circuit; the interface circuit is used to receive a signal from a memory of the smart device, and send a signal to the processor, the signal comprises computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes the endpoint detection method as described in the first aspect.

[0055] The fifth aspect of the present application discloses a computer program product, which, when running on a computer, causes the computer to perform the endpoint detection method of the first aspect.

[0056] The sixth aspect of the present application discloses an apparatus having a function of implementing the intelligent device behavior in the method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0057] It should be understood that the computer readable storage medium provided in the second aspect, the intelligent device in the third aspect, the chip system in the fourth aspect, the computer program product in the fifth aspect, and the apparatus in the sixth aspect all correspond to the method in the first aspect, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is an application scenario diagram of the endpoint detection method provided by the embodiments of the present application.

[0059] Figure 2 is a flowchart of the endpoint detection method provided by the embodiments of the present application.

[0060] Figure 3 is a flowchart of the endpoint detection method provided by the embodiments of the present application.

[0061] Figure 4 is a structural diagram of an intelligent device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0062] For the convenience of understanding, exemplary descriptions of some concepts related to the embodiments of the present application are given for reference.

[0063] It should be noted that "at least one" in the present application means one or more, and "multiple" means two or more than two. "And / or" describes the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0064] In order to better understand the endpoint detection method and related devices disclosed in the embodiments of the present application, first, the application scenario of the endpoint detection method of the present application will be described.

[0065] Figure 1 is a schematic diagram of an application scenario of the endpoint detection method provided in the embodiments of the present application.

[0066] As shown in Figure 1 The endpoint detection method provided in the embodiments of the present application can be applied to a terminal device, and the terminal device can store an interval duration corresponding to a user. When the user performs voice interaction with the terminal device, the terminal device determines a tail endpoint of voice endpoint detection according to the interval duration. As can be easily understood, different users correspond to different interval durations, and the terminal device can determine tail endpoints of voice output by different users based on the interval durations of the different users when processing voice signals. However, the speaking speed of each user is not fixed. As can be easily understood, the speaking speed of the user will change when the user is in different scenarios or has different moods. If the terminal device uses a fixed interval duration to collect and recognize voice information input by the user, there is a situation that the voice of the user cannot be completely collected.

[0067] Please refer to Figure 1 , the user says to the intelligent device "I want to go to A place, first navigate to B place", and if the interval duration corresponding to the user stored in the intelligent device is too short, the intelligent device only collects "I want to go to A place, first navigate to", and the intelligent device understands that it is to navigate to A place, misinterprets the original intention of the user and performs an incorrect operation, which affects the user experience.

[0068] In this embodiment, the intelligent device is a smart speaker.

[0069] The terminal device includes a voice collection device (for example, a microphone array), which is configured to collect voice.

[0070] It can be understood that the terminal device in the embodiments of the present application can refer to a user equipment, an access terminal, a user unit, a user station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent or a user apparatus. The terminal device can be a mobile phone, a pad, a computer with wireless transceiver function, a session initiation protocol (SIP) phone, a personal digital assistant (PDA), a handheld device with wireless communication function, a computer or other processing device, a vehicle-mounted device, a wearable device, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in a smart home, a terminal device in a future 5G network or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application do not limit this.

[0071] In other embodiments of the present application, the smart device (for example, a smart sound box) can also be in communication connection with a server. The server can provide content services for the smart device, for example, providing songs, stories, movies and the like that are on demand by the user for the smart device. The smart device can also send the collected voice to the server for processing of the collected voice. For example, the smart device can send the collected voice to the server for voice recognition.

[0072] Figure 2 is a flowchart of an endpoint detection method provided by an embodiment of the present application. The endpoint detection method provided by the embodiment of the present application is applied to a smart device, for example, a smart device in Figure 1 The endpoint detection method provided by the embodiment of the present application includes the following steps:

[0073] S201, collecting initial voice of a user.

[0074] In an embodiment of the present application, the initial voice is the voice collected by the smart device when the user first performs voice interaction with the smart device.

[0075] When the smart device does not store the voice feature information corresponding to the user, the voice of the user and the smart device in voice interaction can be referred to as initial voice. The voice feature information can include voiceprint information of the user and / or interval duration corresponding to the user.

[0076] For example, in the process of using the smart device, the voice information input by the user, for example, "Xiaoyi Xiaoyi, play music", is collected by the voice collection device of the smart device.

[0077] When the method provided in the application is applied to the management server, in the process that the user uses the smart device, the smart device collects the voice information input by the user and sends the voice information to the management server, and the management server starts to collect the voice information.

[0078] For the smart device with the wake-up function (for example, a smart sound box), the operation of the smart device collecting voice information can be realized by starting to collect voice information after detecting a preset wake-up word.

[0079] S202, first voiceprint information of the initial voice is extracted.

[0080] The voiceprint information includes cepstrum, formant, pitch, etc., and the smart device can uniquely identify the corresponding user according to the voiceprint information. In this way, each user has unique voiceprint information.

[0081] The voiceprint information of multiple users can be stored in the smart device, and the smart device can identify the users according to the voiceprint information and distinguish different users based on different voiceprint information.

[0082] S203, a first interval duration of the initial voice is obtained.

[0083] The smart device identifies the voice input by the user according to a voice endpoint detection technology, and when the user's voice is detected within a preset time, the smart device determines the tail endpoint, and the preset time is the interval duration.

[0084] Optionally, when the corresponding interval duration of the user is not stored in the smart device, when the user interacts with the smart device by voice, the smart device determines the head endpoint and the tail endpoint of the voice input by the user according to the stored preset interval duration, identifies the text information of the initial voice, determines the number of texts in the text information, determines the speech rate information according to the number of texts and the preset interval duration, and determines the first interval duration of the initial voice according to the speech rate information.

[0085] In an embodiment, the speech rate information is calculated by the following formula: speech rate = interval duration / number of texts, wherein the interval duration is the preset interval duration in the smart device.

[0086] For example, the user says three words "school" in 2 seconds, and the speech rate of the user is 2 / 3 seconds per word, wherein 2 / 3 seconds is 666 milliseconds, and the first interval duration can be 666 milliseconds. Of course, according to different user habits, the corresponding relationship between the speech rate and the interval duration can be set according to the actual scene.

[0087] Further, the smart terminal stores a correction value, and the smart terminal adjusts the speech rate information according to the correction value to obtain the first interval duration.

[0088] The first interval duration is the sum of the correction value and the speech speed value. For example, the correction value is 100 milliseconds, and the speech speed is 300 milliseconds. Thus, the first interval duration is 400 milliseconds.

[0089] In some embodiments, the smart device can employ an automatic speech recognition technology to convert the user's speech into text.

[0090] S204, establishing a first corresponding relationship between the first voiceprint information and the first interval duration.

[0091] Optionally, a corresponding relationship table can be established between the first voiceprint information and the first interval duration. The first voiceprint information and the first interval duration can be identified by an identification bit, so that the first voiceprint information and the corresponding first interval duration are bound.

[0092] The smart device stores the first voiceprint information and the first interval duration of multiple users. The relationship between the first voiceprint information and the first interval duration of the multiple users forms the first corresponding relationship.

[0093] Further, the voiceprint information is used to bind the user information instead of other user identifiers such as phone numbers, account numbers, etc. Thus, the smart device can interact with multiple users by voice, not limited to the owner of the smart device.

[0094] S205, collecting the detection speech of the user.

[0095] The detection speech is the speech information input by the user collected by the smart device during the non-first voice interaction between the user and the smart device.

[0096] Optionally, in order to save the power of the smart device, the smart device can collect the detection speech periodically. For example, the collection period can be 24 hours. The user can interact with the smart device multiple times within 24 hours. The smart device can collect the speech of one of the voice interactions as the detection speech.

[0097] Optionally, the smart device can collect the interaction speech between the user and the smart device within a preset time period as the detection speech. For example, the speech between 12:00 and 13:00 is the interaction speech.

[0098] Optionally, the detection speech can be one or multiple pieces of speech.

[0099] S206, extracting the second voiceprint information of the detection speech, obtaining the second interval duration of the detection speech, and establishing a second corresponding relationship between the second voiceprint information and the second interval duration.

[0100] The method for obtaining the second interval duration is the same as the method for obtaining the first interval duration in S203. The smart device can determine the tail end point of the collected voice based on the first interval duration.

[0101] In S207, the first voiceprint information in the first corresponding relationship that matches the second voiceprint information is determined.

[0102] Optionally, since the voiceprint information uniquely identifies a user, by matching the first voiceprint information and the second voiceprint information, it is ensured that the first voiceprint information determined by the initial voice and the second voiceprint information determined based on the detected voice belong to the same user output.

[0103] Further, the matching method includes: calculating the similarity of the second voiceprint information and the first voiceprint information in the first corresponding relationship, and judging whether the similarity is greater than a voiceprint threshold; determining that the second voiceprint information and the first voiceprint information are matched if the similarity is greater than the voiceprint threshold, i.e., the second voiceprint information and the first voiceprint information identify the same user.

[0104] Further, if the similarity of the second voiceprint information and the first voiceprint information in the corresponding relationship is less than or equal to the voiceprint threshold, it is determined that there is no input voice feature of the user in the first corresponding relationship, it is determined that the user is a new user of the smart device, and the first corresponding relationship is updated according to the second voice feature, i.e., the second voiceprint information and the second interval duration of the new user are added to the first corresponding relationship.

[0105] The calculation method of the similarity includes but is not limited to Minkowski distance, Manhattan distance, Euclidean distance, and Chebyshev distance.

[0106] In S208, based on the first voiceprint information that matches the second voiceprint information, the first interval duration corresponding to the second interval duration is determined according to the first corresponding relationship and the second corresponding relationship.

[0107] If there is first voiceprint information in the first corresponding relationship that matches the second voiceprint information, the first interval duration corresponding to the first voiceprint information in the first corresponding relationship and the second interval duration corresponding to the second voiceprint information in the second corresponding relationship belong to the voice features of the same user. According to the matched first voiceprint information and the second voiceprint information, the matched first interval duration and the second interval duration are determined.

[0108] The first interval duration and the second interval duration can be the interval durations corresponding to the voices output by the same user at different times or in different scenes.

[0109] In S209, the first interval duration is updated according to the second interval duration to obtain an actual interval duration.

[0110] The second interval duration corresponds to the first interval duration, i.e., the interval duration of the voice input by the same user in different scenarios.

[0111] Specifically, the actual interval duration is obtained by calculating the average of the second interval duration and the first interval duration. Of course, the actual interval duration can also be obtained by other methods, such as a weight function.

[0112] For example, the first interval duration is 500 milliseconds, and the second interval duration is 600 milliseconds. The actual interval duration is 550 milliseconds.

[0113] In an embodiment, the smart device determines a difference value between the second interval duration and the first interval duration, and determines whether the difference value is within a difference threshold range. If the difference value is not within the difference threshold range, it is determined that the voice feature of the voice collected by the smart device this time belongs to a special scenario, and the first interval duration is replaced by the actual interval duration, and the first interval duration is determined as the actual interval duration. If the difference value is within the difference threshold, the actual interval duration is obtained by calculating the average of the second interval duration and the first interval duration.

[0114] The difference value can be a ratio or a specific duration, for example, the second interval duration is 89% or 500 ms larger than the first interval duration.

[0115] For example, the first interval duration of user A is 500 milliseconds, and the difference threshold is 60 milliseconds. If the second interval duration is 300 milliseconds, the difference value between the first interval duration and the second interval duration is 300 milliseconds and greater than 60 milliseconds, and the actual interval duration is 500 milliseconds. If the second interval duration is 450 milliseconds, the difference value between the first interval duration and the second interval duration is 50 milliseconds and less than 60 milliseconds, and the actual interval duration is 475 milliseconds.

[0116] In this way, when the user first interacts with the smart device by voice, the smart device records the feature information of the voice input by the user, wherein the feature information includes voiceprint information and interval duration. When the user interacts with the smart device for the second time, the smart device collects and recognizes the voice input by the corresponding user according to the stored feature information, so that different users use different interval durations. Further, the smart device updates the feature information of the corresponding user stored in the smart device according to the feature information of the voice input by the user for the second time, so that the interval duration stored in the smart device matches the actual interval duration input by the user, and the accuracy of the smart device is improved.

[0117] Optionally, the smart device stores the feature information corresponding to the user after the first voice interaction between the user and the smart device, and the user can input an indication signal through the keys, touch screen or voice interaction of the smart device, and the smart device adjusts the actual interval duration according to the indication signal. The adjustment includes increasing the actual interval duration and decreasing the actual interval duration.

[0118] For example, the smart device stores the actual interval duration corresponding to the user, and the user of the smart device speaks slowly due to recent tongue pain, so the user can input an indication signal by touching the display screen of the smart device to increase the actual interval duration.

[0119] During the voice interaction between the user and the smart device, there are many factors that affect the user's speaking speed, such as the user's mood. For example, the user's speaking speed when excited is much faster than when the user's mood is low, for example, "I want that clothes, please hand it over", the user finishes speaking in about 3 seconds in an urgent situation, but when the user's mood is stable, it may take 5 seconds or even more.

[0120] In some embodiments, the smart device collects user input to detect speech and extracts voice features of the detected speech, wherein the voice features further include emotion types, wherein the emotion types can include positive emotions and negative emotions, and of course the emotion types can also include other types. The smart device stores a preset adjustment corresponding relationship, and the preset adjustment corresponding relationship includes each emotion type and the corresponding interval adjustment amount. The smart device adjusts the actual interval duration according to the interval adjustment amount.

[0121] For example, the actual interval duration is 500 milliseconds, the user's emotion type is negative emotion, and the interval adjustment amount corresponding to the negative emotion is 30 milliseconds, then the smart device adjusts the actual interval duration according to the interval adjustment amount, and the actual interval duration of the smart device collecting the user's input speech this time is 530 milliseconds.

[0122] In an embodiment, when the user interacts with the smart device for the first time, the smart device is also used to extract the first emotion type in the initial voice, and establish a corresponding relationship between each first emotion type and the first voiceprint information and the first interval duration.

[0123] In this way, the voiceprint information is used to identify the user, the user has multiple emotion types, and each emotion type corresponds to an interval duration.

[0124] Optionally, the smart device collects the speech of multiple users and determines the first emotion type of each speech and the first voiceprint information and the first interval duration corresponding to the first emotion type, and then requests the average value of the first interval duration of multiple speeches of the same emotion type. The average value is the first interval duration corresponding to the emotion type stored in the smart device.

[0125] Optionally, the smart device is further configured to extract a second emotion type of the detected voice, determine a second emotion type corresponding to the second voiceprint information and a first emotion type corresponding to the first voiceprint information according to the first corresponding relationship in which the first voiceprint information matches the second voiceprint information, determine a first interval duration of the first emotion type pair and a second interval duration corresponding to the second emotion type, and update the first interval duration according to the second interval duration. In this way, the interval duration corresponding to the user is further limited by the emotion type, and the interval duration corresponding to the characteristic emotion type can be updated according to the detected voice.

[0126] During the voice interaction between the user and the smart device, when the user controls the smart device to perform a specific operation through voice, for example, "play A Love Song", after the user says "play", most users habitually pause for a preset duration, think about which song to play, and then output "A Love Song". Of course, there are other similar scenarios, for example, "navigate to Erqi Square", after the user says "navigate to", pause for a preset duration, think about the place to navigate to, and then output "Erqi Square".

[0127] In an embodiment, the smart device is further configured to recognize the detected voice to obtain text information, determine whether the text information contains a first keyword, obtain an adjustment increment corresponding to the first keyword, and adjust the actual interval duration according to the adjustment increment.

[0128] For example, if the keyword in the detected voice is "navigate", and the adjustment increment corresponding to the keyword "navigate" is 60 milliseconds, then the actual interval duration is increased by 60 milliseconds.

[0129] In an embodiment, the keyword is a verb, and the smart device stores adjustment increments corresponding to the keyword. Each keyword corresponds to a unique adjustment increment.

[0130] In an embodiment, the smart device determines the type of the keyword, and determines the corresponding adjustment increment according to the type.

[0131] Specifically, the text information contains a keyword, and the keyword is a verb, i.e., the operation performed by the user through voice control of the smart device, such as navigating to Nanjing or broadcasting news. If the verb can be followed by different nouns, the time for the user to pause and think when inputting the voice containing the verb is different. For example, playing music, playing news, telling a story, and playing A Love Song, playing real-time news in Zhengzhou, and telling the story of Snow White, the time for the user to pause when inputting the voice is different. Therefore, the smart device is further configured to determine the type of the keyword, and determine the corresponding adjustment increment according to the type of the keyword.

[0132] Further, the adjustment increments of all keywords are the same.

[0133] Please refer toFigure 3 An endpoint detection method provided by the present application.

[0134] S301, collect the voice input by the user.

[0135] S302, determine the start endpoint and the end endpoint of the voice of the user according to the preset interval duration.

[0136] S303, buffer the voice of the preset length.

[0137] S304, extract the voiceprint information of the voice of the preset length.

[0138] S305, obtain the similarity between the voiceprint information and the standard voiceprint information stored by the intelligent terminal, and determine whether the similarity is greater than a voiceprint threshold.

[0139] The standard voiceprint information can be the voiceprint information corresponding to the user stored by the intelligent terminal. Each user has unique voiceprint information, the intelligent terminal stores the voiceprint information of multiple users, and the intelligent terminal stores the interval duration corresponding to each voiceprint information.

[0140] If the similarity is greater than the voiceprint threshold, execute step S306, obtain the actual interval duration corresponding to the standard voiceprint information.

[0141] If the similarity is less than or equal to the voiceprint threshold, execute step 307, determine the interval duration of the voice as the preset interval duration.

[0142] S308, determine the start endpoint and the end endpoint of the voice of the user according to the actual interval duration.

[0143] S309, recognize the text information of the voice.

[0144] S310, determine whether there is a keyword in the text information.

[0145] If there is a keyword, S311, obtain the adjustment increment corresponding to the keyword, and adjust the actual interval duration according to the adjustment increment.

[0146] If there is no keyword, the process ends.

[0147] Figure 4 is a structural schematic diagram of an intelligent device provided by an embodiment of the present application. As shown in Figure 4As shown, the smart device 40 can include a processor 401, a memory 402, a wireless communication module 403, an audio module 404, a microphone 405, a sensor 406, a camera 407, a display screen 408, and the like. It can be understood that the structure shown in the embodiment does not constitute a specific limitation on the smart device 40. In other embodiments of the present application, the smart device 40 can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0148] The processor 401 can include one or more processing units. For example, the processor 401 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a display process unit (DPU), and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors.

[0149] The memory in the processor 401 can be configured to store instructions and data. In some embodiments, the memory in the processor 401 is a cache memory. The memory can save instructions or data that the processor 401 has just used or repeatedly uses. If the processor 401 needs to use the instructions or data again, it can be directly called from the memory, avoiding repeated access and reducing the waiting time of the processor 401, thereby improving the efficiency of the smart device 40.

[0150] In some embodiments, the processor 401 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative, and does not constitute a structural limitation of the smart device 40. In another embodiment of the present application, the smart device 40 can also use different interface connection modes or a combination of multiple interface connection modes in the above embodiments.

[0151] The memory 402 can be used to store one or more computer programs including instructions. The processor 401 can execute the instructions stored in the memory 402, so as to make the smart device 40 perform the related actions in the embodiments of the present application. The memory 402 can include a program storage area and a data storage area. The program storage area can store an operating system; the program storage area can also store one or more application programs. The data storage area can store data created during the use of the smart device 40 (such as photos, contacts), etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. In some embodiments, the processor 401 can execute the instructions stored in the memory 402 and / or the instructions stored in the memory arranged in the processor 401, so as to make the smart device 40 perform various functional applications and data processing.

[0152] The wireless communication function of the smart device 40 can be implemented through a wireless communication module 403. The wireless communication module 403 can provide a solution for wireless communication applied to the smart device 40, including wireless local area networks (WLAN), Bluetooth, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and the like. The wireless communication module 403 can be one or more devices integrated with at least one communication processing module. The wireless communication module 403 in the embodiments of the present application is used to implement the transceiver function of the electronic device.

[0153] The smart device 40 can implement audio functions such as music playing, recording, etc. through an audio module 404, a microphone 405, etc. The audio module 404 is used to convert digital audio information into an analog audio signal output, and is also used to convert analog audio input into a digital audio signal. The audio module 404 can also be used to encode and decode audio signals. In some embodiments, the audio module 404 can be disposed in the processor 401, or part of the function modules of the audio module 404 can be disposed in the processor 401. The smart device 40 can be provided with at least one microphone 405. In other embodiments, the smart device 40 can be provided with two microphones 405, which can not only collect sound signals, but also implement noise reduction functions. In other embodiments, the smart device 40 can also be provided with three, four or more microphones 405, which can not only collect sound signals and reduce noise, but also identify the source of the sound, implement directional recording functions, etc.

[0154] The sensor 406 can include a pressure sensor 406A, a distance sensor 406B, a proximity light sensor 406C, etc. The pressure sensor 406A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 406A can be disposed on the display screen 408, and the smart device 40 detects the touch operation intensity according to the pressure sensor 406A. The distance sensor 406B is used to measure distance. The smart device 40 can measure distance through infrared or laser. The proximity light sensor 406C can include a light-emitting diode (LED) and a light detector. The light-emitting diode can be an infrared light-emitting diode. The light detector can be a photodiode. The smart device 40 emits infrared light outwardly through the light-emitting diode. The smart device 40 detects infrared reflected light from nearby objects using the light detector. When sufficient reflected light is detected, it can be determined that there is an object near the smart device 40. When insufficient reflected light is detected, the smart device 40 can determine that there is no object near the smart device 40.

[0155] The smart device 40 can perform a shooting function through one or more cameras 407. Additionally, the smart device 40 can perform a display function through a display screen 408. The display screen 408 is used to display images, videos, etc. The display screen 408 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a Micro LED, a Micro OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the smart device 40 may include one or more display screens 408.

[0156] In this embodiment, the presence of a user around the smart device 40 can be detected by the distance sensor 406B and the proximity light sensor 406C. The camera 407 can collect a set of images of the user, and the processor 401 is used to execute the actions described in the above embodiment. Based on Figure 4 The smart device 40 shown can implement the endpoint detection method in the above embodiments. It should be understood that... Figure 4 The structural description in the text is an example of a smart device 40.

[0157] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on a smart device, the smart device performs the aforementioned method steps to implement the endpoint detection method in the above embodiment.

[0158] This embodiment also provides a computer program product that, when run on a smart device, causes the smart device to perform the aforementioned steps to implement the endpoint detection method in the above embodiment.

[0159] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the endpoint detection method in the above method embodiments.

[0160] The intelligent device, the computer storage medium, the computer program product or the chip provided in the embodiment are used for executing the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0162] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another device, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0163] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0164] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0165] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or in other words the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0166] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for endpoint detection, applied to a smart device, comprising: The method comprises: collecting initial voice of user interacting with the smart device; extracting first voiceprint information of the initial voice; acquiring first interval duration of the initial voice; establishing first corresponding relationship between the first voiceprint information and the first interval duration; collecting detection voice of user interacting with the smart device; extracting second voiceprint information of the detection voice; acquiring second interval duration of the detection voice; establishing second corresponding relationship between the second voiceprint information and the second interval duration; determining first voiceprint information in the first corresponding relationship matching the second voiceprint information; determining first interval duration corresponding to the second interval duration according to the first corresponding relationship and the second corresponding relationship based on the first voiceprint information matching the second voiceprint information; updating the corresponding first interval duration according to the second interval duration to obtain actual interval duration, comprising: determining difference value between the first interval duration and the second interval duration; if the difference value is greater than or equal to difference threshold value, determining the first interval duration as the actual interval duration; if the difference value is less than the difference threshold value, acquiring average value of the first interval duration and the second interval duration, and determining the average value as actual interval duration.

2. The method of endpoint detection of claim 1, wherein, The method further comprises: extracting emotion type of the detection voice; determining corresponding interval adjustment amount according to the emotion type and preset adjustment corresponding relationship, wherein the preset adjustment corresponding relationship comprises each emotion type and corresponding interval adjustment amount; adjusting the actual interval duration according to the interval adjustment amount.

3. The method of endpoint detection of claim 1, wherein, The method further comprises: acquiring indication signal input by the user; adjusting the actual interval duration according to the indication signal.

4. The method of endpoint detection of claim 1, wherein, The determination of the first voiceprint information in the first corresponding relationship matching the second voiceprint information comprises: determining similarity of the second voiceprint information and the first voiceprint information in the first corresponding relationship; determining that the first voiceprint information with the similarity greater than voiceprint threshold value matches the second voiceprint information.

5. The method of endpoint detection of claim 1, wherein, The method further comprises: acquiring text information of the detection voice; determining that there is a keyword in the text information; acquiring adjustment increment corresponding to the keyword; adjusting the actual interval duration according to the adjustment increment.

6. The method of endpoint detection of claim 1, wherein, The acquisition of the first interval duration of the initial voice comprises: recognizing text information of the initial voice; determining text quantity in the text information; determining speech speed information according to the text quantity and preset interval duration; determining the first interval duration of the initial voice according to the speech speed information.

7. The method of endpoint detection of claim 1, wherein, The method further comprises: extracting first emotion type of the initial voice, wherein the first corresponding relationship further comprises corresponding relationship between the first emotion type and the first voiceprint information and the first interval duration; extracting second emotion type of the detection voice, wherein the second corresponding relationship further comprises corresponding relationship between the second emotion type and the second voiceprint information and the second interval duration; The first interval time length corresponding to the second interval time length is determined according to the first correspondence relationship and the second correspondence relationship based on the first voiceprint information matched with the second voiceprint information. The first emotion type corresponding to the second emotion type is determined according to the first correspondence relationship and the second correspondence relationship based on the first voiceprint information matched with the second voiceprint information. The second interval time length and the first interval time length corresponding to the second emotion type and the first emotion type are determined.

8. A computer-readable storage medium, characterized in that, The computer program product comprises computer instructions, when the computer instructions are executed on the smart device, the smart device executes the endpoint detection method as claimed in any one of claims 1 to 7.

9. A smart device, comprising: The smart device comprises a processor and a memory, the memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the smart device executes the endpoint detection method as claimed in any one of claims 1 to 7.

10. A chip system applied to a smart device; the chip system comprises an interface circuit and a processor; the interface circuit and the processor are interconnected through a line; the interface circuit is used to receive a signal from a memory of the smart device and send a signal to the processor, the signal comprises computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes the endpoint detection method as claimed in any one of claims 1 to 7.

11. A computer program product, characterised in that, When the computer program product is executed on the smart device, the smart device executes the endpoint detection method as claimed in any one of claims 1 to 7.

Images