Language recognition method and device, electronic equipment and product

Abstract

The application provides a language recognition method and device, electronic equipment, storage medium and product. The method extracts a speech segment from the to-be-recognized speech according to a preset sliding window, and detects a voiceprint turning point from the to-be-recognized speech. In the case that the voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint of the first sliding window. The speech segment in the first sliding window after the position adjustment is subjected to language recognition, and a language recognition result is obtained. The voiceprint turning point in the sliding window can be detected, the speaker jump moment in the sliding window can be recognized, the sliding window is moved according to the jump moment to divide the speech segment, the sliding window can avoid containing multiple languages, the speech segment with a single language in the sliding window is subjected to language recognition, the accuracy of the language recognition of the to-be-recognized speech is higher, and the accuracy of the speech translation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech processing, and in particular to a target tracking method and device, electronic equipment and product. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, intelligent translation machines and other products have gradually become important tools for cross-language communication. These devices can convert one language into another in real time through speech recognition, machine translation, and speech synthesis, greatly facilitating user communication in different language environments.

[0003] In actual use, the intelligent translation machine needs to quickly identify the language used by the current speech of the user, and then translate the current speech based on the language used to ensure the accuracy of the translation result. The existing language identification method uses a traditional sliding window method to identify the language in the window. However, when the window contains multiple languages, only the language with more content can be identified as the language corresponding to the window, thereby affecting the accuracy of language identification and further affecting the accuracy of speech translation. SUMMARY

[0004] Based on the above needs, the present application proposes a language identification method, device, electronic equipment and product, which can improve the accuracy of language identification and further improve the accuracy of speech translation.

[0005] To achieve the above object, the present application proposes the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, a language identification method is provided, comprising:

[0007] A sliding window of a preset size is used to extract a speech segment from the to-be-identified speech, and a voiceprint turning point is detected from the to-be-identified speech;

[0008] In the case where a voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint position of the first sliding window;

[0009] The speech segment in the first sliding window after the position adjustment is subjected to language identification, and a corresponding language identification result is obtained.

[0010] Optionally, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint position of the first sliding window, comprising:

[0011] in a case where the ending point position of the second sliding window is not adjacent to and does not coincide with the voiceprint turning point, shifting the first sliding window forward so that the voiceprint turning point is located at the ending point position of the first sliding window;

[0012] or,

[0013] in a case where the ending point position of the second sliding window is adjacent to or coincides with the voiceprint turning point, shifting the first sliding window backward so that the voiceprint turning point is located at the starting point position of the first sliding window; the second sliding window is a previous sliding window of the first sliding window.

[0014] Optionally, the language recognition method further comprises:

[0015] in a case where no voiceprint turning point is detected from the voice segment in the first sliding window, performing language recognition on the voice segment in the first sliding window to obtain a corresponding language recognition result.

[0016] Optionally, detecting the voiceprint turning point from the voice to be recognized comprises:

[0017] inputting a voice data stream of the voice to be recognized into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects a voiceprint turning point from the voice to be recognized and outputs voiceprint turning point information when the voiceprint turning point is detected.

[0018] Optionally, a sliding window of a preset size is used to extract a voice segment from the voice to be recognized; in a case where a voiceprint turning point is detected from the voice segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point so that the voiceprint turning point is located at an endpoint position of the first sliding window; and language recognition is performed on the voice segment in the first sliding window after the adjustment in position to obtain a corresponding language recognition result, comprising:

[0019] inputting the voice to be recognized into a pre-trained multi-language recognition model, so that the multi-language recognition model extracts a voice segment from the voice to be recognized using a sliding window of a preset size; in a case where voiceprint turning point information output by the voiceprint turning point detection model is received, adjusting the first sliding window containing the voiceprint turning point in position according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at an endpoint position of the first sliding window; and performing language recognition on the voice segment in the first sliding window after the adjustment in position to obtain a corresponding language recognition result.

[0020] Optionally, performing language recognition on the voice segment in the first sliding window after the adjustment in position to obtain a corresponding language recognition result comprises:

[0021] performing overlap analysis on the speech segment in the second sliding window and the speech segment in the first sliding window after the shift adjustment, to obtain a speech overlap degree;

[0022] if the speech overlap degree reaches a preset overlap degree threshold, taking the language recognition result corresponding to the speech segment in the second sliding window as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment;

[0023] if the speech overlap degree does not reach the preset overlap degree threshold, performing language recognition on the speech segment in the first sliding window after the shift adjustment, to obtain a corresponding language recognition result.

[0024] Optionally, in a case where the voiceprint turning point detection model is deployed on a remote server, a size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size is a hybrid expert model;

[0025] in a case where the voiceprint turning point detection model is deployed on a user equipment side, a size of the voiceprint turning point detection model is a second size;

[0026] wherein the first size is greater than the second size.

[0027] According to a second aspect of an embodiment of the present application, a language recognition device is provided, comprising:

[0028] a voiceprint turning point detection module, configured to extract a speech segment from a to-be-recognized speech according to a sliding window of a preset size, and detect a voiceprint turning point from the to-be-recognized speech;

[0029] a sliding window shift module, configured to, in a case where a voiceprint turning point is detected from a speech segment in a first sliding window, shift adjust the first sliding window according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window;

[0030] a language recognition module, configured to perform language recognition on the speech segment in the first sliding window after the shift adjustment, to obtain a corresponding language recognition result.

[0031] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor;

[0032] the memory is connected with the processor, and is configured to store a program;

[0033] the processor is configured to realize the above language recognition method by running the program in the memory.

[0034] According to a fourth aspect of the embodiments of the present application, a computer program product is provided, comprising computer program instructions which, when executed by a processor, cause the processor to implement the above language identification method.

[0035] The language identification method provided in the present application extracts a speech segment from the to-be-identified speech in a sliding window of a preset size, and detects a voiceprint turning point from the to-be-identified speech. In the case where the voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint of the first sliding window. The speech segment in the first sliding window after the position adjustment is subjected to language identification, and a corresponding language identification result is obtained. The technical solution of the present application can detect the voiceprint turning point in the sliding window, thereby identifying the speaker transition moment in the sliding window, i.e., the language transition moment. The sliding window is moved according to the transition moment to divide the speech segment, which can avoid the situation that the sliding window contains multiple languages. The speech segment of a single language in the sliding window is subjected to language identification, so that the accuracy of language identification of the to-be-identified speech is higher, and the accuracy of speech translation is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0037] Figure 1 A schematic diagram of a conventional sliding window language identification provided for the embodiments of the present application.

[0038] Figure 2 A flowchart of a language identification method provided for the embodiments of the present application.

[0039] Figure 3 A flowchart of another language identification method provided for the embodiments of the present application.

[0040] Figure 4 A schematic diagram of sliding window language identification corresponding to the language identification method provided for the embodiments of the present application.

[0041] Figure 5 A structural schematic diagram of a language identification device provided for the embodiments of the present application.

[0042] Figure 6 A structural schematic diagram of a language identification system provided for the embodiments of the present application.

[0043] Figure 7A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solution of the embodiment of the present application is applicable to the application scenario of voice processing, and is specifically used in the application scenario of judging the language in the voice. The technical solution of the embodiment of the present application can improve the accuracy of language recognition, and further improve the accuracy of voice translation.

[0045] With the rapid development of artificial intelligence technology, intelligent translation machines and other products have gradually become important tools for cross-language communication. These devices can convert one language into another in real time through voice recognition, machine translation, and speech synthesis technologies, greatly facilitating user communication in different language environments. However, one of the core functions of intelligent translation machines, language judgment capability, that is, accurately identifying the type of language input by the user, is still a key factor affecting user experience.

[0046] In actual use, the intelligent translation machine needs to quickly identify the language used by the current voice of the user, and then translate the current voice based on the used language to ensure the accuracy of the translation result. The existing language recognition method is to recognize the language in the window using the traditional sliding window method. However, when the window contains multiple languages, only the language with more content can be recognized as the language corresponding to the window, thereby affecting the accuracy of language recognition and further affecting the accuracy of voice translation.

[0047] The traditional language recognition adopts a sliding window method for processing, such as Figure 1As shown, assuming the window length of the sliding window is 2s and the window shift is 1s, the language of the speech segments in the first window (window 1) is all Chinese, so when the speech segments in window 1 are subjected to language recognition, the recognition result is probably Chinese. The language of most of the speech segments in the second window (window 2) is Chinese, and the language of a small part of the speech segments is English, so when the speech segments in window 2 are subjected to language recognition, the recognition result is probably Chinese. The language of most of the speech segments in the third window (window 3) is English, and the language of a small part of the speech segments is Chinese, so when the speech segments in window 3 are subjected to language recognition, the recognition result is probably English. Therefore, the final language recognition result is that the speech segments in window 1 and window 2 are both identified as Chinese, and the speech segments in window 3 are identified as English. It can be seen that there is a deviation in the recognition result of the speech. By analogy, the language recognition result of the speech segments in window 4 is English, the language recognition result of the speech segments in window 5 is English, the language recognition result of the speech segments in window 6 is English, the language recognition result of the speech segments in window 7 is Chinese, the language recognition result of the speech segments in window 8 is Chinese, and the language recognition result of the speech segments in window 9 is Chinese. The language recognition results of the speech segments in window 4 to window 6 are all English, which deviates from the true language. There is no clear boundary between English and Chinese in the speech. Because the window length is fixed, the point where the language changes cannot be recognized, resulting in an unavoidable deviation in the final recognition result. This deviation will affect the language recognition accuracy of the speech to be recognized, and further affect the accuracy of the speech translation according to the language of the speech.

[0048] Based on this, the present application proposes a language recognition method, which can detect the jumping moment of the language in the speech, and move the sliding window according to the jumping moment, so that the speech segments in the sliding window do not contain language jumps, thereby solving the problems of low language recognition accuracy and low speech translation accuracy in the prior art.

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0050] Exemplary method

[0051] Referring to Figure 2 As shown, the embodiments of the present application propose a language recognition method. The method comprises:

[0052] S201, a speech segment is extracted from the speech to be recognized according to a sliding window of a preset size, and a voiceprint turning point is detected from the speech to be recognized.

[0053] The voice requiring language recognition in the embodiment is determined as the voice to be recognized, wherein the voice to be recognized is preferably a real-time voice stream processed by voice activity detection (VAD), that is, a data stream after real-time separation of valid voice segments (removal of non-voice parts such as silence and noise) from continuous audio signals.

[0054] The embodiment is provided with a sliding window and a preset size of the sliding window, that is, a preset size. The embodiment preferably sets the preset size to a time length of 2s. The embodiment first needs to extract a voice segment from the voice to be recognized according to the sliding window of the preset size, wherein the sliding window extracts a voice segment each time, and slides according to a preset sliding size, which is smaller than the preset size of the sliding window, and is preferably set to a time length of 1s.

[0055] The embodiment needs to perform voiceprint turning point detection on the voice to be recognized while extracting a voice segment from the voice to be recognized, and the voice segment extraction and the voiceprint turning point detection need to be performed simultaneously, that is, the voice content in the first voice segment is detected for voiceprint turning point while the first voice segment is extracted from the voice to be recognized, the voice content in the second voice segment is detected for voiceprint turning point while the second voice segment is extracted from the voice to be recognized, and so on.

[0056] In the embodiment, the voiceprint turning point in the voice to be recognized refers to the moment when the speaker changes, that is, the moment when the speaker jumps. Because the language may change when the speaker changes, the embodiment needs to detect whether there is a voiceprint turning point in the voice segment. If there is a voiceprint turning point, it means that there may be multiple languages in the voice segment. If the voice segment is directly recognized for language, only the language with a large content proportion can be recognized, resulting in recognition deviation. Only when the voice segment is adjusted according to the voiceprint turning point so that there is no voiceprint turning point in the voice segment, it is ensured that there is only one language in the voice segment, so as to ensure the accuracy of language recognition.

[0057] The voiceprint turning point detection of the to-be-recognized voice is performed, and first, voice features of the to-be-recognized voice are extracted, that is, key feature points in a voice signal of the to-be-recognized voice are extracted. The extracted voice features of the to-be-recognized voice include timbre features, frequency features, energy change features, and the like. Then, the voice features of the to-be-recognized voice are analyzed, and a time when the key feature points in the to-be-recognized voice change significantly is analyzed, that is, a voiceprint turning point. In addition, the voiceprint turning point detection of the to-be-recognized voice can also be performed in a manner of spectrogram analysis, a spectrogram of the to-be-recognized voice is constructed, and the position of the voiceprint turning point is determined by analyzing the change of the fundamental frequency and the harmonic frequency in the spectrogram.

[0058] In a specific embodiment, a voiceprint turning point detection model can also be constructed and trained, the to-be-recognized voice is input into the voiceprint turning point detection model, so that the voiceprint turning point detection model detects the voiceprint turning point from the to-be-recognized voice, and outputs voiceprint turning point information corresponding to the voiceprint turning point when the voiceprint turning point is detected, wherein the voiceprint turning point information includes a position (for example, a time) of the voiceprint turning point in the to-be-recognized voice. In the embodiment, the training process of the voiceprint turning point detection model is as follows: first, a training sample of the voiceprint turning point detection model, that is, a sample voice, is obtained, wherein the sample voice carries labeled voiceprint turning point information, and the labeled voiceprint turning point information is a voiceprint turning point pre-labeled in the sample voice. Then, the sample voice is input into a pre-constructed voiceprint turning point detection model, the voiceprint turning point detection model is used to detect the voiceprint turning point from the sample voice, and corresponding predicted voiceprint turning point information is output. Finally, the model parameters in the voiceprint turning point detection model are adjusted to make the predicted voiceprint turning point information the same as the labeled voiceprint turning point information.

[0059] The language recognition method of the embodiment can be applied to a remote server or a user equipment end. When the language recognition method of the embodiment is applied to the remote server, the voiceprint turning point detection model is also deployed on the remote server, so that online deployment in the cloud is realized. When the language recognition method of the embodiment is applied to the user equipment end, the voiceprint turning point detection model is also deployed on the user equipment end, so that offline deployment locally is realized.

[0060] When the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model can be set as a first size, and when the voiceprint turning point detection model is deployed on a user equipment, the size of the voiceprint turning point detection model can be set as a second size, wherein the first size is greater than the second size, that is, when the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model can be appropriately enlarged due to sufficient computing power of the remote server, thereby improving the accuracy of the voiceprint turning point detection model. When the voiceprint turning point detection model is deployed on a user equipment, voiceprint turning point detection can be performed without networking, thereby enabling voiceprint turning point detection in an offline situation. In addition, since the computing power is sufficient when the voiceprint turning point detection model is deployed on a remote server, a mixed expert model MOE can also be used as the voiceprint turning point detection model to further improve the voiceprint turning point detection accuracy.

[0061] S202, in the case that a voiceprint turning point is detected in the voice segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint position of the first sliding window.

[0062] If a voiceprint turning point is detected in the voice segment in the first sliding window when the voiceprint turning point detection is performed on the voice to be recognized, the first sliding window needs to be adjusted in position according to the detected voiceprint turning point, so that the voiceprint turning point is located at the endpoint position of the first sliding window. At this time, the voiceprint turning point is not located in the voice segment in the first sliding window, so that there is no speaker jump in the voice segment in the first sliding window, and there is also no language jump, that is, there is only one language in the voice segment in the first sliding window, thereby ensuring the accuracy of language recognition. Wherein, the first sliding window needs to be adjusted in position so that the voiceprint turning point is located at the endpoint position of the first sliding window, and it also needs to be ensured that there is no unextracted content between the first sliding window and the second sliding window extracted last time, that is, the first sliding window and the second sliding window can be adjacent in head and tail, or there can be overlapping content (that is, the end point position of the second sliding window is after the starting point position of the first sliding window in the voice to be recognized), but there cannot be interval between the first sliding window and the second sliding window (that is, there cannot be unextracted voice content between the end point of the second sliding window and the starting point of the first sliding window). Wherein, the first sliding window is the sliding window after the second sliding window is slid once according to the preset sliding size.

[0063] In a specific embodiment, if the position of the detected voiceprint turning point existing in the first sliding window is not adjacent to and coincides with the end point position of the second sliding window, the first sliding window needs to be shifted forward at this time so that the voiceprint turning point is located at the end point position of the first sliding window. Wherein, the second sliding window is the previous sliding window of the first sliding window.

[0064] In another specific embodiment, if the position of the detected voiceprint turning point existing in the first sliding window is adjacent to or coincides with the ending position of the second sliding window, then the first sliding window needs to be shifted backward so that the voiceprint turning point is located at the starting position of the first sliding window. Here, the second sliding window is the previous sliding window of the first sliding window.

[0065] S203, performing language recognition on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0066] After the first sliding window is shifted and adjusted according to the above steps, language recognition is performed on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result of the speech segment. Language recognition can be performed on the speech segment using a direct comparison method based on acoustic features. Speech signals of different languages differ in acoustic features such as pitch, intensity, duration, timbre, and spectral distribution. By directly comparing these acoustic features, different languages can be distinguished, i.e., the acoustic features of the speech segment are extracted, and then the acoustic feature ranges of various languages are preset. By matching and analyzing the acoustic features of the speech segment with the preset acoustic feature ranges of various languages, the language recognition result corresponding to the speech segment is determined. A recognition method based on prosodic features can also be used. Prosodic features include intonation, stress, rhythm, etc. These features show obvious differences in different languages. For example, Chinese is a tonal language, while English is a stress language. The prosodic features of the speech segment are extracted, and then the prosodic feature ranges of various languages are preset. By matching and analyzing the prosodic features of the speech segment with the preset prosodic feature ranges of various languages, the language recognition result corresponding to the speech segment is determined. A rule method using linguistic knowledge can also be used. Linguistic knowledge such as grammatical rules and lexical features is used to construct language recognition rules. These rules can be based on specific properties of a language such as word order and morphological changes. By performing grammatical rule analysis and lexical feature analysis on the speech segment to determine the speech features of the speech segment, and then analyzing the speech features according to the constructed speech recognition rules to determine the language recognition result corresponding to the speech segment.

[0067] In a specific embodiment, a language recognition model can be pre-built and trained, and the voice segment in the first sliding window after the shift adjustment is input into the language recognition model, so that the language recognition model performs language recognition on the input voice segment to obtain a corresponding language recognition result. The training process of the language recognition model is as follows: first, obtain the training sample of the language recognition model, i.e., the sample voice, wherein the sample voice carries a labeled language, which is a language labeled in advance in the sample voice. Then, input the sample voice into the pre-built language recognition model, use the language recognition model to recognize the language of the sample voice, and output the corresponding predicted language. Finally, adjust the model parameters in the language recognition model with the goal of making the predicted language the same as the labeled language. In addition, a multi-language mixed modeling strategy can be used when training the language recognition model, for example, collecting sample voices of several different languages as training samples, so that the trained language recognition model can support multi-language recognition.

[0068] Since the existing language recognition uses a fixed window length for sliding window, and does not adjust the sliding window according to the voiceprint turning point, the voiceprint in the sliding window may not be pure (containing the voiceprint of multiple people), which will affect the accuracy of language detection. At this time, only by improving the accuracy of the language recognition model can the accuracy of language detection be guaranteed as much as possible. Therefore, the existing language recognition model is usually trained separately for different language pairs, i.e., different language pairs need to be trained with different language recognition models to improve the accuracy of the language recognition model. However, the model trained by the separate training method for different language pairs has a large amount of calculation, which is not suitable for local offline deployment. However, by detecting the voiceprint turning point and adjusting the sliding window, the purity of the voiceprint in the sliding window can be improved, and the accuracy of language recognition can be improved. Therefore, there is no need to train language recognition models for different language pairs, and a language recognition model suitable for multi-language recognition can be directly trained, which reduces the model calculation amount and does not need to load complex models, and is suitable for running on offline devices, i.e., both online and offline deployment are applicable.

[0069] In addition, in a specific embodiment, if a voiceprint turning point is detected in the first sliding window, and the voiceprint turning point is located at the ending point of the first sliding window, or the voiceprint turning point is located at the ending point of the first sliding window by adjusting the displacement of the first sliding window, after the language recognition of the speech segment in the first sliding window is completed, the sliding window is slid to extract the next speech segment, at this time, the first sliding window can be directly slid by a preset size to obtain a third sliding window, that is, the starting point of the third sliding window is located at a position adjacent to the ending point of the first sliding window. At this time, the speech segment extracted from the third sliding window is processed according to the language recognition method, that is, whether a voiceprint turning point exists in the speech segment in the third sliding window is detected, in the case that a voiceprint turning point is detected from the speech segment in the third sliding window, the displacement of the third sliding window is adjusted according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint of the third sliding window; the speech segment in the third sliding window after the displacement adjustment is subjected to language recognition to obtain the corresponding language recognition result.

[0070] As can be seen from the above introduction, the language recognition method proposed in the embodiments of the present application extracts a speech segment from the to-be-recognized speech according to a sliding window with a preset size, and detects a voiceprint turning point from the to-be-recognized speech; in the case that a voiceprint turning point is detected from the speech segment in the first sliding window, the displacement of the first sliding window is adjusted according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint of the first sliding window; the speech segment in the first sliding window after the displacement adjustment is subjected to language recognition to obtain the corresponding language recognition result. By adopting the technical solution of the embodiments, the voiceprint turning point in the sliding window can be detected, so that the speaker transition time in the sliding window can be recognized, that is, the language transition time can be recognized, the sliding window is moved according to the transition time to divide the speech segment, the sliding window can avoid containing multiple languages, the speech segment in the sliding window with a single language is subjected to language recognition, so that the accuracy of the language recognition of the to-be-recognized speech is higher, and the accuracy of the speech translation is further improved.

[0071] As an optional implementation, the embodiments of the present application further propose a language recognition method. Referring to FIG. 8, the method comprises the following steps. Figure 3

[0072] S301, a speech segment is extracted from to-be-recognized speech according to a sliding window with a preset size, and a voiceprint turning point is detected from the to-be-recognized speech.

[0073] S302, in the case that a voiceprint turning point is detected from the speech segment in the first sliding window, the displacement of the first sliding window is adjusted according to the voiceprint turning point, so that the voiceprint turning point is located at the endpoint of the first sliding window.

[0074] ​S303, language recognition is performed on the speech segment in the first sliding window after the shift adjustment, to obtain a corresponding language recognition result.

[0075] S304, in the case where no voiceprint turning point is detected from the speech segment in the first sliding window, language recognition is performed on the speech segment in the first sliding window, to obtain a corresponding language recognition result.

[0076] If no voiceprint turning point is detected from the speech segment in the first sliding window, it indicates that there is only one language in the speech segment in the first sliding window, and thus there is no need to perform shift adjustment on the first sliding window, and language recognition is directly performed on the speech segment in the first sliding window, to obtain a corresponding language recognition result.

[0077] The specific execution manners of steps S301-S303 of this embodiment are the same as those of steps S201-S203 of the above-described embodiment, and thus will not be elaborated herein.

[0078] As an optional implementation, in another embodiment of the present application, a speech segment is extracted from the speech to be recognized by a sliding window with a preset size, in the case where a voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window, language recognition is performed on the speech segment in the first sliding window after the shift adjustment, to obtain a corresponding language recognition result, which includes:

[0079] The speech to be recognized is input into a pre-trained multi-language recognition model, so that the multi-language recognition model extracts a speech segment from the speech to be recognized by a sliding window with a preset size, in the case where voiceprint turning point information output by a voiceprint turning point detection model is received, the first sliding window containing the voiceprint turning point is adjusted in position according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at an endpoint position of the first sliding window, language recognition is performed on the speech segment in the first sliding window after the shift adjustment, to obtain a corresponding language recognition result.

[0080] Specifically, the voice to be recognized is input into a pre-trained multi-language recognition model and a voiceprint turning point detection model. The multi-language recognition model extracts a voice segment from the voice to be recognized according to a preset size of a sliding window. Meanwhile, the voiceprint turning point detection model also detects a voiceprint turning point in the voice segment in the voice to be recognized. When a voiceprint turning point in the voice segment is detected, voiceprint turning point information corresponding to the voiceprint turning point is transmitted to the multi-language recognition model. The multi-language recognition model receives the voiceprint turning point information in the voice segment, adjusts a first sliding window containing the voiceprint turning point according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at an endpoint of the first sliding window, and performs language recognition on a voice segment in the first sliding window after the adjustment to obtain a corresponding language recognition result. The training process of the multi-language recognition model in this embodiment has been specifically described in the above embodiment, and will not be described herein again.

[0081] As an optional implementation, in another embodiment of the present application, the language recognition on the voice segment in the first sliding window after the adjustment to obtain a corresponding language recognition result specifically includes the following steps:

[0082] First, a degree of coincidence between the voice segment in the second sliding window and the voice segment in the first sliding window after the adjustment is analyzed to obtain a voice coincidence degree.

[0083] In this embodiment, the voice segment in the second sliding window and the voice segment in the first sliding window after the adjustment are analyzed for the degree of coincidence, that is, it is analyzed how many voice segments in the second sliding window are contained in the first sliding window after the adjustment, so as to determine the voice coincidence degree between the two.

[0084] Second, if the voice coincidence degree reaches a preset coincidence threshold, a language recognition result corresponding to the voice segment in the second sliding window is taken as a language recognition result corresponding to the voice segment in the first sliding window after the adjustment.

[0085] If it is determined that the voice coincidence degree reaches the preset coincidence threshold, it is indicated that the voice segment in the first sliding window after the adjustment and the voice segment in the second sliding window have a large part of coincidence, and the voice segment in the first sliding window after the adjustment also does not have a voiceprint turning point. At this time, the language recognition result corresponding to the voice segment in the second sliding window can be directly taken as the language recognition result corresponding to the voice segment in the first sliding window after the adjustment. Since the second sliding window is the sliding window before the first sliding window after the adjustment, the voice segment in the second sliding window has a corresponding language recognition result. In this embodiment, the preset coincidence threshold is preferably set to 70%.

[0086] Third, if the speech overlap does not reach the preset overlap threshold, the speech segment in the first sliding window after shifting and adjustment is used for language identification to obtain the corresponding language identification result.

[0087] If it is determined that the speech overlap does not reach the preset overlap threshold, then the speech segment in the first sliding window after shift adjustment is identified according to the language identification method described in the above embodiment, and the corresponding language identification result is obtained.

[0088] In addition, when the first sliding window is the first sliding window in the speech to be recognized, the second sliding window does not exist. That is to say, the second sliding window is empty at this time, so the speech overlap between the speech segment in the second sliding window and the speech segment in the first sliding window is zero.

[0089] In a specific embodiment, such as Figure 4 As shown, when the first sliding window is window 1, no voiceprint inflection point is detected in the speech segment within window 1. Language recognition is performed on the speech segment within window 1, and the corresponding language recognition result, i.e., Chinese, is obtained. The first sliding window slides to the right once according to a preset sliding size to obtain the original window 2. A voiceprint inflection point is detected in the speech segment within the original window 2, i.e., Figure 2 At the character turning point 1, the original window 2 is shifted and adjusted based on the character turning point 1. At this time, the end point of the second sliding window (i.e., window 1) is neither adjacent to nor coincides with the character turning point 1. Therefore, the original window 2 is moved forward so that the character turning point 1 is located at the end point of the original window 2, resulting in the first sliding window after the shift adjustment. Figure 2 Window 2 in the middle performs language identification on the speech segment within Window 2, obtaining the corresponding language identification result, i.e., Chinese. The first sliding window slides to the right once according to a preset sliding size to obtain the original window 3. The voiceprint inflection point is detected from the speech segment within the original window 2, i.e. Figure 2 At the character turning point 1, the original window 3 is shifted and adjusted based on the character turning point 1. At this time, the end point of the second sliding window (i.e., window 2) coincides with the character turning point 1. The original window 3 is then moved backward so that the character turning point 1 is located at the starting point of the original window 3, resulting in the first sliding window after the shift adjustment. Figure 2 Window 3 performs language identification on the speech segment within it, obtaining the corresponding language identification result, i.e., English. The first sliding window slides right once by a preset sliding size to obtain window 4. No voiceprint inflection point is detected in the speech segment within window 4. Language identification is then performed on the speech segment within window 4, obtaining the corresponding language identification result, i.e., English. The first sliding window slides right once by a preset sliding size to obtain window 5. A voiceprint inflection point is detected in the speech segment within window 5, i.e., Figure 2In the case that the end point position of the second sliding window (i.e., the window 5) is not adjacent to and does not coincide with the character turning point 2, the character turning point 2 needs to be located at the end point position of the window 5, and the character turning point is located at the end point position of the window 5 at this time, so that the window 5 does not need to be shifted and adjusted, and the language recognition is directly performed on the speech segments in the window 5 to obtain the corresponding language recognition result, i.e., English. The first sliding window is slid right once according to the preset sliding size, to obtain the original window 6. The voiceprint turning point is detected from the speech segments in the original window 6, i.e. Figure 2 In the case that the end point position of the second sliding window (i.e., the window 5) is not adjacent to and does not coincide with the character turning point 2, the character turning point 2 needs to be located at the end point position of the window 5, and the character turning point is located at the end point position of the window 5 at this time, so that the window 5 does not need to be shifted and adjusted, and the language recognition is directly performed on the speech segments in the window 5 to obtain the corresponding language recognition result, i.e., English. The first sliding window is slid right once according to the preset sliding size, to obtain the original window 6. The voiceprint turning point is detected from the speech segments in the original window 6, i.e. Figure 2 In the case that the end point position of the second sliding window (i.e., the window 5) is not adjacent to and does not coincide with the character turning point 2, the character turning point 2 needs to be located at the end point position of the window 5, and the character turning point is located at the end point position of the window 5 at this time, so that the window 5 does not need to be shifted and adjusted, and the language recognition is directly performed on the speech segments in the window 5 to obtain the corresponding language recognition result, i.e., English. The first sliding window is slid right once according to the preset sliding size, to obtain the original window 6. The voiceprint turning point is detected from the speech segments in the original window 6, i.e. Figure 5 In the case that the end point position of the second sliding window (i.e., the window 5) is not adjacent to and does not coincide with the character turning point 2, the character turning point 2 needs to be located at the end point position of the window 5, and the character turning point is located at the end point position of the window 5 at this time, so that the window 5 does not need to be shifted and adjusted, and the language recognition is directly performed on the speech segments in the window 5 to obtain the corresponding language recognition result, i.e., English. The first sliding window is slid right once according to the preset sliding size, to obtain the original window 6. The voiceprint turning point is detected from the speech segments in the original window 6, i.e.

[0090] Exemplary device

[0091] Accordingly, the embodiment of the present application further provides a language recognition device, which refers to Figure 6 as shown in the figure, the device comprises:

[0092] The voiceprint turning point detection module 100 is configured to extract speech segments from the to-be-recognized voice according to the preset sliding window, and detect a voiceprint turning point from the to-be-recognized voice.

[0093] The sliding window shifting module 110 is configured to, in the case that the voiceprint turning point is detected from the speech segments in the first sliding window, shift and adjust the first sliding window according to the voiceprint turning point, so that the voiceprint turning point is located at the end point position of the first sliding window.

[0094] The language recognition module 120 is configured to perform language recognition on the speech segments in the first sliding window after the shifting and adjustment, to obtain the corresponding language recognition result.

[0095] As can be seen from the above introduction, the language recognition device provided by the embodiment of the present application can detect the voiceprint turning point in the sliding window, thereby recognizing the speaker jump time in the sliding window, i.e., recognizing the language jump time, moving the sliding window according to the jump time to divide the speech segment, avoiding the situation that the sliding window contains multiple languages, and performing language recognition on the speech segment in the sliding window with a single language, so that the accuracy of language recognition of the speech to be recognized is higher, and the accuracy of speech translation is further improved.

[0096] As an optional implementation, in another embodiment of the present application, the sliding window shifting module 110 is specifically used for:

[0097] in the case that the end point position of the second sliding window is not adjacent to and does not coincide with the voiceprint turning point, shifting the first sliding window forward so that the voiceprint turning point is located at the end point position of the first sliding window;

[0098] or,

[0099] in the case that the end point position of the second sliding window is adjacent to or coincides with the voiceprint turning point, shifting the first sliding window backward so that the voiceprint turning point is located at the start point position of the first sliding window; the second sliding window is the previous sliding window of the first sliding window.

[0100] As an optional implementation, in another embodiment of the present application, the language recognition module 120 is further used for, in the case that no voiceprint turning point is detected from the speech segment in the first sliding window, performing language recognition on the speech segment in the first sliding window to obtain the corresponding language recognition result.

[0101] As an optional implementation, in another embodiment of the present application, the voiceprint turning point detection module 100 is specifically further used for inputting the speech data stream of the speech to be recognized into the pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects the voiceprint turning point from the speech to be recognized and outputs the voiceprint turning point information when the voiceprint turning point is detected.

[0102] As an optional implementation, in another embodiment of the present application, the language recognition module 120 is specifically used for:

[0103] The voice to be recognized is input into the pre-trained multi-language recognition model to enable the multi-language recognition model to extract voice segments from the voice to be recognized in a sliding window of a preset size. In a case where voiceprint turning point information output by the voiceprint turning point detection model is received, a first sliding window containing a voiceprint turning point is adjusted in position according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at an endpoint position of the first sliding window. The voice segments in the first sliding window after the position adjustment are subjected to language recognition to obtain corresponding language recognition results.

[0104] As an optional implementation, in another embodiment of the present application, a language recognition module 120 is disclosed, and is specifically configured to:

[0105] The voice segments in the second sliding window and the voice segments in the first sliding window after the position adjustment are subjected to coincidence degree analysis to obtain a voice coincidence degree; the second sliding window is a previous sliding window of the first sliding window;

[0106] If the voice coincidence degree reaches a preset coincidence degree threshold, the language recognition result corresponding to the voice segments in the second sliding window is taken as the language recognition result corresponding to the voice segments in the first sliding window after the position adjustment;

[0107] If the voice coincidence degree does not reach the preset coincidence degree threshold, the voice segments in the first sliding window after the position adjustment are subjected to language recognition to obtain corresponding language recognition results.

[0108] As an optional implementation, in another embodiment of the present application, in a case where the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model;

[0109] In a case where the voiceprint turning point detection model is deployed on a user device, the size of the voiceprint turning point detection model is a second size;

[0110] The first size is greater than the second size.

[0111] The language recognition device provided in the embodiment belongs to the same application concept as the language recognition method provided in the above embodiments of the present application, can execute the language recognition method provided in any of the above embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the language recognition method. Technical details not described in detail in the embodiment can be referred to the specific processing content of the language recognition method provided in the above embodiments of the present application, which will not be described here.

[0112] Exemplary system

[0113] Optionally, the embodiment of the present application further provides a language recognition system, which is described with reference to Figure 7As shown, the language recognition system comprises a voiceprint detection subsystem 200 and a language recognition subsystem 210.

[0114] The voiceprint detection subsystem 200 is configured to detect a voiceprint turning point from the to-be-recognized speech, and output voiceprint turning point information of the voiceprint turning point to the language recognition subsystem when the voiceprint turning point is detected.

[0115] The language recognition subsystem 210 is configured to extract a speech segment from the to-be-recognized speech in a sliding window of a preset size, and in a case where the voiceprint turning point information sent by the voiceprint detection subsystem is received, adjust a first sliding window containing the voiceprint turning point by shifting according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at an endpoint position of the first sliding window; and perform language recognition on the speech segment in the first sliding window after the shifting, to obtain a corresponding language recognition result.

[0116] As can be seen from the above description, the language recognition system disclosed by the embodiments of the present application can detect a voiceprint turning point in a sliding window, thereby recognizing a speaker transition time in the sliding window, i.e., a language transition time, moving the sliding window according to the transition time to divide the speech segment, which can avoid the sliding window containing multiple languages, and perform language recognition on the speech segment of a single language in the sliding window, so that the accuracy of language recognition of the to-be-recognized speech is higher, and the accuracy of speech translation is further improved.

[0117] As an optional implementation, in another embodiment of the present application, the voiceprint detection subsystem 200 is specifically configured to, in a case where the endpoint position of the second sliding window and the voiceprint turning point are not adjacent and do not coincide, shift the first sliding window forward, so that the voiceprint turning point is located at the endpoint position of the first sliding window.

[0118] Or,

[0119] In a case where the endpoint position of the second sliding window and the voiceprint turning point are adjacent or coincide, the first sliding window is shifted backward, so that the voiceprint turning point is located at the starting point position of the first sliding window; and the second sliding window is a previous sliding window of the first sliding window.

[0120] As an optional implementation, in another embodiment of the present application, the language recognition subsystem 210 is further configured to, in a case where no voiceprint turning point is detected from the speech segment in the first sliding window, perform language recognition on the speech segment in the first sliding window, to obtain a corresponding language recognition result.

[0121] As an optional implementation, in another embodiment of the present application, it is disclosed that the voiceprint detection subsystem 200 is specifically used for inputting the voice data stream of the voice to be recognized into the pre-trained voiceprint turning point detection model, so that the voiceprint turning point detection model detects the voiceprint turning point from the voice to be recognized, and outputs the voiceprint turning point information when the voiceprint turning point is detected.

[0122] As an optional implementation, in another embodiment of the present application, it is disclosed that the language recognition subsystem 210 is specifically used for inputting the voice to be recognized into the pre-trained multi-language recognition model, so that the multi-language recognition model extracts the voice segment from the voice to be recognized in a sliding window of a preset size, in the case that the voiceprint turning point information output by the voiceprint turning point detection model is received, the first sliding window containing the voiceprint turning point is adjusted in position according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at the endpoint position of the first sliding window, and the voice segment in the first sliding window after the position adjustment is subjected to language recognition to obtain the corresponding language recognition result.

[0123] As an optional implementation, in another embodiment of the present application, it is disclosed that the language recognition subsystem 210 is specifically used for:

[0124] analyzing the coincidence degree of the voice segment in the second sliding window and the voice segment in the first sliding window after the position adjustment to obtain a voice coincidence degree; the second sliding window is a previous sliding window of the first sliding window;

[0125] if the voice coincidence degree reaches a preset coincidence degree threshold, the language recognition result corresponding to the voice segment in the second sliding window is taken as the language recognition result corresponding to the voice segment in the first sliding window after the position adjustment;

[0126] if the voice coincidence degree does not reach the preset coincidence degree threshold, the voice segment in the first sliding window after the position adjustment is subjected to language recognition to obtain the corresponding language recognition result.

[0127] As an optional implementation, in another embodiment of the present application, it is disclosed that, in the case that the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model;

[0128] in the case that the voiceprint turning point detection model is deployed on a user equipment side, the size of the voiceprint turning point detection model is a second size;

[0129] wherein the first size is greater than the second size.

[0130] The language recognition system provided by the embodiment belongs to the same application concept as the language recognition method provided by the embodiments of the present application, can execute the language recognition method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of executing the language recognition method. The technical details not described in detail in the embodiment can refer to the specific processing content of the language recognition method provided by the embodiments of the present application, and will not be described here.

[0131] Exemplary electronic device

[0132] Another embodiment of the present application also provides an electronic device, as shown in ​ The device comprises:

[0133] a memory 300 and a processor 310;

[0134] The memory 300 is connected with the processor 310, and is configured to store a program.

[0135] The processor 310 is configured to realize the language recognition method disclosed in any of the embodiments by running the program stored in the memory 300.

[0136] Specifically, the electronic device can further comprise a bus, a communication interface 320, an input device 330 and an output device 340.

[0137] The processor 310, the memory 300, the communication interface 320, the input device 330 and the output device 340 are connected with each other through the bus.

[0138] The bus can comprise a path for transmitting information between various components of the computer system.

[0139] The processor 310 can be a general-purpose processor, such as a central processing unit (CPU), a microprocessor, etc., or can be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling program execution of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-use programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0140] The processor 310 can include a main processor, and can also include a baseband chip, a modem, etc.

[0141] The memory 300 stores programs for implementing the technical solutions of the present application, and can also store an operating system and other key services. Specifically, the programs can include program codes, which include computer operation instructions. More specifically, the memory 300 can include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash, and the like.

[0142] The input device 330 can include devices that receive data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, and the like.

[0143] The output device 340 can include devices that allow information to be output to a user, such as a display screen, a printer, a speaker, and the like.

[0144] The communication interface 320 can include devices using any transceiver to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), and the like.

[0145] The processor 310 executes programs stored in the memory 300 and calls other devices, which can be used to implement each step of any language recognition method provided by the above-described embodiments of the present application.

[0146] Exemplary computer program products and storage media

[0147] In addition to the above-described methods and devices, embodiments of the present application can also be computer program products that include computer program instructions that, when executed by a processor, cause the processor to perform steps of the language recognition methods according to various embodiments of the present application described in the above “Exemplary Methods” section of the present specification.

[0148] The computer program product can be written in any combination of one or more programming languages to perform the operations of embodiments of the present application, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as the “C” language or a similar programming language. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0149] In addition, an embodiment of the present application can also be a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the language recognition method according to various embodiments of the present application described in the above "Exemplary Method" section of the specification.

[0150] For each of the above method embodiments, in order to simply describe, each of them is described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0151] It should be noted that each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, it is described relatively simply, and the relevant parts are referred to the part of the method embodiment.

[0152] The steps in the method of each embodiment of the present application can be adjusted, combined and reduced in sequence according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0153] The modules and sub-modules in the device and terminal of each embodiment of the present application can be combined, divided and reduced according to actual needs.

[0154] In several embodiments provided by the present application, it should be understood that the disclosed terminal, device and method can be implemented by other ways. For example, the terminal embodiments described above are only schematic, and the division of the modules or sub-modules is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0155] The modules or sub-modules described as separate components can or can not be physically separated, and the components of the modules or sub-modules can or can not be physical modules or sub-modules, that is, they can be located in one place, or can be distributed to a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0156] In addition, each functional module or sub-module in each embodiment of the present application can be integrated in one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated in one module. The integrated module or sub-module can be realized in the form of hardware or in the form of a software functional module or sub-module.

[0157] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various examples have been described generally in terms of their functionality, without limitation to the corresponding description in any particular combination of hardware and software. Those skilled in the art will recognize many modifications and variations of this example that are apparent in light of the above teachings. It is therefore intended that such modifications and variations not depart from the scope of the application.

[0158] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0159] Finally, it should be noted that the terms "first" and "second" and the like are used merely to distinguish one element from another, and do not necessarily indicate a physical or chronological order. Furthermore, the terms "comprise", "include", or their variants, mean "consisting of" or "consisting essentially of", so that when these terms are used, an open-ended inclusion is intended, such that the process, method, article, or apparatus includes not only those elements expressly listed, but also other elements inherent in such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0160] The above description of disclosed embodiments enables one of ordinary skill in the art to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A language identification method characterized by, The method comprises the following steps: A sliding window of a preset size is used to extract a speech segment from the to-be-recognized speech, and a voiceprint turning point is detected from the to-be-recognized speech; In a case where the voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window; According to the coincidence degree between the first sliding window after the position adjustment and a previous sliding window of the first sliding window after the position adjustment, a language recognition result corresponding to the speech segment in the first sliding window after the position adjustment or a language recognition result corresponding to the previous sliding window is taken as a language recognition result corresponding to the first sliding window after the position adjustment.

2. The method of claim 1, wherein, The first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window, and the method comprises the following steps: In a case where the endpoint position of the second sliding window is not adjacent to and does not coincide with the voiceprint turning point, the first sliding window is adjusted forward, so that the voiceprint turning point is located at the endpoint position of the first sliding window; or In a case where the endpoint position of the second sliding window is adjacent to or coincides with the voiceprint turning point, the first sliding window is adjusted backward, so that the voiceprint turning point is located at the starting point position of the first sliding window; the second sliding window is a previous sliding window of the first sliding window. The method further comprises the following steps:

3. The method of claim 1, wherein, In a case where the voiceprint turning point is not detected from the speech segment in the first sliding window, a language recognition is performed on the speech segment in the first sliding window, to obtain a corresponding language recognition result. The voiceprint turning point is detected from the to-be-recognized speech, and the method comprises the following steps:

4. The method of claim 1, wherein, A speech data stream of the to-be-recognized speech is input into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects a voiceprint turning point from the to-be-recognized speech, and outputs voiceprint turning point information when the voiceprint turning point is detected. A sliding window of a preset size is used to extract a speech segment from the to-be-recognized speech; in a case where the voiceprint turning point is detected from the speech segment in the first sliding window, the first sliding window is adjusted in position according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window; according to the coincidence degree between the first sliding window after the position adjustment and a previous sliding window of the first sliding window after the position adjustment, a language recognition result corresponding to the speech segment in the first sliding window after the position adjustment or a language recognition result corresponding to the previous sliding window is taken as a language recognition result corresponding to the first sliding window after the position adjustment, and the method comprises the following steps:

5. The method of claim 4, wherein, ​ The voice to be recognized is input into a pre-trained multi-language recognition model, so that the multi-language recognition model extracts voice segments from the voice to be recognized in a sliding window of a preset size. In a case where voiceprint turning point information output by the voiceprint turning point detection model is received, a first sliding window containing a voiceprint turning point is adjusted in position according to the voiceprint turning point information corresponding to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window. According to the coincidence degree between the first sliding window after the position adjustment and a previous sliding window of the first sliding window after the position adjustment, a language recognition result corresponding to a voice segment in the first sliding window after the position adjustment or a language recognition result corresponding to the previous sliding window is taken as a language recognition result corresponding to the first sliding window after the position adjustment.

6. The method of claim 1, wherein, According to the coincidence degree between the first sliding window after the position adjustment and a previous sliding window of the first sliding window after the position adjustment, a language recognition result corresponding to a voice segment in the first sliding window after the position adjustment or a language recognition result corresponding to the previous sliding window is taken as a language recognition result corresponding to the first sliding window after the position adjustment, comprising: performing coincidence degree analysis on the voice segment in the second sliding window and the voice segment in the first sliding window after the position adjustment to obtain a voice coincidence degree; the second sliding window is a previous sliding window of the first sliding window; if the voice coincidence degree reaches a preset coincidence degree threshold, a language recognition result corresponding to the voice segment in the second sliding window is taken as a language recognition result corresponding to the voice segment in the first sliding window after the position adjustment; if the voice coincidence degree does not reach the preset coincidence degree threshold, language recognition is performed on the voice segment in the first sliding window after the position adjustment to obtain a corresponding language recognition result.

7. The method of claim 4, wherein, In a case where the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model; In a case where the voiceprint turning point detection model is deployed on a user equipment side, the size of the voiceprint turning point detection model is a second size; wherein the first size is greater than the second size.

8. A language identification apparatus characterized by comprising: comprising: a voiceprint turning point detection module, configured to extract voice segments from a voice to be recognized in a sliding window of a preset size, and detect a voiceprint turning point from the voice to be recognized; a sliding window position adjustment module, configured to, in a case where a voiceprint turning point is detected from a voice segment in a first sliding window, adjust the position of the first sliding window according to the voiceprint turning point, so that the voiceprint turning point is located at an endpoint position of the first sliding window; a language recognition module, configured to, according to a coincidence degree between the first sliding window after the position adjustment and a previous sliding window of the first sliding window after the position adjustment, take a language recognition result corresponding to a voice segment in the first sliding window after the position adjustment or a language recognition result corresponding to the previous sliding window as a language recognition result corresponding to the first sliding window after the position adjustment.

9. An electronic device, comprising: comprising: a memory and a processor; The memory is connected with the processor and is used for storing programs; The processor is used for realizing the language recognition method in any one of claims 1 to 7 by running the programs in the memory.

10. A computer program product, characterised in that, Computer program instructions are included, which make the processor realize the language recognition method in any one of claims 1 to 7 when the computer program instructions are run by the processor.

Images