Language recognition method and device, electronic equipment and product

Abstract

The invention provides a language recognition method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: carrying out the sliding extraction of a voice segment from a to-be-recognized voice according to a sliding window of a preset size, and detecting a voiceprint turning point from the to-be-recognized voice; under the condition that the voiceprint turning point is detected from the voice segment in the first sliding window, carrying out displacement adjustment on the first sliding window according to the voiceprint turning point so as to enable the voiceprint turning point to be located at the end point position of the first sliding window; and performing language recognition on the shifted voice segment in the first sliding window to obtain a language recognition result. According to the scheme, the voiceprint turning point in the sliding window can be detected, the jumping moment of the speaker in the sliding window can be recognized, the sliding window is moved according to the jumping moment to divide the voice segments, language recognition of the voice segments of multiple languages and a single language in the sliding window can be avoided, and the user experience is improved. Therefore, the accuracy of language recognition of the to-be-recognized speech is higher, and the accuracy of speech translation is further improved.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology, and in particular to a target tracking method, device, electronic device and product. Background Art

[0002] With the rapid development of artificial intelligence (AI), products like smart translators have become essential tools for cross-language communication. These devices, leveraging technologies like speech recognition, machine translation, and speech synthesis, can convert from one language to another in real time, greatly facilitating communication across multiple languages.

[0003] In actual use, intelligent translation machines need to quickly identify the language of the user's current speech and then translate the current speech based on the language used to ensure the accuracy of the translation result. Existing language recognition methods use a traditional sliding window method to identify the language within the window. However, when the window contains multiple languages, only the language that appears most frequently in the content is identified as the language corresponding to the window, which affects the accuracy of language recognition and, in turn, the accuracy of speech translation. Summary of the Invention

[0004] Based on the above needs, the present application proposes a language recognition method, device, electronic device and product, which can improve the accuracy of language recognition and thus improve the accuracy of speech translation.

[0005] To achieve the above objectives, this application proposes the following technical solutions: According to a first aspect of an embodiment of the present application, a language identification method is provided, comprising: Sliding and extracting voice segments from the speech to be recognized according to a sliding window of a preset size, and detecting voiceprint turning points from the speech to be recognized; When a voiceprint turning point is detected from the speech segment within the first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; Language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0006] Optionally, shifting and adjusting the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window includes: If the end point of the second sliding window is not adjacent to or coincident with the voiceprint turning point, shifting the first sliding window forward so that the voiceprint turning point is located at the end point of the first sliding window; or, When the end point of the second sliding window is adjacent to or coincides with the voiceprint turning point, the first sliding window is shifted backward so that the voiceprint turning point is located at the starting point of the first sliding window; the second sliding window is the previous sliding window of the first sliding window.

[0007] Optionally, the language identification method further includes: When no voiceprint turning point is detected from the speech segment in the first sliding window, language recognition is performed on the speech segment in the first sliding window to obtain a corresponding language recognition result.

[0008] Optionally, detecting a voiceprint turning point from the speech to be recognized includes: The voice data stream of the speech to be recognized is input into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects the voiceprint turning point from the speech to be recognized, and outputs voiceprint turning point information when the voiceprint turning point is detected.

[0009] Optionally, a voice segment is extracted from the speech to be recognized by sliding a sliding window of a preset size; when a voiceprint turning point is detected in the voice segment within the first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at an endpoint of the first sliding window; and language recognition is performed on the voice segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result, including: The speech to be recognized is input into a pre-trained multilingual recognition model so that the multilingual recognition model slides and extracts speech segments from the speech to be recognized according to a sliding window of preset size. Upon receiving voiceprint turning point information output by the voiceprint turning point detection model, a first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information so that the voiceprint turning point is located at the endpoint position of the first sliding window. Language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0010] Optionally, performing language recognition on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result includes: performing overlap analysis on the speech segments in the second sliding window and the speech segments in the first sliding window after the shift adjustment to obtain speech overlap, wherein the second sliding window is a previous sliding window of the first sliding window; If the speech overlap reaches a preset overlap threshold, the language recognition result corresponding to the speech segment in the second sliding window is used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment; If the speech overlap does not reach a preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0011] Optionally, when the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model; When the voiceprint turning point detection model is deployed on a user device, the size of the voiceprint turning point detection model is a second size; The first size is greater than the second size.

[0012] According to a second aspect of an embodiment of the present application, a language identification device is provided, comprising: A voiceprint turning point detection module, configured to extract voice segments from the speech to be recognized by sliding a window of a preset size, and detect voiceprint turning points from the speech to be recognized; a sliding window shifting module, configured to, when a voiceprint turning point is detected from a speech segment within a first sliding window, shift and adjust the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; The language recognition module is used to perform language recognition on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0013] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including: a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the above-mentioned language recognition method by running the program in the memory.

[0014] According to a fourth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to implement the above-mentioned language recognition method.

[0015] The language recognition method proposed in this application extracts speech segments from the speech to be recognized according to a sliding window of a preset size, and detects voiceprint turning points from the speech to be recognized; when a voiceprint turning point is detected from a speech segment within a first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at the end point of the first sliding window; the language recognition is performed on the speech segment within the shifted and adjusted first sliding window to obtain a corresponding language recognition result. The technical solution of this application can detect the voiceprint turning point within the sliding window, thereby identifying the speaker jump moment within the sliding window, that is, it can identify the language jump moment, and move the sliding window according to the jump moment to divide the speech segments. This can avoid the sliding window containing multiple languages. The language recognition is performed on the speech segments of a single language within the sliding window, so that the accuracy of language recognition of the speech to be recognized is higher, thereby improving the accuracy of speech translation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0017] Figure 1 A schematic diagram of traditional sliding window language recognition provided in an embodiment of the present application.

[0018] Figure 2 A flow chart of a language recognition method provided in an embodiment of the present application.

[0019] Figure 3 A flowchart of another language recognition method provided in an embodiment of the present application.

[0020] Figure 4 A schematic diagram of sliding window language recognition corresponding to the language recognition method provided in an embodiment of the present application.

[0021] Figure 5 A schematic diagram of the structure of a language recognition device provided in an embodiment of the present application.

[0022] Figure 6 A schematic diagram of the structure of a language recognition system provided in an embodiment of the present application.

[0023] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solution of the embodiment of the present application is applicable to the application scenario of speech processing, specifically for the application scenario of determining the language in the speech. The technical solution of the embodiment of the present application can improve the accuracy of language recognition and thus improve the accuracy of speech translation.

[0025] With the rapid development of artificial intelligence (AI), products like smart translators are becoming essential tools for cross-language communication. These devices utilize technologies like speech recognition, machine translation, and speech synthesis to convert from one language to another in real time, greatly facilitating communication across multiple languages. However, one of the core functions of smart translators—the ability to accurately identify the language of user input—remains a key factor impacting user experience.

[0026] In actual use, intelligent translation machines need to quickly identify the language of the user's current speech and then translate the current speech based on the language used to ensure the accuracy of the translation result. Existing language recognition methods use a traditional sliding window method to identify the language within the window. However, when the window contains multiple languages, only the language that appears most frequently in the content is identified as the language corresponding to the window, which affects the accuracy of language recognition and, in turn, the accuracy of speech translation.

[0027] Traditional language recognition uses a sliding window approach, such as Figure 1As shown in the figure, assuming a sliding window length of 2 seconds and a window shift of 1 second, the language of the speech segments in the first window (window 1) is all Chinese. Therefore, when performing language recognition on the speech segments in window 1, the probability of recognition is Chinese. The language of the speech segments in the second window (window 2) is mostly Chinese, with a small portion in English. Therefore, when performing language recognition on the speech segments in window 2, the probability of recognition is Chinese. The language of the speech segments in the third window (window 3) is mostly English, with a small portion in Chinese. Therefore, when performing language recognition on the speech segments in window 3, the probability of recognition is English. Therefore, the final language recognition result shows that the speech segments in windows 1 and 2 are both identified as Chinese, while the speech segment in window 3 is identified as English. This shows that there is a deviation in the speech recognition results. By analogy, the language recognition result for the speech segment in window 4 is English, the language recognition result for the speech segment in window 5 is English, the language recognition result for the speech segment in window 6 is English, the language recognition result for the speech segment in window 7 is Chinese, the language recognition result for the speech segment in window 8 is Chinese, and the language recognition result for the speech segment in window 9 is Chinese. The language recognition results for the speech segments in windows 4-6 are all in English, which also deviates from the true language. English and Chinese are inseparable in speech. Because the window length is fixed, the points where the language changes cannot be recognized, resulting in inevitable deviations in the final recognition result. This deviation will affect the accuracy of the language recognition of the speech to be recognized, and will also affect the accuracy of speech translation when speech translation is performed according to the language corresponding to the speech.

[0028] Based on this, the present application proposes a language recognition method. This technical solution can detect the language jump moment in speech and move the sliding window according to the jump moment so that the speech segment within the sliding window does not contain language jump, thereby solving the problem of low accuracy of language recognition and low accuracy of speech translation in the existing technology.

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] Exemplary Methods See also Figure 2 As shown, the embodiment of the present application proposes a language recognition method. The method includes: S201 : Slide and extract voice segments from the speech to be recognized according to a sliding window of a preset size, and detect voiceprint turning points from the speech to be recognized.

[0031] In this embodiment, the speech requiring language recognition is determined as the speech to be recognized. The speech to be recognized is preferably a real-time speech stream processed by Voice Activity Detection (VAD), that is, a data stream obtained by separating valid speech segments (excluding non-speech parts such as silence and noise) from a continuous audio signal in real time.

[0032] In this embodiment, a sliding window is pre-set, and a sliding window size, i.e., a preset size, is preset. In this embodiment, the preset size is preferably set to a time length of 2 seconds. In this embodiment, a voice segment is first extracted from the speech to be recognized according to a sliding window of the preset size. Each time the sliding window slides to extract a voice segment, it slides according to the preset sliding size, which is smaller than the preset size of the sliding window. The preset sliding size is preferably set to a time length of 1 second.

[0033] This embodiment needs to perform voiceprint turning point detection on the speech to be recognized while slidingly extracting speech segments from the speech to be recognized, and the extraction of speech segments from the speech to be recognized and the performance of language recognition and the performance of voiceprint turning point detection on the speech to be recognized need to be performed simultaneously, that is, while extracting the first speech segment from the speech to be recognized, the voiceprint turning point detection is also performed on the speech content in the first speech segment; while extracting the second speech segment from the speech to be recognized, the voiceprint turning point detection is also performed on the speech content in the second speech segment, and so on.

[0034] In this embodiment, the voiceprint turning point in the speech to be recognized refers to the moment when the speaker changes, that is, the moment when the speaker jumps. Because the language may change when the speaker changes, this embodiment needs to detect whether there is a voiceprint turning point in the speech segment. If there is a voiceprint turning point, it means that there may be multiple languages ​​in the speech segment. If the language recognition is directly performed on the speech segment, only the language with a larger content proportion can be recognized, thereby causing recognition deviation. Only by adjusting the speech segment according to the voiceprint turning point so that there is no voiceprint turning point in the speech segment, ensuring that there is only one language in the speech segment, can the accuracy of language recognition be guaranteed.

[0035] This embodiment performs voiceprint turning point detection on the speech to be recognized. First, voice feature extraction is performed on the speech to be recognized, that is, key feature points are extracted from the speech signal of the speech to be recognized. The extracted voice features of the speech to be recognized include timbre features, frequency features, and energy change features. The speech features of the speech to be recognized are then analyzed, and the moment when the key feature points in the speech to be recognized change significantly is identified as the voiceprint turning point. Alternatively, voiceprint turning point detection on the speech to be recognized can also be performed using spectrogram analysis. A spectrogram of the speech to be recognized is constructed, and the location of the voiceprint turning point is determined by analyzing changes in the fundamental frequency and harmonic frequencies in the spectrogram.

[0036] In a specific embodiment, a voiceprint turning point detection model can also be constructed and trained. The speech to be recognized is input into the voiceprint turning point detection model, so that the model detects voiceprint turning points in the speech to be recognized and, upon detection, outputs voiceprint turning point information corresponding to the voiceprint turning point. The voiceprint turning point information includes the location of the voiceprint turning point in the speech to be recognized (e.g., the time at which the voiceprint turning point occurs in the speech to be recognized). In this embodiment, the training process of the voiceprint turning point detection model is as follows: First, a training sample for the voiceprint turning point detection model is obtained, i.e., a speech sample. The speech sample carries annotated voiceprint turning point information, which is pre-annotated voiceprint turning points in the speech sample. Then, the speech sample is input into the pre-constructed voiceprint turning point detection model. The model detects voiceprint turning points in the speech sample using the voiceprint turning point detection model and outputs corresponding predicted voiceprint turning point information. Finally, the model parameters of the voiceprint turning point detection model are adjusted with the goal of ensuring that the predicted voiceprint turning point information is consistent with the annotated voiceprint turning point information.

[0037] The language recognition method of this embodiment can be applied on a remote server or on a user device. When the language recognition method of this embodiment is applied on a remote server, the voiceprint turning point detection model is also deployed on the remote server, thereby realizing cloud-based online deployment. When the language recognition method of this embodiment is applied on a user device, the voiceprint turning point detection model is also deployed on the user device, thereby realizing local offline deployment.

[0038] When the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model can be set to a first size. When the voiceprint turning point detection model is deployed on a user device, the size of the voiceprint turning point detection model can be set to a second size, where the first size is larger than the second size. That is, when the voiceprint turning point detection model is deployed on a remote server, due to the sufficient computing power of the remote server, the size of the voiceprint turning point detection model can be appropriately enlarged, thereby improving the accuracy of the voiceprint turning point detection model. When the voiceprint turning point detection model is deployed on the user device, voiceprint turning point detection can be performed without an internet connection, thereby enabling offline voiceprint turning point detection. In addition, due to the sufficient computing power when the voiceprint turning point detection model is deployed on a remote server, a hybrid expert model (MOE) can also be used as the voiceprint turning point detection model to further improve the accuracy of voiceprint turning point detection.

[0039] S202: When a voiceprint turning point is detected from a speech segment within a first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window.

[0040] If, during voiceprint turning point detection for the speech to be recognized, a voiceprint turning point is detected in the speech segment within the first sliding window, the first sliding window needs to be shifted and adjusted based on the detected voiceprint turning point so that the voiceprint turning point is located at the endpoint of the first sliding window. In this case, if the voiceprint turning point is not located in the speech segment within the first sliding window, the speech segment within the first sliding window will not experience a speaker change or a language change. That is, the speech segment within the first sliding window will only contain one language, thereby ensuring the accuracy of language recognition. The shift adjustment of the first sliding window requires that the voiceprint turning point is located at the endpoint of the first sliding window. It is also necessary to ensure that there is no unextracted content between the first sliding window and the previously extracted second sliding window. That is, the first and second sliding windows can be adjacent end to end or have overlapping content (that is, in the speech to be recognized, the end point of the second sliding window is after the starting point of the first sliding window). However, there must be no gap between the first and second sliding windows (that is, there must be no unextracted speech content between the end point of the second sliding window and the starting point of the first sliding window). The first sliding window is a sliding window obtained by sliding the second sliding window once according to a preset sliding size.

[0041] In a specific embodiment, if the position of the voiceprint turning point detected in the first sliding window is not adjacent to and does not overlap with the end point of the second sliding window, then the first sliding window needs to be shifted forward so that the voiceprint turning point is located at the end point of the first sliding window. The second sliding window is the sliding window before the first sliding window.

[0042] In another specific embodiment, if the position of the voiceprint turning point detected in the first sliding window is adjacent to or coincides with the end point of the second sliding window, then the first sliding window needs to be shifted backward so that the voiceprint turning point is located at the starting point of the first sliding window. The second sliding window is the sliding window preceding the first sliding window.

[0043] S203: Perform language recognition on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0044] After the first sliding window is shifted and adjusted through the above steps, language recognition is performed on the speech segment within the shifted first sliding window to obtain the language recognition result corresponding to the speech segment. Language recognition of speech segments can be performed using a direct comparison method based on acoustic features. Speech signals of different languages ​​differ in acoustic features, such as pitch, intensity, duration, timbre, and spectral distribution. By directly comparing these acoustic features, different languages ​​can be distinguished. Specifically, the acoustic features of the speech segment are extracted, and then the acoustic feature ranges for each language are preset. By matching and analyzing the acoustic features of the speech segment with the preset acoustic feature ranges for each language, the language recognition result corresponding to the speech segment is determined. Recognition methods based on prosodic features can also be used. Prosodic features include intonation, stress, rhythm, etc. These features exhibit significant differences between different languages. For example, Chinese is a tonal language, while English is a stress language. The prosodic features of a speech segment are extracted, and then pre-set prosodic feature ranges for various languages ​​are determined. By matching the prosodic features of the speech segment with the pre-set prosodic feature ranges, the corresponding language recognition result is determined. A rule-based approach based on linguistic knowledge (such as grammatical rules and lexical features) can also be employed, constructing language recognition rules based on specific language attributes (such as word order and inflection). By analyzing the speech segment's grammatical rules and lexical features, the speech features of the speech segment are determined. The speech features are then analyzed according to the constructed speech recognition rules to determine the corresponding language recognition result.

[0045] In a specific embodiment, a language recognition model can be pre-built and trained. The speech segment within the shifted first sliding window is then input into the language recognition model, causing the model to perform language recognition on the input speech segment and obtain a corresponding language recognition result. The language recognition model training process involves the following steps: first, obtaining training samples for the language recognition model, namely, sample speech, wherein the sample speech carries an annotated language, where the annotated language is pre-annotated in the sample speech. Then, the sample speech is input into the pre-built language recognition model, and the language recognition model is used to perform language recognition on the sample speech, outputting a corresponding predicted language. Finally, with the goal of ensuring that the predicted language matches the annotated language, the model parameters in the language recognition model are adjusted. Furthermore, a multilingual hybrid modeling strategy can be employed when training the language recognition model. For example, sample speech in several different languages ​​can be collected as training samples, thereby enabling the trained language recognition model to support multilingual recognition.

[0046] Because existing language recognition uses a fixed window length for sliding the window and does not shift and adjust the sliding window according to the voiceprint turning point, the voiceprint within the sliding window may be impure (including the voiceprints of multiple people), which will affect the accuracy of language detection. In this case, the only way to ensure the accuracy of language detection as much as possible is to improve the accuracy of the language recognition model. Therefore, existing language recognition models are usually trained separately for different language pairs, that is, different language recognition models need to be trained for different language pairs to improve the accuracy of the language recognition model. However, the computational complexity of the model trained by training different language pairs separately is large and is not suitable for local offline deployment. However, the present application can improve the purity of the voiceprint within the sliding window by detecting the voiceprint turning point and shifting the sliding window, thereby improving the accuracy of language recognition. There is no need to train language recognition models separately for different language pairs, and a language recognition model suitable for multi-language recognition can be directly trained. This reduces the computational complexity of the model and does not require loading complex models. It is suitable for running on offline devices, that is, it is applicable to both cloud-based online and local offline deployment.

[0047] In a specific embodiment, if a voiceprint turning point is detected within the first sliding window and is located at the end point of the first sliding window, or if the first sliding window is shifted and adjusted so that the voiceprint turning point is located at the end point of the first sliding window, then after completing language identification for the speech segment within the first sliding window, the sliding window needs to be slid. When extracting the next speech segment, the first sliding window can be directly slid by a preset size to obtain a third sliding window, i.e., the starting point of the third sliding window is located at a position adjacent to the end point of the first sliding window. The speech segment extracted from the third sliding window is then processed according to the language identification method, i.e., the speech segment within the third sliding window is detected for a voiceprint turning point. If a voiceprint turning point is detected within the speech segment within the third sliding window, the third sliding window is shifted and adjusted based on the voiceprint turning point so that the voiceprint turning point is located at the end point of the third sliding window. Language identification is then performed on the speech segment within the shifted third sliding window to obtain a corresponding language identification result.

[0048] As can be seen from the above description, the language identification method proposed in the embodiment of the present application extracts speech segments from the speech to be identified according to a sliding window of a preset size, and detects voiceprint turning points from the speech to be identified; when a voiceprint turning point is detected from a speech segment within a first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at the end point of the first sliding window; the language of the speech segment within the shifted first sliding window is identified to obtain a corresponding language identification result. The technical solution of this embodiment can detect the voiceprint turning point within the sliding window, thereby identifying the speaker jump moment within the sliding window, that is, it can identify the language jump moment, and shift the sliding window according to the jump moment to divide the speech segments. This can avoid the sliding window containing multiple languages. The language recognition is performed on the speech segments of a single language within the sliding window, thereby improving the accuracy of language recognition of the speech to be identified, thereby improving the accuracy of speech translation.

[0049] As an optional implementation, the present application embodiment also proposes a language identification method. Figure 3 As shown, the method includes: S301: Slide and extract voice segments from the speech to be recognized according to a sliding window of a preset size, and detect voiceprint turning points from the speech to be recognized.

[0050] S302: When a voiceprint turning point is detected from a speech segment within a first sliding window, shift and adjust the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window.

[0051] S303: Perform language recognition on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0052] S304: When no voiceprint turning point is detected from the speech segment in the first sliding window, perform language recognition on the speech segment in the first sliding window to obtain a corresponding language recognition result.

[0053] If no voiceprint turning point is detected from the speech segment in the first sliding window, it means that there is only one language in the speech segment in the first sliding window. At this time, there is no need to shift the first sliding window, and the language recognition of the speech segment in the first sliding window can be directly performed to obtain the corresponding language recognition result.

[0054] The specific execution method of steps S301 - S303 of this embodiment is the same as the specific execution method of steps S201 - S203 of the above embodiment, and will not be elaborated in detail in this embodiment.

[0055] As an optional implementation, another embodiment of the present application discloses that a voice segment is extracted from the speech to be recognized by sliding a sliding window of a preset size. When a voiceprint turning point is detected in the voice segment within a first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at the endpoint of the first sliding window. Language recognition is performed on the voice segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result, including: The speech to be recognized is input into a pre-trained multilingual recognition model so that the multilingual recognition model slides and extracts speech segments from the speech to be recognized according to a sliding window of preset size. When the voiceprint turning point information output by the voiceprint turning point detection model is received, the first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information so that the voiceprint turning point is located at the end point position of the first sliding window. The speech segment within the first sliding window after the shift adjustment is subjected to language recognition to obtain the corresponding language recognition result.

[0056] Specifically, in this embodiment, the speech to be recognized is input into a pre-trained multilingual recognition model and a voiceprint turning point detection model. The multilingual recognition model extracts a speech segment from the speech to be recognized using a sliding window of a preset size. Simultaneously, the voiceprint turning point detection model also detects voiceprint turning points in the speech segment. Upon detecting a voiceprint turning point in the speech segment, the voiceprint turning point information corresponding to the voiceprint turning point is transmitted to the multilingual recognition model. The multilingual recognition model receives the voiceprint turning point information in the speech segment and, based on the voiceprint turning point information corresponding to the voiceprint turning point, shifts a first sliding window containing the voiceprint turning point so that the voiceprint turning point is located at the endpoint of the first sliding window. Language recognition is then performed on the speech segment within the shifted first sliding window to obtain the corresponding language recognition result. The training process of the multilingual recognition model in this embodiment has been specifically described in the above embodiment and will not be repeated in this embodiment.

[0057] As an optional implementation, another embodiment of the present application discloses performing language recognition on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result, which specifically includes the following steps: First, the speech segment in the second sliding window and the speech segment in the first sliding window after the shift adjustment are analyzed for overlap to obtain speech overlap.

[0058] In this embodiment, the previous sliding window of the first sliding window after shift adjustment is used as the second sliding window, and the overlap analysis is performed on the voice segments in the second sliding window and the voice segments in the first sliding window after shift adjustment, that is, the number of voice segments in the second sliding window contained in the first sliding window after shift adjustment is analyzed, so as to determine the voice overlap between the two.

[0059] Second, if the speech overlap reaches a preset overlap threshold, the language recognition result corresponding to the speech segment in the second sliding window is used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment.

[0060] If it is determined that the speech overlap reaches the preset overlap threshold, it means that the speech segment in the first sliding window after the shift adjustment overlaps with the speech segment in the second sliding window to a large extent, and the speech segment in the first sliding window after the shift adjustment does not have a voiceprint turning point. In this case, the language recognition result corresponding to the speech segment in the second sliding window can be directly used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment. Since the second sliding window is the previous sliding window of the first sliding window after the shift adjustment, the corresponding language recognition result has been determined for the speech segment in the second sliding window. In this embodiment, the preset overlap threshold is preferably set to 70%.

[0061] Third, if the speech overlap does not reach the preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0062] If it is determined that the speech overlap does not reach the preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment according to the language recognition method described in the above embodiment to obtain a corresponding language recognition result.

[0063] In addition, when the first sliding window is the first sliding window in the speech to be recognized, the second sliding window does not exist, that is, the second sliding window is empty at this time, then the speech overlap between the speech segment in the second sliding window and the speech segment in the first sliding window is zero.

[0064] In a specific embodiment, Figure 4 As shown, when the first sliding window is window 1, no voiceprint turning point is detected from the speech segment in window 1, and the language recognition is performed on the speech segment in window 1 to obtain the corresponding language recognition result, which is Chinese. The first sliding window slides right once according to the preset sliding size to obtain the original window 2, and the voiceprint turning point is detected from the speech segment in the original window 2, which is Figure 2 The character turning point 1 is taken as the middle character, and the original window 2 is shifted and adjusted according to the character turning point 1. At this time, the end point of the second sliding window (i.e., window 1) is not adjacent to and does not overlap with the character turning point 1. Then the original window 2 is moved forward so that the character turning point 1 is located at the end point of the original window 2, and the first sliding window after shift adjustment is obtained, i.e. Figure 2 The first sliding window slides right once according to the preset sliding size to obtain the original window 3, and the voiceprint turning point is detected from the voice segment in the original window 2, that is, Figure 2 The character turning point 1 is taken as the middle character. The original window 3 is shifted and adjusted according to the character turning point 1. At this time, the end point of the second sliding window (i.e., window 2) coincides with the character turning point 1. The original window 3 is moved backward so that the character turning point 1 is located at the starting point of the original window 3. The first sliding window after shift adjustment is obtained, i.e. Figure 2 Window 3 in the image is used to identify the language of the speech segment in window 3, and the corresponding language identification result is obtained, namely English. The first sliding window slides rightward once according to the preset sliding size to obtain window 4. No voiceprint turning point is detected from the speech segment in window 4. Language identification is performed on the speech segment in window 4, and the corresponding language recognition result is obtained, namely English. The first sliding window slides rightward once according to the preset sliding size to obtain window 5. A voiceprint turning point is detected from the speech segment in window 5, namely Figure 2The character turning point 2 is in the middle. Window 5 is shifted and adjusted according to the character turning point 2. At this time, the end point of the second sliding window (i.e., window 4) is not adjacent to and does not overlap with the character turning point 2. Therefore, it is necessary to make the character turning point 2 located at the end point of window 5. The character turning point is located at the end point of window 5 at this time. Therefore, there is no need to shift and adjust window 5. The language recognition is directly performed on the voice segment in window 5 to obtain the corresponding language recognition result, i.e. English. The first sliding window slides right once according to the preset sliding size to obtain the original window 6. The voiceprint turning point is detected from the voice segment in the original window 6, i.e. Figure 2 The character turning point 2 is selected, and the original window 6 is shifted and adjusted according to the character turning point 2. At this time, the end point of the second sliding window (i.e., window 5) coincides with the character turning point 1. The original window 6 is moved backward so that the character turning point 2 is located at the starting point of the original window 6, and the first sliding window after shift adjustment is obtained, i.e. Figure 2 Window 6 in the image is used, and language recognition is performed on the speech segment in window 6 to obtain the corresponding language recognition result, which is Russian. The first sliding window slides right once according to the preset sliding size to obtain window 7. No voiceprint turning point is detected from the speech segment in window 7. Language recognition is performed on the speech segment in window 7 to obtain the corresponding language recognition result, which is Russian. The first sliding window slides right once according to the preset sliding size to obtain original window 8. A voiceprint turning point is detected from the speech segment in original window 8. The original window 8 is shifted and adjusted according to the voiceprint turning point. At this time, the end point position of the second sliding window (i.e., window 7) is not adjacent to and does not overlap with the voiceprint turning point. The original window 8 is moved forward so that the voiceprint turning point is located at the end point position of the original window 8, and the first sliding window after shift adjustment is obtained, i.e. Figure 2 Window 8 in the figure performs language recognition on the speech segment in window 8 to obtain the corresponding language recognition result, namely Russian.

[0065] Exemplary devices Accordingly, the present application also provides a language recognition device, see Figure 5 As shown, the device includes: The voiceprint turning point detection module 100 is used to extract voice segments from the speech to be recognized by sliding a sliding window of a preset size, and detect voiceprint turning points from the speech to be recognized; The sliding window shifting module 110 is configured to, when a voiceprint turning point is detected from a speech segment within a first sliding window, shift and adjust the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; The language recognition module 120 is configured to perform language recognition on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0066] From the above introduction, it can be seen that the language recognition device proposed in the embodiment of the present application can detect the turning point of the voiceprint within the sliding window, thereby identifying the moment when the speaker jumps within the sliding window, that is, it can identify the language jump moment, and move the sliding window according to the jump moment to divide the voice segments, which can avoid the sliding window containing multiple languages ​​and perform language recognition on the voice segments of a single language in the sliding window, so that the accuracy of language recognition of the speech to be recognized is higher, thereby improving the accuracy of speech translation.

[0067] As an optional implementation, another embodiment of the present application discloses a sliding window shifting module 110, which is specifically configured to: If the end point of the second sliding window is not adjacent to or coincident with the voiceprint turning point, the first sliding window is shifted forward so that the voiceprint turning point is located at the end point of the first sliding window. or, When the end point of the second sliding window is adjacent to or coincides with the voiceprint turning point, the first sliding window is shifted backward so that the voiceprint turning point is located at the starting point of the first sliding window; the second sliding window is the previous sliding window of the first sliding window.

[0068] As an optional implementation, another embodiment of the present application discloses that the language recognition module 120 is also used to perform language recognition on the voice segment in the first sliding window when no voiceprint turning point is detected in the voice segment in the first sliding window to obtain a corresponding language recognition result.

[0069] As an optional implementation, disclosed in another embodiment of the present application, a voiceprint turning point detection module 100 is specifically used to input the voice data stream of the speech to be recognized into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects the voiceprint turning point from the speech to be recognized, and outputs the voiceprint turning point information when the voiceprint turning point is detected.

[0070] As an optional implementation, another embodiment of the present application discloses that the language identification module 120 is specifically configured to: The speech to be recognized is input into a pre-trained multilingual recognition model so that the multilingual recognition model slides and extracts speech segments from the speech to be recognized according to a sliding window of preset size. When the voiceprint turning point information output by the voiceprint turning point detection model is received, the first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information so that the voiceprint turning point is located at the end point position of the first sliding window. The speech segment within the first sliding window after the shift adjustment is subjected to language recognition to obtain the corresponding language recognition result.

[0071] As an optional implementation, another embodiment of the present application discloses that the language identification module 120 is specifically configured to: Performing overlap analysis on the speech segments in the second sliding window and the speech segments in the first sliding window after the shift adjustment to obtain speech overlap, wherein the second sliding window is the previous sliding window of the first sliding window; If the speech overlap reaches a preset overlap threshold, the language recognition result corresponding to the speech segment in the second sliding window is used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment; If the speech overlap does not reach the preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0072] As an optional implementation, another embodiment of the present application discloses that, when the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model; When the voiceprint turning point detection model is deployed on the user device, the size of the voiceprint turning point detection model is the second size; The first size is greater than the second size.

[0073] The language recognition device provided in this embodiment is based on the same concept as the language recognition method provided in the aforementioned embodiments of this application. It can execute the language recognition method provided in any of the aforementioned embodiments of this application and has the corresponding functional modules and beneficial effects of executing the language recognition method. For technical details not fully described in this embodiment, please refer to the specific processing content of the language recognition method provided in the aforementioned embodiments of this application and will not be repeated here.

[0074] Exemplary Systems Optionally, the present application also provides a language recognition system, see Figure 6 As shown, the language recognition system includes: a voiceprint detection subsystem 200 and a language recognition subsystem 210.

[0075] The voiceprint detection subsystem 200 is used to detect voiceprint turning points from the speech to be recognized, and when a voiceprint turning point is detected, output the voiceprint turning point information to the language recognition subsystem; The language recognition subsystem 210 is used to slide and extract voice segments from the speech to be recognized according to a sliding window of a preset size. When receiving the voiceprint turning point information sent by the voiceprint detection subsystem, the first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at the end point position of the first sliding window; the language recognition is performed on the voice segment in the first sliding window after the shift adjustment to obtain the corresponding language recognition result.

[0076] From the above introduction, it can be seen that the language recognition system proposed in the embodiment of the present application can detect the turning point of the voiceprint within the sliding window, thereby identifying the moment when the speaker jumps within the sliding window, that is, it can identify the language jump moment, and move the sliding window according to the jump moment to divide the voice segments, which can avoid the sliding window containing multiple languages ​​and perform language recognition on the voice segments of a single language in the sliding window, so that the accuracy of language recognition of the voice to be recognized is higher, thereby improving the accuracy of voice translation.

[0077] As an optional implementation, another embodiment of the present application discloses a voiceprint detection subsystem 200, which is specifically configured to shift the first sliding window forward so that the voiceprint turning point is located at the end point of the first sliding window when the end point of the second sliding window is not adjacent to and does not overlap with the voiceprint turning point. or, When the end point of the second sliding window is adjacent to or coincides with the voiceprint turning point, the first sliding window is shifted backward so that the voiceprint turning point is located at the starting point of the first sliding window; the second sliding window is the previous sliding window of the first sliding window.

[0078] As an optional implementation, another embodiment of the present application discloses that the language recognition subsystem 210 is also used to perform language recognition on the speech segment within the first sliding window when no voiceprint turning point is detected in the speech segment within the first sliding window to obtain a corresponding language recognition result.

[0079] As an optional implementation, disclosed in another embodiment of the present application, a voiceprint detection subsystem 200 is specifically used to input the voice data stream of the speech to be recognized into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects the voiceprint turning point from the speech to be recognized, and outputs the voiceprint turning point information when the voiceprint turning point is detected.

[0080] As an optional implementation, another embodiment of the present application discloses a language recognition subsystem 210, which is specifically used to input the speech to be recognized into a pre-trained multilingual recognition model, so that the multilingual recognition model slides and extracts speech segments from the speech to be recognized according to a sliding window of a preset size. When the voiceprint turning point information output by the voiceprint turning point detection model is received, the first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information, so that the voiceprint turning point is located at the end point position of the first sliding window, and the speech segment in the first sliding window after the shift adjustment is performed language recognition to obtain the corresponding language recognition result.

[0081] As an optional implementation, another embodiment of the present application discloses a language identification subsystem 210, which is specifically configured to: Performing overlap analysis on the speech segments in the second sliding window and the speech segments in the first sliding window after the shift adjustment to obtain speech overlap, wherein the second sliding window is the previous sliding window of the first sliding window; If the speech overlap reaches a preset overlap threshold, the language recognition result corresponding to the speech segment in the second sliding window is used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment; If the speech overlap does not reach the preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

[0082] As an optional implementation, another embodiment of the present application discloses that, when the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model; When the voiceprint turning point detection model is deployed on the user device, the size of the voiceprint turning point detection model is the second size; The first size is greater than the second size.

[0083] The language recognition system provided in this embodiment is based on the same concept as the language recognition method provided in the aforementioned embodiments of this application. It can execute the language recognition method provided in any of the aforementioned embodiments of this application and has the corresponding functional modules and beneficial effects of executing the language recognition method. For technical details not fully described in this embodiment, please refer to the specific processing content of the language recognition method provided in the aforementioned embodiments of this application and will not be repeated here.

[0084] Exemplary electronic devices Another embodiment of the present application further provides an electronic device, see Figure 7 As shown, the device includes: Memory 300 and processor 310; The memory 300 is connected to the processor 310 and is used to store programs; The processor 310 is configured to implement the language recognition method disclosed in any of the above embodiments by running the program stored in the memory 300 .

[0085] Specifically, the electronic device may further include: a bus, a communication interface 320 , an input device 330 and an output device 340 .

[0086] The processor 310, the memory 300, the communication interface 320, the input device 330 and the output device 340 are interconnected via a bus. A bus may include a pathway that transfers information between components of a computer system.

[0087] Processor 310 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, or the like. It can also be an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components.

[0088] The processor 310 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0089] The memory 300 stores a program for executing the technical solution of the present invention and may also store an operating system and other key services. Specifically, the program may include program code, which includes computer operating instructions. More specifically, the memory 300 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, and the like.

[0090] The input device 330 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0091] Output device 340 may include devices that allow information to be output to a user, such as a display screen, printer, speakers, etc.

[0092] The communication interface 320 may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0093] The processor 310 executes the program stored in the memory 300 and calls other devices to implement the various steps of any language recognition method provided in the above embodiments of the present application.

[0094] Exemplary computer program products and storage media In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the language recognition method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0095] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0096] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is used by a processor to execute the steps of the language recognition method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0097] For the sake of simplicity, the aforementioned method embodiments are described as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0098] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similarities between the various embodiments can be referred to in conjunction with each other. For device embodiments, since they are generally similar to method embodiments, their description is relatively simple, and for relevant details, reference can be made to the description of the method embodiments.

[0099] The steps in the methods of each embodiment of the present application can be adjusted in sequence, merged, and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0100] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be merged, divided, and deleted according to actual needs.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of modules or submodules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or module, which can be electrical, mechanical or other forms.

[0102] The modules or submodules described as separate components may or may not be physically separate, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules may be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, each functional module or submodule in each embodiment of the present application may be integrated into a processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into a single module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or software functional modules or submodules.

[0104] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, software executed by a processor, or a combination of the two. The software may be stored in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0106] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0107] The above description of the disclosed embodiments will enable those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. A language identification method, characterized in that: include: Sliding and extracting voice segments from the speech to be recognized according to a sliding window of a preset size, and detecting voiceprint turning points from the speech to be recognized; When a voiceprint turning point is detected from the speech segment within the first sliding window, the first sliding window is shifted and adjusted according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; Language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

2. The method according to claim 1, characterized in that Shifting and adjusting the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window includes: If the end point of the second sliding window is not adjacent to or coincident with the voiceprint turning point, shifting the first sliding window forward so that the voiceprint turning point is located at the end point of the first sliding window; or, When the end point of the second sliding window is adjacent to or coincides with the voiceprint turning point, the first sliding window is shifted backward so that the voiceprint turning point is located at the starting point of the first sliding window; the second sliding window is the previous sliding window of the first sliding window.

3. The method according to claim 1, characterized in that Also includes: When no voiceprint turning point is detected from the speech segment in the first sliding window, language recognition is performed on the speech segment in the first sliding window to obtain a corresponding language recognition result.

4. The method according to claim 1, wherein Detecting a voiceprint turning point from the speech to be recognized includes: The voice data stream of the speech to be recognized is input into a pre-trained voiceprint turning point detection model, so that the voiceprint turning point model detects the voiceprint turning point from the speech to be recognized, and outputs voiceprint turning point information when the voiceprint turning point is detected.

5. The method according to claim 4, characterized in that Extracting a speech segment from the speech to be recognized by sliding a sliding window of a preset size; when a voiceprint turning point is detected in the speech segment within a first sliding window, shifting and adjusting the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; Performing language recognition on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result includes: The speech to be recognized is input into a pre-trained multilingual recognition model so that the multilingual recognition model slides and extracts speech segments from the speech to be recognized according to a sliding window of preset size. Upon receiving voiceprint turning point information output by the voiceprint turning point detection model, a first sliding window containing the voiceprint turning point is shifted and adjusted according to the voiceprint turning point corresponding to the voiceprint turning point information so that the voiceprint turning point is located at the endpoint position of the first sliding window. Language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

6. The method according to claim 1, characterized in that Performing language recognition on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result includes: performing overlap analysis on the speech segments in the second sliding window and the speech segments in the first sliding window after the shift adjustment to obtain speech overlap, wherein the second sliding window is a previous sliding window of the first sliding window; If the speech overlap reaches a preset overlap threshold, the language recognition result corresponding to the speech segment in the second sliding window is used as the language recognition result corresponding to the speech segment in the first sliding window after the shift adjustment; If the speech overlap does not reach a preset overlap threshold, language recognition is performed on the speech segment within the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

7. The method according to claim 4, characterized in that In the case where the voiceprint turning point detection model is deployed on a remote server, the size of the voiceprint turning point detection model is a first size; the voiceprint turning point detection model of the first size adopts a hybrid expert model; When the voiceprint turning point detection model is deployed on a user device, the size of the voiceprint turning point detection model is a second size; The first size is greater than the second size.

8. A language recognition device, characterized in that: include: A voiceprint turning point detection module, configured to extract voice segments from the speech to be recognized by sliding a window of a preset size, and detect voiceprint turning points from the speech to be recognized; a sliding window shifting module, configured to, when a voiceprint turning point is detected from a speech segment within a first sliding window, shift and adjust the first sliding window according to the voiceprint turning point so that the voiceprint turning point is located at an end point of the first sliding window; The language recognition module is used to perform language recognition on the speech segment in the first sliding window after the shift adjustment to obtain a corresponding language recognition result.

9. An electronic device, characterized in that: include: memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the language recognition method according to any one of claims 1 to 7 by running the program in the memory.

10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to implement the language recognition method according to any one of claims 1 to 7.

Images