Method and device for converting multi-language mashup text into voice and medium

Abstract

The invention discloses a method and device for converting a multi-language mashup text into voice and a medium. The method comprises the following steps: carrying out data preprocessing on collected text data of Chinese and foreign languages to obtain a training data set; performing feature vector extraction on the training data set to obtain a feature vector data set of the training data set; inputting the feature vector data set into a deep learning model or a machine learning model for training, and generating a language recognition model; and recognizing the to-be-converted text by using the language recognition model to obtain language data of the to-be-converted text.

Description

Technical Field

[0001] The present invention relates to the field of speech conversion technology, and more specifically, to a method, device and medium for converting multi-language mixed text into speech. Background Art

[0002] When using the voice platform, some articles may be a mix of Chinese and foreign languages. In the process of converting the articles into voice, some foreign language characters are the same as Chinese. For example, some Japanese characters are the same or similar to Chinese. During the recognition process, the foreign language will be read as Chinese, resulting in the converted voice being unable to accurately express the text of the article. Summary of the invention

[0003] In view of the deficiencies of the prior art, the present invention provides a method, device and medium for converting multi-language mixed text into speech.

[0004] According to one aspect of the present invention, a method for converting multi-language mashup text into speech is provided, comprising: Preprocess the collected Chinese and foreign text data to obtain a training data set; Extracting feature vectors from the training data set to obtain a feature vector data set of the training data set; Input the feature vector data set into the deep learning model or machine learning model for training to generate a language recognition model; The language recognition model is used to recognize the text to be converted and obtain the language data of the text to be converted, wherein: Extract feature vectors from the training data set to obtain a feature vector data set of the training data set, including: Extract n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; Extract TF-IDF feature vectors based on the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; Determine the feature vector data set of the training data set based on the n-gram feature vector set and the TF-IDF feature vector set; Extract TF-IDF feature vectors based on the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set, including: Count the number of times each n-gram feature vector appears in the n-gram feature vector set; Calculate the total number of n-grams in the training dataset; Calculate the number of text samples that the training dataset contains for each n-gram feature vector; Determine the TF feature vector based on the number of times and the total number; Determine the IDF feature vector according to the number of text samples; According to the TF feature vector and the IDF feature vector, the TF-IDF feature vector set of the training data set is obtained.

[0005] Optionally, the collected Chinese and foreign text data are preprocessed to obtain a training data set, including: Clean the text data to obtain a valid text data set; Use the pre-trained sentence segmentation model and BERT language representation model to segment the valid text dataset and obtain the training sentence set; Use the built-in word segmenter of the BERT language model to segment the training sentence set and obtain the word segmentation sequence of each sentence in the training sentence set; Generate a training data set based on the word segmentation sequence of each sentence; Optionally, clean the text data to obtain a valid text data set, including: Remove special characters and punctuation marks from different languages ​​in text data to obtain a clean text data set; Unify the capitalization of special language texts in the cleaned text dataset to obtain a unified text dataset; Remove stop words from the unified text dataset to obtain a valid text dataset.

[0006] According to another aspect of the present invention, there is provided a device for converting multi-language mashup text into speech, comprising: A preprocessing module is used to preprocess the collected Chinese and foreign text data to obtain a training data set; A feature extraction module is used to extract feature vectors from a training data set to obtain a feature vector data set of the training data set; The model training module is used to input the feature vector data set into the deep learning model or machine learning model for training to generate a language recognition model; The recognition module is used to recognize the text to be converted by using the language recognition model to obtain the language data of the text to be converted, wherein: Feature extraction module, including: The first extraction submodule is used to extract n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; The second extraction submodule is used to extract TF-IDF feature vectors according to the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; A determination submodule is used to determine a feature vector data set of a training data set according to an n-gram feature vector set and a TF-IDF feature vector set; The second extraction submodule includes: A statistical unit, used to count the number of occurrences of each n-gram feature vector in the n-gram feature vector set; The first computing unit is used to calculate the total number of n-grams in the training data set; The second calculation unit is used to calculate the number of text samples containing each n-gram feature vector in the training data set; A first determining unit is used to determine a TF feature vector according to the number of times and the total number; A second determining unit is used to determine an IDF feature vector according to the number of text samples; The first acquisition unit is used to acquire a TF-IDF feature vector set of the training data set according to the TF feature vector and the IDF feature vector.

[0007] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method described in any one of the above aspects of the present invention.

[0008] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of the above aspects of the present invention.

[0009] Thus, the present invention preprocesses the collected Chinese and foreign text data by machine learning to obtain a training data set; extracts feature vectors from the training data set to obtain a feature vector data set of the training data set; inputs the feature vector data set into a deep learning model or a machine learning model for training to generate a language recognition model; uses the language recognition model to recognize the text to be converted to obtain the language data of the text to be converted. Accurate recognition of foreign languages ​​is achieved, the accuracy and success rate of speech conversion are improved, the cost of later speech production and correction is saved, and work efficiency is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings: Figure 1 is a flow chart of a method for converting multi-language mashup text into speech provided by an exemplary embodiment of the present invention; Figure 2It is a schematic diagram of the structure of a device for converting multi-language mashup text into speech provided by an exemplary embodiment of the present invention; Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0011] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0012] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0013] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0014] It should also be understood that, in the embodiments of the present invention, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0015] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0016] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.

[0017] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0018] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0019] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0020] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0021] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0022] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0023] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0024] Exemplary Methods Figure 1 FIG. 1 is a flow chart of a method for converting multilingual mashup text into speech according to an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices such as Figure 1 As shown, the method 100 for converting multi-language mashup text into speech includes the following steps: Step 101, preprocessing the collected Chinese and foreign text data to obtain a training data set; Step 102, extracting feature vectors from the training data set to obtain a feature vector data set of the training data set; Step 103, inputting the feature vector data set into a deep learning model or a machine learning model for training to generate a language recognition model; Step 104, using a language recognition model to recognize the text to be converted, and obtaining language data of the text to be converted, wherein: Extract feature vectors from the training data set to obtain a feature vector data set of the training data set, including: Extract n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; Extract TF-IDF feature vectors based on the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; Determine the feature vector data set of the training data set based on the n-gram feature vector set and the TF-IDF feature vector set; Extract TF-IDF feature vectors based on the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set, including: Count the number of times each n-gram feature vector appears in the n-gram feature vector set; Calculate the total number of n-grams in the training dataset; Calculate the number of text samples that the training dataset contains for each n-gram feature vector; Determine the TF feature vector based on the number of times and the total number; Determine the IDF feature vector according to the number of text samples; According to the TF feature vector and the IDF feature vector, the TF-IDF feature vector set of the training data set is obtained.

[0025] Optionally, the collected Chinese and foreign text data are preprocessed to obtain a training data set, including: Clean the text data to obtain a valid text data set; Use the pre-trained sentence segmentation model and BERT language representation model to segment the valid text dataset and obtain the training sentence set; Use the built-in word segmenter of the BERT language model to segment the training sentence set and obtain the word segmentation sequence of each sentence in the training sentence set; Generate a training data set based on the word segmentation sequence of each sentence; Optionally, clean the text data to obtain a valid text data set, including: Remove special characters and punctuation marks from different languages ​​in text data to obtain a clean text data set; Unify the capitalization of special language texts in the cleaned text dataset to obtain a unified text dataset; Remove stop words from the unified text dataset to obtain a valid text dataset.

[0026] Specifically, the present invention uses machine learning to perform Chinese and foreign language recognition, annotates a large amount of training data, and performs some data preprocessing and model training steps. The implementation steps are as follows: 1. Data collection: Collect a large amount of text data in Chinese and foreign languages. For example, you can find it on the Internet or use a crawler to crawl it from the website.

[0027] 2. Data preprocessing: preprocess the collected text data, including cleaning data, cutting sentences, and word segmentation. The purpose of this step is to convert the original text data into data that can be input into the model.

[0028] (1) Loading a pre-trained language model: First, load and initialize the selected pre-trained language model, such as BERT.

[0029] (2) Text cleaning and preprocessing: Perform basic cleaning and preprocessing on the collected text data, such as removing special characters, HTML tags, URLs, etc., to ensure the purity and consistency of the text.

[0030] Among them, the cleaning scheme is as follows: a. Remove special characters and punctuation marks: Different languages ​​may use different punctuation marks, so they need to be cleaned up according to the characteristics of the specific language.

[0031] For example: Chinese: you need to remove Chinese punctuation marks, such as ",",".","!"; Before cleaning: "Hello! How is the weather today?"; After cleaning: "Hello, how is the weather today"; English: Remove English punctuation marks, such as “,”, “.”, “!”, and remove extra spaces.

[0032] Before cleaning: "Hello! How are you today?"; After cleaning: "Hello How are you today".

[0033] b. Unify capitalization: In languages ​​like English, capitalization can reduce the number of synonyms that are repeated.

[0034] For example: English: Convert all text to lowercase.

[0035] Before cleaning: "Hello World"; After cleaning: "hello world"; German: Nouns in German usually start with a capital letter, and you can decide whether to use the same capital letter as needed.

[0036] Before cleaning: "Die Katze sitzt auf dem Tisch"; After cleaning: "die katze sitzt auf dem tisch" (for example, it can be unified to lowercase).

[0037] c. Remove stop words: Different languages have different stop words (such as "的", "是", "and", "the", etc.), which need to be removed during cleaning.

[0038] For example: Chinese: Remove common stop words.

[0039] Before cleaning: "我爱自然语言处理"; After cleaning: "爱 自然 语言 处理"; French: Remove French stop words.

[0040] Before cleaning: "Je suis très heureux de vous rencontrer"; After cleaning: "suis heureux rencontrer".

[0041] (3)Sentence splitting: Using the most innovative language model BERT, a pre-trained sentence splitting model can be used to split text data into sentences. This method can handle complex language structures, including ellipsis, quotation marks, and parentheses, etc.

[0042] Using the BERT model for sentence splitting mainly relies on a pre-trained sentence splitting tool, and BERT itself does not directly provide the sentence splitting function. This function can be achieved by combining an NLP toolkit and the tokenizer of the BERT model. The following is a detailed description and example: a. Select a sentence splitting tool: Use natural language processing libraries like NLTK or spaCy, which provide ready-made sentence splitting functions.

[0043] b. Load the BERT model and tokenizer: Use the transformers library of Hugging Face to load the pre-trained BERT model and the corresponding tokenizer.

[0044] c. Split the text into sentences: Use the selected tool to split the text into sentences.

[0045] d. Tokenize the sentences: Input the split sentences into the BERT tokenizer to convert them into a format that the model can handle.

[0046] (4) Word segmentation: Using the pre-trained language model BERT, you can use the built-in word segmenter to segment each sentence into words or subwords. This word segmentation method can handle complex lexical structures such as compound words, abbreviations, and proper nouns.

[0047] (5) Generate word segmentation sequence: Convert the word segmentation results of each sentence into a sequence for subsequent processing and analysis. You can use predefined encoding schemes such as WordPiece or Byte-Pair Encoding (BPE) to convert word segmentation into an input form acceptable to the model.

[0048] 3. Feature engineering: Extract feature vectors from preprocessed text data, including extracting N-gram features and TF-IDF features.

[0049] a. n-gram model: (1) Divide the text into a sequence of n consecutive words.

[0050] (2) Build an n-gram vocabulary, mapping each n-gram to a unique integer index.

[0051] (3) For each text sample, count the number of times each n-gram appears in the text or use TF-IDF to weight it.

[0052] Count the number of times each n-gram appears in the text: Input text sample: text; Input n-gram: n_gram; Count the number of times each n-gram appears in the text: count = count the number of times n-gram appears in text; b. Use TF-IDF for weighting: Input text sample: text; Input n-gram: n_gram; Input corpus: corpus; Count the number of times each n-gram appears in the text: count = count the number of times n-gram appears in text; Calculate the total number of all n-grams in the text sample: total_ngrams = Calculate the total number of n-grams in all text samples in the corpus; Calculate the number of text samples in the corpus that contain the current n-gram: num_documents = Calculate the number of text samples in the corpus that contain the current n-gram; Calculate TF (Term Frequency): tf = count / total_ngrams; Calculate IDF (Inverse Document Frequency): idf = log(total number of text samples in the corpus / (1 + num_documents)); Calculate the TF-IDF weighted value: tf_idf = tf idf.

[0053] (4) Represent each text sample as a vector, where each dimension of the vector corresponds to the number of occurrences of an n-gram in the text or the TF-IDF value.

[0054] 4. Model training: Select a suitable machine learning model or deep learning model, such as SVM, and input feature data into the model for training.

[0055] 5. Model evaluation: After training the model, you need to evaluate the model to see how effective it is. Common evaluation indicators include accuracy, precision, recall, F1 value, etc.

[0056] 6. Model application: Apply the trained model to actual projects for language recognition.

[0057] Therefore, the present invention can accurately recognize foreign languages ​​through machine learning, improve the accuracy and success rate of speech conversion, save the cost of later speech production and correction, and greatly improve work efficiency.

[0058] Exemplary Devices Figure 2 FIG. 1 is a schematic diagram of a structure of an apparatus for converting multilingual mashup text into speech provided by an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes: A preprocessing module is used to preprocess the collected Chinese and foreign text data to obtain a training data set; A feature extraction module is used to extract feature vectors from a training data set to obtain a feature vector data set of the training data set; The model training module is used to input the feature vector data set into the deep learning model or machine learning model for training to generate a language recognition model; The recognition module is used to recognize the text to be converted by using the language recognition model to obtain the language data of the text to be converted, wherein: Feature extraction module, including: The first extraction submodule is used to extract n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; The second extraction submodule is used to extract TF-IDF feature vectors according to the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; A determination submodule is used to determine a feature vector data set of a training data set according to an n-gram feature vector set and a TF-IDF feature vector set; The second extraction submodule includes: A statistical unit, used to count the number of occurrences of each n-gram feature vector in the n-gram feature vector set; The first computing unit is used to calculate the total number of n-grams in the training data set; The second calculation unit is used to calculate the number of text samples containing each n-gram feature vector in the training data set; A first determining unit is used to determine a TF feature vector according to the number of times and the total number; A second determining unit is used to determine an IDF feature vector according to the number of text samples; The first acquisition unit is used to acquire a TF-IDF feature vector set of the training data set according to the TF feature vector and the IDF feature vector.

[0059] Optionally, the preprocessing module includes: The first acquisition submodule is used to clean the text data and obtain a valid text data set; The second acquisition submodule is used to use the pre-trained sentence segmentation model and the BERT language representation model to perform sentence segmentation on the valid text data set to obtain a training sentence set; The third acquisition submodule is used to use the built-in word segmenter of the BERT language model to segment the training sentence set and obtain the word segmentation sequence of each sentence in the training sentence set; The generation submodule is used to generate a training data set based on the word segmentation sequence of each sentence; Optionally, the first acquisition submodule includes: A second acquisition unit is used to remove special characters and punctuation marks according to different languages ​​in the text data to obtain a cleaned text data set; A third acquisition unit is used to unify the capitalization of the special language text in the cleaned text data set to obtain a unified text data set; The fourth acquisition unit is used to remove stop words in the unified text data set to acquire a valid text data set.

[0060] Exemplary Electronic Devices Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes one or more processors 31 and a memory 32 .

[0061] The processor 31 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0062] The memory 32 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may run the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may also include: an input device 33 and an output device 34, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0063] In addition, the input device 33 may also include, for example, a keyboard, a mouse, and the like.

[0064] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0065] Of course, to simplify, Figure 3 Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device may also include any other appropriate components.

[0066] Exemplary computer program products and computer-readable storage media In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification.

[0067] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0068] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for information mining of historical change records according to various embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0069] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0070] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0071] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0072] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0073] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0074] It should also be noted that in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present invention. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but in accordance with the widest range consistent with the principles disclosed herein and novel feature vectors.

[0075] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for converting multi-language mixed text into speech, wherein the feature vector is: include: Preprocess the collected Chinese and foreign text data to obtain a training data set; Extracting feature vectors from the training data set to obtain a feature vector data set of the training data set; Inputting the feature vector data set into a deep learning model or a machine learning model for training to generate a language recognition model; The language recognition model is used to recognize the text to be converted, and the language data of the text to be converted is obtained, wherein: Extracting feature vectors from the training data set to obtain a feature vector data set of the training data set includes: Extracting n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; Extract TF-IDF feature vectors according to the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; Determine the feature vector data set of the training data set according to the n-gram feature vector set and the TF-IDF feature vector set; Extracting TF-IDF feature vectors according to the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set includes: Counting the number of occurrences of each n-gram feature vector in the n-gram feature vector set; Calculate the total number of n-grams in the training data set; Calculate the number of text samples that the training data set contains each n-gram feature vector; Determine a TF feature vector according to the number of times and the total number; Determine an IDF feature vector according to the number of text samples; The TF-IDF feature vector set of the training data set is obtained according to the TF feature vector and the IDF feature vector.

2. The method according to claim 1, characterized in that The collected Chinese and foreign text data are preprocessed to obtain training data sets, including: Cleaning the text data to obtain a valid text data set; Using a pre-trained sentence segmentation model and a BERT language representation model to perform sentence segmentation on the valid text dataset to obtain a training sentence set; The training sentence set is segmented using the built-in segmenter of the BERT language model to obtain a segmentation sequence of each sentence in the training sentence set; The training data set is generated according to the word segmentation sequence of each sentence.

3. The method according to claim 2, characterized in that Cleaning the text data to obtain a valid text data set includes: Removing special characters and punctuation marks for different languages ​​in the text data to obtain a cleaned text data set; Unifying the case of special language texts in the cleaned text data set to obtain a unified text data set; Stop words in the unified text data set are removed to obtain the valid text data set.

4. A device for converting multi-language mixed text into speech, Its eigenvectors include: A preprocessing module is used to preprocess the collected Chinese and foreign text data to obtain a training data set; A feature extraction module is used to extract feature vectors from the training data set to obtain a feature vector data set of the training data set; A model training module, used to input the feature vector data set into a deep learning model or a machine learning model for training to generate a language recognition model; A recognition module is used to recognize the text to be converted by using the language recognition model to obtain the language data of the text to be converted, wherein: Feature extraction module, including: A first extraction submodule is used to extract n-gram feature vectors from the training data set to obtain an n-gram feature vector set of the training data set; A second extraction submodule is used to extract TF-IDF feature vectors according to the n-gram feature vector set to obtain the TF-IDF feature vector set of the training data set; A determination submodule, configured to determine the feature vector data set of the training data set according to the n-gram feature vector set and the TF-IDF feature vector set; The second extraction submodule includes: A statistical unit, used for counting the number of occurrences of each n-gram feature vector in the n-gram feature vector set; A first computing unit, configured to compute the total number of n-grams in the training data set; A second calculation unit is used to calculate the number of text samples contained in each n-gram feature vector in the training data set; A first determining unit, configured to determine a TF feature vector according to the number of times and the total number; A second determining unit is used to determine an IDF feature vector according to the number of text samples; The first acquisition unit is used to acquire the TF-IDF feature vector set of the training data set according to the TF feature vector and the IDF feature vector.

5. The device according to claim 4, characterized in that Preprocessing modules include: A first acquisition submodule is used to clean the text data to obtain a valid text data set; A second acquisition submodule is used to perform sentence segmentation on the valid text data set using a pre-trained sentence segmentation model and a BERT language representation model to obtain a training sentence set; The third acquisition submodule is used to perform word segmentation processing on the training sentence set using the word segmenter built into the BERT language model to obtain a word segmentation sequence of each sentence in the training sentence set; The generating submodule is used to generate the training data set according to the word segmentation sequence of each sentence.

6. The device according to claim 5, characterized in that The first acquisition submodule includes: A second acquisition unit is used to remove special characters and punctuation marks in different languages ​​in the text data to obtain a cleaned text data set; A third acquisition unit is used to unify the capitalization of the special language text in the cleaned text data set to obtain a unified text data set; The fourth acquisition unit is used to remove stop words in the unified text data set to acquire the valid text data set.

7. A computer-readable storage medium, wherein the characteristic vector is: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 3.

8. An electronic device, whose characteristic vector is, The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 3.

Images