Training method, device, listening and speaking practice method, storage medium and electronic device

By obtaining and converting textbook content and teacher-student dialogue texts and training listening and speaking practice models, the problems of insufficient time in traditional classroom practice and the inability to connect with textbooks are solved, and personalized oral practices that synchronize courses are realized, and students are improved listening and speaking ability.

CN118643904BActive Publication Date: 2024-11-05CHENGDU JIAFAANTAI EDUCATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411125569.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-05
Estimated Expiration
2044-08-16

AI Technical Summary

Technical Problem

When synchronous listening and speaking exercises in traditional foreign language texts are carried out in class, some students have insufficient practice time and cannot provide sufficient personalized exercises and feedback, which leads to slow improvement of listening and speaking ability, and the existing dialogue language model for assisted language learning is difficult to match with Textbook content is closely connected and cannot provide synchronous personalized oral exercises.

Method used

By obtaining the chapter content and teacher-student dialogue text in the textbook, converting it into training samples, training listening and speaking practice models, using embedded networks, coding networks and language networks, combining with Transformer derivative layers, optimizing the teacher-student dialogue coding sequence, and generating textbook content with Match speaking exercises.

Benefits of technology

It enables students to perform listening and speaking exercises simultaneously during the course learning process, improves the effect and listening and speaking ability of personalized oral exercises, and meets the customized training needs for oral skills in course learning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118643904B_ABST
    Figure CN118643904B_ABST
Patent Text Reader

Abstract

The present invention provides a training method, device, listening and speaking practice method, storage medium and electronic device, which relate to the field of natural language processing. The electronic device obtains the chapter contents and teacher-student dialogue texts of multiple chapters in a textbook, and converts the chapter contents and teacher-student dialogue texts of each chapter into a group of training samples, wherein each group of training samples includes an input coding sequence and a target sequence, the input coding sequence includes the teacher-student dialogue text of the corresponding chapter and the coding sequence of the chapter contents, and the target sequence includes the coding sequence of the teacher dialogue content in the teacher-student dialogue text of the corresponding chapter; the input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain a listening and speaking practice model. In this way, the listening and speaking practice model enables students to perform listening and speaking practice synchronously with the text during the course learning process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a training method, a device, a listening and speaking practice method, a storage medium and an electronic device. Background Art

[0002] Synchronous listening and speaking exercises for foreign language texts are a teaching method that combines listening and speaking. It helps students deeply understand the text content and improve their foreign language listening and speaking skills by having them perform oral exercises such as imitation, retelling or interaction while listening to the text recording or the teacher’s reading.

[0003] Traditional foreign language text synchronous listening and speaking practice is usually carried out in class, relying on the interaction between teachers and students or between students. However, due to limited class time, students have very few opportunities to actually practice listening and speaking. Usually, each student can only take turns to speak in class, resulting in a serious lack of individual practice time. This method cannot provide each student with enough personalized practice and feedback, resulting in a slow improvement in their listening and speaking skills, and the practice effect is therefore not ideal.

[0004] Although there are currently a variety of conversational language models that assist language learning, these models are generally too open in the design of conversational content, which makes it difficult for these models to be closely connected with the content in textbooks, and thus unable to provide users with personalized oral practice that is synchronized with textbook knowledge. Summary of the invention

[0005] In order to overcome at least one of the deficiencies in the prior art, the present invention provides a training method, device, listening and speaking practice method, storage medium and electronic device, specifically comprising:

[0006] In a first aspect, the present invention provides a training method, the method comprising:

[0007] Obtain the chapter contents and teacher-student dialogue texts of multiple chapters in the textbook, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter;

[0008] Convert the chapter content and the teacher-student dialogue text of each chapter into a set of training samples, wherein each set of training samples includes an input coding sequence and a target sequence, the input coding sequence includes the teacher-student dialogue text of the corresponding chapter and the coding sequence of the chapter content, and the target sequence includes the coding sequence of the teacher's dialogue content in the teacher-student dialogue text of the corresponding chapter;

[0009] The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain the listening and speaking practice model.

[0010] In combination with the optional implementation manner of the first aspect, converting the chapter content of each chapter and the teacher-student dialogue text into a set of training samples includes:

[0011] For each chapter content and the teacher-student dialogue text, convert the chapter content into a chapter coding sequence;

[0012] Converting the teacher-student dialogue text into a teacher-student dialogue code sequence, and appending it to the chapter code sequence in the order of dialogue to obtain the input code sequence;

[0013] Replacing the coding of the student dialogue content in the teacher-student dialogue coding sequence by ignoring coding, thereby obtaining an optimized teacher-student dialogue coding sequence, wherein the ignoring coding represents coding without calculating model loss;

[0014] The target sequence is obtained according to the optimized teacher-student dialogue coding sequence.

[0015] In conjunction with the optional implementation manner of the first aspect, obtaining the target sequence according to the optimized teacher-student dialogue coding sequence includes:

[0016] By replacing all codes in the chapter code sequence by ignoring the codes, an auxiliary sequence is obtained;

[0017] The optimized teacher-student dialogue coding sequence is appended to the auxiliary sequence to obtain the target sequence.

[0018] In combination with the optional implementation manner of the first aspect, the input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, so as to obtain the listening and speaking practice model, including:

[0019] The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and the output data of the model to be trained respectively to iterate the parameters of the model to be trained, and multiple groups of model parameters in the iterative process are retained;

[0020] According to the multiple groups of model parameters, multiple candidate models are obtained;

[0021] The multiple candidate models are tested using a validation set, and the one with the best test result is selected as the listening and speaking practice model.

[0022] In combination with the optional implementation manner of the first aspect, the model to be trained includes an embedding network, an encoding network, and a language network, the encoding network includes a plurality of Transformer derivative layers, each Transformer derivative layer includes a Transformer layer and a bypass network parallel to the Transformer layer;

[0023] The step of using the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and the output data of the model to be trained to iterate the parameters of the model to be trained comprises:

[0024] For each group of training samples, inputting the input coding sequence in the training samples into the embedding network to obtain an embedding representation feature sequence;

[0025] Inputting the embedding representation feature sequence into the encoding network to obtain encoding features;

[0026] Inputting the coding features into the language network to obtain a predicted coding sequence;

[0027] Obtaining a model loss of the model to be trained according to the predicted coding sequence and the target sequence in the training sample;

[0028] The bypass network of each Transformer derivative layer and the parameters in the language network are updated according to the model loss.

[0029] In conjunction with the optional implementation manner of the first aspect, before inputting the input coding sequence in the training sample into the embedding network to obtain the embedding representation feature sequence, the method further includes:

[0030] Get the weights of a pre-trained large language model;

[0031] At least a portion of the embedding network, the encoding network, and the language network are initialized by weights of the pre-trained large language model.

[0032] In a second aspect, the present invention further provides a listening and speaking practice method, the method comprising:

[0033] Obtain the text of historical dialogues between teachers and students;

[0034] According to the historical dialogue text between the teacher and the students, a code sequence to be processed is obtained, wherein the code sequence to be processed includes the code sequence of the historical dialogue text between the teacher and the students and the code sequence of the chapter content in the chapter to be practiced;

[0035] Inputting the to-be-processed coding sequence into the listening and speaking practice model obtained by the training method to obtain a predicted coding sequence corresponding to the teacher-student historical dialogue text;

[0036] The predictive coding sequence of the teacher-student historical dialogue text is converted into a predictive text, and the predictive text is played in a voice manner.

[0037] In a third aspect, the present invention further provides a training device, the device comprising:

[0038] A text acquisition module is used to acquire the chapter contents of multiple chapters in the textbook and the teacher-student dialogue texts, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter;

[0039] A sample encoding module is used to convert the chapter content and the teacher-student dialogue text of each chapter into a set of training samples, wherein each set of training samples includes an input encoding sequence and a target sequence, the input encoding sequence includes the teacher-student dialogue text of the corresponding chapter and the encoding sequence of the chapter content, and the target sequence includes the encoding sequence of the teacher's dialogue content in the teacher-student dialogue text of the corresponding chapter;

[0040] The model training module is used to train the model to be trained by taking the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and output data of the model to be trained, so as to obtain the listening and speaking practice model.

[0041] In conjunction with the optional implementation manner of the third aspect, the sample encoding module is further specifically configured to:

[0042] For each chapter content and the teacher-student dialogue text, convert the chapter content into a chapter coding sequence;

[0043] Converting the teacher-student dialogue text into a teacher-student dialogue code sequence, and appending it to the chapter code sequence in the order of dialogue to obtain the input code sequence;

[0044] Replacing the coding of the student dialogue content in the teacher-student dialogue coding sequence by ignoring coding, thereby obtaining an optimized teacher-student dialogue coding sequence, wherein the ignoring coding represents coding without calculating model loss;

[0045] The target sequence is obtained according to the optimized teacher-student dialogue coding sequence.

[0046] In conjunction with the optional implementation manner of the third aspect, the sample encoding module is further specifically configured to:

[0047] By replacing all codes in the chapter code sequence by ignoring the codes, an auxiliary sequence is obtained;

[0048] The optimized teacher-student dialogue coding sequence is appended to the auxiliary sequence to obtain the target sequence.

[0049] In conjunction with the optional implementation manner of the third aspect, the model training module is further specifically used for:

[0050] The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and the output data of the model to be trained respectively to iterate the parameters of the model to be trained, and multiple groups of model parameters in the iterative process are retained;

[0051] According to the multiple groups of model parameters, multiple candidate models are obtained;

[0052] The multiple candidate models are tested using a validation set, and the one with the best test result is selected as the listening and speaking practice model.

[0053] In combination with an optional implementation manner of the third aspect, the model to be trained includes an embedding network, an encoding network, and a language network, the encoding network includes a plurality of Transformer derivative layers, each Transformer derivative layer includes a Transformer layer and a bypass network parallel to the Transformer layer;

[0054] The model training module is also specifically used for:

[0055] For each group of training samples, inputting the input coding sequence in the training samples into the embedding network to obtain an embedding representation feature sequence;

[0056] Inputting the embedding representation feature sequence into the encoding network to obtain encoding features;

[0057] Inputting the coding features into the language network to obtain a predicted coding sequence;

[0058] Obtaining a model loss of the model to be trained according to the predicted coding sequence and the target sequence in the training sample;

[0059] The bypass network of each Transformer derivative layer and the parameters in the language network are updated according to the model loss.

[0060] In conjunction with the optional implementation manner of the third aspect, before inputting the input coding sequence in the training sample into the embedding network to obtain the embedding representation feature sequence, the model training module is further used to:

[0061] Get the weights of a pre-trained large language model;

[0062] At least a portion of the embedding network, the encoding network, and the language network are initialized by weights of the pre-trained large language model.

[0063] In a fourth aspect, the present invention further provides a storage medium storing a computer program, wherein when the computer program is executed by a processor, the training method or the listening and speaking practice method is implemented.

[0064] In a fifth aspect, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the training method or the listening and speaking practice method is implemented.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention provides a training method, device, listening and speaking practice method, storage medium and electronic device. The electronic device obtains the chapter contents and teacher-student dialogue texts of multiple chapters in a textbook, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter; the chapter contents and teacher-student dialogue text of each chapter are respectively converted into a group of training samples, wherein each group of training samples includes an input coding sequence and a target sequence, the input coding sequence includes the teacher-student dialogue text of the corresponding chapter and the coding sequence of the chapter contents, and the target sequence includes the coding sequence of the teacher dialogue contents in the teacher-student dialogue text of the corresponding chapter; the input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain a listening and speaking practice model. In this way, the listening and speaking practice model enables students to perform listening and speaking practice synchronously with the text during the course learning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A flowchart of a training method provided by an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of a coding sequence of chapter contents provided by an embodiment of the present invention;

[0070] Figure 3 A schematic diagram of an input coding sequence provided by an embodiment of the present invention;

[0071] Figure 4 A schematic diagram of a target sequence provided by an embodiment of the present invention;

[0072] Figure 5 A schematic diagram of the structure of a model to be trained provided in an embodiment of the present invention;

[0073] Figure 6A schematic diagram of the structure of the Transformer derivative layer provided in an embodiment of the present invention;

[0074] Figure 7 A schematic diagram of the structure of a training device provided by an embodiment of the present invention;

[0075] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0076] Icons: 11-text acquisition module; 12-sample encoding module; 13-model training module; 21-memory; 22-processor; 23-communication unit; 24-system bus. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0078] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0080] Based on the above statement, as introduced in the background technology, although there are many conversational language models that assist language learning, these models are generally too open in the design of conversational content, which makes it difficult for these models to be closely connected with the content in textbooks, and thus cannot provide users with personalized oral practice that is synchronized with textbook knowledge.

[0081] For example, taking learning spoken English through ChatGPT as an example, users can communicate extensively with ChatGPT and discuss various topics, but this open-ended conversation cannot meet the needs of learners who want to conduct specialized oral training based on the textbook content. In other words, although ChatGPT can interact with users in English, the content of ChatGPT's conversations lacks pertinence and systematicness. This means that although ChatGPT can provide opportunities to practice English conversations, it cannot provide customized oral practice based on the content of specific textbooks, which is an obvious shortcoming for learners who hope to improve their oral skills while learning the course.

[0082] Based on the discovery of the above technical problems, the inventors have proposed the following technical solutions to solve or improve the above problems through creative work. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventors after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present invention for the above problems below should all be the contributions made by the inventors to the present invention in the process of invention and creation, and should not be understood as the technical contents known to those skilled in the art.

[0083] In view of this, the present embodiment provides a training method applied to electronic devices. In the method, the electronic device obtains the chapter contents and teacher-student dialogue texts of multiple chapters in the textbook, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter; the chapter contents and teacher-student dialogue text of each chapter are respectively converted into a set of training samples, wherein each set of training samples includes an input coding sequence and a target sequence, the input coding sequence includes the teacher-student dialogue text of the corresponding chapter and the coding sequence of the chapter content, and the target sequence includes the coding sequence of the teacher dialogue content in the teacher-student dialogue text of the corresponding chapter; the input coding sequence and the target sequence in each set of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain a listening and speaking practice model. In this way, the listening and speaking practice model enables students to perform listening and speaking exercises synchronously with the text during the course learning process.

[0084] It should be noted that the electronic device implementing the above training method may be, but is not limited to, a mobile terminal, a tablet computer, a laptop computer, a desktop computer, and a server that meets the computing power requirements. When it is a server, the server may be a single server or a server group. The server group may be centralized or distributed (for example, the server may be a distributed system). In some embodiments, the server may be local or remote relative to the user terminal. In some embodiments, the server may be implemented on a cloud platform; as an example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or any combination thereof. In some embodiments, the server may be implemented on an electronic device having one or more components.

[0085] To make the solution provided by this embodiment clearer, the following takes the server as an example and combines Figure 1 Each step of the method is described in detail. However, it should be understood that the operations of the flowchart may not be implemented in order, and steps without logical contextual relationships may be reversed in order or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart, or may remove one or more operations from the flowchart under the guidance of the content of the present invention. Figure 2 As shown, the method includes:

[0086] S1, obtains the chapter contents of multiple chapters in the textbook and the text of teacher-student dialogues.

[0087] The teacher-student dialogue text of each chapter matches the teaching requirements of this chapter. It should be noted that the textbooks in this embodiment are not limited to English textbooks, but can also be textbooks for learning foreign languages ​​such as Korean, Japanese, and German. In addition, the number of the above textbooks can be multiple, for example, these textbooks can come from different learning stages. The teacher-student dialogue text of each chapter can be obtained by inviting teachers and students to conduct dialogue exercises based on the chapter content.

[0088] For example, let's take the People's Education Press English textbook as an example. First, the textbook is organized, mainly the words, phrases, sentence patterns, grammar and teaching objectives to be learned in each chapter. After obtaining these contents, teachers and students are invited to conduct dialogue exercises based on the textbook content for each chapter, and these dialogues are recorded in text form and then associated with the textbook content. For example, the chapter content of a chapter in the People's Education Press English textbook may include:

[0089] {

[0090] "book": {

[0091] "words": ["textbook", "conversation", "aloud", "pronunciation", "sentence", "patient", "expression", "discover", "secret", "grammar", "repeat", "note", "pal", "pattern", "physics", "chemistry", "partner", "pronounce", "increase", "speed", "ability", "brain", "active", "attention", "connect", "review", "knowledge"],

[0092] “phrases”: [“fall in love with”, “pay attention to”, “connect with”, “overnight”, “wisely”, “ask for”, “give a report”],

[0093] “sentences”: [“—How do you learn English?

[0094] —I learn by studying with a group. I learn by working in a group. ”, “—Do you learn English by reading aloud?

[0095] —Yes,I do.It helps my pronunciation. Yes,I do.It helps my pronunciation. ”],

[0096] "grammar": "Master the special questions introduced by "how", their answers and the usage of the present perfect tense",

[0097] “object”: “Be able to understand and learn to talk about daily study methods; be able to read articles describing study methods; and be able to write a short article that simply introduces study methods or gives reasonable suggestions to others for difficulties in learning.”

[0098] }

[0099] The teacher-student dialogue text for this chapter can include:

[0100] "conversation": [

[0101] [“teacher: Hi there! How do you learn English?”, “student: take English class”],

[0102] [“teacher: OK, but let’s answer the question in complete sentences. You could say: \“I learnEnglish by attending English classes.\” Try this answer.”, “student: I learnEnglish by attending English classes.”] ]

[0104] Based on the above embodiment, the chapter content and the teacher-student dialogue text are further referred to. Figure 1 , the training method provided in this embodiment also includes:

[0105] S2, converts the chapter content of each chapter and the teacher-student dialogue text into a set of training samples.

[0106] Each set of training samples includes an input coding sequence and a target sequence, wherein the input coding sequence includes the coding sequence of the teacher-student dialogue text of the corresponding chapter and the chapter content, and the target sequence includes the coding sequence of the teacher's dialogue content in the teacher-student dialogue text of the corresponding chapter. In this embodiment, in order to enable the model to be trained to learn how to generate dialogue content according to the teaching requirements of the chapter from the chapter content and the dialogue text, the coding sequence of the teacher-student dialogue text and the chapter content is organized in a specific manner. Therefore, this embodiment also provides the following optional implementation of step S2:

[0107] S2-1, for each chapter content and teacher-student dialogue text, convert the chapter content into a chapter coding sequence.

[0108] S2-2, converting the teacher-student dialogue text into a teacher-student dialogue coding sequence, and appending it to the chapter coding sequence according to the dialogue order to obtain an input coding sequence.

[0109] It should be understood that the model to be trained in this embodiment is a model based on the principle of neural network, which cannot directly process the chapter content and the teacher-student dialogue text, but needs to convert it into a digitally encoded form. In this regard, the currently mature word embedding tool can be used to process the chapter content and the teacher-student dialogue text.

[0110] For example, let's continue to take the content of a chapter in the above-mentioned People's Education Press English textbook as an example. After adjusting its format, the content of the chapter can be expressed as:

[0111] The unit contents are as follows:

[0112] Words: ["textbook", "conversation", "aloud", "pronunciation", "sentence", "patient", "expression", "discover", "secret", "grammar", "repeat", "note", "pal", "pattern", "physics", "chemistry", "partner", "pronounce", "increase", "speed", "ability", "brain", "active", "attention", "connect", "review", "knowledge"] Phrases: ["fall in love with", "pay attentionto", "connect with", "overnight", "wisely", "ask for", "give a report"],

[0113] Sentence pattern: [“—How do you learn English? How do you learn English?

[0114] —I learn by studying with a group. I learn by working in a group. ”, “—Do you learn English by reading aloud?

[0115] —Yes,I do.It helps my pronunciation. Yes,I do.It helps my pronunciation. ”],

[0116] Grammar: "Master the special interrogative sentences introduced by "how", their answers and the usage of the present perfect tense",

[0117] Objective: "Be able to understand and talk about daily study methods; be able to read articles describing study methods; and be able to write a short article that simply introduces study methods or gives reasonable suggestions to others for difficulties in learning."

[0118] After the above chapter contents are encoded by Qwen1.5 word segmenter, the chapter encoding sequence is as follows Figure 2 Similarly, the server converts the teacher-student dialogue text into a teacher-student dialogue coding sequence, and appends it to the chapter coding sequence according to the dialogue order to obtain an input coding sequence.

[0119] For example, let's continue to take the teacher-student dialogue text in the above example. For the dialogue text:

[0120] Teacher: "Hi there! How do you learn English?"

[0121] Student: “Taking English class.”

[0122] The teacher's dialogue content in the above dialogue text is encoded by Qwen1.5 word segmenter and is encoded as:

[0123] [198, 101049, 5122, 13048, 1052, 0, 2585, 653, 498, 3960, 6364, 30].

[0124] Then, append the coding sequence of the teacher's dialogue content to Figure 2 In the chapter coding sequence shown in Figure 3 The processing method of other dialogue contents in the teacher-student dialogue text is the same as that of the above dialogue text, until the coding sequence of all the teacher-student dialogue texts is appended to Figure 2 After the chapter code sequence shown is obtained, the input code sequence is obtained. This embodiment will not be described in detail.

[0125] In combination with the input coding sequence obtained in the above embodiment, an optional implementation of step S2 further includes:

[0126] S2-3, by ignoring the coding and replacing the coding of the student dialogue content in the teacher-student dialogue coding sequence, an optimized teacher-student dialogue coding sequence is obtained.

[0127] Among them, ignoring the encoding means that there is no need to calculate the encoding of the model loss.

[0128] S2-4, obtain the target sequence based on the optimized teacher-student dialogue encoding sequence.

[0129] In this regard, it can be understood that only the coding of the teacher's dialogue content is retained in the optimized teacher-student dialogue coding sequence, and the coding of the student's dialogue content is replaced with the ignore code. Therefore, in some embodiments, the optimized teacher-student dialogue coding sequence can be directly used as the target sequence. In some other embodiments, in order to keep the target sequence and the input coding sequence with the same sequence length, the server can also replace all the codes in the chapter coding sequence with the ignore code to obtain an auxiliary sequence; the optimized teacher-student dialogue coding sequence is appended to the auxiliary sequence to obtain the target sequence.

[0130] It should be noted that, for the above-mentioned ignore code, any value outside the code range output by Qwen1.5 can be selected as the ignore code. For example, assuming that the value -100 is used as the ignore code, Figure 2 Replace the code in the code sequence of the chapter shown in the figure to obtain the auxiliary sequence. In this way, during the training process, there is no need to calculate the model loss with a code value of -100. Continuing with the code sequence of the teacher's dialogue content in the above dialogue text [198, 101049, 5122, 13048,1052, 0, 2585, 653, 498, 3960, 6364, 30] as an example, append it to the auxiliary sequence to obtain the following Figure 4 The coding sequence shown. The student dialogue content in the teacher-student dialogue text is replaced by -100. After all the optimized teacher-student dialogue coding sequences are appended to the auxiliary sequence, the target sequence can be obtained.

[0131] This embodiment uses the input coding sequence and the corresponding target sequence obtained in each of the above sections as a set of training samples for training the model to be trained. Figure 1 Based on the above training samples, the training method provided in this embodiment also includes:

[0132] S3, using the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain the listening and speaking practice model.

[0133] In this regard, it was found in practice that the training of the model to be trained is not a simple proportional relationship between the number of iterations and the performance improvement. In fact, when the model is trained, its performance will gradually improve with the increase in the number of iterations until it reaches an optimal performance. The number of iterations at this time is the optimal number of iterations. However, if the training process exceeds this optimal number of iterations, it will be found that the performance of the model not only no longer improves, but may show a downward trend. The reason behind this phenomenon may be related to the overfitting of the model to the training data, that is, in the early stages of training, the model improves performance by learning the rules in the training data. However, when the number of iterations is too many, the model may pay too much attention to the details in the training data and cannot generalize well to new, unseen data, resulting in performance degradation. Based on the above findings, this embodiment also provides the following optional implementation of step S3:

[0134] S3-1, using the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and output data of the model to be trained respectively, iterating the parameters of the model to be trained, and retaining multiple groups of model parameters during the iteration process.

[0135] Since there are many mature natural language processing models, the model to be trained can select these mature natural language processing models for fine-tuning. Figure 5 The model to be trained is shown, which includes an embedding network, an encoding network, and a language network.

[0136] The embedding network includes a word embedding layer and a position embedding layer, which are used to receive input encoding sequences and position encoding sequences, and output embedding representation feature sequences.

[0137] The encoding network includes multiple Transformer-derived layers. It should be noted that Transformer is a deep learning model based on the self-attention mechanism, which has achieved remarkable results in fields such as natural language processing and computer vision. Unlike traditional recurrent neural networks (RNNs) and convolutional neural networks (CNNs), the Transformer model uses a self-attention mechanism to capture long-distance dependencies in sequences, while introducing techniques such as multi-head attention mechanisms and position encoding, so that it has higher efficiency and accuracy when processing long sequence data. The so-called Transformer-derived layer means that while retaining the Transformer structure, additional network layers are introduced according to the needs of the training task.

[0138] like Figure 6As shown, each Transformer derivative layer includes a Transformer layer and a bypass network parallel to the Transformer layer, and the bypass network includes multiple bypass matrices. The first bypass matrix receives the same input as the attention matrix in the Transformer layer, and the output features of the two are added and normalized; then, the added and normalized features are input into the second bypass matrix and the feedforward matrix in the Transformer layer, and then the output features of the two are further added and normalized. In this embodiment, the hyperparameter of the bypass network can select 256 as the dimension of the intermediate hidden layer, that is, the bypass network is a low-rank autoencoder network of 4096-256-4096.

[0139] The language network is composed of a linear network and a softmax activation function. It receives the hidden feature representation sequence output by the encoding network and outputs a predicted coding sequence.

[0140] Based on the introduction of the model to be trained in the above embodiment, for each group of training samples, the server inputs the input coding sequence in the training sample into the embedding network to obtain the embedding representation feature sequence; inputs the embedding representation feature sequence into the encoding network to obtain the encoding feature; inputs the encoding feature into the language network to obtain the predicted coding sequence; obtains the model loss of the model to be trained based on the predicted coding sequence and the target sequence in the training sample; updates the bypass network of each Transformer derivative layer and the parameters in the language network based on the model loss. As an optional implementation method, in order to keep the number of inputs of the training samples to be trained balanced, the training samples can be sampled without replacement.

[0141] It can be understood that in the process of model training, this embodiment only updates the parameters in the bypass network and the language network, while keeping the parameters of other parts of the model unchanged. The parameters of other parts of the model to be trained are derived from the weights of the pre-trained large language model. In a specific implementation, the server can obtain the weights of the pre-trained large language model; initialize at least a part of the embedding network, the encoding network, and the language network through the weights of the pre-trained large language model.

[0142] Exemplarily, the open source model Qwen-7B-Chat can be selected, and the weights of the open source model can be used as the initial weights of the embedded network and the non-bypass network part in the encoding network in the model to be trained, and during the model training process, the non-bypass part of the embedded network and the encoding network in the model to be trained is frozen.

[0143] Through the parameter updating method introduced in the embodiment, multiple groups of model parameters can be obtained. Based on these model parameters, step S3 also includes:

[0144] S3-2, obtaining multiple candidate models based on multiple groups of model parameters.

[0145] S3-3, test multiple candidate models using the validation set, and select the one with the best test results as the listening and speaking practice model.

[0146] Exemplarily, it is assumed that the training parameters of the model to be trained are as follows:

[0147] Optimizer: Adam optimizer with weight decay, learning rate of 1e-5, weight decay rate of 1e-3;

[0148] Batch size: 8

[0149] Learning rate scheduling: linear decay scheduler with preheating, 100 preheating steps, and a decay rate of 0.1;

[0150] Number of training steps: 1200;

[0151] Weight preservation and verification steps: every 300 steps.

[0152] In this case, 4 sets of model parameters can be retained during the entire iteration process, which means that 4 candidate models can be obtained. Then, the performance of these 4 candidate models is screened through the validation set, and the one with the best test result is selected as the final listening and speaking practice model.

[0153] Based on the listening and speaking practice model obtained in the above embodiment, this embodiment also provides a listening and speaking practice method. In the method, the electronic device obtains the historical conversation text between the teacher and the student; obtains the code sequence to be processed according to the historical conversation text between the teacher and the student, wherein the code sequence to be processed includes the code sequence of the historical conversation text between the teacher and the student and the code sequence of the chapter content in the chapter to be practiced; inputs the code sequence to be processed into the listening and speaking practice model to obtain the predicted code sequence corresponding to the historical conversation text between the teacher and the student; converts the predicted code sequence of the historical conversation text between the teacher and the student into the predicted text, and plays the predicted text in the form of voice.

[0154] For example, it is assumed that the chapter contents in the chapter to be practiced include the following:

[0155] {

[0156] "words": ["textbook", "conversation", "aloud", "pronunciation", "sentence", "patient", "expression", "discover", "secret", "grammar", "repeat", "note", "pal", "pattern", "physics", "chemistry", "partner", "pronounce", "increase", "speed", "ability", "brain", "active", "attention", "connect", "review", "knowledge"],

[0157] “phrases”: [“fall in love with”, “pay attention to”, “connect with”, “overnight”, “wisely”, “ask for”, “give a report”],

[0158] “sentences”: [“—How do you learn English?

[0159] —I learn by studying with a group. I learn by working in a group. ”, “—Do you learn English by reading aloud?

[0160] —Yes,I do.It helps my pronunciation. Yes,I do.It helps my pronunciation. ”],

[0161] "grammar": "Master the special questions introduced by "how", their answers and the usage of the present perfect tense",

[0162] “object”: “Be able to understand and learn to talk about daily study methods; be able to read articles describing study methods; and be able to write a short article that simply introduces study methods or gives reasonable suggestions to others for difficulties in learning.”

[0163] }

[0164] The current historical dialogue text between teachers and students is:

[0165] [Teacher: Hi there! How do you learn English?

[0166] Student: English class]

[0167] At this time, the electronic device obtains the coding sequence of the historical dialogue text between the teacher and the student and the coding sequence of the chapter content in the chapter to be practiced, and inputs them into the listening and speaking practice model as the coding sequence to be processed. The model outputs the predicted coding sequence corresponding to the current historical dialogue text between the teacher and the student:

[0168] [198, 101049, 5122, 562, 11, 498, 646, 1977, 40727, 40, 3960, 6364,553, 23218, 6364, 6846, 10465, 1430, 432]

[0169] Then, the above predicted coding sequence is decoded into the following reply text by the word segmenter:

[0170] "Teacher: ok, you can say: \"I learn English by attending English classes.\", try it".

[0171] Next, the reply text "ok, you can say: \"I learn English by attending English classes.\", try it" is transcribed into speech through TTS and played. After hearing the speech, the student generates a voice reply and transcribes it into text through ASR as user input to the listening and speaking practice model for the next round of dialogue.

[0172] Based on the same inventive concept as the training method provided in this embodiment, this embodiment also provides a training device, which includes at least one software function module that can be stored in a memory or fixed in an electronic device in the form of software. The processor in the electronic device is used to execute the executable module stored in the memory. For example, the software function module and computer program included in the device. Please refer to Figure 7 , functionally speaking, the device may include:

[0173] The text acquisition module 11 is used to acquire the chapter contents of multiple chapters in the textbook and the teacher-student dialogue texts, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter;

[0174] The sample encoding module 12 is used to convert the chapter content and the teacher-student dialogue text of each chapter into a set of training samples, wherein each set of training samples includes an input encoding sequence and a target sequence, the input encoding sequence includes the teacher-student dialogue text of the corresponding chapter and the encoding sequence of the chapter content, and the target sequence includes the encoding sequence of the teacher's dialogue content in the teacher-student dialogue text of the corresponding chapter;

[0175] The model training module 13 is used to train the model to be trained by using the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and output data of the model to be trained, so as to obtain a listening and speaking practice model.

[0176] In this implementation, the text acquisition module 11 is used to implement Figure 1 In step S1, the sample encoding module 12 is used to implement Figure 1 In step S2, the model training module 13 is used to implement Figure 1 Therefore, for the detailed description of the above modules, please refer to the specific implementation of the corresponding steps. In addition, in view of the same inventive concept as the training method, the above modules can also be used to implement other steps or sub-steps of the method, which will not be described in detail in this embodiment.

[0177] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0178] It should also be understood that if the above embodiments are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0179] Therefore, this embodiment further provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the training method or the listening and speaking practice method provided in this embodiment is implemented. Among them, the storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, etc., which can store program codes.

[0180] This embodiment provides an electronic device. Figure 8 As shown, the electronic device may include a processor 22 and a memory 21. In addition, the memory 21 stores a computer program, and the processor implements the training method or listening and speaking practice method provided in this embodiment by reading and executing the computer program corresponding to the above implementation in the memory 21.

[0181] Continue to see Figure 8 The electronic device further includes a communication unit 23. The memory 21, the processor 22 and the communication unit 23 are electrically connected to each other directly or indirectly through a system bus 24 to achieve data transmission or interaction.

[0182] The memory 21 may be an information recording device based on any electronic, magnetic, optical or other physical principle, used to record execution instructions, data, etc. In some embodiments, the memory 21 may be, but is not limited to, a volatile memory, a non-volatile memory, a storage drive, etc.

[0183] In some embodiments, the volatile memory may be a random access memory (RAM); in some embodiments, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a flash memory, etc.; in some embodiments, the storage drive may be a disk drive, a solid-state drive, any type of storage disk (such as a CD, a DVD, etc.), or a similar storage medium, or a combination thereof, etc.

[0184] The communication unit 23 is used to send and receive data through a network. In some embodiments, the network may include a wired network, a wireless network, a fiber optic network, a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof. In some embodiments, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station and / or a network switching node, and one or more components of the service request processing system may be connected to the network through the access point to exchange data and / or information.

[0185] The processor 22 may be an integrated circuit chip with signal processing capability, and the processor may include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), or a microprocessor, or any combination thereof.

[0186] Understandably, Figure 8The structure shown is for illustration only. The electronic device may also have Figure 8 More or fewer components than shown, or with Figure 8 Different configurations shown. Figure 8 The components shown may be implemented in hardware, software or a combination thereof.

[0187] It should be understood that the apparatus and method disclosed in the above-mentioned embodiments can also be implemented in other ways. The apparatus embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0188] The above are only various embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A training method, characterized in that: The method comprises: Obtain the chapter contents and teacher-student dialogue texts of multiple chapters in the textbook, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter; The chapter content of each chapter and the teacher-student dialogue text are converted into a set of training samples, where each set of training samples includes an input coding sequence and a target sequence; this step specifically includes: For each chapter content and the teacher-student dialogue text, convert the chapter content into a chapter coding sequence; Converting the teacher-student dialogue text into a teacher-student dialogue code sequence, and appending it to the chapter code sequence in the order of dialogue to obtain the input code sequence; Replacing the coding of the student dialogue content in the teacher-student dialogue coding sequence by ignoring coding, thereby obtaining an optimized teacher-student dialogue coding sequence, wherein the ignoring coding represents coding without calculating model loss; According to the optimized teacher-student dialogue coding sequence, the target sequence is obtained; The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and output data of the model to be trained, respectively, to train the model to be trained, and obtain the listening and speaking practice model.

2. The training method according to claim 1, characterized in that: According to the optimized teacher-student dialogue coding sequence, the target sequence is obtained, including: The auxiliary sequence is obtained by replacing all the codes in the chapter code sequence by ignoring the codes; The optimized teacher-student dialogue coding sequence is appended to the auxiliary sequence to obtain the target sequence.

3. The training method according to claim 1, characterized in that: The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and the output data of the model to be trained, respectively, to train the model to be trained, and obtain the listening and speaking practice model, including: The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and the output data of the model to be trained respectively to iterate the parameters of the model to be trained, and multiple groups of model parameters in the iterative process are retained; According to the multiple groups of model parameters, multiple candidate models are obtained; The multiple candidate models are tested using a validation set, and the one with the best test result is selected as the listening and speaking practice model.

4. The training method according to claim 3, characterized in that: The model to be trained includes an embedding network, an encoding network and a language network, the encoding network includes a plurality of Transformer derivative layers, each Transformer derivative layer includes a Transformer layer and a bypass network parallel to the Transformer layer; The input coding sequence and the target sequence in each group of training samples are used as the supervision information of the input data and the output data of the model to be trained respectively to iterate the parameters of the model to be trained, including: For each group of training samples, inputting the input coding sequence in the training samples into the embedding network to obtain an embedding representation feature sequence; Inputting the embedding representation feature sequence into the encoding network to obtain encoding features; Inputting the coding features into the language network to obtain a predicted coding sequence; Obtaining a model loss of the model to be trained according to the predicted coding sequence and the target sequence in the training sample; The bypass network of each Transformer derivative layer and the parameters in the language network are updated according to the model loss.

5. The training method according to claim 4, characterized in that: Before inputting the input coding sequence in the training sample into the embedding network to obtain the embedding representation feature sequence, the method further includes: Get the weights of a pre-trained large language model; At least a portion of the embedding network, the encoding network, and the language network are initialized by weights of the pre-trained large language model.

6. A listening and speaking practice method, characterized in that: The method comprises: Obtain the text of historical dialogues between teachers and students; According to the historical dialogue text between the teacher and the students, a code sequence to be processed is obtained, wherein the code sequence to be processed includes the code sequence of the historical dialogue text between the teacher and the students and the code sequence of the chapter content in the chapter to be practiced; Inputting the to-be-processed coding sequence into the listening and speaking practice model obtained by the training method according to any one of claims 1 to 5 to obtain a predicted coding sequence corresponding to the teacher-student historical dialogue text; The predictive coding sequence of the teacher-student historical dialogue text is converted into a predictive text, and the predictive text is played in a voice manner.

7. A training device, characterized in that: The device comprises: The text acquisition module is used to acquire the chapter contents of multiple chapters in the textbook and the teacher-student dialogue texts, wherein the teacher-student dialogue text of each chapter matches the teaching requirements of the chapter; The sample encoding module is used to convert the chapter content of each chapter and the teacher-student dialogue text into a set of training samples, wherein each set of training samples includes an input coding sequence and a target sequence; the sample encoding module is also specifically used to: For each chapter content and the teacher-student dialogue text, convert the chapter content into a chapter coding sequence; Converting the teacher-student dialogue text into a teacher-student dialogue code sequence, and appending it to the chapter code sequence in the order of dialogue to obtain the input code sequence; Replacing the coding of the student dialogue content in the teacher-student dialogue coding sequence by ignoring coding, thereby obtaining an optimized teacher-student dialogue coding sequence, wherein the ignoring coding represents coding without calculating model loss; According to the optimized teacher-student dialogue coding sequence, the target sequence is obtained; The model training module is used to train the model to be trained by taking the input coding sequence and the target sequence in each group of training samples as the supervision information of the input data and output data of the model to be trained, so as to obtain the listening and speaking practice model.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the training method according to any one of claims 1 to 5 or the listening and speaking practice method according to claim 6 is implemented.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the training method according to any one of claims 1 to 5 or the listening and speaking practice method according to claim 6 is implemented.