Oral language evaluation method, oral language evaluation model training method and related device

Abstract

The spoken language evaluation method is applied to a server and comprises the following steps: acquiring basic information, to-be-evaluated voice and a historical evaluation result of a user; obtaining a voice score based on the to-be-evaluated voice; a text score is obtained based on a to-be-evaluated text, and the to-be-evaluated text is obtained according to the to-be-evaluated voice; based on the voice score and the text score, obtaining a current evaluation result; and inputting the basic information, the historical evaluation result and the current evaluation result into a spoken language evaluation model to output spoken language evaluation information of the to-be-evaluated speech, the spoken language evaluation information representing evaluation results of the to-be-evaluated speech in multiple dimensions. According to the spoken language evaluation method and device, the spoken language evaluation information is a multi-dimensional comprehensive result, and personalized feedback and targeted guidance can be carried out on the user. The spoken language evaluation information is not limited to the style of scores any more, follow-up promotion suggestions and guidance can be given like teachers, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a spoken language assessment method, a training method for a spoken language assessment model, and related devices. Background Art

[0002] With the acceleration of globalization, language learning has become a trend and plays an important role in daily life, academics, work, etc. Therefore, a technical solution for oral assessment is needed to help users improve their oral skills. Summary of the invention

[0003] In order to solve the above problems, the embodiments of the present application provide a spoken language assessment method, a training method of a spoken language assessment model and related devices, which, combined with a large language model and the user's basic information and historical assessment results, can provide users with multi-dimensional spoken language assessment information.

[0004] In a first aspect, an embodiment of the present application provides a spoken language evaluation method, which is applied to a server, and includes: obtaining basic information of a user, a speech to be evaluated, and historical evaluation results; obtaining a speech score based on the speech to be evaluated, wherein the speech score represents the speech score of the speech to be evaluated; obtaining a text score based on the text to be evaluated, and obtaining a current evaluation result based on the speech score and the text score, wherein the text to be evaluated is obtained based on the text information in the speech to be evaluated, and the text score represents the score of the text accuracy of the speech to be evaluated; inputting the basic information, the historical evaluation results, and the current evaluation results into a spoken language evaluation model to output spoken language evaluation information of the speech to be evaluated, wherein the spoken language evaluation information represents the evaluation results of the speech to be evaluated in multiple dimensions.

[0005] The technical solution of the embodiment of the present application can obtain oral evaluation information based on the user's speech to be evaluated and historical data. The oral evaluation information is a comprehensive result of evaluating the user's spoken speech in multiple dimensions. The oral evaluation information can provide personalized feedback and targeted guidance to the user, so that the user can improve his oral skills more efficiently. The oral evaluation information can represent the user's oral level from multiple dimensions, thereby improving the accuracy of the oral evaluation. The oral evaluation model is a large language model. Based on the generative ability unique to the large language model, the oral evaluation information is no longer limited to the style of scores, but automatically obtains multi-dimensional oral evaluation information based on the input data. For example, subsequent improvement suggestions and guidance can be given like a teacher, which improves the interactive effect of the evaluation effect and improves the user experience. The technical solution provided in the embodiment of the present application is characterized by advancement, comprehensiveness, personalization and interactivity.

[0006] In one possible implementation, the oral assessment information includes at least one of an assessment report and learning suggestions. The assessment report represents changes in the user's oral ability from multiple dimensions, and the multiple dimensions include voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy, and the learning suggestions represent suggestions for the user on oral learning.

[0007] In this implementation, the oral evaluation information includes a multi-dimensional evaluation report and subsequent learning suggestions. The multi-dimensional evaluation report shows the advantages and disadvantages of the user's speech to be evaluated in multiple dimensions, so as to help the user correct the shortcomings and improve the oral ability. The subsequent learning suggestions are a comprehensive evaluation of the user's historical performance and current performance in combination with the user's historical data, so as to help the user formulate a learning plan and improve the oral ability in an orderly manner. The evaluation report shows the changes in the user's oral ability from multiple dimensions, reflecting the comprehensiveness of the oral evaluation information; since the evaluation report sub-tags are multi-dimensional, the learning suggestions can provide users with more detailed suggestions from multiple dimensions, reflecting the personalization of the oral evaluation information.

[0008] In a possible implementation, the above-mentioned obtaining a text score based on the text to be evaluated includes: inputting the text to be evaluated into a spoken language evaluation model to output a text score.

[0009] In this implementation, another use of the spoken language evaluation model is provided. The spoken language evaluation model is a large language model with comprehensive analysis functions, which can complete multiple tasks at the same time. For example, the spoken language evaluation model can form a user's evaluation report and learning suggestions, and can also analyze the text accuracy score of the speech to be evaluated.

[0010] In one possible implementation, the method of obtaining a speech score based on the speech to be evaluated includes: extracting features of the speech to be evaluated to obtain acoustic features of the speech to be evaluated, wherein the acoustic features represent features of the speech to be evaluated in an acoustic dimension, wherein the acoustic dimension includes at least one of sound quality and rhythm; and inputting the acoustic features into a speech evaluation model to output a speech score.

[0011] In this implementation, the speech evaluation model is a neural network model with good learning ability. It can analyze the user's speech performance from multiple dimensions such as pronunciation, vocabulary, fluency, speed, etc., and output a speech score. Compared with the speech to be evaluated, the acoustic features can better express the information of the acoustic dimension, which is convenient for the speech evaluation model to analyze, and can effectively reduce the parameters of the speech evaluation model, while the output results are more accurate.

[0012] In a possible implementation, the evaluation result includes at least one of a voice score, a text score, and a comprehensive score, and the comprehensive score is obtained by weighting the voice score and the text score.

[0013] In this implementation, the evaluation result includes at least one of a voice score, a text score, and a comprehensive score. The comprehensive score can be understood as the result of combining voice and text, so the user's oral ability can be analyzed from a more comprehensive perspective.

[0014] In a possible implementation, the speech to be evaluated is obtained based on the user completing the test questions to be evaluated; the method also includes: determining the user's oral ability level based on the user's basic information and historical evaluation results; determining the difficulty of the questions corresponding to the user based on the oral ability level; and determining the test questions based on the difficulty of the questions.

[0015] In this implementation, a method for determining the speech to be evaluated is provided. Based on the user's basic information and historical evaluation results, the topic to be evaluated is determined, so that the topic to be evaluated comprehensively considers the user's personal characteristics and historical performance, and has the characteristics of high personalization and strong pertinence, so that the oral evaluation information can provide personalized feedback and targeted guidance to the user. In addition, according to the oral ability level, the difficulty of the question corresponding to the user is determined, and a variety of questions to be predicted can be set according to the user's needs, which improves the user experience. For example, the level of the student and the level of the difficulty of the question are divided into K levels; if the user focuses on consolidating the current oral ability while improving the oral ability, assuming that the current student's level is level i, then the question difficulty is determined to be level i-1 and level i; if the user likes challenging questions, assuming that the current student's level is level i, then the question difficulty is determined to be level i and level i+1.

[0016] In one possible implementation, the user's oral ability level is determined based on the user's basic information and historical evaluation results, including: adding the user's basic information and historical evaluation results to a first prompt word template to obtain first prompt information, the first prompt word template is used to guide the generation process of the oral evaluation model; the first prompt information is input into the oral evaluation model to output the user's oral ability level.

[0017] In this implementation, the first prompt word can enable the oral evaluation model to more accurately understand the basic information in the input content and the historical evaluation results, perform the task more accurately, and thus obtain a more accurate oral ability level.

[0018] In a possible implementation, the test questions are determined according to the difficulty of the questions, including: generating the test questions by a spoken language assessment model according to the difficulty of the questions; or matching the test questions from an exercise database according to the difficulty of the questions.

[0019] In this implementation, another use of the oral evaluation model is provided: generating test questions. That is, the learning ability of the large language model can be utilized to generate more accurate test questions using the oral evaluation model. Alternatively, the test questions can be matched from the exercise database according to the difficulty of the questions. Exemplarily, when the number of exercises in the exercise data set is small, it may not meet the use requirements, and the oral evaluation model can be used to generate the test questions; when the number of exercises in the exercise data set is large, the exercise database can meet the use requirements, and the test questions can be matched from the exercise database according to the difficulty of the questions.

[0020] In one possible implementation, the oral evaluation model generates a question to be evaluated based on the difficulty of the question, including: adding the difficulty of the question to a second prompt word template to obtain second prompt information, the second prompt word template is used to guide the generation process of the oral evaluation model; and inputting the second prompt information into the oral evaluation model to output the question to be evaluated.

[0021] In this implementation, the second prompt word template can enable the oral evaluation model to more accurately understand the difficulty of the questions in the input content, perform the task more accurately, and thus obtain more accurate questions to be evaluated.

[0022] In a possible implementation, the method further includes: based on the topic to be evaluated and in combination with the digital human, in the process of interacting with the user, providing the topic to be evaluated to the user and acquiring the user's voice to be evaluated.

[0023] In this implementation, the digital human can simulate a real English teacher and have natural oral interactions with users, improving the interactive experience of the oral assessment process.

[0024] In a possible implementation, in the process of interacting with the user, the digital human is combined to display the topic to be evaluated to the user, and the user's voice to be evaluated is obtained, including: adding the topic to be evaluated in the third prompt word template to obtain the third prompt information, and the third prompt word template is used to guide the generation process of the oral evaluation model; inputting the third prompt information into the oral evaluation model to output the live stream information of the topic to be evaluated; through the digital human, the live stream information is interacted with the user, and the user's voice to be evaluated is obtained.

[0025] In this implementation, the third prompt word can enable the oral evaluation model to more accurately understand the to-be-evaluated topic in the input content, perform the task more accurately, and thus obtain more accurate live streaming information.

[0026] In one possible implementation, basic information, historical evaluation results, and current evaluation results are input into an oral evaluation model to output oral evaluation information of the speech to be evaluated, including: in a fourth prompt word template, basic information, historical evaluation results, and current evaluation results are input to obtain fourth prompt information, and the fourth prompt word template is used to guide the generation process of the oral evaluation model; the fourth prompt information is input into the oral evaluation model to output the oral evaluation information of the speech to be evaluated.

[0027] In this implementation, the fourth prompt word can enable the oral evaluation model to more accurately understand the basic information in the input content, historical evaluation results, and current evaluation results, and perform tasks more accurately, thereby obtaining more accurate oral evaluation information.

[0028] In a second aspect, an embodiment of the present application provides a training method for a spoken language evaluation model, comprising: constructing a spoken language evaluation data set, the spoken language evaluation data set comprising a user's spoken language evaluation sample and a spoken language evaluation information label, the spoken language evaluation sample comprising the user's basic information and several evaluation results thereof, the spoken language evaluation information label representing the evaluation results of the spoken language evaluation sample in multiple dimensions, the multiple dimensions comprising a speech dimension and / or a text dimension, the speech dimension comprising at least one of pronunciation, vocabulary, fluency, and speed, and the text speech dimension comprising at least one of answer accuracy and word accuracy; taking the user's basic information and several evaluation results thereof as input of the spoken language evaluation model to be trained, and outputting the user's spoken language evaluation information prediction result; obtaining an evaluation loss value according to the spoken language evaluation information label and the spoken language evaluation information prediction result; adjusting the parameters of the spoken language evaluation model to be trained according to the evaluation loss value to obtain a trained spoken language evaluation model.

[0029] In the embodiment of the present application, a training method for a spoken language evaluation model is provided. The spoken language evaluation data set includes a user's spoken language evaluation sample and a spoken language evaluation information label. The spoken language evaluation sample includes the user's basic information and several evaluation results. Therefore, the trained spoken language evaluation model can generate multi-dimensional spoken language evaluation information, such as analyzing the changes in the user's spoken language ability based on several evaluation results, and making targeted suggestions.

[0030] In a possible implementation, the oral evaluation model to be trained is obtained by adding a low-rank matrix layer to the output layer of several modules in the pre-trained large model, and the several modules are neural network modules in the pre-trained large model for realizing specific functions. According to the evaluation loss value, the parameters of the oral evaluation model to be trained are adjusted to obtain the trained oral evaluation model, including: according to the evaluation loss value, the parameters of the low-rank matrix layer are adjusted to obtain the trained oral evaluation model.

[0031] In this implementation, the pre-trained large model can be fine-tuned to obtain a spoken language evaluation model, so that the learning and analysis capabilities of the pre-trained large model can be utilized, and the spoken language evaluation model that meets the application scenario of the embodiment of the present application can be fine-tuned. For example, adding a LoRA (Low-Rank Adaptation, low-rank matrix) layer to the pre-trained large model and adjusting the LoRA layer can reduce computing resource consumption while maintaining good performance improvement.

[0032] In one possible implementation, constructing a spoken language evaluation data set includes: designing a seed data set, the seed data set includes evaluation sample examples and their corresponding evaluation information examples, the seed data set is an example of generating a spoken language evaluation data set, the evaluation sample examples are examples of generating spoken language evaluation samples, and the evaluation information samples are examples of generating spoken language evaluation information labels; inputting the seed data set as a sample generation model, so that the sample generation model generates spoken language evaluation samples according to the evaluation sample examples, and generates spoken language evaluation information labels according to the evaluation information samples; and obtaining a spoken language evaluation data set according to the spoken language evaluation samples and their corresponding spoken language evaluation information labels.

[0033] In this implementation, the method of generating a training data set based on a seed data set saves the time of sample collection, improves the efficiency of constructing a spoken language evaluation data set, increases the diversity of the spoken language evaluation data set, saves development costs, and solves the problem of fewer data samples and difficulty in obtaining samples. Among them, the sample generation model can be any large language model, and the embodiments of this application do not limit this.

[0034] In one possible implementation, the oral assessment information tag includes at least one of an assessment report sub-tag and a learning suggestion sub-tag, the assessment report sub-tag represents the changes in the user's oral ability from multiple dimensions, the multiple dimensions including voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy; the learning suggestion sub-tag represents suggestions for the user on oral learning.

[0035] In this implementation, the oral evaluation information tag includes at least one of an evaluation report subtag and a learning suggestion subtag, and the multiple dimensions of the evaluation report subtag are exemplarily described. The evaluation report subtag represents the changes in the user's oral ability from multiple dimensions, reflecting the comprehensiveness of the oral evaluation information tag; since the evaluation report subtag is multi-dimensional, the learning suggestion subtag can provide users with more detailed suggestions from multiple dimensions, reflecting the personalization of the oral evaluation information tag.

[0036] In one possible implementation, the oral evaluation data set includes a first data subset, a second data subset and a third data subset. The first data subset includes the user's oral evaluation samples and oral evaluation information labels, the second data subset includes text evaluation samples and text scoring labels, and the third data subset is obtained based on the general task data set.

[0037] In this implementation, a method for constructing a spoken language evaluation data set is provided. According to the method and implementation provided by the embodiment of the present application, there are two main tasks for the spoken language evaluation model, which are to comprehensively evaluate the user and to score the predicted text in multiple dimensions by combining basic information, historical evaluation results and this evaluation results. Therefore, it is necessary to construct data sets for two tasks: a first data subset and a second data subset. Since the large language model will have the problem of catastrophic forgetting when fine-tuning, fine-tuning using only the data sets of the above two tasks will reduce the general performance of the large speech model. Therefore, when constructing a complete training data set, a data set of general tasks is usually added: a third data subset. The data set of general tasks is generally obtained through an open source general task data set. Exemplarily, a data ratio of 4:4:2 can be used to mix the data of the first task, the second task and the general task together, and after random shuffling, a spoken language evaluation data set used for the final training is formed.

[0038] In a third aspect, an embodiment of the present application provides a method for training a speech evaluation model, comprising: constructing a speech evaluation data set; and training the above-mentioned speech evaluation model based on the speech evaluation data set.

[0039] In an embodiment of the present application, a method for training a speech evaluation model is provided. The speech evaluation model obtained by training a speech evaluation data set analyzes the user's speech performance from multiple dimensions such as pronunciation, vocabulary, fluency, speed, etc., and outputs a speech score.

[0040] In a possible implementation, the speech evaluation data set includes acoustic feature samples and speech scoring labels. Constructing the speech evaluation data set includes: obtaining speech samples and speech scoring labels; and performing feature extraction on the speech samples to obtain acoustic feature samples.

[0041] In this implementation, compared with the speech to be evaluated, the acoustic feature samples can better express the information of the acoustic dimension, which is convenient for the speech evaluation model to analyze, and can effectively improve the training efficiency and reduce the number of training rounds.

[0042] In a fourth aspect, an embodiment of the present application provides a spoken language evaluation device, comprising: an acquisition module, used to acquire basic information of a user, a speech to be evaluated, and historical evaluation results; a processing module, used to obtain a speech score based on the speech to be evaluated, the speech score representing the score of the speech to be evaluated in terms of speech; and, based on the text to be evaluated, to obtain a text score, the text to be evaluated is obtained based on the text information in the speech to be evaluated, the text score representing the score of the text accuracy of the speech to be evaluated; and, based on the speech score and the text score, to obtain a current evaluation result; and, inputting the basic information, historical evaluation results, and current evaluation results into an oral evaluation model to output oral evaluation information of the speech to be evaluated, the oral evaluation information representing the evaluation results of the speech to be evaluated in multiple dimensions.

[0043] In one possible implementation, the oral assessment information includes at least one of an assessment report and learning suggestions. The assessment report represents changes in the user's oral ability from multiple dimensions, and the multiple dimensions include voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy, and the learning suggestions represent suggestions for the user on oral learning.

[0044] In a possible implementation, the processing module is specifically used to: input the text to be evaluated into the spoken language evaluation model to output a text score.

[0045] In one possible implementation, the processing module is specifically used to: extract features of the speech to be evaluated to obtain acoustic features of the speech to be evaluated, where the acoustic features represent features of the speech to be evaluated in an acoustic dimension, where the acoustic dimension includes at least one of sound quality and rhythm; and input the acoustic features into a speech evaluation model to output a speech score.

[0046] In a possible implementation, the evaluation result includes at least one of a voice score, a text score, and a comprehensive score, and the comprehensive score is obtained by weighting the voice score and the text score.

[0047] In a possible implementation, the speech to be evaluated is obtained based on the user completing the test questions to be evaluated. The processing module is also used to: determine the user's oral ability level based on the user's basic information and historical evaluation results; determine the difficulty of the questions corresponding to the user based on the oral ability level; and determine the test questions based on the difficulty of the questions.

[0048] In a possible implementation, the processing module is specifically used to: add the user's basic information and historical evaluation results to a first prompt word template to obtain first prompt information, where the first prompt word template is used to guide the generation process of the oral evaluation model; input the first prompt information into the oral evaluation model to output the user's oral ability level.

[0049] In a possible implementation, the processing module is specifically used to: generate the test questions by the oral evaluation model according to the difficulty of the questions; or, according to the difficulty of the questions, obtain the test questions by matching from the exercise database.

[0050] In a possible implementation, the processing module is specifically used to: add the difficulty of the question to the second prompt word template to obtain the second prompt information, the second prompt word template is used to guide the generation process of the oral evaluation model; input the second prompt information into the oral evaluation model to output the question to be evaluated.

[0051] In a possible implementation, the processing module is further used to: based on the question to be evaluated and in combination with the digital human, provide the question to be evaluated to the user and obtain the user's voice to be evaluated during the interaction with the user.

[0052] In a possible implementation, the processing module is specifically used to: add the question to be evaluated in the third prompt word template to obtain the third prompt information, and the third prompt word template is used to guide the generation process of the oral evaluation model; input the third prompt information into the oral evaluation model to output the live stream information of the question to be evaluated; interact with the user through the digital human to provide the live stream information and obtain the user's voice to be evaluated.

[0053] In one possible implementation, the processing module is specifically used to: in a fourth prompt word template, basic information, historical evaluation results, and current evaluation results, obtain fourth prompt information, and the fourth prompt word template is used to guide the generation process of the oral evaluation model; input the fourth prompt information into the oral evaluation model to output the oral evaluation information of the speech to be evaluated.

[0054] In a fifth aspect, an embodiment of the present application provides a training device for a spoken language evaluation model, comprising: a first data set construction module, used to construct a spoken language evaluation data set, the spoken language evaluation data set comprising a spoken language evaluation sample and a spoken language evaluation information label of a user, the spoken language evaluation sample comprising the user's basic information and several evaluation results thereof, the spoken language evaluation information label representing the evaluation results of the spoken language evaluation sample in multiple dimensions, the multiple dimensions comprising a speech dimension and / or a text dimension, the speech dimension comprising at least one of pronunciation, vocabulary, fluency, and speed, and the text speech dimension comprising at least one of answer accuracy and word accuracy; a first training module, used to take the user's basic information and several evaluation results thereof as input of the spoken language evaluation model to be trained, and output a predicted result of the user's spoken language evaluation information; obtain an evaluation loss value according to the spoken language evaluation information label and the predicted result of the spoken language evaluation information; and adjust the parameters of the spoken language evaluation model to be trained according to the evaluation loss value to obtain a trained spoken language evaluation model.

[0055] In a possible implementation, the spoken language evaluation model to be trained is obtained by adding a low-rank matrix layer to the output layers of several modules in the pre-trained large model, and the several modules are neural network modules in the pre-trained large model for realizing specific functions. The first training module is specifically used to adjust the parameters of the low-rank matrix layer according to the evaluation loss value to obtain the trained spoken language evaluation model.

[0056] In one possible implementation, the first data set construction module is specifically used to: design a seed data set, the seed data set includes an evaluation sample example and its corresponding evaluation information example, the seed data set is an example of generating a spoken evaluation data set, the evaluation sample example is an example of generating a spoken evaluation sample, and the evaluation information sample is an example of generating a spoken evaluation information label; input the seed data set as a sample generation model, so that the sample generation model generates a spoken evaluation sample according to the evaluation sample example, and generates a spoken evaluation information label according to the evaluation information sample; obtain a spoken evaluation data set according to the spoken evaluation sample and its corresponding spoken evaluation information label.

[0057] In one possible implementation, the oral assessment information tag includes at least one of an assessment report sub-tag and a learning suggestion sub-tag, the assessment report sub-tag represents the changes in the user's oral ability from multiple dimensions, the multiple dimensions including voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy; the learning suggestion sub-tag represents suggestions for the user on oral learning.

[0058] In a possible implementation, the spoken language evaluation data set further includes a second data subset and a third data subset, the second data subset includes text evaluation samples and text scoring labels, and the third data subset is obtained according to the general task data set.

[0059] In a sixth aspect, an embodiment of the present application provides a training device for a speech evaluation model, comprising: a second data set construction module, used to construct a speech evaluation data set; and a second training module, used to train the above-mentioned speech evaluation model based on the speech evaluation data set.

[0060] In a possible implementation, the speech evaluation data set includes acoustic feature samples and speech scoring labels. The second data set construction module is specifically used to: obtain speech samples and speech scoring labels; and perform feature extraction on the speech samples to obtain acoustic feature samples.

[0061] In a seventh aspect, the present application provides a server, on which a spoken language assessment model is deployed, and the server is used to execute the method described in the above first aspect and any possible implementation method thereof.

[0062] In an eighth aspect, the present application provides a spoken language evaluation system, which includes a terminal device and the server provided in the third aspect. The server is used to determine the topic to be predicted based on the basic information of the user and the historical evaluation results, and to determine the user's spoken language evaluation information based on the speech to be evaluated and the basic information of the user and the historical evaluation results of the user; the terminal device is used to provide the user with the topic to be evaluated and obtain the user's speech to be evaluated based on the topic to be evaluated in the process of interacting with the user.

[0063] In a ninth aspect, the present application provides a chip system, the chip system comprising a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the method described in the above first aspect and any possible implementation thereof.

[0064] In a tenth aspect, the present application provides a computing device, the computing device comprising a processor and a memory. The processor is used to execute instructions stored in the memory, so that the computing device executes the method described in the first aspect and any possible implementation thereof.

[0065] In an eleventh aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method described in the above first aspect and any possible implementation thereof.

[0066] In a twelfth aspect, the present application provides a computer-readable storage medium comprising computer program instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method described in the above-mentioned first aspect and any possible implementation thereof, wherein the computing device cluster comprises at least one computing device.

[0067] In a thirteenth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method described in the above first aspect and any possible implementation thereof, wherein the computing device cluster includes at least one computing device.

[0068] It can be understood that the beneficial effects of the fourth to thirteenth aspects can be found in the relevant descriptions of the first, second and third aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0070] Figure 1 A schematic diagram of the architecture of a spoken language assessment system provided in an embodiment of the present application;

[0071] Figure 2 A flowchart of a spoken language assessment method provided in an embodiment of the present application;

[0072] Figure 3 A flowchart of a method for determining a question to be evaluated provided in an embodiment of the present application;

[0073] Figure 4 A flowchart of a method for determining a speech to be evaluated provided in an embodiment of the present application;

[0074] Figure 5 A flowchart of a method for determining spoken language evaluation information provided in an embodiment of the present application;

[0075] Figure 6 A flowchart of a method for training a speech evaluation model provided in an embodiment of the present application;

[0076] Figure 7 A flowchart of a training method for a spoken language assessment model provided in an embodiment of the present application;

[0077] Figure 8 This is a schematic diagram of the architecture of another spoken language assessment system provided in an embodiment of the present application;

[0078] Fig. 9 A schematic diagram of the composition of a spoken language evaluation device provided in an embodiment of the present application;

[0079] Fig.10 A schematic diagram of the composition of a training device for a spoken language assessment model provided in an embodiment of the present application;

[0080] Fig.11 A schematic diagram of the composition of a training device for a speech evaluation model provided in an embodiment of the present application;

[0081] Fig.12 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0082] Fig.13 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0083] Fig.14 It is a structural diagram of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an "or" relationship, for example, A / B means A or B.

[0085] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0086] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0087] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0088] In order to facilitate understanding of the solution provided by the embodiment of the present application, some terms involved in the solution are first briefly introduced.

[0089] Large Language Model (LLM): usually refers to a neural network model that contains ultra-large-scale parameters (usually more than one billion). Large language models have excellent performance in various natural language processing tasks, such as text classification, sentiment analysis, summary generation, translation, etc. Large language models are also called large models, language large models, etc.

[0090] Prompt: Also known as a prompt template, it refers to the text input provided by the user when interacting with a large language model (LLM). These inputs are designed to guide the model to generate a specific type of response. Prompts can be questions, commands, parts of conversations, or other forms of text. They provide contextual information and directional guidance to the model, helping the model to better understand and respond to user needs. In the application of large language models, the design and optimization of prompts is an important task. Effective prompts can improve the quality of the model's response, enabling it to solve problems or provide information more accurately and in a more targeted manner.

[0091] With the acceleration of globalization, language learning has become a trend and plays an important role in daily life, academics, work, etc. Therefore, a technical solution for oral assessment is needed to help users improve their oral ability in the process of language learning.

[0092] In view of this, the embodiment of the present application provides a technical solution for oral evaluation, which can obtain oral evaluation information based on the user's speech to be evaluated and historical data. The oral evaluation information is a comprehensive result of evaluating the user's oral speech in multiple dimensions. The oral evaluation information can provide personalized feedback and targeted guidance to the user, so that the user can improve his oral skills more efficiently.

[0093] Exemplarily, the oral evaluation information includes evaluation reports of various dimensions and subsequent learning suggestions; wherein, the evaluation reports of various dimensions indicate the advantages and disadvantages of the user's speech to be evaluated in multiple dimensions, so as to help the user correct the disadvantages and improve the oral ability; the subsequent learning suggestions are a comprehensive evaluation of the user's historical performance and current performance in combination with the user's historical data, so as to help the user develop a learning plan and improve the oral ability in an orderly manner.

[0094] In some implementations, the embodiments of the present application provide a training method for a speech evaluation model and a spoken language evaluation model. The trained speech evaluation model and spoken language evaluation model can evaluate the spoken language level from multiple dimensions, thereby improving the accuracy of the spoken language evaluation.

[0095] In some implementations, the oral assessment model is a large language model. Based on the generative capabilities of the large language model, the assessment results are not limited to scores, but text-based oral assessment information. In other words, it can give follow-up improvement suggestions like a teacher, which improves the assessment effect.

[0096] The technical solution provided in the embodiment of the present application is characterized by its advancedness. Specifically, the large language model technology is used to use its capabilities in different scenarios at the question-setting end, the interaction end, and the evaluation end. Due to its deep learning characteristics, the large language model can understand and process natural language more accurately, and can generate questions, conduct oral interactions, and evaluate students' oral abilities more accurately. At the same time, the large language model can provide more accurate evaluations and personalized feedback based on the subtle differences in students' answers and oral expressions.

[0097] The technical solution provided by the embodiment of the present application is comprehensive. Specifically, by utilizing the capabilities of a large language model, a series of oral evaluation processes such as question setting, answering, evaluation, and feedback are covered to form a complete closed loop. The entire system not only considers students' pronunciation, vocabulary, fluency, and speed, but also integrates students' accuracy in answering questions, accuracy in word usage, etc., so as to more comprehensively evaluate students' oral ability. In comparison, existing technical solutions only focus on individual aspects of oral skills and cannot comprehensively evaluate students' oral ability.

[0098] The technical solution provided by the embodiment of the present application has the characteristics of personalization. Taking students as an example of users, specifically, personalized questions are set according to the historical evaluation results of students. This personalized strategy helps to better meet the learning needs and ability levels of different students. In contrast, existing technical solutions usually use standardized questions, which cannot meet the personalized learning needs of students.

[0099] The technical solution provided in the embodiment of the present application has the characteristics of interactivity. Taking students as an example of users, specifically, the oral evaluation system provided in the embodiment of the present application includes an AI (Artificial Intelligence) digital human teacher who interacts with students orally. This method enhances interactivity and makes the oral evaluation process closer to a real oral communication environment. In contrast, the existing technical solutions lack this interactivity, which may make the oral evaluation process lack practicality and interest.

[0100] That is to say, compared with the existing technical solutions, the embodiment of the present application can comprehensively evaluate the student's oral level through a large language model, combined with the student's basic information, historical evaluation results and this evaluation result, improve the accuracy of the oral evaluation, and can give subsequent learning suggestions to improve the effect of the evaluation. In addition, the embodiment of the present application simulates a real English teacher and a student to perform natural oral interaction through a large language model, and combines digital human technology to enhance the interactive experience of the oral evaluation process. Therefore, the embodiment of the present application uses the latest AI algorithm technology (large language model, digital human, etc.) to improve the calculation method of oral evaluation information, and on this basis, combined with the basic information and historical evaluation results of the evaluator, gives text evaluations of each dimension and subsequent learning suggestions, and comprehensively improves the evaluation effect.

[0101] For example, Figure 1 FIG. 1 is a schematic diagram showing the architecture of a spoken language assessment system provided in an embodiment of the present application. Figure 1As shown, the embodiment of the present application provides a spoken language assessment system 100, which mainly includes: a terminal device 110 and a server 120. Among them, the method provided by the embodiment of the present application can be implemented by software, and the software has a terminal display module and a service operation module. The terminal device 110 runs the terminal display module of the software, and the server 120 runs the service operation module of the software. In the process of the terminal device 110 running the terminal display module of the software, the service operation module running on the server 120 can be called to implement the method provided by the embodiment of the present application.

[0102] Furthermore, the server 120 determines the topic to be evaluated based on the user's basic information and historical evaluation information; the terminal device 110 can obtain the topic to be evaluated from the server 120 and display the topic to be evaluated to the user; the user starts the oral evaluation according to the evaluation topic, the terminal device 110 obtains the user's voice to be evaluated, and sends the voice to be evaluated to the server 120; the server 120 evaluates the voice to be evaluated, obtains oral evaluation information, and sends the oral evaluation information to the terminal device 110; the terminal device 110 displays the oral evaluation information to the user to feedback the evaluation result to the user.

[0103] Alternatively, if Figure 1 As shown, combined with digital human technology, the user is provided with test questions and the user's test voice is obtained. The digital human has good interactive performance and can guide the user to conduct multiple rounds of dialogue and obtain the user's test voice for multiple rounds of dialogue. The digital human can simulate a real English teacher and interact with the user in a natural way, which improves the interactive experience of the oral evaluation process.

[0104] Furthermore, the server 120 may include one or more servers ( Figure 1 In the example, a server is included, and the server 120 can provide the method and / or apparatus provided in the embodiment of the present application for one or more terminal devices.

[0105] Furthermore, the terminal device 110 may be installed with an application program related to the method and / or device of the present application. The application program or webpage may provide an interface. The terminal device 110 may receive input information input by the user on the interface and send the input information to the server 120. The input information may be, for example, speech to be evaluated, interaction information, etc. The server 120 is deployed with a spoken language evaluation model, and the received input information may be input into the spoken language evaluation model to output spoken language evaluation information, thereby realizing information interaction between the terminal device 110 and the server 120. The server 120 may also return the spoken language evaluation information to the terminal device 110, and feedback the spoken language evaluation information to the user. The spoken language evaluation information may be an evaluation report, learning suggestions, etc.

[0106] It should be understood that in some optional implementations, the terminal device 110 is deployed with a spoken language assessment model, and the terminal device 110 can also complete the action of obtaining the processing result based on the received input information by itself without the need for the cooperation of the server. The embodiments of the present application are not limited to this.

[0107] Next, the product form of the terminal device 110 is described. The terminal device 110 in the embodiment of the present application may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this. Exemplary embodiments of the terminal device 110 involved in this solution include but are not limited to electronic devices equipped with iOS, Android, Windows, Harmony OS or other operating systems. The embodiment of the present application does not specifically limit the type of electronic device.

[0108] Next, the product form of the server 120 is described. It can be further understood that the server 120 can be a variety of servers, such as an X86 architecture server, and can specifically be a whole cabinet server, a blade server, a high-density server, a rack server, or a high-performance server. In other words, the embodiment of the present application does not specifically limit the specific type of the server. Further, it can be understood that, Figure 1 The structure of the server shown does not constitute a limitation on the structure of the server. The server may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0109] Furthermore, server 120 can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers. It can also be configured as a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN) and big data and artificial intelligence platforms. The cloud server cluster is deployed in several cloud data centers; the software can be an application that implements object control methods, etc., but is not limited to the above forms.

[0110] Next, the communication connection mode between the terminal device 110 and the server 120 is described. Exemplarily, the terminal device 110 and the server 120 are connected via a network, so that the terminal device 110 can access the cloud management platform deployed by the cloud server cluster. Among them, the network can be a wired network or a wireless network. For example, the wired network can be a cable network, an optical fiber network, a digital data network (Digital Data Network, DDN), etc., and the wireless network can be a telecommunications network, an internal network, the Internet, a local area network (Local Area Network, LAN), a wide area network (Wide Area Network, WAN), a wireless local area network (Wireless Local Area Network, WLAN), a metropolitan area network (Metropolitan Area Network, MAN), a public switched telephone network (Public Service Telephone Network, PSTN), a Bluetooth network, a ZigBee network (ZigBee), a mobile phone (Global System for Mobile Communications, GSM), a CDMA (Code Division Multiple Access) network, a CPRS (General Packet Radio Service) network, etc. or any combination thereof.

[0111] It is understandable that the network can use any known network communication protocol to achieve communication between different terminal device layers and gateways. The above-mentioned network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), Firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR), Bluetooth (blue tooth), wireless fidelity (Wi-Fi) and other communication protocols.

[0112] In one possible scenario, the server 120 can be used as a cloud (a software platform that uses application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup). In specific use, the server 120 can deploy a cloud management platform and a data center, and the terminal device 110 and the cloud interact through the cloud management platform. In addition, the data center can deploy nodes, where the nodes in the data center can be virtual machine instances, container instances, physical servers, etc.

[0113] In another possible scenario, the method provided by the embodiment of the present application can be implemented by software. The software has a terminal display module and a service operation module, the terminal device 110 runs the terminal display module of the software, and the server 120 runs the service operation module of the software. During the process of the terminal device 110 running the terminal display module of the software, the service operation module running on the server 120 can be called to implement the method provided by the embodiment of the present application.

[0114] That is to say, the method provided in the embodiment of the present application can be applied to the above-mentioned terminal device 110, and can also be applied to the server 120. When it is specifically implemented, it can be run on the terminal device 110 or the server 120 in the form of software. For example, the software can be a service or an application. The embodiment of the present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, a program module includes routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The embodiment of the present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0115] The above is an introduction to a spoken language evaluation system 100 provided in an embodiment of the present application. Next, based on the above content, a spoken language evaluation method provided in an embodiment of the present application is introduced. It can be understood that the above method is proposed based on the spoken language evaluation system 100 described above, and part or all of the content of the above method can refer to the above description of the spoken language evaluation system 100.

[0116] For example, Figure 2 The flowchart of a spoken language assessment method provided by an embodiment of the present application is shown. It can be understood that the method is executed by a computing device, which can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1The server 120 shown. Figure 2 As shown, a spoken language evaluation method mainly includes the following steps:

[0117] Step S210, obtaining the user's basic information, the voice to be evaluated, and historical evaluation results.

[0118] For example, basic information may include the user's age, grade, etc. Historical evaluation results may include historical scores, which are used to represent the user's oral evaluation information over the past period of time, so that the large language model can grasp the changes in the user's oral ability. The speech to be evaluated refers to the speech data corresponding to the answer generated by the user during the interaction process, which is used to evaluate the user's oral ability.

[0119] Step S220, based on the speech to be evaluated, a speech score is obtained, where the speech score represents the score of the speech to be evaluated in terms of speech.

[0120] Exemplarily, the speech score includes speech scores in multiple dimensions such as pronunciation, vocabulary, fluency, and speed.

[0121] Optionally, the speech to be evaluated is used as the input of the speech evaluation model, and the speech score is output. The speech evaluation model is a neural network model with good learning ability, which can analyze the user's speech evaluation results from multiple dimensions such as pronunciation, vocabulary, fluency, speed, etc.

[0122] Optionally, feature extraction is performed on the speech to obtain acoustic features of the speech to be evaluated; the acoustic features are input into the speech evaluation model to output a speech score. The acoustic features represent the characteristics of the speech to be evaluated in terms of sound quality and rhythm, which improves the prediction accuracy of the sound evaluation model.

[0123] Step S230, based on the text to be evaluated, a text score is obtained; and based on the voice score and the text score, the evaluation result is obtained. The text to be evaluated is obtained based on the text information in the voice to be evaluated, and the text score represents the score of the text accuracy of the voice to be evaluated.

[0124] Exemplarily, the text score includes the accuracy of the answer text, grammatical accuracy, etc.

[0125] Optionally, the text to be evaluated is obtained based on the speech to be evaluated. For example, the speech to be evaluated is used as the input of a speech recognition model (Automatic Speech Recogn it ion, ASR), and the text to be evaluated is output. Alternatively, the text to be evaluated is directly obtained and generated.

[0126] Optionally, the text to be evaluated is input into a spoken language evaluation model to output a text score, where the text score represents a score of the text accuracy of the speech to be evaluated.

[0127] Furthermore, the oral evaluation model is a neural network model with good learning ability, which can analyze the user's text evaluation results from multiple dimensions such as words, sentence patterns, and grammar. In other words, the oral evaluation model can process any text data to achieve different tasks. For example, under the guidance of the prompt word template, the oral evaluation model can form an evaluation report for the user, and determine subsequent learning suggestions based on the student's historical data and the oral evaluation information of this evaluation; and generate a text score based on the text to be evaluated.

[0128] For example, in some scenarios, there is a corresponding reference text for the speech to be evaluated, and the reference text is the correct text corresponding to the speech to be evaluated. For example, the speech to be evaluated is the speech generated by the user reading a certain text, and the certain text is the reference text. In this scenario, the reference text corresponding to the speech to be evaluated can be obtained; and then the text score can be obtained by comparing the text to be evaluated with the reference text. For example, the text to be evaluated is input into the oral evaluation model, and the oral evaluation model compares the text to be evaluated with the reference text to output the text score.

[0129] For example, in some other scenarios, there is no corresponding reference text for the speech to be evaluated. For example, the speech to be evaluated is the speech generated by the user speaking a random sentence. In this scenario, the text to be evaluated itself can be scored to obtain a text score. For example, the text to be evaluated is input into the spoken language evaluation model to output a text score.

[0130] Optionally, the spoken language evaluation model may also be a large multimodal model, which can be used to respectively implement the tasks of generating speech scores, generating text scores, and generating spoken language evaluation information.

[0131] Optionally, a comprehensive score is obtained based on the voice score and the text score. The voice score, the text score, and the comprehensive score can be used as part of the historical evaluation results when the oral evaluation is performed next time. In other words, the historical evaluation results include the voice scores, text scores, and comprehensive scores before the current evaluation for several times, so that the changes in each score can be obtained by combining the historical evaluation results and the current evaluation information. Exemplarily, the voice score and the text score are weighted to obtain a comprehensive score.

[0132] Furthermore, the evaluation results of this time include the scores of each dimension obtained in this evaluation. In the next evaluation process, the evaluation results of this time will be used as part of the historical evaluation results and as the basis for generating the next oral evaluation information.

[0133] Step S240, input the basic information, historical evaluation results, and current evaluation results into the spoken language evaluation model to output spoken language evaluation information of the speech to be evaluated, wherein the spoken language evaluation information represents the evaluation results of the speech to be evaluated in multiple dimensions.

[0134] Optionally, the input of the oral evaluation model also includes a comprehensive score. The oral evaluation model can obtain the change trend of the comprehensive score based on the comprehensive scores in the historical evaluation results and the current comprehensive score, so as to analyze the change trend of the user's oral ability from a comprehensive perspective.

[0135] Exemplarily, the spoken language evaluation information may include: at least one of a speech evaluation result, a text evaluation result, and a learning suggestion, so that the spoken language evaluation information may represent evaluation information of the speech to be evaluated in multiple dimensions.

[0136] The speech evaluation result indicates how the speech score changes over time. For example, the speech score includes at least one of the multiple scores such as pronunciation score, vocabulary score, fluency score, speed score, etc. The speech evaluation result may indicate how each score in the speech score changes over time.

[0137] The text evaluation result indicates how the text score changes over time. For example, the text score includes at least one of multiple scores such as answer accuracy and word accuracy; the text evaluation result indicates how each score in the text score changes over time.

[0138] For example, Figure 3 A flow chart of a method for determining a topic to be evaluated provided by an embodiment of the present application is shown. Optionally, the speech to be evaluated is obtained based on the user completing the topic to be evaluated. It can be understood that the method is executed by a computing device, and the computing device can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1 The server 120 shown. Figure 3 As shown, a method for determining a topic to be evaluated mainly includes the following steps:

[0139] Step S310, determining the user's oral proficiency level based on the user's basic information and historical evaluation results.

[0140] Optionally, obtain the student's historical data and exercise database. Design a first prompt word template, process the student's historical data through the first prompt word template, and input it into the oral evaluation model to determine the current student's level (i.e., oral ability level). And set the question difficulty (i.e., question difficulty) according to the oral ability level. Among them, the student is an example of a user; the historical data includes basic information, historical evaluation results, etc.

[0141] Exemplarily, the first prompt word template can be designed as: "You are a professional English teacher, and you will conduct oral assessments on Chinese students. Now you need to assess the students' spoken English proficiency in order to match questions of corresponding difficulty. The information provided to you includes: basic information of the students, including age, grade, etc.; historical assessment results of the students, including historical scores in multiple dimensions. For multiple dimensions, please refer to the discussion on the classification categories of voice scoring and text scoring, which will not be repeated here. Please evaluate the students' levels based on the above information. The students' levels are represented by A, B, C, D, and E, with A representing the highest level. Please return the evaluation results, and you only need to return the letters representing the levels."

[0142] Step S320, determining the difficulty of the questions corresponding to the user according to the oral proficiency level.

[0143] For example, the level of the students and the difficulty of the questions are divided into K levels. If the user focuses on consolidating the current oral ability while improving the oral ability, assuming that the current level of the student is level i, then the questions with difficulty levels i-1 and i are selected from the exercise database; if the user likes challenging questions, assuming that the current level of the student is level i, then the questions with difficulty levels i and i+1 are selected from the exercise database.

[0144] Step S330, determining the questions to be evaluated according to the difficulty of the questions.

[0145] Optionally, the user's test questions are determined from the question database according to the difficulty of the questions. Determining the test questions from the question database uses fewer computing resources and has higher work efficiency, which can be applied to scenarios where there are a large number of questions in the question data set.

[0146] Exemplarily, the number of questions is set, and random sampling is performed from all questions with difficulty levels i and i+1 according to the number of questions to obtain questions to be evaluated.

[0147] Optionally, the difficulty of the questions can also be input into the oral evaluation model to output the questions to be evaluated. Optionally, the oral evaluation model can generate similar questions as the questions to be evaluated based on the known exercise data. The exercise data represents a number of exercise examples corresponding to the difficulty of the questions. The way in which the oral evaluation model generates the questions to be evaluated can utilize the learning ability of the large language model to generate more accurate questions to be evaluated, which can be applicable to scenarios where the number of exercises in the exercise data set is small, for example.

[0148] Optionally, when using the oral evaluation model to generate the test questions, in a possible implementation, a second prompt word template is designed, and the current student's level and / or the difficulty of the question are processed by the second prompt word template and input into the oral evaluation model, and the oral evaluation model generates the test questions. Exemplarily, the oral evaluation model generates similar questions as the test questions based on known examples of exercises of corresponding difficulty.

[0149] For example, the second prompt word template can be designed as: "You are a professional English teacher, and you will conduct oral assessments on Chinese students. Now that you know the students' oral English proficiency and the corresponding question difficulty, you are now asked to set the question difficulty and design the test questions in advance. The requirements for the generated test questions are: no biased questions can be generated; the answers to these questions should be accurate, and there should not be multiple answers. The format of the returned questions should be in json format, and the returned questions need to contain the question content information, the corresponding answer information, and the corresponding difficulty information."

[0150] For example, Figure 4 The flowchart of a method for determining a speech to be evaluated provided by an embodiment of the present application is shown. Optionally, in the process of interacting with the user, the user can be provided with a topic to be evaluated through a user interface, and the user's speech to be evaluated can be obtained. It can be understood that the method is executed by a computing device, and the computing device can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1 The server 120 shown in the figure; the computing device interacts with the user through the user interface on the terminal side, such as providing the user with the test questions, obtaining the user's voice to be tested, and displaying the oral test information to the user. Figure 4 As shown, a method for determining a speech to be evaluated mainly includes the following steps:

[0151] Step S410, obtaining the title text of the question to be predicted.

[0152] Step S420: The user interface configures the digital human and converts the title text into the voice output by the digital human model to form a live streaming of the digital human. Figure 4 Optionally, the speech output by the digital human model matches the lip shape of the digital human model.

[0153] Step S430, obtaining the speech to be evaluated provided by the student.

[0154] Optionally, the speech to be evaluated provided by the student may be converted into text to be evaluated through a speech recognition module.

[0155] Step S440: Based on the text to be evaluated provided by the student, the oral interaction module outputs a reply text.

[0156] Step S450: input the reply text into the digital human model to form a live streaming of the digital human. Figure 4 As shown by the solid line in .

[0157] Steps S410 to S450 are repeated until the oral interaction module completes the oral interaction task with the student and obtains the student's speech to be evaluated and the corresponding text to be evaluated.

[0158] Optionally, the topic to be evaluated can also be input into the oral evaluation model to output the topic text of the topic to be predicted. The topic text generated by the oral evaluation model has better interactive effect and higher user experience.

[0159] Exemplarily, a third prompt word template is designed, and the question to be evaluated is processed by the third prompt word template and then input into the oral evaluation model. The oral evaluation model interacts with the student orally to obtain the student's speech to be evaluated.

[0160] Optionally, the third prompt word template can be designed as: "You are a professional English teacher, and you are conducting an oral assessment on Chinese students. Now you know the test topic, and you need to generate a topic text for interaction with the user based on the test topic. You will ask students questions in order based on the following list of questions to be assessed: Question 1 is..., Question 2 is..., etc. After students answer your questions, please naturally connect with their responses. Your response requirements are: If the students answer correctly, then please ask the next question; if the students answer incorrectly, then give the students a hint and ask them to answer again; if there are no more questions, please naturally end this test. Now, please ask your first question."

[0161] For example, Figure 5 The flowchart of a method for determining oral evaluation information provided by an embodiment of the present application is shown. It can be understood that the method is executed by a computing device, and the computing device can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1 The server 120 shown in FIG. 1 is equipped with a spoken language evaluation model. Figure 5 As shown, a method for determining oral evaluation information mainly includes the following steps:

[0162] Step S510: designing a fourth prompt word template.

[0163] Optionally, the fourth prompt word template can be designed as: "You are a professional English teacher. Now you are going to conduct an oral assessment on Chinese students and comprehensively evaluate the results of this assessment. The information provided to you includes: students' text scores, including multi-dimensional scores such as answer accuracy and word accuracy; students' voice assessment results (i.e., voice scores), including multi-dimensional voice scores such as pronunciation, vocabulary, fluency, speed, etc.; students' basic information, including age, grade, etc.; students' historical assessment records, including historical scores. Finally, please describe the students' assessment results in natural language based on the above information to form an assessment report, and give subsequent learning suggestions based on the students' historical data and combined with the results of this assessment. Please return to the assessment report and subsequent learning suggestions."

[0164] Step S520, through the fourth prompt word template, the basic information, the historical evaluation results, and the current evaluation results are processed to obtain the information to be evaluated. The information to be evaluated is also called the fourth prompt information. Among them, the information to be evaluated belongs to the templated expression form of human-computer interaction, which is more convenient for the oral evaluation model to understand the input information and output more accurate oral evaluation information. Optionally, the evaluation result of this time includes at least one of the voice score, text score and comprehensive score.

[0165] Step S530: input the information to be evaluated into the spoken language evaluation model to output the user's spoken language evaluation information. The spoken language evaluation information may represent the user's speech score in the speech to be evaluated and the corresponding text score in the text to be evaluated.

[0166] For example, Figure 6 The flowchart of a method for training a speech evaluation model provided by an embodiment of the present application is shown. It can be understood that the method is executed by a training device, which can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities. The trained speech evaluation model can be deployed on a computing device, which can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1 The server 120 shown. Figure 6 As shown, a training method for a speech evaluation model mainly includes the following steps:

[0167] Step S610: Obtain a training data set, which includes training samples and speech scoring labels.

[0168] Optionally, the training samples in the training data set may be audio recording samples. For example, assuming that the language of the assessment is English, a large number of audio recording samples of Chinese students speaking English may be collected, including samples of different English proficiency levels, accents, and speaking speeds.

[0169] Optionally, the training data may be diversified by changing the speed and pitch of the audio of the training samples, and data enhancement may be performed on the training data set, which may reduce the possibility of overfitting during the training process.

[0170] Alternatively, the labels can be obtained through data annotation. For example, the labels of the training samples are obtained through manual annotation, or the training samples are annotated by a machine learning system. For example, assuming that the language of the evaluation is English, each training sample can be scored by an English pronunciation expert or through a verified scoring system, and the scores of the scoring cover scores of multiple dimensions such as accuracy, clarity, fluency, etc.

[0171] Step S620: extract features from the training samples to obtain acoustic features of the training samples.

[0172] Exemplarily, a possible implementation of the extracted acoustic features is to extract acoustic features such as Mel-Frequency Cepstrum Coefficient (MFCC), fundamental frequency (F0), and spectral envelope, which can reflect the sound quality and rhythmic characteristics of pronunciation.

[0173] Step S630, training the speech evaluation model according to the acoustic features and speech scoring labels of the training samples.

[0174] Exemplarily, the training process is implemented through multiple iteration steps, and the training process of one iteration step mainly includes the following steps: using the extracted acoustic features as the input of the speech evaluation model, and outputting the predicted value of the speech score; obtaining the loss function of the speech evaluation model based on the predicted value and the speech score label; adjusting the parameters of the speech evaluation model with the goal of minimizing the loss function.

[0175] Optionally, the speech evaluation model may select a convolutional neural network, and the multi-layer convolution structure of the convolutional neural network is defined by the user.

[0176] Among them, step S610 can be understood as the stage of acquiring the training data set; step S620 can be understood as the stage of preprocessing the training samples to extract features in the audio; step S630 can be understood as the training stage of the speech evaluation model, and the training stage can include multiple iterations of model training.

[0177] For example, Figure 7The flowchart of a method for training a spoken language assessment model provided in an embodiment of the present application is shown. It can be understood that the method is executed by a training device, which can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities. The trained spoken language assessment model can be deployed on a computing device, which can be implemented by any device, equipment, platform, or device cluster with computing and processing capabilities, such as Figure 1 The server 120 shown. Figure 7 As shown, a training method for a spoken language evaluation model mainly includes the following steps:

[0178] Step S710, constructing a spoken language evaluation data set. The spoken language evaluation data set includes a user's spoken language evaluation sample and a spoken language evaluation information tag; the spoken language evaluation sample includes the user's basic information and several evaluation results; the spoken language evaluation information tag indicates the evaluation results of the spoken language evaluation sample in multiple dimensions; the multiple dimensions include a voice dimension and / or a text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy.

[0179] In one possible implementation, constructing a spoken language evaluation data set includes: designing a seed data set, the seed data set includes evaluation sample examples and their corresponding evaluation information examples, the seed data set is an example of generating a spoken language evaluation data set, the evaluation sample examples are examples of generating spoken language evaluation samples, and the evaluation information samples are examples of generating spoken language evaluation information labels; inputting the seed data set as a sample generation model, so that the sample generation model generates spoken language evaluation samples according to the evaluation sample examples, and generates spoken language evaluation information labels according to the evaluation information samples; and obtaining a spoken language evaluation data set according to the spoken language evaluation samples and their corresponding spoken language evaluation information labels.

[0180] In this implementation, the method of generating a training data set based on a seed data set saves the time of sample collection, improves the efficiency of constructing a spoken language evaluation data set, increases the diversity of the spoken language evaluation data set, saves development costs, and solves the problem of fewer data samples and difficulty in obtaining samples. Among them, the sample generation model can be any large language model, and the embodiments of this application do not limit this.

[0181] In one possible implementation, the oral evaluation data set includes a first data subset, a second data subset and a third data subset. The first data subset includes the user's oral evaluation samples and oral evaluation information labels, the second data subset includes text evaluation samples and text scoring labels, and the third data subset is obtained based on the general task data set.

[0182] In this implementation, for example, for the oral evaluation model, there are two main tasks: the first task and the second task. The first task refers to the multi-dimensional scoring of the predicted text; the second task refers to the comprehensive evaluation of the user based on the user's basic information, historical evaluation results, and this evaluation result to obtain oral evaluation information. Therefore, it is necessary to construct data sets for the two tasks. The evaluation results of this time include the voice score, text score, comprehensive score, etc.

[0183] For example, for the multi-dimensional scoring of the text to be predicted, it is necessary to clarify the specific standards of the spoken language evaluation and provide an exemplary description of the training samples and labels of the first task. For example, according to the output format of the speech evaluation model, determine the evaluation paradigm of the text content to be evaluated, and design the seed data set of the first task.

[0184] The example of the oral assessment model input (i.e. the example of the training sample) is: You are an English oral assessment teacher, and you need to score the oral test results of the candidates in multiple dimensions. The assessment dimensions are {dimension one} and {dimension two}, and the assessment standard is {assessment standard}; the title of this exam is: {title}, the reference answer is: {reference answer}, and the candidate's answer is: {candidate's answer}. Please score the candidate's answer in different dimensions.

[0185] The output examples of the oral assessment model (i.e., label examples) are: {dimension one}, the examinee's answer basically meets the reference answer, and the score is {score 1}, where score 1 is, for example, 90 points; {dimension two}, the examinee's answer is not precise in terms of wording, but can be expressed in a good way, and the score is {score 2}, where score 2 is, for example, 60 points, etc. In this data sample, {*} indicates the edited part of the data sample, and different content is set for different training samples.

[0186] Exemplarily, the training samples and labels of the second task are illustrated.

[0187] The example of the oral assessment model input (i.e. the training sample example) is: You are an oral English assessment teacher, and you need to make a comprehensive evaluation of the oral test results of the candidates, and give the candidates' basic information and historical assessment results, and give subsequent improvement suggestions. The candidate's test results this time are: {multi-dimensional assessment results of this test}; the candidate's historical assessment results are: {assessment results of the past 5 tests}; the candidate's basic information is: {basic information of the candidate}.

[0188] Among them, the output example of the oral evaluation model (i.e., the label example) is: According to the test results of the examinee this time and the historical evaluation results, it can be seen that in {dimension one}, the examinee has made obvious progress; in {dimension two}, the examinee's score performance is relatively stable, and there is still room for improvement. In general, the examinee's test results this time are improved compared with the historical evaluation, and the comprehensive evaluation result is excellent. Subsequent suggestions: In {dimension two}, you can focus on strengthening practice in order to achieve greater progress. In this data sample, {*} represents the editing part of the data sample, and for different training samples, the editing part is set with different content.

[0189] Therefore, we can create a seed data set based on the above standards and data samples. For example, we can manually write some high-quality oral evaluation samples; or, we can obtain oral evaluation samples by combining the above standards and data samples with a neural network model.

[0190] In a possible implementation, more training samples are generated based on the seed data set. For example, a large model with hundreds of billions of parameters is used to generate expanded training samples based on the seed data set.

[0191] Furthermore, it is necessary to select a suitable large model. A large model with sufficient parameters is selected to generate more diverse training sample data.

[0192] Furthermore, it is necessary to guide the generation of the large model. Specifically, the training samples and corresponding labels in the seed dataset are used as input to guide the large model to generate training samples and corresponding labels of similar expressions. Optionally, the style and difficulty of the generated content can be adjusted by setting different prompts.

[0193] Furthermore, it is necessary to control the quality of the data set generated by the large model. For example, the generated data can be manually reviewed to ensure that it meets the requirements of oral assessment, and if necessary, corrections can be made or unqualified samples can be removed.

[0194] In a possible implementation, different training samples are combined according to different tasks to obtain a training data set.

[0195] It should be noted that, since large language models will have catastrophic forgetting problems when fine-tuning, fine-tuning using only the datasets of the above two tasks will reduce the general performance of the large model. Therefore, when building a complete training dataset, a dataset of general tasks is usually added. The dataset of general tasks generally uses an open source general task dataset. Exemplarily, a data ratio of 4:4:2 is used to mix the training data of the first task, the second task, and the general task together, and then randomly shuffle them to form the training dataset used for the final training.

[0196] Step S720: taking the basic information of the user and several evaluation results thereof as inputs of the oral evaluation model to be trained, and outputting the prediction result of the oral evaluation information of the user.

[0197] Step S730, obtaining an evaluation loss value according to the spoken language evaluation information label and the spoken language evaluation information prediction result.

[0198] Step S740: adjusting the parameters of the spoken language evaluation model to be trained according to the evaluation loss value to obtain a trained spoken language evaluation model.

[0199] In a possible implementation, the oral evaluation model to be trained is obtained by adding a low-rank matrix layer to the output layer of several modules in the pre-trained large model, and the several modules are neural network modules in the pre-trained large model for realizing specific functions. According to the evaluation loss value, the parameters of the oral evaluation model to be trained are adjusted to obtain the trained oral evaluation model, including: according to the evaluation loss value, the parameters of the low-rank matrix layer are adjusted to obtain the trained oral evaluation model.

[0200] Exemplarily, a suitable pre-trained large model is selected as the spoken language evaluation model to be trained, and a LoRA (Low-Rank Adaptation, low-rank matrix) layer is added to the spoken language evaluation model to be trained.

[0201] For example, according to the project requirements (such as model size, inference speed, etc.), select the most suitable transformer-based pre-trained large model from the existing open source models as the basic model. Evaluate the compatibility of the pre-trained large model to ensure that the selected pre-trained large model can support the subsequent LoRA layer addition and fine-tuning operations.

[0202] Furthermore, adding a LoRA layer can be achieved by inserting a low-rank matrix into the key layer of a pre-trained large model (such as the oral evaluation model to be trained in the embodiment of the present application). Among them, the key layer refers to the output layer of several neural network modules of the pre-trained large model. Exemplarily, depending on the different functions implemented, a pre-trained large model includes multiple neural network modules (blocks), and a LoRA layer is added to the output layer of each block. During the fine-tuning process, the parameters of the original pre-trained large model are not adjusted, only the parameters of the LoRA layer are adjusted, thereby achieving the purpose of fine-tuning the pre-trained large model.

[0203] In other words, LoRA is an efficient fine-tuning technique that is implemented by inserting low-rank matrices into the key layers of the pre-trained large model. This approach can reduce the consumption of computing resources while maintaining good performance improvements. For example, add Lora matrices to the output layer of the block and set appropriate rank values. These matrices will be optimized during the training process, while the original pre-trained large model weights remain unchanged or change slightly.

[0204] Exemplarily, the steps of fine-tuning the pre-trained large model may include: preparing the data set, for example, converting the previously generated extended training data set into a format suitable for model input; configuring the training environment, for example, setting hyperparameters such as GPU / CPU resources, batch size, and learning rate; performing LoRA layer fine-tuning, for example, using the prepared data set to fine-tune the LoRA layer of the model, monitoring the changes in the loss function during training, and adjusting the hyperparameters in time to obtain better results; verification and testing, for example, regularly evaluating the model performance on the verification set, and finally conducting a comprehensive evaluation on the test set to ensure the generalization ability of the model.

[0205] The above is an introduction to a spoken language assessment system 100 and a spoken language assessment method provided in an embodiment of the present application. Next, based on the above content, from the perspective of software implementation, another spoken language assessment system is introduced to further illustrate the specific implementation of the above spoken language assessment system and the above spoken language assessment method in terms of software modules. It can be understood that part or all of the content of the above-mentioned another spoken language assessment system can refer to the above description of the spoken language assessment system 100.

[0206] For example, Figure 8 FIG. 2 shows a schematic diagram of the architecture of another oral evaluation system provided in an embodiment of the present application. Figure 8 As shown, the embodiment of the present application provides another oral evaluation system, which mainly includes: a question-setting end, an interactive end and an evaluation end. The question-setting end is used to construct the questions required for each evaluation according to the student's historical data to complete the question setting; the interactive end is used to configure the AI ​​digital human teacher, and the student and the configured AI digital human teacher perform oral interaction according to the questions provided by the question-setting end to obtain the student's voice to be evaluated and the corresponding text to be evaluated; the evaluation end is used to evaluate the student's voice to be evaluated and the corresponding text to be evaluated to obtain multi-dimensional oral evaluation information. Among them, the question-setting end, the interactive end and the evaluation end are an example of functional modules divided according to the functions implemented, and do not constitute a limitation to the embodiments of the present application.

[0207] It is understandable that if Figure 8 Another oral evaluation system shown is equivalent to a software running on the server 120. Exemplarily, the software has a terminal display module and a service operation module, such as Figure 1The terminal device 110 shown in FIG. 1 is a terminal display module running the software, such as Figure 1 The server 120 shown runs the service running module of the software. During the process of the terminal device 110 running the terminal display module of the software, the service running module running on the server 120 can be called to implement the method provided in the embodiment of the present application.

[0208] like Figure 8 As shown, the question-generating end is used to construct assessment questions based on the historical data of students to complete personalized question-generating. Exemplarily, the question-generating end includes a data storage module, an exercise construction module, and an exercise recommendation module.

[0209] Furthermore, the data storage module is used to store student history data and exercise database; the student history data includes: student basic information and historical assessment results. The exercise construction module is used to generate questions based on the difficulty of questions, exercise data and prompt words (prompt) engineering through a pre-trained large language model in the education field (such as an oral assessment model); the questions include information such as the stem, answer, and difficulty. The exercise recommendation module is used to judge the current student's level based on the student history data and determine the difficulty of questions.

[0210] It should be noted that, after determining the current student's level, the exercise recommendation module can confirm the difficulty of the exercises corresponding to the level, so that the exercise construction module can select the questions to be evaluated from the exercise database. The difficulty of the exercises is also called the difficulty of the questions.

[0211] Optionally, the exercise construction module can call the oral assessment model to generate questions to be assessed based on the difficulty of the exercises. Optionally, the exercise construction module can combine the prompt word prompt to generate prompt information based on the difficulty of the exercises; the oral assessment model generates questions to be assessed based on the prompt information. Exemplarily, the prompt word prompt corresponding to the question to be assessed can be designed as: "You are a professional English teacher. You will conduct oral assessments on Chinese students. Now you pre-design questions based on the students' levels and the corresponding difficulty of the exercises. The requirements for the questions are: biased questions cannot be generated; the answers to these questions should be accurate, and there cannot be multiple answers; the format of the questions returned should be in json format, containing the questions, answers, and difficulty". json (JavaScript Object Notation) is a lightweight data exchange format. It should be noted that the above is only an example of the prompt word prompt, and technicians in this field can flexibly adjust it according to actual conditions.

[0212] Specifically, the process of generating questions through the oral evaluation model according to the difficulty of the questions includes: for example, in the embodiment of the present application, the student's level is divided into 5 levels A, B, C, D, and E, and the difficulty of the questions is pre-set to have 5 levels of 1 star, 2 stars, 3 stars, 4 stars, and 5 stars. If the current student's level is C, then 3-star and 4-star questions can be screened out from the exercise database, and then according to the preset number of questions, random sampling can be performed to obtain the final recommended questions. Optionally, in the embodiment of the present application, questions of equal difficulty and one level higher can be recommended, which can increase the challenge and help improve the students' level. Optionally, in the embodiment of the present application, questions of equal difficulty and one level lower can be recommended, and the questions that have been evaluated can be reviewed, which is conducive to students consolidating their existing oral proficiency.

[0213] In other implementations, other rules may also be used to match the student's level and the difficulty of the questions.

[0214] Furthermore, in an embodiment of the present application, the oral assessment model can also determine the current student's level based on the student's historical data. The corresponding prompt word prompt can be designed as: "You are a professional English teacher. You will conduct oral assessments on Chinese students. Now you need to assess the students' spoken English level to match questions of corresponding difficulty. The information provided to you includes: basic information of the students, including age, grade, etc.; historical assessment records of the students, including historical scores. Please evaluate the students' level based on the above information. The student's level is represented by A, B, C, D, and E, with A representing the highest level. Please return the evaluation results, and you only need to return the letters representing the level." It should be noted that the above is only an example of the prompt word prompt, and the information required to be provided in the prompt word prompt can be flexibly adjusted by technicians in this field according to actual conditions.

[0215] like Figure 8 As shown, the interactive end is used to configure an AI digital human teacher. Students interact with the configured AI digital human teacher orally according to the questions provided by the question-setting end to obtain the student's voice to be evaluated and the corresponding text to be evaluated; the interactive end includes an oral interaction module, a digital human module and a speech recognition module.

[0216] Furthermore, the oral interaction module is used to output the question text (i.e., the question text of the question to be predicted) through the oral evaluation model according to the question provided by the question-setting end, and provide a reply text based on the text to be evaluated provided by the student. The digital human module is used to input the question text or the reply text into the digital human model to form a digital human live streaming. Optionally, the question text or the reply text is converted into a voice output by the AI ​​digital human teacher, and the voice output by the AI ​​digital human teacher matches the lip shape of the AI ​​digital human teacher. The speech recognition module is used to convert the voice to be evaluated provided by the student into the text to be evaluated, so as to facilitate the accuracy of the evaluation of the answer.

[0217] Furthermore, in an embodiment of the present application, in order to generate a question text with a better interactive effect, the question to be evaluated can be input into the oral evaluation model, and the oral evaluation model will generate a corresponding question text in the form of a dialogue, so that the digital human teacher can guide students to conduct oral questions and answers more naturally.

[0218] Among them, the prompt for inputting the test questions into the oral assessment model can be designed as: "You are a professional English teacher, and you are conducting oral assessments on Chinese students. You will ask students questions in order according to the test questions: Question 1,...; Question 2,...;... After the students answer your questions, please naturally connect with the students' replies. Your response requirements are: If the students answer correctly, then please ask the next question; if the students answer incorrectly, then give the students some hints and ask them to answer again; if there are no more questions, please naturally end this test. Now, please ask your first question."

[0219] like Figure 8 As shown, the evaluation end is used to evaluate the student's speech to be evaluated and the corresponding text to be evaluated to obtain oral evaluation information, which is used to represent the accuracy of oral answers from multiple dimensions. The evaluation end includes a speech evaluation module and an oral evaluation module.

[0220] Furthermore, the speech evaluation module is used to evaluate the student's speech to be evaluated from multiple dimensions such as pronunciation, vocabulary, fluency, speed, etc. through a pre-trained large language model in the field of education (i.e., a speech evaluation model). The oral evaluation module is used to evaluate the text to be evaluated through a pre-trained large language model in the field of education (i.e., an oral evaluation model). The oral evaluation module is also used to generate oral evaluation information by combining the student's basic information, historical evaluation results, and current evaluation results through the oral evaluation model.

[0221] Exemplarily, in an embodiment of the present application, the student's speech to be evaluated and the corresponding text to be evaluated are evaluated through a speech evaluation model to generate oral evaluation information, and the prompt word prompt is designed as: "You are a professional English teacher. You have conducted oral evaluations on Chinese students. Now you need to comprehensively evaluate the results of this evaluation. The information provided to you includes: the student's text score, including multi-dimensional scores such as answer accuracy and word accuracy; the student's speech evaluation results, including multi-dimensional speech scores such as pronunciation, vocabulary, fluency, speed, etc.; the student's basic information, including age, grade, etc.; the student's historical evaluation records, including historical scores. Based on the above information, please describe the student's evaluation results in natural language to form an evaluation report, and at the same time, based on the student's historical data and combined with the results of this evaluation, give subsequent learning suggestions. Please return to the evaluation report and subsequent learning suggestions."

[0222] Next, based on the same inventive concept, combined Fig. 9 , Fig.10 and Fig.11 , a detailed example description is given of the oral evaluation device, the oral evaluation model training device and the speech evaluation model training device provided in the embodiments of the present application.

[0223] For example, Fig. 9 FIG. 1 is a schematic diagram showing the composition of a spoken language evaluation device provided in an embodiment of the present application. Fig. 9 As shown, the embodiment of the present application provides a spoken language evaluation device 900, comprising:

[0224] The acquisition module 910 is used to obtain the user's basic information, the voice to be evaluated, and the historical evaluation results.

[0225] Processing module 920 is used to obtain a speech score based on the speech to be evaluated, where the speech score represents the speech score of the speech to be evaluated; and, based on the text to be evaluated, obtain a text score, where the text to be evaluated is obtained based on the text information in the speech to be evaluated, and the text score represents the score of the text accuracy of the speech to be evaluated; and, based on the speech score and the text score, obtain the current evaluation result; and, input the basic information, historical evaluation results, and current evaluation results into the oral evaluation model to output oral evaluation information of the speech to be evaluated, where the oral evaluation information represents the evaluation results of the speech to be evaluated in multiple dimensions.

[0226] In one possible implementation, the oral assessment information includes at least one of an assessment report and learning suggestions. The assessment report represents changes in the user's oral ability from multiple dimensions, and the multiple dimensions include voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy, and the learning suggestions represent suggestions for the user on oral learning.

[0227] In a possible implementation, the processing module 920 is specifically configured to: input the text to be evaluated into the spoken language evaluation model to output a text score.

[0228] In one possible implementation, the processing module 920 is specifically used to: extract features of the speech to be evaluated to obtain acoustic features of the speech to be evaluated, where the acoustic features represent features of the speech to be evaluated in an acoustic dimension, where the acoustic dimension includes at least one of sound quality and rhythm; and input the acoustic features into a speech evaluation model to output a speech score.

[0229] In a possible implementation, the evaluation result includes at least one of a voice score, a text score, and a comprehensive score, and the comprehensive score is obtained by weighting the voice score and the text score.

[0230] In a possible implementation, the speech to be evaluated is obtained based on the user completing the test questions to be evaluated. The processing module 920 is also used to: determine the user's oral proficiency level based on the user's basic information and historical evaluation results; determine the difficulty of the questions corresponding to the user based on the oral proficiency level; and match the test questions from the exercise database based on the difficulty of the questions.

[0231] In one possible implementation, the processing module 920 is specifically used to: add the user's basic information and historical evaluation results to a first prompt word template to obtain first prompt information, where the first prompt word template is used to guide the generation process of the oral evaluation model; and input the first prompt information into the oral evaluation model to output the user's oral ability level.

[0232] In a possible implementation, the processing module 920 is specifically used to: generate the test questions by the oral evaluation model according to the difficulty of the questions; or, obtain the test questions by matching from the exercise database according to the difficulty of the questions.

[0233] In a possible implementation, the processing module 920 is specifically used to: add the difficulty of the question to the second prompt word template to obtain the second prompt information, the second prompt word template is used to guide the generation process of the oral evaluation model; input the second prompt information into the oral evaluation model to output the question to be evaluated.

[0234] In a possible implementation, the processing module 920 is further configured to: based on the topic to be evaluated and in combination with the digital human, provide the topic to be evaluated to the user and obtain the user's voice to be evaluated during the interaction with the user.

[0235] In a possible implementation, the processing module 920 is specifically used to: add the question to be evaluated in the third prompt word template to obtain the third prompt information, where the third prompt word template is used to guide the generation process of the oral evaluation model; input the third prompt information into the oral evaluation model to output the live stream information of the question to be evaluated; interact with the user through the digital human to provide the live stream information and obtain the user's voice to be evaluated.

[0236] In one possible implementation, the processing module 920 is specifically used to: in the fourth prompt word template, basic information, historical evaluation results, and current evaluation results, obtain fourth prompt information, and the fourth prompt word template is used to guide the generation process of the oral evaluation model; input the fourth prompt information into the oral evaluation model to output the oral evaluation information of the speech to be evaluated.

[0237] In a possible implementation, the processing module 920 is further used to: divide the difficulty of the questions to obtain at least one question difficulty; construct a set of exercise examples for each question difficulty; input each question difficulty and the corresponding exercise example set into the oral evaluation model to output a number of questions corresponding to each question difficulty; and construct an exercise database based on the number of questions corresponding to each question difficulty. The processing module 920 is specifically used to: determine the corresponding question difficulty based on the question difficulty; select a number of candidate questions corresponding to the question difficulty from the exercise database based on the question difficulty; and select a question to be evaluated from the number of candidate questions.

[0238] In a possible implementation, the processing module 920 is further configured to: based on the topic to be evaluated, the oral evaluation model provides the user with the topic to be evaluated and obtains the user's speech to be evaluated during the interaction with the user.

[0239] For example, Fig.10 The figure shows a schematic diagram of the composition of a training device for a spoken language evaluation model provided in an embodiment of the present application. Fig.10 As shown, the embodiment of the present application provides a training device 1000 for a spoken language assessment model, comprising:

[0240] The first data set construction module 1010 is used to construct a spoken language evaluation data set. The spoken language evaluation data set includes a user's spoken language evaluation sample and a spoken language evaluation information tag. The spoken language evaluation sample includes the user's basic information and several evaluation results. The spoken language evaluation information tag indicates the evaluation results of the spoken language evaluation sample in multiple dimensions. The multiple dimensions include a speech dimension and / or a text dimension. The speech dimension includes at least one of pronunciation, vocabulary, fluency, and speed. The text speech dimension includes at least one of answer accuracy and word accuracy.

[0241] The first training module 1020 is used to take the basic information of the user and several evaluation results of the user as the input of the oral evaluation model to be trained, and output the predicted result of the oral evaluation information of the user; obtain the evaluation loss value according to the oral evaluation information label and the predicted result of the oral evaluation information; and adjust the parameters of the oral evaluation model to be trained according to the evaluation loss value to obtain a trained oral evaluation model.

[0242] In a possible implementation, the spoken language evaluation model to be trained is obtained by adding a low-rank matrix layer to the output layer of several modules in the pre-trained large model, and the several modules are neural network modules in the pre-trained large model for implementing specific functions. The first training module 1020 is specifically used to: adjust the parameters of the low-rank matrix layer according to the evaluation loss value to obtain the trained spoken language evaluation model.

[0243] In one possible implementation, the first data set construction module 1010 is specifically used to: design a seed data set, the seed data set includes an evaluation sample example and its corresponding evaluation information example, the seed data set is an example of generating a spoken evaluation data set, the evaluation sample example is an example of generating a spoken evaluation sample, and the evaluation information sample is an example of generating a spoken evaluation information label; input the seed data set as a sample generation model, so that the sample generation model generates a spoken evaluation sample according to the evaluation sample example, and generates a spoken evaluation information label according to the evaluation information sample; obtain a spoken evaluation data set according to the spoken evaluation sample and its corresponding spoken evaluation information label.

[0244] In one possible implementation, the oral assessment information tag includes at least one of an assessment report sub-tag and a learning suggestion sub-tag, the assessment report sub-tag represents the changes in the user's oral ability from multiple dimensions, the multiple dimensions including voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy; the learning suggestion sub-tag represents suggestions for the user on oral learning.

[0245] In a possible implementation, the spoken language evaluation data set further includes a second data subset and a third data subset, the second data subset includes text evaluation samples and text scoring labels, and the third data subset is obtained according to the general task data set.

[0246] For example, Fig.11 FIG. 1 is a schematic diagram showing the composition of a training device for a speech evaluation model provided in an embodiment of the present application. Fig.11 As shown, the embodiment of the present application provides a training device 1100 for a speech evaluation model, comprising:

[0247] The second data set construction module 1110 is used to construct a speech evaluation data set.

[0248] The second training module 1120 is used to train the above-mentioned speech evaluation model based on the speech evaluation data set.

[0249] In a possible implementation, the speech evaluation data set includes acoustic feature samples and speech scoring labels. The second data set construction module 1110 is specifically used to: obtain speech samples and speech scoring labels; and perform feature extraction on the speech samples to obtain acoustic feature samples.

[0250] In one possible implementation, Fig. 9 , Fig.10 and Fig.11 The multiple functional modules shown in can be implemented by software or hardware. Exemplarily, the implementation of the processing module 920 is described below by taking the processing module 920 as an example. Similarly, the implementation of other functional modules in the acquisition module 910, the training device 1000 of the spoken language evaluation model, and the training device 1100 of the speech evaluation model can refer to the implementation of the processing module 920.

[0251] As an example of a software functional unit, the processing module 920 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the processing module 920 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0252] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0253] As an example of a hardware functional unit, the processing module 920 may include at least one computing device, such as a server, etc. Alternatively, the processing module 920 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0254] The multiple computing devices included in the processing module 920 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 920 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the processing module 920 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0255] It should be noted that, in other embodiments, the above-mentioned device (i.e., any one of the oral evaluation device 900, the oral evaluation model training device 1000, and the speech evaluation model training device 1100) may be additionally provided with one or more modules for executing any steps included in the above-mentioned implementation method, and the above-mentioned device is also referred to as a related device of the oral evaluation method. The steps that one or more modules in the above-mentioned device are responsible for implementing can be specified as needed. The functions of the above-mentioned device can also be implemented using more or fewer modules than the embodiments of the present application. One or more modules in the above-mentioned device are used to respectively implement different steps in the above-mentioned method, thereby realizing all the functions of the above-mentioned device.

[0256] The present application also provides a computing device 1200. Fig.12 As shown, the computing device 1200 includes: a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. The processor 1204, the memory 1206, and the communication interface 1208 communicate through the bus 1202. The computing device 1200 can be a server, such as a central server, an edge server, or a local server in a local data center, or an electronic device such as a desktop computer, a laptop computer, or a smart phone. It should be understood that the present application does not limit the number of processors and memories in the computing device 1200.

[0257] The bus 1202 may be a peripheral component interconnect (PCI) bus, a peripheral component interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. Among them, the unified bus is also called a Lingqu bus. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 The bus 1204 may include a path for transmitting information between various components of the computing device 1200 (eg, the memory 1206, the processor 1204, and the communication interface 1208).

[0258] Processor 1204 may include any one or more computing devices such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP), ASIC, FPGA, CPLD, NPU, SoC, offload card, acceleration card, etc.

[0259] The memory 1206 may include a volatile memory, such as a random access memory (RAM). The processor 1204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). In addition, the memory 1206 may also be implemented by a storage class memory (SCM), a phase change memory (PCM), or other types of storage media.

[0260] It is worth noting that the same type of storage medium can be configured in the same computing device to implement the function of memory 1206, or two or more types of storage media can be configured to implement the function of memory 1206, and this application does not limit this.

[0261] The memory 1206 stores executable program codes, and the processor 1204 executes the executable program codes to respectively implement the functions of the aforementioned one or more modules, thereby implementing the method described in the above embodiment. That is, the memory 1206 stores instructions for executing the method described in the above embodiment.

[0262] Alternatively, the memory 1206 stores executable codes, and the processor 1204 executes the executable codes to respectively implement the functions of the above-mentioned devices, thereby implementing the methods described in the above-mentioned embodiments. That is, the memory 1206 stores instructions for executing the methods described in the above-mentioned embodiments.

[0263] The communication interface 1208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1200 and other devices or communication networks.

[0264] As a possible implementation, the computing device 1200 may also include a chip system, the chip system includes a processor and a power supply circuit, the power supply circuit is used to power the processor, and the processor is used to execute the operation steps corresponding to the method of the embodiment of the present application. For the sake of brevity, it is not repeated here. Among them, the processor can be implemented by a GPU, and can also be implemented by a computing device or AI chip such as a DPU, NPU, XPU, SoC, offload card, acceleration card, etc.

[0265] As a possible implementation, the computing device 1200 may include multiple types of processors 1204, that is, the computing device 1200 is a heterogeneous device, for example, the computing device 1200 includes a CPU and a GPU, and at least one of the processors 1204 may perform the operation steps corresponding to the method of the embodiment of the present application. For the sake of brevity, it will not be repeated here.

[0266] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be an electronic device such as a desktop computer, a laptop computer, or a smart phone.

[0267] like Fig.13 As shown, the computing device cluster includes at least one computing device 1200. The memory 1206 in one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the method described in the above embodiment.

[0268] In some possible implementations, the memory 1206 of one or more computing devices 1200 in the computing device cluster may also store partial instructions for executing the method described in the above embodiment. In other words, the combination of one or more computing devices 1200 may jointly execute instructions for executing the method described in the above embodiment.

[0269] It should be noted that the memory 1206 in different computing devices 1200 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the above-mentioned device. That is, the instructions stored in the memory 1206 in different computing devices 1200 can implement the functions of one or more modules in the above-mentioned device.

[0270] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.14 A possible implementation is shown. Fig.14 As shown, two computing devices 1200A and 1200B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1206 in the computing device 1200A stores instructions for the functions of one or more modules in the above-mentioned device. At the same time, the memory 1206 in the computing device 1200B stores instructions for the functions of another one or more modules in the above-mentioned device.

[0271] It should be understood that Fig.14The functions of the computing device 1200A shown in FIG. 1200A may also be completed by multiple computing devices 1200. Similarly, the functions of the computing device 1200B may also be completed by multiple computing devices 1200.

[0272] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.13 and Fig.14 The connection mode of the computing device cluster is different in that the memory 1206 in one or more computing devices 1200 in the computing device cluster may store the same instructions for executing the method in the above embodiment.

[0273] In some possible implementations, the memory 1206 of one or more computing devices 1200 in the computing device cluster may also store some instructions for executing the aforementioned data processing method. In other words, the combination of one or more computing devices 1200 may jointly execute instructions for executing the aforementioned method.

[0274] In addition to the above method and electronic device, the embodiment of the present application may also provide a computer program product, which includes computer program instructions, which, when executed by a processor, cause the processor to execute the steps in the method according to various embodiments of the present application described in the above "Method" section of this specification. Wherein, the computer program product can be written in any combination of one or more programming languages ​​to perform computer program code for performing the operation of the embodiment of the present application, and the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or similar programming language. Wherein, the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer program code can be executed completely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device, partially on the remote computing device, or completely on the remote computing device or server. In some examples, when the computer program instruction is executed by a computing device cluster comprising at least one computing device, at least one computing device in the computing device cluster is caused to execute the method in the above embodiment.

[0275] In addition, the embodiment of the present application can also provide a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when executed by the processor, cause the processor to execute the steps in the method according to various embodiments of the present application described in the above "Method" section of this specification. The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals. In some examples, when the instructions are executed by a computing device cluster including at least one computing device, at least one computing device in the computing device cluster executes the method in the above embodiments.

[0276] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0277] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0278] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0279] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0280] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0281] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0282] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0283] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0284] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0285] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A spoken language assessment method, characterized in that: Applied to a server, the spoken language evaluation method comprises: Obtain the user's basic information, the voice to be evaluated, and historical evaluation results; Based on the speech to be evaluated, a speech score is obtained, wherein the speech score represents the score of the speech to be evaluated in terms of speech; Based on the text to be evaluated, a text score is obtained, and based on the voice score and the text score, a result of this evaluation is obtained, the text to be evaluated is obtained according to the voice to be evaluated, and the text score represents a score of the text accuracy of the voice to be evaluated; The basic information, the historical evaluation results, and the current evaluation results are input into a spoken language evaluation model to output spoken language evaluation information of the speech to be evaluated, wherein the spoken language evaluation information represents evaluation results of the speech to be evaluated in multiple dimensions.

2. The method according to claim 1, characterized in that: The oral evaluation information includes at least one of an evaluation report and learning suggestions, the evaluation report represents the changes in the user's oral ability from multiple dimensions, the multiple dimensions including voice dimension and / or text dimension; the voice dimension includes at least one of pronunciation, vocabulary, fluency, and speed; the text voice dimension includes at least one of answer accuracy and word accuracy; the learning suggestions represent suggestions for the user in oral learning.

3. The method according to claim 1 or 2, characterized in that: The step of obtaining a text score based on the text to be evaluated includes: The text to be evaluated is input into the spoken language evaluation model to output the text score.

4. The method according to any one of claims 1 to 3, characterized in that: The step of obtaining a speech score based on the speech to be evaluated includes: Extracting features of the speech to be evaluated to obtain acoustic features of the speech to be evaluated, wherein the acoustic features represent features of the speech to be evaluated in an acoustic dimension, wherein the acoustic dimension includes at least one of sound quality and prosody; The acoustic features are input into a speech evaluation model to output the speech score.

5. The method according to any one of claims 1 to 4, characterized in that: The speech to be evaluated is obtained based on the user completing the question to be evaluated; the method further includes: Determining the user's oral proficiency level according to the user's basic information and the historical evaluation results; According to the oral proficiency level, determining the difficulty of the questions corresponding to the user; The question to be evaluated is determined according to the difficulty of the question.

6. The method according to any one of claims 1 to 5, characterized in that: The step of inputting the basic information, the historical evaluation results, and the current evaluation results into a spoken language evaluation model to output spoken language evaluation information of the speech to be evaluated includes: In the prompt word template, the basic information, the historical evaluation results, and the current evaluation results are added to obtain prompt information, and the prompt word template is used to guide the generation process of the oral evaluation model; The prompt information is input into the spoken language evaluation model to output the spoken language evaluation information of the speech to be evaluated.

7. A method for training a spoken language assessment model, characterized in that: include: Constructing a spoken language evaluation data set, wherein the spoken language evaluation data set includes a user's spoken language evaluation sample and a spoken language evaluation information label, wherein the spoken language evaluation sample includes the user's basic information and several evaluation results, and the spoken language evaluation information label indicates the evaluation results of the spoken language evaluation sample in multiple dimensions, wherein the multiple dimensions include a speech dimension and / or a text dimension, wherein the speech dimension includes at least one of pronunciation, vocabulary, fluency, and speed; and the text speech dimension includes at least one of answer accuracy and word accuracy; Taking the basic information of the user and several evaluation results thereof as inputs of the oral evaluation model to be trained, and outputting the prediction results of the oral evaluation information of the user; Obtaining an evaluation loss value according to the spoken language evaluation information label and the spoken language evaluation information prediction result; According to the evaluation loss value, the parameters of the spoken language evaluation model to be trained are adjusted to obtain a trained spoken language evaluation model.

8. The method according to claim 7, characterized in that The spoken language evaluation model to be trained is obtained by adding a low-rank matrix layer to the output layers of several modules in the pre-trained large model, wherein the several modules are neural network modules in the pre-trained large model for realizing specific functions; The step of adjusting the parameters of the spoken language evaluation model to be trained according to the evaluation loss value to obtain the trained spoken language evaluation model comprises: According to the evaluation loss value, the parameters of the low-rank matrix layer are adjusted to obtain a trained spoken language evaluation model.

9. The method according to claim 7 or 8, characterized in that: The step of constructing a spoken language evaluation dataset includes: Designing a seed data set, wherein the seed data set includes an evaluation sample example and its corresponding evaluation information example, the seed data set is an example for generating the spoken language evaluation data set, the evaluation sample example is an example for generating the spoken language evaluation sample, and the evaluation information example is an example for generating the spoken language evaluation information label; Inputting the seed data set into a sample generation model, so that the sample generation model generates the spoken language evaluation sample according to the evaluation sample example, and generates the spoken language evaluation information label according to the evaluation information example; The spoken language evaluation data set is obtained according to the spoken language evaluation samples and the corresponding spoken language evaluation information labels.

10. A server, characterized in that: The server is deployed with a spoken language assessment model, and the server is used to execute the method described in any one of claims 1-9.

Images