Speech evaluation method, speech evaluation device, electronic device, and storage medium

Abstract

A speech evaluation method, a speech evaluation device, an electronic device, and a storage medium. The speech evaluation method includes: obtaining to-be-evaluated audio and a subject corresponding to the to-be-evaluated audio; and outputting a score of semantic correlation between the to-be-evaluated audio and the subject. The score is determined by: performing speech recognition processing on the to-be-evaluated audio to obtain to-be-evaluated text corresponding to the to-be-evaluated audio; performing semantic correlation analysis on the to-be-evaluated text and the subject to obtain semantic similarity between the to-be-evaluated text and the subject; and calculating the score based on the semantic similarity between the to-be-evaluated text and the subject. The speech evaluation method can improve the accuracy, comprehensiveness, and scientificity of speech evaluation.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a speech evaluation method, a speech evaluation device, an electronic device, and a storage medium. Background Art

[0002] In the field of artificial intelligence, speech recognition and natural language processing technologies are rapidly developing and have been widely used in various scenarios, including voice assistants, intelligent customer service, and online education. Speech recognition technology, also known as automatic speech recognition, converts speech signals into text, and therefore plays a vital role in speech evaluation. Speech evaluation involves the process of evaluating and analyzing speech quality, pronunciation accuracy, fluency, intonation, and other aspects, helping users identify pronunciation issues.

[0003] With advances in speech-to-text technology, combined with text analysis and scoring techniques, many scenarios have begun to rely on automatic speech-to-text processing and speech evaluation. For example, in customer service, call quality can be scored by analyzing the text converted from the speech in a call; in language learning, learners' pronunciation and expression can be evaluated based on speech-to-text results.

[0004] However, due to the diversity of speech expressions and the complexity of text processing, achieving accurate, efficient and semantically deep scoring remains a major challenge. Summary of the Invention

[0005] At least one embodiment of the present disclosure provides a speech evaluation method, which includes: obtaining audio to be evaluated and a topic corresponding to the audio to be evaluated; and outputting a score of the semantic relevance between the audio to be evaluated and the topic, wherein determining the score includes: performing speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated; performing semantic relevance analysis on the text to be evaluated and the topic to obtain semantic similarity between the text to be evaluated and the topic; and calculating the score based on the semantic similarity between the text to be evaluated and the topic.

[0006] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, performing speech recognition processing on the audio to be evaluated to obtain the text to be evaluated corresponding to the audio to be evaluated includes: inputting the audio to be evaluated into an end-to-end model; and performing the speech recognition processing on the audio to be evaluated through the end-to-end model, and obtaining the text to be evaluated from the output of the end-to-end model.

[0007] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the end-to-end model includes a first neural network and a second neural network, and the speech recognition processing is performed on the audio to be evaluated by the end-to-end model, and the text to be evaluated is obtained from the output of the end-to-end model, including: extracting features of the data of the audio to be evaluated by the input layer of the first neural network to obtain a word vector; inputting the word vector into the hidden layer of the first neural network, and performing weighted processing on the word vector through the hidden layer to obtain a semantic feature corresponding to the word vector; inputting the semantic feature into the bias layer of the first neural network, and adjusting the semantic feature through the bias layer to obtain activation data; inputting the activation data into the second neural network, and performing time series analysis on the activation data through the second neural network to obtain a time series dependency; and determining the text to be evaluated based on the time series dependency.

[0008] For example, in the speech evaluation method provided by at least one embodiment of the present disclosure, the second neural network includes a recurrent neural network, a long short-term memory network, or a gated recurrent unit network.

[0009] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the semantic correlation analysis of the text to be evaluated and the topic is performed to obtain the semantic similarity between the text to be evaluated and the topic, including: processing the text to be evaluated through a semantic analysis model to obtain a text vector including semantic information; processing the topic through the semantic analysis model to obtain a topic vector including semantic information; and calculating the semantic similarity between the text to be evaluated and the topic based on the text vector and the topic vector.

[0010] For example, the speech evaluation method provided by at least one embodiment of the present disclosure further includes: performing word segmentation processing on the text to be evaluated to obtain multiple word blocks and parts of speech of the multiple word blocks; and performing sentence segmentation processing on the text to be evaluated based on the multiple word blocks and parts of speech of the multiple word blocks to obtain multiple sentences.

[0011] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the text to be evaluated is segmented based on the multiple word blocks and the parts of speech of the multiple word blocks to obtain multiple sentences, including: determining multiple text features corresponding to the multiple word blocks based on a sequence format, wherein the sequence format includes the current word block, the position of the current word block, the part of speech of the current word block, the previous word block, the part of speech of the previous word block, the next word block and the part of speech of the next word block; mapping the multiple text features into multiple feature vectors, inputting the multiple feature vectors into a binary classification model, processing the multiple feature vectors through the binary classification model, and outputting a processing result for determining whether to segment the text; and determining the multiple sentences based on the processing result.

[0012] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the text vector includes multiple word block vectors and multiple sentence vectors.

[0013] The calculating of the semantic similarity between the text to be evaluated and the topic based on the text vector and the topic vector includes: calculating the semantic similarity between the text to be evaluated and the topic based on the multiple word block vectors, the multiple sentence vectors and the topic vector.

[0014] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the semantic similarity between the text to be evaluated and the topic is calculated based on the multiple word block vectors, the multiple sentence vectors and the topic vector, including: calculating the cosine similarity between the multiple word block vectors and the topic vector to obtain a first correlation index; calculating the cosine similarity between the multiple sentence vectors and the topic vector to obtain a second correlation index; and calculating the semantic similarity between the text to be evaluated and the topic based on the first correlation index and the second correlation index.

[0015] For example, the speech evaluation method provided by at least one embodiment of the present disclosure further includes: calculating the cosine similarity between two adjacent sentences in the multiple sentences based on the multiple sentence vectors to obtain a third correlation index, wherein the score is obtained by calculating the semantic similarity between the text to be evaluated and the topic, including: calculating the score based on the first correlation index, the second correlation index and the third correlation index.

[0016] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the score is calculated based on the first correlation index, the second correlation index and the third correlation index, including: normalizing the first correlation index, the second correlation index and the third correlation index; and weighting the normalized first correlation index, the second correlation index and the third correlation index to obtain the score.

[0017] For example, in the speech evaluation method provided in at least one embodiment of the present disclosure, the semantic analysis model includes a word embedding model or a context-related model, wherein the word embedding model includes a word vector model or a global logarithmic bilinear regression model, and the context-related model includes a bidirectional encoding representation model from a transformer.

[0018] At least one embodiment of the present disclosure further provides a speech evaluation device, comprising: an input module, an output module, and a processing module, wherein the input module is configured to obtain an audio to be evaluated and a topic corresponding to the audio to be evaluated; the output module is configured to output a score of the correlation between the audio to be evaluated and the topic; the processing module comprises a speech recognition submodule, a correlation analysis submodule, and a scoring submodule, wherein the speech recognition submodule is configured to perform speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated; the correlation analysis submodule is configured to perform semantic correlation analysis on the text to be evaluated and the topic to obtain a semantic similarity between the text to be evaluated and the topic; and the scoring submodule is configured to calculate the score based on the semantic similarity between the text to be evaluated and the topic.

[0019] For example, in the speech evaluation device provided in at least one embodiment of the present disclosure, the speech recognition submodule includes a speech recognition unit, which is configured to input the audio to be evaluated into an end-to-end model, and perform the speech recognition processing on the audio to be evaluated through the end-to-end model, and obtain the text to be evaluated from the output of the end-to-end model.

[0020] For example, in the speech evaluation device provided in at least one embodiment of the present disclosure, the relevance analysis submodule includes a semantic analysis unit and a relevance calculation unit, the semantic analysis unit is configured to process the text to be evaluated through a semantic analysis model to obtain a text vector including semantic information, and to process the topic through the semantic analysis model to obtain a topic vector including semantic information; the relevance calculation unit is configured to calculate the semantic similarity between the text to be evaluated and the topic based on the text vector and the topic vector.

[0021] For example, the speech evaluation device provided by at least one embodiment of the present disclosure further includes: a text processing sub-module, which is configured to perform word segmentation processing on the text to be evaluated to obtain multiple word blocks and parts of speech of the multiple word blocks, and to perform sentence segmentation processing on the text to be evaluated based on the multiple word blocks and parts of speech of the multiple word blocks to obtain multiple sentences.

[0022] For example, in the speech evaluation device provided in at least one embodiment of the present disclosure, the text vector includes multiple word block vectors and multiple sentence vectors, and the correlation calculation unit is further configured to: calculate the cosine similarity between the multiple word block vectors and the topic vector to obtain a first correlation index; calculate the cosine similarity between the multiple sentence vectors and the topic vector to obtain a second correlation index; and calculate the cosine similarity between two adjacent sentences in the multiple sentences based on the multiple sentence vectors to obtain a third correlation index.

[0023] For example, in the speech evaluation device provided in at least one embodiment of the present disclosure, the scoring submodule includes a normalization processing unit and a scoring unit, the normalization processing unit is configured to perform normalization processing on the first correlation index, the second correlation index and the third correlation index; the scoring unit is configured to perform weighted processing on the normalized first correlation index, the second correlation index and the third correlation index to obtain the score.

[0024] At least one embodiment of the present disclosure further provides an electronic device, comprising: at least one memory storing computer-executable instructions non-transiently; and at least one processor configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the speech evaluation method described in any of the above embodiments.

[0025] At least one embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by at least one processor, the speech evaluation method described in any of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0027] Figure 1 A flowchart of a speech evaluation method provided by at least one embodiment of the present disclosure;

[0028] Figure 2A schematic diagram of an exemplary subject and a text to be evaluated provided for at least one embodiment of the present disclosure;

[0029] Figure 3 A schematic diagram of performing text processing on a text to be evaluated provided in at least one embodiment of the present disclosure;

[0030] Figure 4 A schematic diagram of a speech evaluation process provided by at least one embodiment of the present disclosure;

[0031] Figure 5 A schematic diagram of the structure of a speech evaluation device provided by at least one embodiment of the present disclosure;

[0032] Figure 6 A schematic structural diagram of another speech evaluation device provided by at least one embodiment of the present disclosure;

[0033] Figure 7 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure;

[0034] Figure 8 A schematic block diagram of an electronic device provided in accordance with at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0036] Unless otherwise defined, technical or scientific terms used in this disclosure should have the ordinary meanings understood by a person of ordinary skill in the art to which this disclosure belongs. The terms "first," "second," and similar terms used in this disclosure do not denote any order, quantity, or importance, but are simply used to distinguish different components. Terms such as "include" or "comprising" mean that the element or object preceding the term includes the elements or objects listed after the term, and their equivalents, without excluding other elements or objects. Terms such as "connected" or "connected" are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly. It should be understood that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit steps shown. The scope of this disclosure is not limited in this respect.

[0037] The present disclosure is described below using several specific embodiments. To maintain clarity and conciseness in the following description of the embodiments of the present disclosure, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, that component is represented by the same or similar reference numeral in each drawing.

[0038] Speech assessment technology can be used to convert audio into text using speech recognition technology. This text is then matched with reference text to achieve speech assessment. For example, the input speech signal is preprocessed, including denoising and filtering. Feature extraction is then performed to convert the speech signal into a feature vector. Common features include Mel-Frequency Cepstral Coefficients (MFCCs). The feature vector is then processed and recognized using an acoustic model and a language model. The acoustic model identifies the acoustic features of the speech, while the language model corrects and optimizes the recognition results based on the language's vocabulary and grammar, ultimately outputting the recognized text. Finally, the text generated by speech recognition is matched with the corresponding reference text. Based on the matching results, the audio is scored to assess the user's pronunciation accuracy, intonation, fluency, and other aspects.

[0039] The inventors of the present disclosure found that in the text matching stage of speech evaluation, scoring relies more on keyword matching and simple grammatical rule analysis, which makes it difficult to deeply explore the semantic information of the text. Moreover, for scenarios where there is no reference text as a standard answer or open-ended questions and answers, the overall semantics of the audio cannot be scored. The scoring method is single and the accuracy is low.

[0040] In this regard, at least one embodiment of the present disclosure provides a speech evaluation method, a speech evaluation device, an electronic device and a storage medium. By performing speech recognition on the audio to be evaluated, a text to be evaluated corresponding to the audio to be evaluated is obtained, and then semantic analysis is performed on the text to be evaluated. The correlation between the audio to be evaluated and the topic is evaluated based on the semantics of the text to be evaluated, and a score corresponding to the audio to be evaluated is given. This can effectively reflect the degree of fit between the audio to be evaluated and the topic, and improve the accuracy, comprehensiveness and scientificity of the speech evaluation.

[0041] Figure 1 This is a flowchart of a speech evaluation method provided in at least one embodiment of the present disclosure. For example, the speech evaluation method can be applied to a computing device, which includes any electronic device with computing capabilities, such as a smartphone, laptop, tablet computer, desktop computer, server, etc., although the embodiments of the present disclosure are not limited thereto.

[0042] like Figure 1 As shown, the speech evaluation method provided by at least one embodiment of the present disclosure includes the following steps S101 and S102.

[0043] Step S101: Acquire the audio to be evaluated and the subject corresponding to the audio to be evaluated.

[0044] The audio to be evaluated can be understood as audio that needs to be evaluated or scored. For example, the audio to be evaluated can be audio recorded when a user is practicing speaking or taking an exam, or it can be audio collected in, for example, customer service or other situational conversations. For example, the audio to be evaluated can be voice collected by the audio collection device of the client, or it can be voice obtained from a network server. For example, the client includes but is not limited to smartphones, tablets, personal computers, wearable devices, head-mounted display devices, car-mounted devices, etc., and the audio collection device includes but is not limited to a built-in or external microphone of the client. The audio to be evaluated can be pre-recorded or recorded in real time, and the embodiments of the present disclosure do not limit this.

[0045] In the embodiments of the present disclosure, the subject corresponding to the audio to be evaluated may be a question provided to the user for voice testing, or a product concept required to be introduced in the voice service, or a core topic extracted from reference text or audio content.

[0046] For example, in one example, the oral practice software provides a topic question bank, which includes multiple pre-set candidate topics. The user can select a target topic from the multiple candidate topics in the topic question bank as the object of oral training, and record the audio to be evaluated corresponding to the target topic. For example, in another example, the oral examination software specifies a topic for the examinee, and the examinee gives an oral answer based on the specified topic. The oral examination software collects the examinee's answer audio as the audio to be evaluated. For example, in another example, it is also possible to obtain the topic provided by the user and use the audio uploaded by the user as the audio to be evaluated. For example, in another example, it is also possible to first obtain the conversation audio, and then use the natural language processing technology to identify and analyze the conversation audio, obtain the core topic of the conversation audio as the topic in the embodiment of the present disclosure, and use the target audio segment in the conversation audio (for example, the audio segment corresponding to the customer service's answer) as the audio to be evaluated.

[0047] Step S102: Output the score of the semantic relevance between the audio to be evaluated and the topic.

[0048] Here, semantics can be understood as the meaning of words, sentences, or paragraphs, including, for example, the literal meaning, deeper meaning, referential relationship, and contextual meaning of language units such as words or sentences. For example, the semantics of the word "apple" can include "apple is a fruit." More information can be determined from the semantics of words, sentences, or paragraphs, and the meaning of the recognized text can be determined more accurately based on the semantics of the context. For example, the semantics of the sentences "It rained today" and "Did it rain today" both include "date - today" and "weather - raining," but according to the semantics of the sentences, the two sentences express different meanings.

[0049] In an embodiment of the present disclosure, the semantics of the audio to be evaluated and the semantics of the topic may be determined first, and the correlation between the two may be evaluated to obtain a score of the semantic correlation between the audio to be evaluated and the topic.

[0050] In at least one embodiment of the present disclosure, determining a score for the semantic relevance between the audio to be evaluated and the topic includes: performing speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated; performing semantic relevance analysis on the text to be evaluated and the topic to obtain a semantic similarity between the text to be evaluated and the topic; and calculating a score based on the semantic similarity between the text to be evaluated and the topic.

[0051] For example, in one example, the audio to be evaluated can be subjected to speech recognition processing through an acoustic model, a pronunciation dictionary, and a language model, thereby obtaining a text to be evaluated corresponding to the audio to be evaluated. The text to be evaluated contains the vocabulary and sentence structure of a natural language. For example, the acoustic model can be a Hidden Markov Model (HMM) model or a neural network model, and the language model can be an N-gram model or a deep learning model.

[0052] The inventors of the present disclosure further discovered that acoustic models and language models typically use shallow neural network models, and the conversion accuracy is limited by noise interference and language complexity. In addition, the models are trained independently of each other and are only spliced ​​together after training. This leads to deficiencies in the model's speech recognition capabilities and semantic analysis depth in noisy environments, making it difficult to meet the needs of complex scenarios. In this regard, in at least one example of the embodiments of the present disclosure, speech recognition processing is performed on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated. A specific example of this may include: inputting the audio to be evaluated into an end-to-end model; and performing speech recognition processing on the audio to be evaluated through the end-to-end model to obtain the text to be evaluated from the output of the end-to-end model.

[0053] An end-to-end model is a model architecture that directly maps input to output. The entire model, from input to output, is a complete, end-to-end trainable system. In the disclosed embodiments, the original speech waveform of the audio to be evaluated can be directly input. After a series of calculations and processing within the end-to-end model, the corresponding text transcription result (the text to be evaluated) is directly output. This eliminates the need for intermediate steps such as phoneme recognition and avoids the complex process of splicing acoustic and language models. Moreover, by training the end-to-end model as a whole, the model can be given more room for automatic adjustment, which is conducive to achieving efficient and accurate speech-to-text processing.

[0054] For example, in at least one embodiment of the present disclosure, an end-to-end model includes a first neural network and a second neural network, and speech recognition processing is performed on the audio to be evaluated through the end-to-end model, and the text to be evaluated is obtained from the output of the end-to-end model. A specific example may include: feature extraction of the data of the audio to be evaluated through the input layer of the first neural network to obtain a word vector; inputting the word vector into the hidden layer of the first neural network, performing weighted processing on the word vector through the hidden layer to obtain a semantic feature corresponding to the word vector; inputting the semantic feature into the bias layer of the first neural network, adjusting the semantic feature through the bias layer to obtain activation data; inputting the activation data into the second neural network, performing time series analysis on the activation data through the second neural network to obtain a time series dependency; and determining the text to be evaluated based on the time series dependency.

[0055] In the embodiments of the present disclosure, the first neural network includes but is not limited to convolutional neural networks. For example, the input layer of the first neural network is responsible for processing the raw data extracted from the audio during speech recognition, such as data after feature conversion using Mel-Frequency Cepstral Coefficients (MFCCs) or Filter Bank Features (Fbank). The input layer can decompose the speech signal into processable lexical features and convert them into vectors. The word vector of each word represents the semantic information of the word in the corpus. For example, the vector calculation process of the input layer is as follows:

[0056]

[0057] in, Represents the word vector of the i-th word, w i is the i-th word after speech conversion, and Word2Vec is used to map words into vectors.

[0058] Next, the data enters the hidden layer, the core of the neural network. The hidden layer contains multiple neurons (e.g., n). Each neuron is responsible for receiving information from the input layer and performing feature extraction and transformation. Each neuron generates an output signal through weighted calculation based on the word vectors in the input layer.

[0059] In the weight calculation layer, the vector of the input layer and each neuron of the hidden layer are weighted. The formula is as follows:

[0060]

[0061] Among them, h j represents the output of the jth neuron in the hidden layer, w ij represents the weight between the i-th neuron in the input layer and the j-th neuron in the hidden layer, x i is the word vector of the i-th word in the input layer, b j is the bias of the neuron, σ is the activation function, and the tanh function is usually used to process nonlinear features. Through this calculation process, the hidden layer neurons can extract complex semantic features from the word vectors of the input layer.

[0062] The output after weight calculation enters the bias layer, which provides an offset value for each neuron (the bias b of the neuron j ), which is used to adjust the activation state of each neuron, can make the output of the neuron more stable and help accelerate convergence. The formula is as follows:

[0063]

[0064] in, Represents the activation output after the bias layer. The tanh function compresses the neuron output to the range [-1, 1], thereby improving the nonlinear fitting ability of the neural network.

[0065] Next, the activation data enters the second neural network, which processes sequential data (such as speech signals or text). For example, this second neural network includes a recurrent neural network (RNN). In an RNN, the output at each moment depends not only on the current input but also on the output state at the previous moment. This allows RNNs to effectively capture temporal dependencies in sequential data.

[0066] Since RNNs are prone to gradient vanishing or exploding problems, a Long Short-Term Memory network (LSTM) or a Gated Recurrent Unit (GRU) structure can be used, both of which can better maintain long-term dependencies.

[0067] In LSTM and GRU, the flow of information is primarily controlled through gating mechanisms. GRU uses update and reset gates to control the updating and forgetting of information at each moment, while LSTM uses input, output, and forget gates to control the introduction, transmission, and suppression of information, respectively.

[0068] At a certain time t, the update mechanism of GRU can be expressed as:

[0069]

[0070] Among them, z t is the output of the update gate, x t is the input at the current moment, h t-1 is the hidden state of the previous moment, W z 、U z and b z is the parameter of the update gate, and σ is the activation function. The update gate determines the degree of influence of the current input and the hidden state at the previous moment on the current output.

[0071] GRU and LSTM can capture long-term and short-term dependencies in speech data, effectively perform sequence learning, and generate more accurate text for evaluation.

[0072] In the above-mentioned embodiments of the present disclosure, the audio to be evaluated can be directly converted into accurate text to be evaluated, avoiding the complicated process of splicing the acoustic model and the language model, and improving the training efficiency.

[0073] In order to accurately measure the semantic relevance between the text to be evaluated and the topic, the text to be evaluated and the topic semantics can be represented as high-dimensional vectors respectively. The topic semantics can be a set of semantic features related to a certain field or topic.

[0074] For example, natural language processing techniques can be used to embed words in the text to be evaluated, generating a text vector corresponding to the text to be evaluated. This text vector includes the text's vocabulary, grammar, and contextual information. Topic semantics also need to be converted into vectors, which can usually be extracted by analyzing large corpora or using pre-trained models.

[0075] For example, the text to be evaluated may be processed by a semantic analysis model to obtain a text vector including semantic information, and the topic may be processed by the semantic analysis model to obtain a topic vector including semantic information.

[0076] For example, word embedding models or context-sensitive models can be used to extract vector representations of the text and topic to be evaluated. Examples of word embedding models include but are not limited to the Word2Vec model or the global log bilinear regression model (GloVe). Context-sensitive models include but are not limited to Transformer-based pre-trained language models, such as the Bidirectional Encoder Representations from Transformers (BERT) model.

[0077] Finally, the correlation between the text to be evaluated and the topic semantics is calculated to obtain a score for the topic fit of the text to be evaluated. Calculating the correlation between the text to be evaluated and the topic semantics involves not only word matching between the two, but also contextual understanding and the calculation of topic relevance.

[0078] Cosine similarity can be used to quantify the relevance between the text to be evaluated and the topic semantics. For example, the semantic similarity between the text to be evaluated and the topic can be obtained by calculating the cosine similarity between the text vector and the topic vector. The calculation formula is as follows:

[0079]

[0080] in, is a text vector, is the topic vector, and Perform vector dot product, and are the modulo lengths of the text vector and topic vector respectively.

[0081] A higher cosine similarity value between a text vector and a topic vector indicates a stronger semantic correlation between the text being evaluated and the topic. For example, the cosine similarity between a text being evaluated and a topic ranges from -1 to 1. Values ​​closer to 1 indicate greater semantic similarity between the two, values ​​closer to -1 indicate less semantic correlation, and values ​​close to 0 indicate no significant correlation.

[0082] For example, the calculated cosine similarity between the text to be evaluated and the topic can be used as a relevance index, which can be further aggregated in the scoring module to ultimately derive an automatic score for the text to be evaluated. The scoring module can use multiple relevance indices, combined with methods such as weighted averaging and linear regression, to determine the final score for the audio to be evaluated.

[0083] In the above embodiment, the semantic relevance between the text to be evaluated and the topic is analyzed, which can efficiently and accurately evaluate the fit between the text to be evaluated and the topic, and elevate the scoring from keyword matching to the semantic level. The topic fit is used as one of the criteria for the final scoring of the audio to be evaluated, making the evaluation results more intelligent and improving the accuracy, comprehensiveness and scientificity of the speech evaluation.

[0084] When performing semantic correlation analysis on the evaluation text and the topic, not only the semantics of the words in the evaluation text and the words in the topic can be analyzed for correlation, but also the semantics of the sentences in the evaluation text and the words in the topic can be analyzed for correlation.

[0085] The inventors of the present disclosure noticed that compared with the more standard sentence structure and clear punctuation in the reference text, in spoken expression, people often omit some key components in the sentence or have multiple pauses in a sentence due to expression habits or thinking. This results in a large number of confusing punctuation marks in the text after speech conversion obtained based on methods such as phoneme boundary recognition of pause time. The sentences marked in this way may have reversed word order or be incomplete, which will affect the accuracy of subsequent scoring.

[0086] In this regard, after obtaining the text to be evaluated, the embodiments of the present disclosure can first perform text processing on the text to be evaluated to obtain accurate words and sentences, and then vectorize the words and sentences in the processed text to be evaluated to obtain the above-mentioned text vector.

[0087] The text processing includes word segmentation and sentence segmentation. For example, by performing word segmentation on the evaluation text, multiple word blocks and parts of speech of multiple word blocks can be obtained, and then the evaluation text is segmented based on the multiple word blocks and parts of speech of multiple word blocks to obtain multiple sentences.

[0088] In the embodiments of the present disclosure, a word chunk may include one or more words or lexicons, which are typically relatively fixed language units with specific meanings and functions. These words are often stored and retrieved as a whole during language usage, and have a high frequency of use and specific pragmatic functions. For example, a word chunk may be a fixed phrase or collocation in Chinese or English. The embodiments of the present disclosure do not limit the types of languages ​​or word chunks.

[0089] Based on the characteristics and parts of speech of word chunks, multiple characters can be accurately divided into sentences. For example, word chunk parts of speech include but are not limited to nouns, verbs, adverbs, adjectives, pronouns, prepositions, and conjunctions. Nouns often form subject-verb-object structures with verbs. Conjunctions can indicate the logical relationship between sentence components and serve as an important basis for sentence segmentation.

[0090] Figure 2 A schematic diagram of an exemplary subject and a text to be evaluated is provided for at least one embodiment of the present disclosure. Figure 2 As shown, the text to be evaluated obtained after performing speech recognition on the audio to be evaluated includes multiple consecutive characters A1, A2...A30. These characters are obtained after performing accurate speech recognition on the audio to be evaluated, and correspond one-to-one to each syllable in the audio to be evaluated.

[0091] Figure 3 This is a schematic diagram of text processing for a text to be evaluated provided in at least one embodiment of the present disclosure. Figure 3 As shown, a word segmentation tool can be used to divide the multiple characters in the text to be evaluated into multiple word blocks (e.g., W1, W2, W3, W4, W5, W6, etc.), and obtain the part of speech of each word block. Then, based on the multiple word blocks and the part of speech of the multiple word blocks, the text to be evaluated is segmented to obtain multiple sentences (e.g., S1, S2, etc.).

[0092] For example, in at least one embodiment of the present disclosure, a text to be evaluated is segmented based on multiple word blocks and the parts of speech of the multiple word blocks to obtain multiple sentences. A specific example may include: determining multiple text features corresponding to the multiple word blocks based on a sequence format, wherein the sequence format includes the current word block, the position of the current word block, the part of speech of the current word block, the previous word block, the part of speech of the previous word block, the next word block, and the part of speech of the next word block; mapping the multiple text features into multiple feature vectors, inputting the multiple feature vectors into a binary classification model, processing the multiple feature vectors through the binary classification model, and outputting a processing result for determining whether to segment the text; and determining multiple sentences based on the processing result.

[0093] For example, the text features of each chunk can be determined based on the sequence format of [current chunk, current chunk's position, current chunk's part of speech, previous chunk, previous chunk's part of speech, next chunk, and next chunk's part of speech]. If the chunk is the first in the text to be evaluated, the previous chunk can be set to a null value (e.g., null), and the previous chunk's part of speech is also set to a null value. If the chunk is the last word in the text to be evaluated, the next chunk can be set to a null value, and the next chunk's part of speech is also set to a null value. For example, in one example, the text features corresponding to chunk W1 are [W1, 1, W1's part of speech, null, null, W2, W2's part of speech], and the text features corresponding to chunk W2 are [W2, 2, W2's part of speech, W1, W1's part of speech, W3, W3's part of speech], and so on, to obtain the text features corresponding to all chunks. The above sequence format may also adopt other forms according to actual needs. The embodiments of the present disclosure do not limit the sequence format of text features.

[0094] Then, multiple text features are vectorized to obtain multiple feature vectors that can be processed by a computer. For example, each word chunk can be mapped to a numerical value based on a dictionary-to-numerical value mapping relationship. For example, the dictionary includes N word chunks, and these N word chunks correspond one-to-one to N numerical values. Similarly, the part of speech of each word chunk can also be mapped to a numerical value.

[0095] Multiple feature vectors are input into a pre-trained binary classification model, which then segments the feature vectors into sentences, thereby identifying multiple sentences. For example, if segmentation is required, a 1 is output; if not, a 0 is output. Since only the segmentation of the text to be evaluated is required, precise punctuation is not necessary. Therefore, classification models such as Support Vector Machines (SVMs) can be used to reduce computational overhead and improve execution speed.

[0096] The above-mentioned embodiments of the present disclosure can accurately segment the text to be evaluated through text processing, thereby enhancing the structuring of the text to be evaluated, solving the problem of distorted scoring caused by unclear sentences, and improving the accuracy of subsequent relevance calculation and scoring.

[0097] In the embodiments of the present disclosure, Figure 2 As shown, the topic can also be segmented, and then the word blocks T1, T2, and T3 in the processed topic are semantically vectorized to obtain a topic vector including semantic information.

[0098] For example, in at least one example of the embodiments of the present disclosure, after obtaining the text to be evaluated after text processing, semantic vectorization processing can be performed on multiple word blocks and multiple sentences in the text respectively to obtain multiple word block vectors and multiple sentence vectors including semantic information, thereby calculating the semantic similarity between the text to be evaluated and the topic based on the multiple word block vectors, multiple sentence vectors and topic vectors.

[0099] For example, we can filter out some stop words from the processed text to be evaluated, then extract each remaining word and map it to a semantic vector library to obtain a word chunk vector that includes semantic information. For example, stop words here can be words that are widely used in the language but lack actual semantic meaning, such as modal particles, interjections, or some adverbs and prepositions. Filtering stop words allows the system to focus more on key information, reducing computational effort and improving accuracy.

[0100] For example, a sentence vector including semantic information can be obtained based on semantic vectors corresponding to multiple words in a sentence, thereby enabling effective semantic comparison at the sentence level.

[0101] For example, in at least one embodiment of the present disclosure, the semantic similarity between the text to be evaluated and the topic is calculated based on multiple word block vectors, multiple sentence vectors and topic vectors. A specific example may include: calculating the cosine similarity between multiple word block vectors and topic vectors to obtain a first correlation index; calculating the cosine similarity between multiple sentence vectors and topic vectors to obtain a second correlation index; and calculating the semantic similarity between the text to be evaluated and the topic based on the first correlation index and the second correlation index.

[0102] For example, the cosine similarity value between each word chunk vector and the topic vector in multiple word chunk vectors can be calculated to obtain a similarity vector, which represents the semantic relevance of each word chunk between the text to be evaluated and the topic.

[0103] For example, in one example, a sentence in the text to be evaluated is "Autonomous driving technology has widespread applications in future transportation," and the topic is "autonomous driving technology." After filtering out stop words from the text, words such as "autonomous driving," "technology," "future," "transportation," "widespread," and "application" are extracted and converted into semantic vectors. Next, the similarity between these chunk vectors and the topic vocabulary of "autonomous driving technology" is calculated. For example, the similarity between "autonomous driving" and the topic "autonomous driving technology" is 0.9, and the similarity between "technology" and the topic "autonomous driving technology" is 0.85. Similarly, the resulting similarity vector SV = [0.9, 0.85, ...] is used to calculate the first relevance index. Similarly, the second relevance index is obtained by calculating the cosine similarity between the sentence vector and the topic vector of each sentence.

[0104] Then, a first score can be calculated based on the first and second relevance indices. For example, the first score reflects the semantic relevance between the audio to be evaluated and the topic. A higher first score indicates a higher semantic relevance between the text to be evaluated corresponding to the audio to be evaluated and the topic, and a higher degree of topical fit for the audio to be evaluated. A lower first score indicates a lower semantic relevance between the text to be evaluated corresponding to the audio to be evaluated and the topic, and a lower degree of topical fit for the audio to be evaluated.

[0105] For example, in another example, the paragraph vector or the overall vector of the text to be evaluated can be further obtained based on the sentence vector, and the semantic similarity between the text to be evaluated and the topic can be calculated based on the word block vector, sentence vector, paragraph vector, etc., thereby analyzing the correlation between the text to be evaluated and the topic from different levels.

[0106] While calculating the correlation between the text to be evaluated and the topic, the correlation between adjacent sentences in the text to be evaluated can also be calculated, thereby measuring the internal logical consistency of the text to be evaluated. For example, in at least one embodiment of the present disclosure, the cosine similarity between two adjacent sentences in the multiple sentences can also be calculated based on the multiple sentence vectors to obtain a third correlation index. The third correlation index can reflect the semantic correlation between adjacent sentences in the text to be evaluated. The higher the third correlation index, the better the coherence of the sentences.

[0107] The third relevance index can be used together with the first relevance index and the second relevance index to evaluate the semantic level score of the audio to be evaluated, thereby achieving a comprehensive quantitative analysis of the audio to be evaluated.

[0108] For example, the first correlation index, the second correlation index, and the third correlation index may be normalized, and then the normalized first correlation index, the second correlation index, and the third correlation index may be weighted to obtain a second score.

[0109] For example, the first correlation index, the second correlation index, and the third correlation index may be normalized to map the indexes to a uniform interval (eg, 0 to 1). The normalization formula is:

[0110]

[0111] Among them, Index raw is the original correlation index, Index norm is the normalized correlation index, Index min and Index max are the minimum and maximum values ​​of all calculated correlation indices, respectively. The normalized values ​​facilitate unified processing and eliminate the influence of data scale on the score.

[0112] For example, weighting coefficients may be assigned to the first correlation index, the second correlation index, and the third correlation index according to actual needs, and the embodiments of the present disclosure are not limited thereto. For example, in one example, the weighting coefficient of the first correlation index is 0.5, the weighting coefficient of the second correlation index is 0.3, and the weighting coefficient of the third correlation index is 0.2.

[0113] In the embodiments of the present disclosure, the scoring method is not limited to fixed rules or linear models. After calculating the score of the semantic relevance between the audio to be evaluated and the topic (for example, the first score or the second score), the score can be directly output, or it can be weighted with other scores to obtain the final score of the audio to be evaluated. Here, other scores can be scores for pronunciation accuracy, fluency, intonation, etc. of the audio to be evaluated, which are not limited in the embodiments of the present disclosure.

[0114] The speech evaluation method provided by the embodiment of the present disclosure has strong scalability, good flexibility and adaptability, and can effectively reflect the topic fit and semantic coherence of the speech.

[0115] Figure 4 This is a schematic diagram of a speech evaluation process provided by at least one embodiment of the present disclosure. Figure 4As shown, at least one embodiment of the present disclosure provides a full-process systematic solution from voice input to automatic output of scoring, which realizes a more efficient and accurate automatic scoring function of voice text by converting the audio to be evaluated into text to be evaluated, performing text processing and analysis on the text to be evaluated to obtain text semantics, and performing semantic correlation analysis between the text semantics and the topic semantics, and calculating the score based on the correlation index and automatically outputting the score.

[0116] At least one embodiment of the present disclosure further provides a speech evaluation device. Figure 5 A schematic diagram of the structure of a speech evaluation device provided in at least one embodiment of the present disclosure.

[0117] like Figure 5 As shown, the speech evaluation device 50 provided in at least one embodiment of the present disclosure includes an input module 100 , a processing module 200 and an output module 300 .

[0118] For example, the input module 100 is configured to obtain the audio to be evaluated and the topic corresponding to the audio to be evaluated. The output module 300 is configured to output the score of the association between the audio to be evaluated and the topic.

[0119] For example, the processing module 200 includes a speech recognition submodule 210 , a relevance analysis submodule 220 , and a scoring submodule 230 .

[0120] The speech recognition submodule 210 is configured to perform speech recognition processing on the audio to be evaluated to obtain the text to be evaluated corresponding to the audio to be evaluated; the relevance analysis submodule 220 is configured to perform semantic relevance analysis on the text to be evaluated and the topic to obtain the semantic similarity between the text to be evaluated and the topic; the scoring submodule 230 is configured to calculate the score based on the semantic similarity between the text to be evaluated and the topic.

[0121] For example, in the speech evaluation device 50 provided in at least one embodiment of the present disclosure, the speech recognition submodule 210 includes a speech recognition unit, which is configured to input the audio to be evaluated into the end-to-end model, and perform speech recognition processing on the audio to be evaluated through the end-to-end model, and obtain the text to be evaluated from the output of the end-to-end model.

[0122] For example, in the speech evaluation device 50 provided in at least one embodiment of the present disclosure, the relevance analysis submodule 220 includes a semantic analysis unit and a relevance calculation unit. The semantic analysis unit is configured to process the text to be evaluated through a semantic analysis model to obtain a text vector including semantic information, and to process the topic through a semantic analysis model to obtain a topic vector including semantic information; the relevance calculation unit is configured to calculate the semantic similarity between the text to be evaluated and the topic based on the text vector and the topic vector.

[0123] Figure 6 This is a structural diagram of another speech evaluation device provided by at least one embodiment of the present disclosure. Figure 6 As shown, the speech evaluation device 60 provided in at least one embodiment of the present disclosure includes an input module 100 , a processing module 200 and an output module 300 .

[0124] For example, the processing module 200 includes a speech recognition submodule 210, a relevance analysis submodule 220, a scoring submodule 230, and a text processing submodule 240. Figure 6 The relevant description of the input module 100, output module 300, speech recognition submodule 210, correlation analysis submodule 220 and scoring submodule 230 can be referred to Figure 5 The detailed description of the illustrated embodiments will not be repeated here.

[0125] The text processing submodule 240 is configured to perform word segmentation processing on the evaluation text to obtain multiple word blocks and parts of speech of the multiple word blocks, and to perform sentence segmentation processing on the evaluation text based on the multiple word blocks and parts of speech of the multiple word blocks to obtain multiple sentences.

[0126] For example, in the speech evaluation device 60 provided in at least one embodiment of the present disclosure, the text vector includes multiple word block vectors and multiple sentence vectors, and the correlation calculation unit is further configured to: calculate the cosine similarity between the multiple word block vectors and the topic vector to obtain a first correlation index; calculate the cosine similarity between the multiple sentence vectors and the topic vector to obtain a second correlation index; and calculate the cosine similarity between two adjacent sentences in the multiple sentences based on the multiple sentence vectors to obtain a third correlation index.

[0127] For example, in the speech evaluation device 60 provided in at least one embodiment of the present disclosure, the scoring submodule includes a normalization processing unit and a scoring unit. The normalization processing unit is configured to perform normalization processing on the first correlation index, the second correlation index, and the third correlation index; the scoring unit is configured to perform weighted processing on the normalized first correlation index, the second correlation index, and the third correlation index to obtain a score.

[0128] The speech evaluation devices 50 and 60 provided in the above-mentioned embodiments of the present disclosure can obtain the text to be evaluated corresponding to the audio to be evaluated by performing speech recognition on the audio to be evaluated, and then perform semantic analysis on the text to be evaluated. According to the semantics of the text to be evaluated, the relevance between the audio to be evaluated and the topic is evaluated, and a corresponding score is given, which can effectively reflect the fit between the audio to be evaluated and the topic, and improve the accuracy, comprehensiveness and scientificity of the speech evaluation.

[0129] At least one embodiment of the present disclosure further provides an electronic device comprising a memory and a processor, wherein the memory non-transiently stores computer-executable instructions, and the processor is configured to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the speech evaluation method according to any of the above embodiments is implemented. The technical effects of this electronic device are the same as those of the above-described speech evaluation methods and are not further described here.

[0130] At least one embodiment of the present disclosure also provides a non-transitory computer-readable storage medium.

[0131] Figure 7 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Figure 7 As shown, one or more computer-executable instructions 701 can be non-transitory stored on a storage medium 700. For example, when computer-executable instructions 701 are executed by a processor, one or more steps of the speech evaluation method described above can be performed. The technical effects of this non-transitory storage medium are the same as those of the speech evaluation method described above and are not further described here.

[0132] For example, the aforementioned non-transitory readable storage medium may be implemented as a memory, such as a volatile memory and / or a non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, a flash memory, etc. The memory may also store various applications (code, instructions) and data, as well as various data used and / or generated by the applications.

[0133] The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device executes the voice evaluation method.

[0134] The computer program code for performing the speech evaluation method can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] Some embodiments of the present disclosure further provide an electronic device, which includes the speech evaluation device of any of the above embodiments or can execute the speech evaluation method of any of the above embodiments.

[0136] Figure 8 The following is a schematic block diagram of an electronic device according to at least one embodiment of the present disclosure. The electronic device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device 800 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0137] For example, Figure 8 As shown, in some examples, an electronic device 800 includes a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801. This processing device 801 can include any of the aforementioned embodiments of the speech evaluation device. It can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 802 or programs loaded from a storage device 808 into a random access memory (RAM) 803. RAM 803 also stores various programs and data required for computer system operation. The processing device 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0138] For example, the following components can be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809 which can also include, for example, a network interface card such as a LAN card, a modem, etc. The communication device 809 can allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data, and perform communication processing via a network such as the Internet. The drive 810 is also connected to the I / O interface 805 as needed. Removable storage media 811, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 810 as needed, so that the computer program read therefrom can be installed into the storage device 808 as needed. Although Figure 8 The electronic device 800 is shown as including various devices, but it should be understood that it is not required to implement or include all of the devices shown, and more or fewer devices may be implemented or included instead.

[0139] For example, the electronic device 800 may further include a peripheral interface (not shown in the figure), etc. The peripheral interface may be various types of interfaces, such as a USB interface, a lightning interface, etc. The communication device 809 may communicate with a network and other devices via wireless communication, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communications may use any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0140] For example, the electronic device 800 can be any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, navigator, etc., or it can be any combination of data processing devices and hardware, and the embodiments of the present disclosure are not limited to this.

[0141] Although the present disclosure has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the embodiments of the present disclosure. Therefore, such modifications or improvements, as long as they do not depart from the spirit of the present disclosure, are within the scope of protection claimed by the present disclosure.

[0142] In addition to the above exemplary contents, the following points need to be explained in this disclosure:

[0143] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0144] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present disclosure are enlarged or reduced, that is, these drawings are not drawn according to the actual scale.

[0145] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0146] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A speech evaluation method, comprising: Obtaining an audio to be evaluated and a subject corresponding to the audio to be evaluated; as well as Outputting a score of the semantic relevance between the audio to be evaluated and the topic, Wherein, determining the score includes: performing speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated, wherein the text to be evaluated includes a plurality of consecutive characters obtained by the speech recognition processing, the plurality of consecutive characters correspond one-to-one to a plurality of syllables in the audio to be evaluated, and there are no punctuation marks between the plurality of consecutive characters; Performing word segmentation on the text to be evaluated to obtain a plurality of word chunks and parts of speech of the plurality of word chunks; Determining a plurality of text features corresponding to the plurality of chunks based on a sequence format, the sequence format including a current chunk, a position of the current chunk, a part of speech of the current chunk, a previous chunk, a part of speech of the previous chunk, a next chunk, and a part of speech of the next chunk; Mapping the plurality of text features into a plurality of feature vectors, inputting the plurality of feature vectors into a binary classification model, processing the plurality of feature vectors by the binary classification model, and outputting a processing result for determining whether to break a sentence, wherein the processing result includes a first value for indicating a break and a second value for indicating whether to break a sentence; Perform sentence segmentation processing on the text to be evaluated based on the processing result to determine multiple sentences; Processing the text to be evaluated including the multiple word chunks and the multiple sentences through a semantic analysis model to obtain multiple word chunk vectors and multiple sentence vectors including semantic information; Processing the topic using the semantic analysis model to obtain a topic vector including semantic information; Calculating the semantic similarity between the text to be evaluated and the topic based on the multiple word block vectors, the multiple sentence vectors and the topic vector; and The score is calculated based on the semantic similarity between the text to be evaluated and the topic.

2. The speech evaluation method according to claim 1, wherein: The performing speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated includes: Inputting the audio to be evaluated into the end-to-end model; and The speech recognition process is performed on the audio to be evaluated through the end-to-end model, and the text to be evaluated is obtained from the output of the end-to-end model.

3. The speech evaluation method according to claim 2, wherein: The end-to-end model includes a first neural network and a second neural network, The step of performing the speech recognition process on the audio to be evaluated by using the end-to-end model and obtaining the text to be evaluated from the output of the end-to-end model includes: Performing feature extraction on the audio data to be evaluated through the input layer of the first neural network to obtain a word vector; Inputting the word vector into the hidden layer of the first neural network, performing weighted processing on the word vector through the hidden layer to obtain semantic features corresponding to the word vector; Inputting the semantic feature into a bias layer of the first neural network, and adjusting the semantic feature through the bias layer to obtain activation data; Inputting the activation data into the second neural network, performing time series analysis on the activation data through the second neural network to obtain a time series dependency relationship; and The text to be evaluated is determined based on the temporal dependency.

4. The speech evaluation method according to claim 3, wherein: The second neural network includes a recurrent neural network, a long short-term memory network or a gated recurrent unit network.

5. The speech evaluation method according to claim 1, wherein: The calculating the semantic similarity between the text to be evaluated and the topic based on the multiple word block vectors, the multiple sentence vectors, and the topic vector includes: Calculating cosine similarities between the plurality of chunk vectors and the topic vector to obtain a first relevance index; Calculating cosine similarities between the plurality of sentence vectors and the topic vector to obtain a second relevance index; and The semantic similarity between the text to be evaluated and the topic is calculated based on the first relevance index and the second relevance index.

6. The speech evaluation method according to claim 5, further comprising: Calculate the cosine similarity between two adjacent sentences in the plurality of sentences based on the plurality of sentence vectors to obtain a third relevance index, The step of calculating the score based on the semantic similarity between the text to be evaluated and the topic includes: The score is calculated based on the first correlation index, the second correlation index, and the third correlation index.

7. The speech evaluation method according to claim 6, wherein: The calculating the score based on the first correlation index, the second correlation index, and the third correlation index includes: performing normalization processing on the first correlation index, the second correlation index, and the third correlation index; The normalized first correlation index, the second correlation index, and the third correlation index are weighted to obtain the score.

8. The speech evaluation method according to claim 1, wherein: The semantic analysis model includes a word embedding model or a context-dependent model, wherein the word embedding model includes a word vector model or a global logarithmic bilinear regression model, and the context-dependent model includes a bidirectional encoding representation model from a transformer.

9. A speech evaluation device comprising: Input module, output module and processing module, wherein, The input module is configured to obtain the audio to be evaluated and a subject corresponding to the audio to be evaluated; The output module is configured to output a score of the relevance between the audio to be evaluated and the topic; The processing module includes a speech recognition submodule, a text processing submodule, a relevance analysis submodule and a scoring submodule. The speech recognition submodule is configured to perform speech recognition processing on the audio to be evaluated to obtain a text to be evaluated corresponding to the audio to be evaluated, wherein the text to be evaluated includes a plurality of consecutive characters obtained by the speech recognition processing, the plurality of consecutive characters correspond one-to-one to a plurality of syllables in the audio to be evaluated, and there are no punctuation marks between the plurality of consecutive characters; The text processing submodule is configured to perform word segmentation on the text to be evaluated to obtain a plurality of word blocks and the parts of speech of the plurality of word blocks; determine a plurality of text features corresponding to the plurality of word blocks based on a sequence format, wherein the sequence format includes a current word block, a position of the current word block, the part of speech of the current word block, a previous word block, the part of speech of the previous word block, a next word block, and the part of speech of the next word block; map the plurality of text features into a plurality of feature vectors, input the plurality of feature vectors into a binary classification model, process the plurality of feature vectors through the binary classification model, and output a processing result for determining whether to segment the text, wherein the processing result includes a first value for indicating segmentation and a second value for indicating whether to segment the text; segment the text to be evaluated based on the processing result to determine a plurality of sentences; The relevance analysis submodule includes a semantic analysis unit and a relevance calculation unit. The semantic analysis unit is configured to process the text to be evaluated including the multiple word chunks and the multiple sentences using a semantic analysis model to obtain multiple word chunk vectors and multiple sentence vectors including semantic information, and to process the topic using the semantic analysis model to obtain a topic vector including semantic information; The relevance calculation unit is configured to calculate the semantic similarity between the text to be evaluated and the topic based on the multiple word block vectors, the multiple sentence vectors and the topic vector; The scoring submodule is configured to calculate the score based on the semantic similarity between the text to be evaluated and the topic.

10. The speech evaluation device according to claim 9, wherein: The speech recognition submodule includes a speech recognition unit, which is configured to input the audio to be evaluated into an end-to-end model, perform the speech recognition processing on the audio to be evaluated through the end-to-end model, and obtain the text to be evaluated from the output of the end-to-end model.

11. The speech evaluation device according to claim 9, wherein: The relevance calculation unit is further configured to: Calculating cosine similarities between the plurality of chunk vectors and the topic vector to obtain a first relevance index; Calculating cosine similarities between the plurality of sentence vectors and the topic vector to obtain a second relevance index; as well as The cosine similarity between two adjacent sentences in the plurality of sentences is calculated based on the plurality of sentence vectors to obtain a third relevance index.

12. The speech evaluation device according to claim 11, wherein: The scoring submodule includes a normalization processing unit and a scoring unit. The normalization processing unit is configured to perform normalization processing on the first correlation index, the second correlation index, and the third correlation index; The scoring unit is configured to perform weighted processing on the normalized first correlation index, the second correlation index, and the third correlation index to obtain the score.

13. An electronic device comprising: at least one memory non-transitorily storing computer-executable instructions; at least one processor configured to execute the computer-executable instructions, The computer-executable instructions, when executed by the processor, implement the speech evaluation method according to any one of claims 1 to 8.

14. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, When the computer-executable instructions are executed by at least one processor, the speech evaluation method according to any one of claims 1 to 8 is implemented.

Images