Voice evaluation method and related device

Abstract

The invention discloses a voice evaluation method and a related device, and relates to the technical field of voice evaluation, and the voice evaluation method comprises the steps: obtaining to-be-evaluated voice data; processing the to-be-tested voice data into a phoneme sequence to obtain a target phoneme sequence; based on a large language model, obtaining a source text of a text corresponding to the target phoneme sequence based on the target phoneme sequence, and obtaining a first traceability text; processing the target phoneme sequence into a plurality of phoneme subsequences, determining a source text of a text corresponding to the target phoneme sequence by determining a source text of a text corresponding to each phoneme subsequence, and obtaining a second traceability text; determining a target traceability text according to the correlation degrees between the first traceability text and the target phoneme sequence and between the second traceability text and the target phoneme sequence; and evaluating the to-be-evaluated voice data by referring to the target traceability text. According to the voice evaluation method disclosed by the invention, the to-be-evaluated voice data is evaluated by referring to the target traceability text, and a relatively accurate evaluation result can be obtained.

Description

Technical Field

[0001] The present application relates to the field of speech evaluation technology, and in particular to a speech evaluation method and related devices. Background Art

[0002] In some application scenarios, voice data (such as user's reading or reciting voice data) needs to be evaluated.

[0003] The current evaluation plan is: first obtain the voice data to be evaluated (such as the user's reading or reciting voice data), then perform speech recognition on the voice data to be evaluated based on the speech recognition model to obtain the speech recognition text, and finally evaluate the voice data to be evaluated with reference to the speech recognition text.

[0004] However, the speech recognition text obtained by performing speech recognition on the evaluation speech data based on the speech recognition model may contain errors (for example, the user reads "The spring scenery in the garden cannot be contained, a red apricot comes out of the wall", while the text recognized by the speech recognition model is "The spring scenery in the garden cannot be contained, a red apricot comes out of the wall"). Evaluating the evaluation speech data with reference to the erroneous speech recognition text will lead to inaccurate evaluation results. Summary of the invention

[0005] In view of this, the present application provides a speech evaluation method and related devices, which are used to solve the problem of inaccurate evaluation results of existing speech evaluation methods. The technical solution is as follows:

[0006] The first aspect of the present application provides a speech evaluation method, comprising:

[0007] Obtain the voice data to be evaluated;

[0008] Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence;

[0009] Based on the large language model and taking the target phoneme sequence as a basis, obtaining a source text of the text corresponding to the target phoneme sequence to obtain a first source text;

[0010] Processing the target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text;

[0011] Determining a target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively;

[0012] The speech data to be evaluated is evaluated with reference to the target source text.

[0013] In a possible implementation, the step of obtaining the speech data to be evaluated includes:

[0014] Real-time acquisition of voice data when users continuously read or recite multiple articles;

[0015] Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence includes:

[0016] Processing the real-time acquired speech data into a phoneme sequence to obtain a target phoneme sequence;

[0017] The step of evaluating the speech data to be evaluated by referring to the target source text includes:

[0018] According to the target phoneme sequence, determining whether the user has currently completed reading or reciting an article;

[0019] If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

[0020] In a possible implementation, determining the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively includes:

[0021] Obtaining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities;

[0022] According to the two similarities, the target source tracing text is determined.

[0023] In a possible implementation manner, processing the target phoneme sequence into a plurality of phoneme subsequences includes:

[0024] The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0025] In a possible implementation, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source text includes:

[0026] Based on a pre-constructed reverse index table, determining article identifiers corresponding to the plurality of first n-gram units, wherein the reverse index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and article identifiers corresponding to the plurality of n-gram units;

[0027] Counting the number of occurrences of the article identifiers corresponding to the plurality of first n-gram units to obtain a statistical result;

[0028] Determine the target article identifier according to the statistical results;

[0029] An article in the article library with an article identifier as the target article identifier is determined as the second source tracing text.

[0030] In a possible implementation, determining the target article identifier according to the statistical result includes:

[0031] Determine the K article identifiers with the largest number of occurrences as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1;

[0032] Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article;

[0033] Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier;

[0034] The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

[0035] In a possible implementation, determining the target source tracing text according to the two similarities includes:

[0036] Determine the maximum similarity of the two similarities as the target similarity;

[0037] If the target similarity is greater than or equal to a preset similarity threshold, the source tracing text corresponding to the phoneme sequence whose similarity to the target phoneme sequence is the target similarity is determined as the target source tracing text;

[0038] If the target similarity is less than a preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

[0039] In a possible implementation, judging whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes:

[0040] Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length;

[0041] If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

[0042] In a possible implementation, judging whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes:

[0043] Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1;

[0044] Determine article identifiers corresponding to the plurality of second n-gram units based on a pre-constructed reverse index table, wherein the reverse index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and article identifiers corresponding to the plurality of n-gram units;

[0045] Whether the user has currently completed reading aloud an article is determined according to the article identifiers respectively corresponding to the plurality of second n-gram units.

[0046] In a possible implementation, judging whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes:

[0047] In a case where the target source-tracing text is the first source-tracing text or the second source-tracing text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source-tracing text;

[0048] If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

[0049] In a possible implementation, judging whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes:

[0050] In the case where the target source tracing text is the corrected text, performing sentence-level N-gram processing on the target source tracing text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source tracing text, wherein N is an integer greater than 1;

[0051] Based on the pre-trained classification model, classify each sentence-level N-gram unit to obtain the classification result of each sentence-level N-gram unit, wherein the classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has undergone article switching;

[0052] According to the classification results of N-gram units at each sentence level, it is determined whether the user has currently completed reading aloud an article.

[0053] In a possible implementation, the step of evaluating the speech data to be evaluated by referring to the target source text includes:

[0054] According to the target tracing text, the general decoding network is tailored to obtain a limited decoding network;

[0055] Extracting speech features from the speech data to be evaluated, and decoding the speech features based on the limited decoding network to obtain a decoding result;

[0056] Processing the decoding result through a language model to obtain a phoneme sequence;

[0057] The target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain a scoring result.

[0058] A second aspect of the present application provides a speech evaluation device, comprising: a speech data acquisition module, a speech data processing module, a first tracing module, a second tracing module, a target tracing text determination module and a speech evaluation module;

[0059] The voice data acquisition module is used to acquire the voice data to be evaluated;

[0060] The speech data processing module is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence;

[0061] The first source tracing module is used to obtain the source text of the text corresponding to the target phoneme sequence based on the large language model and the target phoneme sequence, so as to obtain a first source tracing text;

[0062] The second source tracing module is used to process the target phoneme sequence into a plurality of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, so as to obtain a second source tracing text;

[0063] The target source text determination module is used to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively;

[0064] The speech evaluation module is used to evaluate the speech data to be evaluated with reference to the target source tracing text.

[0065] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0066] The memory is used to store computer programs;

[0067] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned speech evaluation methods.

[0068] A fourth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned speech evaluation methods.

[0069] A fifth aspect of the present application provides a computer program product, comprising computer-readable instructions, which, when executed on an electronic device, enables the electronic device to implement the steps of any one of the above-described speech evaluation methods.

[0070] By means of the above technical solution, the speech evaluation method provided by the present application first obtains the speech data to be evaluated, then processes the speech data to be evaluated into a phoneme sequence to obtain a target phoneme sequence, then uses two different tracing methods to trace the target phoneme sequence to obtain a target tracing text, and finally refers to the target tracing text to evaluate the speech data to be evaluated. Since the speech evaluation method provided by the present application evaluates the speech data to be evaluated with reference to the target tracing text, rather than with reference to the speech recognition result of the speech data to be evaluated, a relatively accurate evaluation result can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0072] Figure 1 A schematic diagram of a system architecture involved in this application;

[0073] Figure 2 A schematic diagram of a hardware structure of a terminal provided in an embodiment of the present application;

[0074] Figure 3 A schematic diagram of a hardware structure of a server provided in an embodiment of the present application;

[0075] Figure 4 A flowchart of a speech evaluation method provided in an embodiment of the present application;

[0076] Figure 5 A schematic diagram of an implementation method for evaluating speech data to be evaluated, which is a reference target source text provided in an embodiment of the present application;

[0077] Figure 6 A flow chart of processing a target phoneme sequence into a plurality of phoneme subsequences provided in an embodiment of the present application, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text;

[0078] Figure 7 A flowchart of a specific example of the speech evaluation method provided in the embodiment of the present application;

[0079] Figure 8 A flowchart of another specific example of the speech evaluation method provided in the embodiment of the present application;

[0080] Fig. 9 A schematic diagram of the structure of a speech evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0082] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0083] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0084] In one possible implementation, Figure 1 As shown, the system architecture involved in the present application may include a terminal 101 and a server 102, and the terminal 101 may interact with the server 102 via a network (wired network or wireless network). The server 102 may include one or more servers ( Figure 1 The example includes a server. The terminal 101 can obtain the voice data to be evaluated, and transmit the voice data to be evaluated to the server 102 through the network. The server 102 uses the voice evaluation method provided in the present application to evaluate the voice data to be evaluated, and transmits the voice evaluation result to the terminal 101 through the network. The terminal 101 can output the voice evaluation result.

[0085] In another possible implementation, the system architecture involved in the present application may include a terminal. The terminal has strong data processing capabilities. The terminal may use the voice evaluation method provided in the present application to evaluate the voice data to be evaluated, and then output the voice evaluation result.

[0086] Next, the product form of the above-mentioned terminal is described.

[0087] The above-mentioned terminal can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a robot, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions on this.

[0088] Figure 2 An optional hardware structure diagram of a terminal is shown.

[0089] refer to Figure 2As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will appreciate that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0090] The input unit 230 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 270, and can receive and execute the command sent by the processor 270; the touch signal at least includes the contact point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch screen 231, the input unit 230 can also include other input devices. Specifically, other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0091] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0092] The memory 220 can be used to store instructions and data. The memory 220 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; it provides the processor 270 with hardware, software, and data resources including management of computing and processing equipment, and supports control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0093] The processor 270 is the control center of the terminal. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing instructions stored in the memory 220 and calling data stored in the memory 220, it executes various functions of the terminal and processes data, thereby controlling the terminal as a whole. Optionally, the processor 270 may include one or more processing units; preferably, the processor 270 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 270. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 270 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 220, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0094] The memory 220 may be used to store software codes related to the speech evaluation method, and the processor 270 may execute the software codes in the memory 220 or schedule other units (such as the input unit 230 and the display unit 240) to implement corresponding functions.

[0095] The radio frequency unit 210 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 270 for processing; in addition, the designed uplink data is sent to the base station. Generally, the radio frequency unit 210 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LowNoiseAmplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 210 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General PacketRadio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0096] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices, and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional, and can be replaced by other communication interfaces, such as a network port.

[0097] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0098] The terminal also includes an external interface 280, which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to communicate with other devices, or to connect a charger to charge the terminal.

[0099] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0100] Next, the product form of the above server is described.

[0101] Figure 3 A structural diagram of the above server is provided, such as Figure 3 As shown, the server may include a bus 301, a processor 302, a communication interface 303, and a memory 304. The processor 302, the memory 304, and the communication interface 303 communicate with each other via the bus 301.

[0102] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0104] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0105] The memory 304 may be used to store software codes related to the speech evaluation method. The processor 302 may call the software codes stored in the memory 304 or schedule other units to implement corresponding functions.

[0106] The processors in the above-mentioned terminals and servers (for example, processor 270 and processor 302) may be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the processor may be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0107] The inventors of this case discovered during the process of implementing this case that the current speech evaluation scheme evaluates the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated. Since the speech recognition text of the speech data to be evaluated may contain errors, the evaluation results obtained by evaluating the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated may be inaccurate.

[0108] The inventor of this case conducted research on the problems existing in the current speech evaluation scheme. The initial idea was to pre-set an article library in the speech evaluation system. Before reading or reciting, the user first selects the article to be read or recited from the article library, and then clicks to start recording and reads or recites the selected article. After the user finishes reading or reciting, clicks to end recording and waits for the speech evaluation system to give evaluation feedback. The process of speech evaluation by the speech evaluation system is to obtain the speech data of the user reading or reciting the complete article, and evaluate the obtained speech data with reference to the article selected by the user.

[0109] Since the above idea evaluates the speech data to be evaluated with reference to the article selected by the user, rather than evaluating the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated, a more accurate evaluation result can be obtained.

[0110] The inventor of this case studied the above idea and found that although the above idea can obtain relatively accurate evaluation results, it has a problem: before reading or reciting, the user needs to select the article to be read or recited from the system's article library. It is understandable that if the user wants to read or recite multiple texts, before reading or reciting each article, the user needs to select the article, and before reading or reciting each article, an operation is required to trigger the start of recording, and after reading or reciting each article, an operation is required to trigger the end of recording. It can be seen that the user operation is cumbersome, and it is impossible to achieve continuous reading or reciting by the user, and the user experience is not good.

[0111] In view of the defects of the above ideas, the inventor of this case continued to study and finally proposed a speech evaluation method with better effect. Next, the speech evaluation method provided by this application is introduced through the following embodiments.

[0112] See also Figure 4 , shows a flow chart of a speech evaluation method provided in an embodiment of the present application, which may include:

[0113] Step S401: Acquire speech data to be evaluated.

[0114] The voice data to be evaluated in this embodiment may be the user's reading or reciting voice data.

[0115] Step S402: Process the audio data to be tested into a phoneme sequence to obtain a target phoneme sequence.

[0116] In a possible implementation, the audio data to be tested and evaluated may be processed into a phoneme sequence based on a speech recognition model.

[0117] It should be noted that the process of speech recognition model performing speech recognition on input speech data is to first process the input speech data into a phoneme sequence, and then determine the final speech recognition result based on the obtained phoneme sequence. This application only uses the phoneme sequence acquisition function of the speech recognition model to obtain the phoneme sequence for the speech data to be evaluated.

[0118] Exemplarily, the content of the voice data to be evaluated is "The spring scenery in the garden cannot be contained, and a red apricot comes out of the wall." Based on the speech recognition model, the voice data to be evaluated is processed to obtain the phoneme sequence "man yuan chun se guan buzhu yi zhi hong xing chu qiang lai".

[0119] The speech data to be evaluated in this embodiment can be Chinese speech data or English speech data. If the speech data to be evaluated is Chinese speech data, the phonemes in the target phoneme sequence are pinyin, and if the speech data to be evaluated is English speech data, the phonemes in the target phoneme sequence are phonetic symbols. Of course, this embodiment does not limit the speech data to be evaluated to be Chinese speech data or English speech data, and it can also be speech data in other languages, such as speech data in German, French, etc.

[0120] Step S403a: Based on the large language model and taking the target phoneme sequence as a basis, the source text of the text corresponding to the target phoneme sequence is obtained to obtain a first source text.

[0121] Specifically, the preset first instruction prompt information and the target phoneme sequence are organized into a first prompt instruction, and the first prompt instruction is used to prompt the large language model to trace the target phoneme sequence. After obtaining the first prompt instruction, the first prompt instruction is input into the large language model, and the large language model traces the input target phoneme sequence and outputs a first tracing text.

[0122] Exemplarily, the target phoneme sequence is "man yuan chun se guan bu zhu yi zhi hong xingchu qiang lai", and the first prompt instruction is "If you are a Chinese teacher, please use [man yuan chun se guanbu zhu yi zhi hong xing chu qiang lai] to find the possible poem that this pinyin is reading, which may be missed, misread, or added. Please output the original poem", and input the first prompt instruction into the large language model. The large language model predicts the source text of the text corresponding to [man yuan chun se guan bu zhu yi zhi hong xing chu qiang lai], and outputs "I should pity the imprints of my clogs on the moss, and the fact that the wooden door has not been opened for a long time after I knocked on it. The spring scenery in the garden cannot be contained, and a red apricot branch comes out of the wall." from "Not Worth Visiting the Garden".

[0123] Step S403b: Process the target phoneme sequence into a plurality of phoneme subsequences, determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtain a second source text.

[0124] In order to determine a more accurate source text for the target phoneme sequence, in addition to using a text tracing method based on a large language model to trace the text of the target phoneme sequence, this embodiment also uses another text tracing method to trace the text of the target phoneme sequence, that is, the target phoneme sequence is processed into several phoneme subsequences, and the source text of the text corresponding to the target phoneme sequence is determined by determining the source text of the text corresponding to each phoneme subsequence.

[0125] Step S404: Determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively.

[0126] The degree of relevance between the target phoneme sequence and the first source text can be determined, and the degree of relevance between the target phoneme sequence and the second source text can be determined. Then, the target source text can be determined based on the degree of relevance between the target phoneme sequence and the first source text and the degree of relevance between the target phoneme sequence and the second source text.

[0127] This embodiment uses two different text tracing methods to perform text tracing on the target phoneme sequence respectively, and then determines the final tracing text, that is, the target tracing text, based on the tracing texts obtained by using the two different text tracing methods.

[0128] Step S405: Evaluate the speech data to be evaluated with reference to the target source text.

[0129] There are many ways to evaluate the speech data to be evaluated with reference to the target source text. In one possible implementation, speech features (such as MFCC features) can be extracted from the speech data to be evaluated. Then, the extracted speech features are decoded based on a general decoding network to obtain a decoding result. The decoding result is then processed by a language model to obtain a phoneme sequence. Finally, the target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain an evaluation result.

[0130] In order to improve the evaluation effect, this embodiment provides another implementation method of evaluating the speech data to be evaluated by referring to the target source text, such as Figure 5 As shown, first, the general decoding network is trimmed according to the target source text to obtain a limited decoding network, and then speech features (such as MFCC features) are extracted from the speech data to be evaluated, and the extracted speech features are decoded based on the limited decoding network to obtain a decoding result. The decoding result is then processed by a language model to obtain a phoneme sequence. Finally, the target source text and the phoneme sequence obtained by the language model are input into the scoring model for scoring to obtain an evaluation result.

[0131] The speech evaluation method provided in the embodiment of the present application first obtains the speech data to be evaluated, then processes the speech data to be evaluated into a phoneme sequence to obtain a target phoneme sequence, then traces the target phoneme sequence to obtain a target source text, and finally evaluates the speech data to be evaluated with reference to the target source text. Since the speech evaluation method provided in the embodiment of the present application evaluates the speech data to be evaluated with reference to the target source text, rather than evaluating with reference to the speech recognition result of the speech data to be evaluated, a relatively accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the present application does not require the user to select an article, thereby improving the user experience.

[0132] In another embodiment of the present application, the specific implementation process of "Step S403b: processing the target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second tracing text" in the above embodiment is introduced.

[0133] See also Figure 6 , showing a flow chart of processing a target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text, which may include:

[0134] Step S601: performing phoneme-level n-gram processing on the target phoneme sequence to obtain a plurality of first n-gram units.

[0135] Each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0136] For example, the target phoneme sequence is "man yuan chun se guan bu zhu", assuming n=3, and 3-gram processing is performed on the target phoneme sequence to obtain multiple first 3-gram units: man-yuan-chun, yuan-chun-se, chun-se-guan, se-guan-bu, guan bu zhu.

[0137] Step S602: Based on the pre-constructed inverted index table, determine the article identifiers corresponding to the first n-gram units.

[0138] The article identifier may be, but is not limited to, the article title.

[0139] The reverse index table in this embodiment is constructed based on each article in the article library, and includes multiple n-gram units obtained by performing n-gram processing on the phoneme sequence corresponding to the article content of each article in the article library. The reverse index table also includes article identifiers corresponding to the multiple n-gram units obtained by performing n-gram processing on the phoneme sequence corresponding to the article content of each article in the article library.

[0140] For example, the reverse index table is as follows:

[0141] {

[0142] “man yuan chun”:[“Not Worth Visiting the Garden”],

[0143] “yuan chun se”:[“Not worth visiting the garden”], ...

[0145] }

[0146] In the reverse index table, "man yuan chun", "yuan chun se" and so on are indexes. It should be noted that the article identifier corresponding to any n-gram unit in the reverse index table may be one or more.

[0147] Step S603: Count the number of occurrences of the article identifiers corresponding to the first n-gram units to obtain a statistical result.

[0148] Exemplarily, there are 5 first n-gram units. The article identifiers corresponding to the 1st first n-gram unit are A and B, the article identifier corresponding to the 2nd first n-gram unit is A, the article identifiers corresponding to the 3rd first n-gram unit are A and C, the article identifiers corresponding to the 4th first n-gram unit are A and B, and the article identifier corresponding to the 5th first n-gram unit is A. Then, count the number of occurrences of article identifier A, the number of occurrences of article identifier B, and the number of occurrences of article identifier C. After statistics, article identifier A appears 5 times, article identifier B appears 2 times, and article identifier C appears 1 time.

[0149] Step S604: Determine the target article identifier according to the statistical result.

[0150] There are multiple ways to determine the target article identifier according to the statistical result. This embodiment provides two implementation manners.

[0151] The first implementation manner: Determine the article identifier with the most occurrences (sort the article identifiers in descending order of the number of occurrences, and the article identifier ranked first) as the target article identifier.

[0152] The second implementation manner: Determine the K (K is an integer greater than 1) article identifiers with the most occurrences (sort the article identifiers in descending order of the number of occurrences, and the article identifiers ranked in the top K) as candidate article identifiers, obtaining K candidate article identifiers; From the pre-constructed forward index table, obtain the phoneme sequences corresponding to the K candidate article identifiers respectively; Determine the similarity between the target phoneme sequence and the phoneme sequences corresponding to each candidate article identifier; Determine the candidate article identifier corresponding to the phoneme sequence with the highest similarity to the target phoneme sequence as the target article identifier.

[0153] Among them, the forward index table is constructed based on each article in the article library, and it includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article. The phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of that article.

[0154] Exemplarily, the forward index table is as follows:

[0155] {

[0156] " ":[ "…man yuan chun se guan bu zhu…"],

[0157] " ":[ "…bai yun sheng chu you ren jia…"], ...

[0159] }

[0160] "Not Worth Visiting the Garden" and "Hike in the Mountains" in the forward index table are indexes.

[0161] The similarity between the target phoneme sequence and the phoneme sequence corresponding to any candidate article identifier can be determined as follows: the number of matching phonemes between the target phoneme sequence and the phoneme sequence corresponding to the candidate article identifier / the total number of phonemes in the target phoneme sequence. Exemplarily, the target phoneme sequence includes 7 phonemes, and it matches 6 phonemes with the phoneme sequence corresponding to the candidate article identifier (the phoneme sequence corresponding to the candidate article identifier covers 6 phonemes of the target phoneme sequence), then the similarity between the target phoneme sequence and the phoneme sequence corresponding to the candidate article identifier is 6 / 7.

[0162] Step S605: The article in the article library that is identified as the target article is determined as the second source tracing text.

[0163] In another embodiment of the present application, the specific implementation process of "step S404: determining the target source text according to the degree of relevance of the first source text and the second source text to the target phoneme sequence respectively" is introduced.

[0164] According to the relevance of the first source text and the second source text to the target phoneme sequence, the process of determining the target source text may include:

[0165] Step a1: Obtain the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, and obtain two similarities.

[0166] The method for determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text and the method for determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text is the same as the method for determining the similarity between the above-mentioned target phoneme sequence and the phoneme sequence corresponding to any candidate article identifier, which will not be elaborated in this embodiment.

[0167] Step a2: Determine the target source text based on the two similarities.

[0168] In a possible implementation, the process of determining the target source text according to the two similarities may include:

[0169] Step a21: determine the maximum similarity of the two similarities as the target similarity.

[0170] Step a22-a: If the target similarity is greater than or equal to the preset similarity threshold, the source text corresponding to the phoneme sequence whose similarity to the target phoneme sequence is the target similarity is determined as the target source text.

[0171] Exemplarily, the preset similarity threshold is 0.8, the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text is greater than the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text is greater than 0.8, then the first source text is determined to be the target source text.

[0172] It should be noted that the preset similarity threshold of 0.8 is only an example, and the specific value of the preset similarity threshold can be set according to the actual application scenario.

[0173] Step a22-b: If the target similarity is less than the preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

[0174] In actual applications, there may be a situation where the target similarity is less than the preset similarity threshold. When this happens, the text corresponding to the target phoneme sequence can be predicted based on the large language model, and the predicted text can be corrected and used as the target tracing text.

[0175] When predicting the text corresponding to the target phoneme sequence based on the large language model and correcting the predicted text, the preset second instruction prompt information and the target phoneme sequence can be organized into a second prompt instruction, and the second prompt instruction is used to prompt the large language model to predict the text corresponding to the target phoneme sequence and correct the predicted text. After obtaining the second prompt instruction, the second prompt instruction is input into the large language model, the large language model predicts the text corresponding to the target phoneme sequence, corrects the predicted text, and outputs the corrected text.

[0176] Exemplarily, the second prompt instruction is "If you are a Chinese teacher, please give a prediction through [jin tian tian qiqing lang wo men de xin qing hen yu kuai]: 1. What is the corresponding Chinese text? 2. If there are illogical grammatical errors in the original text, you need to correct it and give the text." The second prompt instruction is input into the large language model, the large language model predicts the text corresponding to "jin tian tian qi qing lang wo men de xin qing hen yukuai", and corrects the predicted text and outputs it.

[0177] Based on the speech evaluation method provided in the above embodiment, a specific example of the speech evaluation method is given below. This specific example is suitable for speech evaluation in the reading or recitation scenario of a single article. Figure 7 , showing a schematic diagram of the process of this specific example, which may include:

[0178] Step S701: Acquire voice data of a user reading or reciting an article.

[0179] When a user's start recording trigger operation is detected, voice data of the user reading or reciting an article is obtained.

[0180] Step S702: after obtaining the voice data of the user reading or reciting the entire article, the voice data of the user reading or reciting the entire article is processed into a phoneme sequence to obtain a target phoneme sequence.

[0181] In a possible implementation, the voice data of the user reading or reciting the entire article may be processed into a phoneme sequence based on a speech recognition model.

[0182] In the scenario of reading or reciting a single article, when the user's end recording trigger operation is detected, it is considered that the user has recited or read the entire article. At this time, the reading or reciting voice data of the entire article can be obtained, and then the reading or reciting voice data of the entire article can be processed into a phoneme sequence to obtain a target phoneme sequence.

[0183] Step S703: Tracing the target phoneme sequence to obtain a target tracing text.

[0184] The target phoneme sequence is traced back, that is, the source of the text corresponding to the target phoneme sequence is found, and the target tracing text is the source text of the text corresponding to the target phoneme sequence.

[0185] The specific implementation process of tracing the target phoneme sequence to obtain the target tracing text can be found in the above embodiment, which will not be described in detail in this embodiment.

[0186] Step S704: evaluating the reading or reciting voice data of the entire article in combination with the target source text.

[0187] According to the target source text, the general decoding network can be tailored to obtain a limited decoding network, and speech features (such as MFCC features) can be extracted from the reading or recitation voice data of the entire article. The extracted speech features are decoded based on the limited decoding network to obtain a decoding result. The decoding result is processed by a language model to obtain a phoneme sequence. The phoneme sequence and the target source text are input into the scoring model for scoring to obtain an evaluation result.

[0188] The speech evaluation method provided in the embodiment of the present application first obtains the speech data when the user reads or recites an article, and then after obtaining the speech data of the user reading or reciting the entire article, the speech data of the user reading or reciting the entire article is processed into a phoneme sequence to obtain a target phoneme sequence, and then the target phoneme sequence is traced to obtain a target source text, and finally the target source text is referred to to evaluate the speech data of the user reading or reciting the entire article. Since the speech evaluation method provided in the embodiment of the present application refers to the target source text obtained by tracing the phoneme sequence corresponding to the user's reading or reciting speech data, the user's reading or reciting speech data is evaluated, rather than referring to the speech recognition result of the user's reading or reciting speech data for evaluation, therefore, a more accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the present application does not require the user to select an article, thereby improving the user experience.

[0189] Based on the speech evaluation method provided in the above embodiment, another specific example of the speech evaluation method is given below. This specific example is suitable for speech evaluation in the scenario of continuous reading or reciting of multiple articles. Figure 8 , showing a schematic diagram of the process of this specific example, which may include:

[0190] Step S801: Acquire in real time the voice data of the user when the user continuously reads or recites multiple articles.

[0191] When the user's start recording trigger operation is detected, the voice data of the user continuously reading or reciting multiple articles is obtained in real time.

[0192] Step S802: Process the voice data acquired in real time into a phoneme sequence to obtain a target phoneme sequence.

[0193] The speech data acquired in real time can be processed into a phoneme sequence in real time based on the speech recognition model to obtain a target phoneme sequence, where the target phoneme sequence is the phoneme sequence obtained up to now.

[0194] Step S803: Tracing the target phoneme sequence to obtain the target tracing text, and judging whether the user has completed reading or reciting an article according to the target phoneme sequence.

[0195] The specific implementation process of tracing the target phoneme sequence to obtain the target source text can be referred to the above embodiment, which will not be described in detail in this embodiment. This embodiment focuses on the implementation process of determining whether the user has completed reading or reciting an article according to the target phoneme sequence.

[0196] In one possible implementation, the process of determining whether the user has currently completed reading or reciting an article based on a target phoneme sequence may include: detecting whether the target phoneme sequence shows that the time interval between adjacent phonemes is greater than or equal to a preset time length (e.g., 10s); if the target phoneme sequence shows that the time interval between adjacent phonemes is greater than or equal to a preset time length, determining that the user has currently completed reading or reciting an article.

[0197] If the target phoneme sequence does not have a time interval between adjacent phonemes greater than or equal to the preset time length, in one possible implementation, it can be directly determined that the user has not yet completed reading or reciting an article. In order to improve the accuracy of the determination, in another possible implementation, the following determination strategy can be further introduced:

[0198] (a) When the target source text is obtained through step a22-a (i.e., the target source text is one of the first source text and the second source text in the above embodiment), the discrimination strategy is:

[0199] It is determined whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text. If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading or reciting an article.

[0200] When there is no phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, in one possible implementation, it can be directly determined that the user has not yet completed the reading or recitation of an article. In order to improve the accuracy of the determination, in another possible implementation, the last M (M is an integer greater than 1) phonemes of the target phoneme sequence can be further processed at the phoneme level to obtain a number of second n-gram units (each second n-gram unit includes n consecutive phonemes among the last M phonemes), and based on a pre-constructed reverse index table, the article identifiers corresponding to the second n-gram units are determined, and then it is determined whether the user has currently completed the reading or recitation of an article based on the article identifiers corresponding to the second n-gram units.

[0201] Among them, the process of determining whether the user has currently completed reading or reciting an article based on the article identifiers corresponding to several second n-gram units may include: determining whether there is a change in the article identifier based on the article identifiers corresponding to several second n-gram units, if there is a change in the article identifier, determining that the user has currently completed reading an article, otherwise, determining that the user has not currently completed reading or reciting an article.

[0202] Exemplarily, the article identifications corresponding to several second n-gram units are respectively […visiting the garden is not worth, visiting the garden is not worth, visiting the garden is not worth, visiting the garden is not worth, fisherman on the river], and based on […visiting the garden is not worth, visiting the garden is not worth, visiting the garden is not worth, visiting the garden is not worth, fisherman on the river], it can be determined that the article identification has changed (from "visiting the garden is not worth" to "fisherman on the river"), and further it can be determined that the user has currently completed the reading of an article.

[0203] It should be noted that if it is determined that the user has not yet completed reading or reciting an article, voice data is continuously acquired and processed into phonemes in real time, and then it is further determined based on the acquired phonemes whether the user has completed reading or reciting an article.

[0204] (b) When the target source text is obtained through step a22-b, the discrimination strategy is:

[0205] The target source text is processed at sentence level N-gram to obtain multiple sentence-level N-gram units (each sentence-level N-gram unit includes N consecutive sentences in the target source text). Based on the pre-trained classification model, each sentence-level N-gram unit is classified to obtain the classification result of each sentence-level N-gram unit. The classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has undergone article switching. According to the classification results of each sentence-level N-gram unit, it is determined whether the user has completed the reading or recitation of an article.

[0206] Exemplarily, the target source text includes 4 sentences, namely sentence 1, sentence 2, sentence 3 and sentence 4. Assuming N=3, the target source text is processed at the sentence level 3-gram to obtain two sentence-level 3-gram units, namely sentence 1-sentence 2-sentence 3 and sentence 2-sentence 3-sentence 4. Sentence 1-sentence 2-sentence 3 is input into the pre-trained classification model to obtain a classification result indicating whether an article switch has occurred in sentence 1-sentence 2-sentence 3. Sentence 2-sentence 3-sentence 4 is input into the pre-trained classification model to obtain a classification result indicating whether an article switch has occurred in sentence 2-sentence 3-sentence 4. Assuming that the classification result of sentence 1-sentence 2-sentence 3 indicates that no article switch has occurred, and the classification result of sentence 2-sentence 3-sentence 4 indicates that an article switch has occurred, it can be determined that the user has currently completed the reading of an article.

[0207] Among them, the classification model may include a semantic feature extraction module and a classification module. The semantic feature extraction module may be a pre-trained language model (such as BERT), and the classification module may be but is not limited to MLP. The semantic feature extraction module is used to extract contextual semantic features from the input text. The contextual semantic features extracted by the semantic feature acquisition module are input into the classification module. Based on the contextual semantic features extracted by the semantic feature extraction module, the classification module further extracts higher-level features and classifies them, and outputs the classification results.

[0208] The above content provides four discrimination strategies. The first discrimination strategy is to judge whether the user has completed the reading or reciting of an article by detecting whether the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to the preset time length. The second discrimination strategy is to judge whether the user has completed the reading or reciting of an article by judging whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text. The third discrimination strategy is to determine whether the user has completed the reading or reciting of an article based on the article identifiers corresponding to several second n-gram units obtained by n-gram processing of the last M phonemes of the target phoneme sequence. The fourth discrimination strategy is to determine whether the user has completed the reading or reciting of an article based on a classification model. It should be noted that in addition to adopting the above-mentioned discrimination strategy based on the first discrimination strategy and combining it with the second discrimination strategy, the present application In addition to the first discrimination strategy and the third discrimination strategy for determining whether the user has currently completed the reading or reciting of an article, as well as the fourth discrimination strategy combined with the first discrimination strategy for determining whether the user has currently completed the reading or reciting of an article, the first discrimination strategy alone can be used to determine whether the user has currently completed the reading or reciting of an article, or the third discrimination strategy can be used to determine whether the user has currently completed the reading or reciting of an article, and the first discrimination strategy can be combined with the third discrimination strategy to determine whether the user has currently completed the reading or reciting of an article. When the target tracing text is obtained through step a22-a, the second discrimination strategy alone can be used to determine whether the user has currently completed the reading or reciting of an article, and when the target tracing text is obtained through step a22-b, the fourth discrimination strategy alone can be used to determine whether the user has currently completed the reading or reciting of an article.

[0209] Step S804: If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article currently completed by the user is evaluated with reference to the target source text.

[0210] Specifically, according to the target source text, the general decoding network is trimmed to obtain a limited decoding network, and speech features (such as MFCC features) are extracted from the reading or reciting voice data of the article currently read or recited by the user. The extracted speech features are decoded based on the limited decoding network to obtain a decoding result, and the decoding result is processed by a language model to obtain a phoneme sequence. The phoneme sequence and the target source text are input into the scoring model for scoring to obtain an evaluation result.

[0211] It should be noted that the voice evaluation method provided in this embodiment is also applicable to the evaluation of the voice data of a user reading or reciting an article (ie, step S801 obtains the voice data of the user reading or reciting an article in real time).

[0212] For the next article, since the acquisition of voice data and the conversion of phonemes are carried out in real time, the phoneme sequence after the switching position can be used as the target phoneme sequence, and step S803 and subsequent steps are executed to complete the evaluation of the reading or reciting voice data of the next article. The evaluation of the reading or reciting voice data of other subsequent articles is similar.

[0213] The speech evaluation method provided in the embodiment of the present application first obtains the speech data of the user when reading or reciting multiple articles continuously in real time, then processes the speech data obtained in real time into a phoneme sequence in real time to obtain a target phoneme sequence, then traces the target phoneme sequence to obtain a target traceable text, and according to the target phoneme sequence, determines whether the user has currently completed the reading or reciting of an article. If the user has currently completed the reading or reciting of an article, the target traceable text is referred to to evaluate the reading or reciting speech data of the article currently read or recited by the user. Since the speech evaluation method provided in the embodiment of the present application refers to the target traceable text obtained by tracing the phoneme sequence corresponding to the user's reading or reciting speech data, the speech data to be evaluated is evaluated, rather than the speech recognition result of the user's reading or reciting speech data is evaluated, therefore, a more accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the embodiment of the present application allows the user to select no article, and the user can read or recite multiple articles continuously, which greatly improves the user experience.

[0214] The above describes the speech evaluation method provided in the embodiment of the present application. The following describes the device corresponding to the above speech evaluation method.

[0215] See also Fig. 9 , Fig. 9A structural schematic diagram of a speech evaluation device provided in an embodiment of the present application, the speech evaluation device may include: a speech data acquisition module 901, a speech data processing module 902, a first tracing module 903a, a second tracing module 903b, a target tracing text determination module 904 and a speech evaluation module 905.

[0216] The voice data acquisition module 901 is used to acquire the voice data to be evaluated.

[0217] The speech data processing module 902 is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence.

[0218] The first source tracing module 903a is used to obtain the source text of the text corresponding to the target phoneme sequence based on the large language model and the target phoneme sequence, and obtain the first source tracing text.

[0219] The second tracing module 903b is used to process the target phoneme sequence into a plurality of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, so as to obtain a second tracing text.

[0220] The target source text determination module 904 is used to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively.

[0221] The speech evaluation module 905 is used to evaluate the speech data to be evaluated with reference to the target source text.

[0222] In a possible implementation, when acquiring the voice data to be evaluated, the voice data acquisition module 901 is specifically used to:

[0223] Acquire voice data in real time when users read or recite multiple articles continuously.

[0224] When the speech data processing module 902 processes the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence, it is specifically used to:

[0225] The voice data acquired in real time is processed into a phoneme sequence to obtain a target phoneme sequence.

[0226] The speech evaluation module 905 is specifically used to:

[0227] Determine whether the user has completed reading or reciting an article;

[0228] If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

[0229] In a possible implementation, when the target source text determination module 904 determines the target source text according to the relevance between the first source text and the second source text and the target phoneme sequence, it is specifically configured to:

[0230] Obtaining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities;

[0231] Used to determine the target source text based on two similarities.

[0232] In a possible implementation, when processing the target phoneme sequence into a plurality of phoneme subsequences, the second source tracing module 903b is specifically configured to:

[0233] The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0234] In a possible implementation, when the second source tracing module 903b determines the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source tracing text, it is specifically used to:

[0235] Based on a pre-constructed reverse index table, determining article identifiers corresponding to a plurality of first n-gram units, wherein the reverse index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and article identifiers corresponding to the plurality of n-gram units;

[0236] Counting the number of occurrences of article identifiers corresponding to a number of first n-gram units to obtain statistical results;

[0237] According to the statistical results, determine the target article identification;

[0238] The article in the article library that is identified as the target article is determined as the second tracing text.

[0239] In a possible implementation, when determining the target article identifier according to the statistical results, the second source tracing module 903b is specifically configured to:

[0240] Determine the K article identifiers with the largest number of occurrences as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1;

[0241] Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article;

[0242] Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier;

[0243] The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

[0244] In a possible implementation, when the target source text determination module 904 determines the target source text according to the two similarities, it is specifically configured to:

[0245] The maximum similarity of the two similarities is determined as the target similarity;

[0246] If the target similarity is greater than or equal to the preset similarity threshold, the source text corresponding to the phoneme sequence whose similarity to the target phoneme sequence is the target similarity is determined as the target source text;

[0247] If the target similarity is less than the preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected. The corrected text is used as the target tracing text.

[0248] The speech evaluation module 905 includes a chapter switching determination submodule. The chapter switching determination submodule is used to determine whether the user has completed reading or reciting an article according to the target phoneme sequence.

[0249] In a possible implementation, the chapter switching determination submodule is specifically used to determine whether the user has completed reading or reciting an article according to the target phoneme sequence:

[0250] Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length;

[0251] If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

[0252] In a possible implementation, when the chapter switching determination submodule determines whether the user has currently completed reading or reciting an article according to the target phoneme sequence, it is specifically used to:

[0253] Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1;

[0254] Based on a pre-constructed inverted index table, determining article identifiers corresponding to a plurality of second n-gram units;

[0255] Whether the user has currently completed reading aloud an article is determined according to the article identifiers respectively corresponding to the plurality of second n-gram units.

[0256] In a possible implementation, the chapter switching determination submodule is specifically used to determine whether the user has completed reading or reciting an article according to the target phoneme sequence:

[0257] When the target source text is the first source text or the second source text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text;

[0258] If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

[0259] In a possible implementation, the chapter switching determination submodule is specifically used to determine whether the user has completed reading or reciting an article according to the target phoneme sequence:

[0260] In the case where the target source text is a revised text, performing sentence-level N-gram processing on the target source text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source text, wherein N is an integer greater than 1;

[0261] Based on the pre-trained classification model, classify each sentence-level N-gram unit to obtain the classification result of each sentence-level N-gram unit, wherein the classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has undergone article switching;

[0262] According to the classification results of N-gram units at each sentence level, it is determined whether the user has currently completed reading aloud an article.

[0263] Since the speech evaluation device provided in the embodiment of the present application evaluates the speech data to be evaluated by referring to the target source text obtained by tracing the phoneme sequence corresponding to the speech data to be evaluated, rather than referring to the speech recognition result of the speech data to be evaluated, a relatively accurate evaluation result can be obtained. In addition, the speech evaluation device provided in the present application eliminates the need for users to select articles, and users can read or recite multiple articles continuously, which greatly improves the user experience.

[0264] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0265] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0266] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0267] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0268] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the speech evaluation method provided in the above embodiment.

[0269] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech evaluation method provided in the above embodiment.

[0270] The embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the speech evaluation method provided in the above embodiment.

[0271] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0272] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0273] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0274] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A speech evaluation method, characterized in that: include: Obtain the voice data to be evaluated; Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence; Based on the large language model and taking the target phoneme sequence as a basis, obtaining a source text of the text corresponding to the target phoneme sequence to obtain a first source text; Processing the target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text; Determining a target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively; The speech data to be evaluated is evaluated with reference to the target source text.

2. The speech evaluation method according to claim 1, characterized in that: The step of obtaining the voice data to be evaluated includes: Real-time acquisition of voice data when users continuously read or recite multiple articles; Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence includes: Processing the real-time acquired speech data into a phoneme sequence to obtain a target phoneme sequence; The step of evaluating the speech data to be evaluated by referring to the target source text includes: According to the target phoneme sequence, determining whether the user has currently completed reading or reciting an article; If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

3. The speech evaluation method according to claim 1 or 2, characterized in that: The step of determining the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively includes: Obtaining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities; According to the two similarities, the target source tracing text is determined.

4. The speech evaluation method according to claim 1, characterized in that: The step of processing the target phoneme sequence into a plurality of phoneme subsequences comprises: The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

5. The speech evaluation method according to claim 4, characterized in that: The step of determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source text includes: Based on a pre-constructed reverse index table, determining article identifiers corresponding to the plurality of first n-gram units, wherein the reverse index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and article identifiers corresponding to the plurality of n-gram units; Counting the number of occurrences of the article identifiers corresponding to the plurality of first n-gram units to obtain a statistical result; Determine the target article identifier according to the statistical results; An article in the article library with an article identifier as the target article identifier is determined as the second source tracing text.

6. The speech evaluation method according to claim 5, characterized in that: Determining the target article identifier according to the statistical result includes: Determine the K article identifiers with the largest number of occurrences as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1; Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article; Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier; The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

7. The speech evaluation method according to claim 3, characterized in that: Determining the target source tracing text according to the two similarities includes: Determine the maximum similarity of the two similarities as the target similarity; If the target similarity is greater than or equal to a preset similarity threshold, the source tracing text corresponding to the phoneme sequence whose similarity to the target phoneme sequence is the target similarity is determined as the target source tracing text; If the target similarity is less than a preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

8. The speech evaluation method according to claim 2, characterized in that: The step of determining whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes: Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length; If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

9. The speech evaluation method according to claim 2, characterized in that: The step of determining whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes: Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1; Determine article identifiers corresponding to the plurality of second n-gram units based on a pre-constructed reverse index table, wherein the reverse index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and article identifiers corresponding to the plurality of n-gram units; Whether the user has currently completed reading aloud an article is determined according to the article identifiers respectively corresponding to the plurality of second n-gram units.

10. The speech evaluation method according to claim 7, characterized in that: The step of determining whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes: In a case where the target source-tracing text is the first source-tracing text or the second source-tracing text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source-tracing text; If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

11. The speech evaluation method according to claim 7, characterized in that: The step of determining whether the user has currently completed reading or reciting an article according to the target phoneme sequence includes: In the case where the target source tracing text is the corrected text, performing sentence-level N-gram processing on the target source tracing text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source tracing text, wherein N is an integer greater than 1; Based on the pre-trained classification model, classify each sentence-level N-gram unit to obtain the classification result of each sentence-level N-gram unit, wherein the classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has undergone article switching; According to the classification results of N-gram units at each sentence level, it is determined whether the user has currently completed reading aloud an article.

12. The speech evaluation method according to claim 1, characterized in that: The step of evaluating the speech data to be evaluated by referring to the target source text includes: According to the target tracing text, the general decoding network is tailored to obtain a limited decoding network; Extracting speech features from the speech data to be evaluated, and decoding the speech features based on the limited decoding network to obtain a decoding result; Processing the decoding result through a language model to obtain a phoneme sequence; The target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain a scoring result.

13. A speech evaluation device, characterized in that: include: A voice data acquisition module, a voice data processing module, a first tracing module, a second tracing module, a target tracing text determination module and a voice evaluation module; The voice data acquisition module is used to acquire the voice data to be evaluated; The speech data processing module is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence; The first source tracing module is used to obtain the source text of the text corresponding to the target phoneme sequence based on the large language model and the target phoneme sequence, so as to obtain a first source tracing text; The second source tracing module is used to process the target phoneme sequence into a plurality of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, so as to obtain a second source tracing text; The target source text determination module is used to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively; The speech evaluation module is used to evaluate the speech data to be evaluated with reference to the target source tracing text.

14. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the speech evaluation method as described in any one of claims 1 to 12.

15. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech evaluation method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the steps of the speech evaluation method according to any one of claims 1 to 12.

Images