A voice evaluation method and related device

Abstract

The present application discloses a speech evaluation method and related devices, which relate to the field of speech evaluation technology. The speech evaluation method includes: obtaining speech data to be evaluated; processing the speech data to be evaluated into a phoneme sequence to obtain a target phoneme sequence; based on a large language model and taking the target phoneme sequence as a basis, obtaining the source text of the text corresponding to the target phoneme sequence to obtain a first tracing text; processing the target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain a second tracing text; determining the target tracing text based on the degree of relevance between the first tracing text and the second tracing text and the target phoneme sequence respectively; and evaluating the speech data to be evaluated with reference to the target tracing text. The speech evaluation method disclosed in the present application evaluates the speech data to be evaluated with reference to the target tracing text, and can obtain relatively accurate evaluation results.

Description

Technical Field

[0001] The present application relates to the field of speech evaluation technology, and in particular to a speech evaluation method and related devices. Background Art

[0002] In some application scenarios, voice data (such as user reading or reciting voice data) needs to be evaluated.

[0003] The current evaluation plan is: first obtain the voice data to be evaluated (such as the user's reading or recitation voice data), then perform voice recognition on the voice data to be evaluated based on the voice recognition model to obtain voice recognition text, and finally evaluate the voice data to be evaluated with reference to the voice recognition text.

[0004] However, the speech recognition text obtained by performing speech recognition on the evaluation speech data based on the speech recognition model may contain errors (for example, the user reads "The spring scenery in the garden cannot be contained, a red apricot branch comes out of the wall", but the text recognized by the speech recognition model is "The spring scenery in the garden cannot be contained, a red apricot branch comes out of the wall"). Evaluating the evaluation speech data with reference to the erroneous speech recognition text will lead to inaccurate evaluation results. Summary of the Invention

[0005] In view of this, the present application provides a speech evaluation method and related devices to solve the problem of inaccurate evaluation results of existing speech evaluation methods. The technical solution is as follows:

[0006] The first aspect of the present application provides a speech evaluation method, comprising:

[0007] Obtain the voice data to be evaluated;

[0008] Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence;

[0009] Based on the large language model and the target phoneme sequence, a source text of the text corresponding to the target phoneme sequence is obtained to obtain a first source text;

[0010] Processing the target phoneme sequence into a plurality of phoneme subsequences, and determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, to obtain a second source text;

[0011] determining a target source text according to the relevance of the first source text and the second source text to the target phoneme sequence;

[0012] The speech data to be evaluated is evaluated with reference to the target source text.

[0013] In a possible implementation, obtaining the speech data to be evaluated includes:

[0014] Real-time acquisition of voice data when users continuously read or recite multiple articles;

[0015] Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence includes:

[0016] Processing the real-time acquired speech data into a phoneme sequence to obtain a target phoneme sequence;

[0017] The step of evaluating the speech data to be evaluated by referring to the target source text includes:

[0018] Determining whether the user has completed reading or reciting an article based on the target phoneme sequence;

[0019] If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

[0020] In a possible implementation, determining the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence includes:

[0021] Obtaining a similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and a similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities;

[0022] The target source text is determined based on the two similarities.

[0023] In a possible implementation, processing the target phoneme sequence into a plurality of phoneme subsequences includes:

[0024] The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0025] In a possible implementation, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source text includes:

[0026] Determining, based on a pre-constructed inverted index table, article identifiers corresponding to the plurality of first n-gram units, wherein the inverted index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and the article identifiers corresponding to the plurality of n-gram units;

[0027] Counting the number of occurrences of the article identifiers corresponding to the plurality of first n-gram units to obtain a statistical result;

[0028] Determining a target article identifier based on the statistical results;

[0029] An article in the article library identified as the target article is determined as the second source text.

[0030] In a possible implementation, determining the target article identifier according to the statistical results includes:

[0031] Determine the K article identifiers that appear the most times as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1;

[0032] Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article;

[0033] Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier;

[0034] The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

[0035] In a possible implementation, determining the target source text according to the two similarities includes:

[0036] Determine the maximum similarity of the two similarities as the target similarity;

[0037] If the target similarity is greater than or equal to a preset similarity threshold, the source text corresponding to the phoneme sequence having the target similarity as the target phoneme sequence is determined as the target source text;

[0038] If the target similarity is less than a preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

[0039] In a possible implementation, determining whether the user has currently completed reading or reciting an article based on the target phoneme sequence includes:

[0040] Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length;

[0041] If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

[0042] In a possible implementation, determining whether the user has currently completed reading or reciting an article based on the target phoneme sequence includes:

[0043] Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1;

[0044] Determining, based on a pre-constructed inverted index table, article identifiers corresponding to the plurality of second n-gram units, wherein the inverted index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and the article identifiers corresponding to the plurality of n-gram units;

[0045] Whether the user has currently completed reading aloud an article is determined based on the article identifiers corresponding to the plurality of second n-gram units.

[0046] In a possible implementation, determining whether the user has currently completed reading or reciting an article based on the target phoneme sequence includes:

[0047] When the target source text is the first source text or the second source text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text;

[0048] If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

[0049] In a possible implementation, determining whether the user has currently completed reading or reciting an article based on the target phoneme sequence includes:

[0050] When the target source text is the revised text, performing sentence-level N-gram processing on the target source text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source text, where N is an integer greater than 1;

[0051] Based on the pre-trained classification model, each sentence-level N-gram unit is classified to obtain a classification result for each sentence-level N-gram unit. The classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has experienced article switching.

[0052] Based on the classification results of N-gram units at each sentence level, it is determined whether the user has completed reading aloud an article.

[0053] In a possible implementation, evaluating the speech data to be evaluated with reference to the target source text includes:

[0054] According to the target tracing text, the general decoding network is tailored to obtain a limited decoding network;

[0055] Extracting speech features from the speech data to be evaluated, and decoding the speech features based on the limited decoding network to obtain a decoding result;

[0056] Processing the decoding result through a language model to obtain a phoneme sequence;

[0057] The target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain an evaluation result.

[0058] A second aspect of the present application provides a speech evaluation device, comprising: a speech data acquisition module, a speech data processing module, a first tracing module, a second tracing module, a target tracing text determination module, and a speech evaluation module;

[0059] The voice data acquisition module is used to acquire the voice data to be evaluated;

[0060] The speech data processing module is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence;

[0061] The first source tracing module is configured to obtain a source text of a text corresponding to the target phoneme sequence based on a large language model and the target phoneme sequence, thereby obtaining a first source tracing text;

[0062] The second source tracing module is configured to process the target phoneme sequence into a plurality of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, thereby obtaining a second source text;

[0063] The target source text determination module is configured to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively;

[0064] The speech evaluation module is used to evaluate the speech data to be evaluated with reference to the target source text.

[0065] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0066] The memory is used to store computer programs;

[0067] The processor is configured to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned speech evaluation methods.

[0068] In a fourth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned speech evaluation methods.

[0069] In a fifth aspect, the present application provides a computer program product comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of any one of the above-described speech evaluation methods.

[0070] By means of the above technical solution, the speech evaluation method provided by this application first obtains the speech data to be evaluated, then processes the speech data to be evaluated into a phoneme sequence to obtain a target phoneme sequence, then uses two different tracing methods to trace the target phoneme sequence to obtain a target source text, and finally evaluates the speech data to be evaluated with reference to the target source text. Because the speech evaluation method provided by this application evaluates the speech data to be evaluated with reference to the target source text, rather than with reference to the speech recognition results of the speech data to be evaluated, a more accurate evaluation result can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0072] Figure 1 A schematic diagram of a system architecture involved in this application;

[0073] Figure 2 A schematic diagram of the hardware structure of a terminal provided in an embodiment of the present application;

[0074] Figure 3 A schematic diagram of the hardware structure of a server provided in an embodiment of the present application;

[0075] Figure 4 A flowchart of the speech evaluation method provided in an embodiment of the present application;

[0076] Figure 5 A schematic diagram of an implementation method for evaluating speech data to be evaluated, which is a reference target source text provided in an embodiment of the present application;

[0077] Figure 6 A flowchart of an embodiment of the present application for processing a target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text;

[0078] Figure 7 A flowchart of a specific example of the speech evaluation method provided in an embodiment of the present application;

[0079] Figure 8 A flowchart of another specific example of the speech evaluation method provided in an embodiment of the present application;

[0080] Figure 9 This is a schematic diagram of the structure of the speech evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0082] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0083] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0084] In one possible implementation, Figure 1 As shown, the system architecture involved in this application may include a terminal 101 and a server 102, and the terminal 101 may interact with the server 102 via a network (wired network or wireless network). The server 102 may include one or more servers ( Figure 1 (The example includes a server.) Terminal 101 can obtain speech data to be evaluated and transmit the speech data to be evaluated to server 102 via a network. Server 102 evaluates the speech data using the speech evaluation method provided in this application and transmits the speech evaluation results to terminal 101 via a network. Terminal 101 can then output the speech evaluation results.

[0085] In another possible implementation, the system architecture involved in this application may include a terminal. The terminal has strong data processing capabilities. The terminal can use the voice evaluation method provided in this application to evaluate the voice data to be evaluated, and then output the voice evaluation results.

[0086] Next, the product form of the above-mentioned terminal will be described.

[0087] The above-mentioned terminal can be a mobile phone, a tablet computer, a wearable device, an in-vehicle device, a robot, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions on this.

[0088] Figure 2 A schematic diagram of an optional hardware structure of a terminal is shown.

[0089] refer to Figure 2As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will understand that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0090] The input unit 230 can be used to receive input digital or character information and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect user touch operations on or near it (for example, operations performed on or near the touch screen using a finger, joint, stylus, or any other suitable object) and drive corresponding connected devices according to a pre-set program. The touch screen can detect user touch actions on the touch screen, convert the touch actions into touch signals and transmit them to the processor 270, and can receive and execute commands sent by the processor 270; the touch signals include at least touch point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, touch screens can be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 231, the input unit 230 may also include other input devices. Specifically, the other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, a joystick, and the like.

[0091] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0092] Memory 220 can be used to store instructions and data. Memory 220 primarily includes an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files and text. The instruction storage area can store software units such as the operating system, applications, and instructions required for at least one function, or subsets or extensions thereof. It may also include non-volatile random access memory (RAM). It provides processor 270 with management functions for the hardware, software, and data resources within the computing and processing device, supporting control software and applications. It is also used to store multimedia files and running programs and applications.

[0093] The processor 270 is the terminal's control center, connecting all components of the terminal using various interfaces and circuits. By executing instructions stored in memory 220 and accessing data stored in memory 220, it performs various terminal functions and processes data, thereby providing overall control of the terminal. Optionally, the processor 270 may include one or more processing units. Preferably, the processor 270 integrates an application processor and a modem processor, with the application processor primarily processing the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 270. In some embodiments, the processor and memory may be implemented on a single chip; in other embodiments, they may be implemented on separate chips. The processor 270 may also generate corresponding operational control signals and send them to corresponding components of the computing and processing device. It may also read and process data in the software, particularly the data and programs in memory 220, to enable the various functional modules therein to perform their corresponding functions, thereby controlling the corresponding components to operate as instructed.

[0094] The memory 220 may be used to store software codes related to the speech evaluation method, and the processor 270 may execute the software codes in the memory 220 or schedule other units (such as the input unit 230 and the display unit 240 ) to implement corresponding functions.

[0095] The RF unit 210 (optional) can be used to send and receive information or receive and send signals during a call. For example, it receives downlink information from the base station and passes it to the processor 270 for processing; in addition, it sends uplink data to the base station. Typically, the RF unit 210 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF unit 210 can communicate with network devices and other devices via wireless communication. This wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0096] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional and can be replaced by other communication interfaces, such as a network port.

[0097] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0098] The terminal further includes an external interface 280 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to other devices for communication, or to connect a charger for charging the terminal.

[0099] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0100] Next, the product form of the above server will be described.

[0101] Figure 3 A structural diagram of the above server is provided, such as Figure 3 As shown, the server may include a bus 301 , a processor 302 , a communication interface 303 , and a memory 304 . The processor 302 , the memory 304 , and the communication interface 303 communicate with each other via the bus 301 .

[0102] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0103] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0104] The memory 304 may include volatile memory, such as random access memory (RAM). The memory 304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0105] The memory 304 may be used to store software codes related to the speech evaluation method. The processor 302 may call the software codes stored in the memory 304 or schedule other units to implement corresponding functions.

[0106] The processors in the above-mentioned terminals and servers (such as processor 270 and processor 302) can be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0107] The inventors of this case discovered during the process of implementing this case that the current speech evaluation scheme evaluates the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated. Since the speech recognition text of the speech data to be evaluated may contain errors, the evaluation results obtained by evaluating the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated may be inaccurate.

[0108] In response to the problems existing in current voice evaluation schemes, the inventors of this case conducted research. The initial idea was to pre-set an article library in the voice evaluation system. Before reading or reciting, the user first selects the article to be read or recited from the article library, then clicks to start recording and reads or recites the selected article. After the user finishes reading or reciting, clicks to end recording and waits for the voice evaluation system to give evaluation feedback. The process of voice evaluation by the voice evaluation system is to obtain voice data of the user reading or reciting the entire article, and evaluate the obtained voice data with reference to the article selected by the user.

[0109] Since the above idea evaluates the speech data to be evaluated with reference to the article selected by the user, rather than evaluating the speech data to be evaluated with reference to the speech recognition text of the speech data to be evaluated, a more accurate evaluation result can be obtained.

[0110] The inventors of this case studied the above ideas and found that although the above ideas can obtain relatively accurate evaluation results, there is a problem: before reading or reciting, the user needs to select the article to be read or recited from the system's article library. It is understandable that if the user wants to read or recite multiple texts, before reading or reciting each article, it is necessary to select the article, and before reading or reciting each article, an operation is required to trigger the start of recording. After reading or reciting each article, an operation is required to trigger the end of recording. It can be seen that the user operation is cumbersome and cannot enable the user to read or recite continuously, resulting in a poor user experience.

[0111] In view of the shortcomings of the above ideas, the inventors of this case continued to study and finally proposed a speech evaluation method with better effect. The speech evaluation method provided by this application is introduced through the following embodiments.

[0112] See also Figure 4 , which shows a flow chart of a voice evaluation method provided in an embodiment of the present application, and which may include:

[0113] Step S401: Acquire speech data to be evaluated.

[0114] The voice data to be evaluated in this embodiment may be the user's reading or reciting voice data.

[0115] Step S402: Process the audio data to be tested and evaluated into a phoneme sequence to obtain a target phoneme sequence.

[0116] In a possible implementation, the audio data to be tested and evaluated may be processed into a phoneme sequence based on a speech recognition model.

[0117] It should be noted that the process of speech recognition model performing speech recognition on input speech data is to first process the input speech data into a phoneme sequence, and then determine the final speech recognition result based on the obtained phoneme sequence. This application only uses the phoneme sequence acquisition function of the speech recognition model to obtain the phoneme sequence for the speech data to be evaluated.

[0118] For example, the content of the speech data to be evaluated is "The spring scenery in the garden cannot be contained, and a red apricot tree comes out of the wall." Based on the speech recognition model, the speech data to be evaluated is processed to obtain the phoneme sequence "man yuan chun se guan buzhu yi zhi hong xing chu qiang lai."

[0119] In this embodiment, the speech data to be evaluated can be either Chinese or English. If the speech data to be evaluated is Chinese, the phonemes in the target phoneme sequence are pinyin. If the speech data to be evaluated is English, the phonemes in the target phoneme sequence are phonetic symbols. Of course, this embodiment is not limited to Chinese or English speech data; the speech data to be evaluated can also be speech data in other languages, such as German, French, etc.

[0120] Step S403a: Based on the large language model and the target phoneme sequence, the source text of the text corresponding to the target phoneme sequence is obtained to obtain a first source text.

[0121] Specifically, the preset first instruction prompt information and the target phoneme sequence are organized into a first prompt instruction. The first prompt instruction is used to prompt the large language model to trace the target phoneme sequence. After obtaining the first prompt instruction, the first prompt instruction is input into the large language model. The large language model traces the input target phoneme sequence and outputs a first tracing text.

[0122] For example, the target phoneme sequence is "man yuan chun se guan bu zhu yi zhi hong xingchu qiang lai", and the first prompt instruction is "If you are a Chinese teacher, please use [man yuan chun se guanbu zhu yi zhi hong xing chu qiang lai] to find the possible poem that this pinyin is reading aloud. There may be omissions, misreadings, or additional readings. Please output the original poem." The first prompt instruction is input into the large language model, and the large language model predicts the source text of the text corresponding to [man yuan chun se guan bu zhu yi zhi hong xing chu qiang lai] and outputs "I should pity the imprints of my clogs on the moss, and the wooden door that has not been opened for a long time after I knocked on it. The spring scenery in the garden cannot be contained, and a red apricot branch comes out of the wall." from "Not Worth Visiting the Garden".

[0123] Step S403b: Process the target phoneme sequence into a number of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain a second source text.

[0124] In order to determine a more accurate source text for the target phoneme sequence, in addition to using a text tracing method based on a large language model to perform text tracing on the target phoneme sequence, this embodiment also uses another text tracing method to perform text tracing on the target phoneme sequence, that is, the target phoneme sequence is processed into several phoneme subsequences, and the source text of the text corresponding to the target phoneme sequence is determined by determining the source text of the text corresponding to each phoneme subsequence.

[0125] Step S404: Determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence.

[0126] The degree of relevance between the target phoneme sequence and the first source text can be determined, and the degree of relevance between the target phoneme sequence and the second source text can be determined. Then, based on the degree of relevance between the target phoneme sequence and the first source text and the degree of relevance between the target phoneme sequence and the second source text, the target source text can be determined.

[0127] This embodiment uses two different text tracing methods to perform text tracing on the target phoneme sequence respectively, and then determines the final tracing text, that is, the target tracing text, based on the tracing texts obtained using the two different text tracing methods.

[0128] Step S405: Evaluate the speech data to be evaluated with reference to the target source text.

[0129] There are many ways to evaluate the speech data to be evaluated with reference to the target source text. In one possible implementation method, speech features (such as MFCC features) can be extracted from the speech data to be evaluated. The extracted speech features are then decoded based on a general decoding network to obtain a decoding result. The decoding result is then processed by a language model to obtain a phoneme sequence. Finally, the target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain an evaluation result.

[0130] In order to improve the evaluation effect, this embodiment provides another implementation method of evaluating the speech data to be evaluated by referring to the target source text, such as Figure 5 As shown, first, the general decoding network is trimmed according to the target source text to obtain a restricted decoding network, and then speech features (such as MFCC features) are extracted from the speech data to be evaluated, and the extracted speech features are decoded based on the restricted decoding network to obtain a decoding result. The obtained decoding result is then processed by a language model to obtain a phoneme sequence. Finally, the target source text and the phoneme sequence obtained by the language model are input into the scoring model for scoring to obtain an evaluation result.

[0131] The speech evaluation method provided in the embodiment of the present application first obtains the speech data to be evaluated, then processes the speech data to be evaluated into a phoneme sequence to obtain a target phoneme sequence, then traces the target phoneme sequence to its source to obtain a target source text, and finally evaluates the speech data to be evaluated with reference to the target source text. Because the speech evaluation method provided in the embodiment of the present application evaluates the speech data to be evaluated with reference to the target source text, rather than with reference to the speech recognition results of the speech data to be evaluated, a more accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the present application does not require the user to select an article, thereby improving the user experience.

[0132] In another embodiment of the present application, the specific implementation process of "Step S403b: processing the target phoneme sequence into several phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second tracing text" in the above embodiment is introduced.

[0133] See also Figure 6 , which shows a flow chart of processing a target phoneme sequence into a plurality of phoneme subsequences, determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, and obtaining a second source text, which may include:

[0134] Step S601: performing phoneme-level n-gram processing on the target phoneme sequence to obtain a plurality of first n-gram units.

[0135] Each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0136] For example, the target phoneme sequence is "man yuan chun se guan bu zhu", assuming n=3, and 3-gram processing is performed on the target phoneme sequence to obtain multiple first 3-gram units: man-yuan-chun, yuan-chun-se, chun-se-guan, se-guan-bu, guan bu zhu.

[0137] Step S602: Based on the pre-built inverted index table, determine the article identifiers corresponding to the first n-gram units.

[0138] The article identifier can be, but is not limited to, the article title.

[0139] The reverse index table in this embodiment is constructed based on each article in the article library, and includes multiple n-gram units obtained by performing n-gram processing on the phoneme sequence corresponding to the article content of each article in the article library. The reverse index table also includes article identifiers corresponding to the multiple n-gram units obtained by performing n-gram processing on the phoneme sequence corresponding to the article content of each article in the article library.

[0140] For example, the reverse index table is as follows:

[0141] {

[0142] “man yuan chun”:[“Not Worth Visiting the Garden”],

[0143] “yuan chun se”:[“Not Worth Visiting the Garden”], ...

[0145] }

[0146] In the inverted index table, "man yuan chun", "yuan chun se" and so on are indexes. It should be noted that the article identifier corresponding to any n-gram unit in the inverted index table may be one or more.

[0147] Step S603: Count the number of occurrences of the article identifiers corresponding to the first n-gram units to obtain a statistical result.

[0148] Exemplarily, there are 5 first n-gram units. The article identifiers corresponding to the 1st first n-gram unit are A and B, the article identifier corresponding to the 2nd first n-gram unit is A, the article identifiers corresponding to the 3rd first n-gram unit are A and C, the article identifiers corresponding to the 4th first n-gram unit are A and B, and the article identifier corresponding to the 5th first n-gram unit is A. Then, count the number of occurrences of article identifier A, the number of occurrences of article identifier B, and the number of occurrences of article identifier C. After statistics, article identifier A appears 5 times, article identifier B appears 2 times, and article identifier C appears 1 time.

[0149] Step S604: Determine the target article identifier according to the statistical result.

[0150] There are multiple ways to determine the target article identifier according to the statistical result. Two implementation manners are provided in this embodiment.

[0151] The first implementation manner: Determine the article identifier with the most occurrences (sort the article identifiers in descending order of the number of occurrences, and the article identifier ranked first) as the target article identifier.

[0152] The second implementation manner: Determine the K (K is an integer greater than 1) article identifiers with the most occurrences (sort the article identifiers in descending order of the number of occurrences, and the article identifiers ranked in the top K) as candidate article identifiers, obtaining K candidate article identifiers; from the pre-constructed forward index table, obtain the phoneme sequences corresponding to the K candidate article identifiers respectively; determine the similarity between the target phoneme sequence and the phoneme sequences corresponding to each candidate article identifier; determine the candidate article identifier corresponding to the phoneme sequence with the highest similarity to the target phoneme sequence as the target article identifier.

[0153] Among them, the forward index table is constructed based on each article in the article library, and it includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article. The phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of this article.

[0154] Exemplarily, the forward index table is as follows:

[0155] {

[0156] " "You Yuan Bu Zhi"":["…man yuan chun se guan bu zhu…"],

[0157] " "Shan Xing"":["…bai yun sheng chu you ren jia…"], ...

[0159] }

[0160] "Not Worth Visiting the Garden" and "Mountain Hike" in the forward index table are indexes.

[0161] The similarity between the target phoneme sequence and the phoneme sequence corresponding to any candidate article identifier can be determined as follows: the number of matching phonemes between the target phoneme sequence and the phoneme sequence corresponding to the candidate article identifier / the total number of phonemes in the target phoneme sequence. For example, if the target phoneme sequence includes 7 phonemes and it matches 6 phonemes with the phoneme sequence corresponding to the candidate article identifier (the phoneme sequence corresponding to the candidate article identifier covers 6 phonemes in the target phoneme sequence), then the similarity between the target phoneme sequence and the phoneme sequence corresponding to the candidate article identifier is 6 / 7.

[0162] Step S605: an article in the article library identified as an article with a target article identification is determined as a second source tracing text.

[0163] In another embodiment of the present application, the specific implementation process of "step S404: determining the target source text according to the degree of relevance of the first source text and the second source text to the target phoneme sequence respectively" is introduced.

[0164] The process of determining the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence may include:

[0165] Step a1: Obtain the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, and obtain two similarities.

[0166] The method for determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text is the same as the method for determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to any candidate article identifier, and this embodiment will not be repeated here.

[0167] Step a2: Determine the target source text based on the two similarities.

[0168] In one possible implementation, the process of determining the target source text based on the two similarities may include:

[0169] Step a21: determine the maximum similarity between the two similarities as the target similarity.

[0170] Step a22-a: If the target similarity is greater than or equal to the preset similarity threshold, the source text corresponding to the phoneme sequence having the target similarity to the target phoneme sequence is determined as the target source text.

[0171] Exemplarily, the preset similarity threshold is 0.8, the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text is greater than the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text is greater than 0.8, then the first source text is determined to be the target source text.

[0172] It should be noted that the preset similarity threshold of 0.8 is only an example, and the specific value of the preset similarity threshold can be set according to actual application scenarios.

[0173] Step a22-b: If the target similarity is less than the preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

[0174] In actual applications, there may be a situation where the target similarity is less than the preset similarity threshold. When this happens, the text corresponding to the target phoneme sequence can be predicted based on the large language model, and the predicted text can be corrected and used as the target tracing text.

[0175] When predicting the text corresponding to the target phoneme sequence based on the large language model and correcting the predicted text, the preset second instruction prompt information and the target phoneme sequence can be organized into a second prompt instruction. The second prompt instruction is used to prompt the large language model to predict the text corresponding to the target phoneme sequence and correct the predicted text. After obtaining the second prompt instruction, the second prompt instruction is input into the large language model. The large language model predicts the text corresponding to the target phoneme sequence, corrects the predicted text, and outputs the corrected text.

[0176] For example, the second prompt instruction is "If you are a Chinese teacher, please give a prediction based on [jin tian tian qiqing lang wo men de xin qing hen yu kuai]: 1. What is the corresponding Chinese text? 2. If there are illogical grammatical errors in the original text, you need to correct them and give the text." The second prompt instruction is input into the large language model, and the large language model predicts the text corresponding to "jin tian tian qi qing lang wo men de xin qing hen yukuai", corrects the predicted text, and outputs it.

[0177] Based on the speech evaluation method provided in the above embodiment, a specific example of the speech evaluation method is given below. This specific example is suitable for speech evaluation in the scenario of reading or reciting a single article. Figure 7 , which shows a flow chart of this specific example, may include:

[0178] Step S701: Acquire voice data of a user reading or reciting an article.

[0179] When a user's start recording trigger operation is detected, voice data of the user reading or reciting an article is obtained.

[0180] Step S702: After obtaining the voice data of the user reading or reciting the entire article, the voice data of the user reading or reciting the entire article is processed into a phoneme sequence to obtain a target phoneme sequence.

[0181] In one possible implementation, the user's voice data of reading or reciting the entire article may be processed into a phoneme sequence based on a speech recognition model.

[0182] In the scenario of reading or reciting a single article, when the user's end recording trigger operation is detected, it is considered that the user has recited or read the entire article. At this time, the reading or reciting voice data of the entire article can be obtained, and then the reading or reciting voice data of the entire article can be processed into a phoneme sequence to obtain the target phoneme sequence.

[0183] Step S703: Tracing the target phoneme sequence to obtain the target tracing text.

[0184] The target phoneme sequence is traced back to its source, that is, the source of the text corresponding to the target phoneme sequence is found. The target traced back text is the source text of the text corresponding to the target phoneme sequence.

[0185] The specific implementation process of tracing the target phoneme sequence to obtain the target source text can be found in the above embodiment, which will not be described in detail in this embodiment.

[0186] Step S704: Evaluate the reading or recitation voice data of the entire article in combination with the target source text.

[0187] The general decoding network can be tailored according to the target source text to obtain a restricted decoding network. Speech features (such as MFCC features) can be extracted from the reading or recitation voice data of the entire article, and the extracted speech features can be decoded based on the restricted decoding network to obtain a decoding result. The decoding result is processed by a language model to obtain a phoneme sequence. The phoneme sequence and the target source text are input into the scoring model for scoring to obtain the evaluation result.

[0188] The speech evaluation method provided in the embodiment of the present application first obtains the speech data when the user reads aloud or recites an article, and then after obtaining the speech data of the user reading aloud or reciting the entire article, the speech data of the user reading aloud or reciting the entire article is processed into a phoneme sequence to obtain a target phoneme sequence, and then the target phoneme sequence is traced to obtain a target source text, and finally, the speech data of the user reading aloud or reciting the entire article is evaluated with reference to the target source text. Since the speech evaluation method provided in the embodiment of the present application evaluates the user's reading aloud or reciting speech data with reference to the target source text obtained by tracing the phoneme sequence corresponding to the user's reading aloud or reciting speech data, rather than evaluating with reference to the speech recognition result of the user's reading aloud or reciting speech data, a more accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the present application does not require the user to select an article, thereby improving the user experience.

[0189] Based on the speech evaluation method provided in the above embodiment, another specific example of the speech evaluation method is given below. This specific example is suitable for speech evaluation in the scenario of continuous reading or reciting of multiple articles. Figure 8 , which shows a flow chart of this specific example, may include:

[0190] Step S801: Acquire in real time the voice data of the user when he or she continuously reads or recites multiple articles.

[0191] When the user's start recording trigger operation is detected, the voice data of the user continuously reading or reciting multiple articles is obtained in real time.

[0192] Step S802: Processing the real-time acquired speech data into a phoneme sequence to obtain a target phoneme sequence.

[0193] The speech data acquired in real time can be processed into a phoneme sequence in real time based on the speech recognition model to obtain a target phoneme sequence, which is the phoneme sequence obtained up to now.

[0194] Step S803: Tracing the target phoneme sequence to obtain the target source text, and judging whether the user has completed reading or reciting an article based on the target phoneme sequence.

[0195] The specific implementation process of tracing the target phoneme sequence to obtain the target source text can be found in the above embodiment, which will not be described in detail in this embodiment. This embodiment focuses on the implementation process of determining whether the user has completed reading or reciting an article based on the target phoneme sequence.

[0196] In one possible implementation, the process of determining whether the user has currently completed reading or reciting an article based on a target phoneme sequence may include: detecting whether the target phoneme sequence has a time interval between adjacent phonemes that is greater than or equal to a preset duration (e.g., 10s); if the target phoneme sequence has a time interval between adjacent phonemes that is greater than or equal to the preset duration, determining that the user has currently completed reading or reciting an article.

[0197] If the target phoneme sequence does not have a time interval between adjacent phonemes that is greater than or equal to the preset time length, in one possible implementation, it can be directly determined that the user has not yet completed reading or reciting the article. To improve the accuracy of the determination, in another possible implementation, the following determination strategy can be further introduced:

[0198] (a) When the target source text is obtained through step a22-a (i.e., the target source text is one of the first source text and the second source text in the above embodiment), the discrimination strategy is:

[0199] It is determined whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text. If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading or reciting an article.

[0200] When there is no phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, in one possible implementation, it can be directly determined that the user has not yet completed reading or reciting an article. In order to improve the accuracy of the judgment, in another possible implementation, the last M (M is an integer greater than 1) phonemes of the target phoneme sequence can be further processed at the phoneme level to obtain several second n-gram units (each second n-gram unit includes n consecutive phonemes among the last M phonemes), and based on a pre-constructed reverse index table, the article identifiers corresponding to the several second n-gram units are determined, and then it is determined whether the user has currently completed reading or reciting an article based on the article identifiers corresponding to the several second n-gram units.

[0201] Among them, the process of determining whether the user has currently completed reading or reciting an article based on the article identifiers corresponding to several second n-gram units may include: determining whether there has been a change in the article identifier based on the article identifiers corresponding to several second n-gram units; if there has been a change in the article identifier, it is determined that the user has currently completed reading or reciting an article; otherwise, it is determined that the user has not currently completed reading or reciting an article.

[0202] Exemplarily, the article identifications corresponding to several second n-gram units are [… visiting the garden is not worth it, visiting the garden is not worth it, visiting the garden is not worth it, visiting the garden is not worth it, fisherman on the river]. Based on [… visiting the garden is not worth it, visiting the garden is not worth it, visiting the garden is not worth it, visiting the garden is not worth it, fisherman on the river], it can be determined that the article identification has changed (from "visiting the garden is not worth it" to "fisherman on the river"), and further, it can be determined that the user has currently completed reading an article.

[0203] It should be noted that if it is determined that the user has not yet completed reading or reciting an article, voice data will continue to be acquired and processed into phonemes in real time, and then the acquired phonemes will be used to determine whether the user has completed reading or reciting an article.

[0204] (b) When the target source text is obtained through step a22-b, the discrimination strategy is:

[0205] The target source text is processed at the sentence level N-gram to obtain multiple sentence-level N-gram units (each sentence-level N-gram unit includes N consecutive sentences in the target source text). Based on the pre-trained classification model, each sentence-level N-gram unit is classified to obtain the classification result of each sentence-level N-gram unit. The classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has undergone article switching. Based on the classification results of each sentence-level N-gram unit, it is determined whether the user has completed reading or reciting an article.

[0206] Exemplarily, the target tracing text includes 4 sentences, namely sentence 1, sentence 2, sentence 3 and sentence 4. Assuming N=3, the target tracing text is processed at the sentence level 3-gram to obtain two sentence-level 3-gram units, namely sentence 1-sentence 2-sentence 3 and sentence 2-sentence 3-sentence 4. Sentence 1-sentence 2-sentence 3 is input into the pre-trained classification model to obtain a classification result indicating whether an article switch has occurred in sentence 1-sentence 2-sentence 3. Sentence 2-sentence 3-sentence 4 is input into the pre-trained classification model to obtain a classification result indicating whether an article switch has occurred in sentence 2-sentence 3-sentence 4. Assuming that the classification result of sentence 1-sentence 2-sentence 3 indicates that no article switch has occurred, and the classification result of sentence 2-sentence 3-sentence 4 indicates that an article switch has occurred, it can be determined that the user has currently completed the reading of an article.

[0207] Among them, the classification model can include a semantic feature extraction module and a classification module. The semantic feature extraction module can be a pre-trained language model (such as BERT), and the classification module can be but is not limited to MLP. The semantic feature extraction module is used to extract contextual semantic features of the input text. The contextual semantic features extracted by the semantic feature acquisition module are input into the classification module. Based on the contextual semantic features extracted by the semantic feature extraction module, the classification module further extracts higher-level features and performs classification, and outputs the classification results.

[0208] The above content provides four discrimination strategies. The first discrimination strategy is to judge whether the user has completed the reading or recitation of an article by detecting whether the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to the preset time length. The second discrimination strategy is to judge whether the user has completed the reading or recitation of an article by judging whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text. The third discrimination strategy is to determine whether the user has completed the reading or recitation of an article based on the article identifiers corresponding to several second n-gram units obtained by n-gram processing of the last M phonemes of the target phoneme sequence. The fourth discrimination strategy is to determine whether the user has completed the reading or recitation of an article based on a classification model. It should be noted that in addition to adopting the above-mentioned first discrimination strategy combined with the second discrimination strategy, this application In addition to the above-mentioned combination of the first discrimination strategy and the fourth discrimination strategy to determine whether the user has currently completed the reading or reciting of an article, only the first discrimination strategy can be used to determine whether the user has currently completed the reading or reciting of an article, or only the third discrimination strategy can be used to determine whether the user has currently completed the reading or reciting of an article. In addition, the first discrimination strategy can be combined with the third discrimination strategy to determine whether the user has currently completed the reading or reciting of an article. In the case where the target tracing text is obtained through step a22-a, only the second discrimination strategy can be used to determine whether the user has currently completed the reading or reciting of an article. In the case where the target tracing text is obtained through step a22-b, only the fourth discrimination strategy can be used to determine whether the user has currently completed the reading or reciting of an article.

[0209] Step S804: If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article currently completed by the user is evaluated with reference to the target source text.

[0210] Specifically, according to the target source text, the general decoding network is trimmed to obtain a restricted decoding network, and speech features (such as MFCC features) are extracted from the reading or reciting voice data of the article currently read or recited by the user. The extracted speech features are decoded based on the restricted decoding network to obtain a decoding result. The decoding result is processed by a language model to obtain a phoneme sequence. The phoneme sequence and the target source text are input into the scoring model for scoring to obtain an evaluation result.

[0211] It should be noted that the voice evaluation method provided in this embodiment is also applicable to the evaluation of the voice data of a user reading or reciting an article (ie, step S801 obtains the voice data of the user reading or reciting an article in real time).

[0212] For the next article, since the acquisition of voice data and the conversion of phonemes are carried out in real time, the phoneme sequence after the switching position can be used as the target phoneme sequence, and step S803 and subsequent steps are executed to complete the evaluation of the reading or recitation voice data of the next article. The evaluation of the reading or recitation voice data of other subsequent articles is similar.

[0213] The speech evaluation method provided in the embodiment of the present application first obtains the speech data of the user when continuously reading or reciting multiple articles in real time, then processes the real-time acquired speech data into a phoneme sequence in real time to obtain a target phoneme sequence, then traces the target phoneme sequence to obtain a target source text, and according to the target phoneme sequence, determines whether the user has currently completed the reading or recitation of an article. If the user has currently completed the reading or recitation of an article, the target source text is referenced to evaluate the reading or recitation speech data of the article currently read or recited by the user. Since the speech evaluation method provided in the embodiment of the present application evaluates the speech data to be evaluated with reference to the target source text obtained by tracing the phoneme sequence corresponding to the user's reading or reciting speech data, rather than evaluating with reference to the speech recognition result of the user's reading or reciting speech data, a more accurate evaluation result can be obtained. In addition, the speech evaluation method provided in the embodiment of the present application makes it possible for the user to not select an article, and the user can read or recite multiple articles continuously, which greatly improves the user experience.

[0214] The above describes the speech evaluation method provided in the embodiments of the present application. The following describes the device corresponding to the above speech evaluation method.

[0215] See also Figure 9 , Figure 9This is a structural diagram of a speech evaluation device provided in an embodiment of the present application. The speech evaluation device may include: a speech data acquisition module 901, a speech data processing module 902, a first tracing module 903a, a second tracing module 903b, a target tracing text determination module 904 and a speech evaluation module 905.

[0216] The voice data acquisition module 901 is used to acquire the voice data to be evaluated.

[0217] The speech data processing module 902 is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence.

[0218] The first source tracing module 903a is used to obtain the source text of the text corresponding to the target phoneme sequence based on the large language model and the target phoneme sequence, and obtain the first source tracing text.

[0219] The second tracing module 903b is used to process the target phoneme sequence into several phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain a second tracing text.

[0220] The target source text determination module 904 is configured to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence.

[0221] The speech evaluation module 905 is used to evaluate the speech data to be evaluated with reference to the target source text.

[0222] In a possible implementation, when acquiring the voice data to be evaluated, the voice data acquisition module 901 is specifically configured to:

[0223] Acquire voice data in real time when users continuously read or recite multiple articles.

[0224] The speech data processing module 902 is specifically used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence:

[0225] The real-time acquired speech data is processed into a phoneme sequence to obtain a target phoneme sequence.

[0226] The speech evaluation module 905 is specifically used to evaluate the speech data to be evaluated by referring to the target source text:

[0227] Determine whether the user has completed reading or reciting an article;

[0228] If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

[0229] In a possible implementation, when determining the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence, the target source text determination module 904 is specifically configured to:

[0230] Obtaining the similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and the similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities;

[0231] Used to determine the target source text based on two similarities.

[0232] In a possible implementation, when processing the target phoneme sequence into a plurality of phoneme subsequences, the second tracing module 903b is specifically configured to:

[0233] The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

[0234] In a possible implementation, when the second source tracing module 903b determines the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source text, it is specifically configured to:

[0235] Determining article identifiers corresponding to the plurality of first n-gram units based on a pre-constructed inverted index table, wherein the inverted index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and the article identifiers corresponding to the plurality of n-gram units;

[0236] Counting the number of occurrences of article identifiers corresponding to a plurality of first n-gram units to obtain statistical results;

[0237] According to the statistical results, determine the target article identification;

[0238] An article in the article library identified as a target article is determined as the second tracing text.

[0239] In a possible implementation, when determining the target article identifier based on the statistical results, the second source tracing module 903b is specifically configured to:

[0240] Determine the K article identifiers that appear the most times as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1;

[0241] Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article;

[0242] Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier;

[0243] The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

[0244] In a possible implementation, when determining the target source text based on the two similarities, the target source text determination module 904 is specifically configured to:

[0245] The maximum similarity between the two similarities is determined as the target similarity;

[0246] If the target similarity is greater than or equal to the preset similarity threshold, the source text corresponding to the phoneme sequence having the target similarity as the target phoneme sequence is determined as the target source text;

[0247] If the target similarity is less than the preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected. The corrected text is used as the target tracing text.

[0248] The speech evaluation module 905 includes a chapter switching determination submodule, which is used to determine whether the user has completed reading or reciting an article based on the target phoneme sequence.

[0249] In one possible implementation, the chapter switching determination submodule is specifically configured to:

[0250] Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length;

[0251] If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

[0252] In one possible implementation, the chapter switching determination submodule is specifically configured to:

[0253] Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1;

[0254] Based on a pre-built inverted index table, determining the article identifiers corresponding to the plurality of second n-gram units;

[0255] Whether the user has currently completed reading aloud an article is determined based on the article identifiers corresponding to the plurality of second n-gram units.

[0256] In one possible implementation, the chapter switching determination submodule is specifically configured to:

[0257] When the target source text is the first source text or the second source text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text;

[0258] If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

[0259] In one possible implementation, the chapter switching determination submodule is specifically configured to:

[0260] When the target source text is a revised text, sentence-level N-gram processing is performed on the target source text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source text, where N is an integer greater than 1;

[0261] Based on the pre-trained classification model, each sentence-level N-gram unit is classified to obtain a classification result for each sentence-level N-gram unit. The classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has experienced article switching.

[0262] Based on the classification results of N-gram units at each sentence level, it is determined whether the user has completed reading aloud an article.

[0263] Since the speech evaluation device provided in the embodiment of the present application evaluates the speech data to be evaluated with reference to the target source text obtained by tracing the phoneme sequence corresponding to the speech data to be evaluated, rather than evaluating with reference to the speech recognition result of the speech data to be evaluated, a more accurate evaluation result can be obtained. In addition, the speech evaluation device provided in the present application eliminates the need for users to select articles, and users can read or recite multiple articles continuously, which greatly improves the user experience.

[0264] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0265] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0266] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0267] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0268] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the voice evaluation method provided in the above embodiment.

[0269] An embodiment of the present application further provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech evaluation method provided in the above embodiment.

[0270] An embodiment of the present application further provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the speech evaluation method provided in the above embodiment.

[0271] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0272] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0273] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0274] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A speech evaluation method, characterized in that: include: Obtain the voice data to be evaluated; Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence; Based on the large language model and the target phoneme sequence, a source text of the text corresponding to the target phoneme sequence is obtained to obtain a first source text; Processing the target phoneme sequence into a plurality of phoneme subsequences, and determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, to obtain a second source text; determining a target source text according to the relevance of the first source text and the second source text to the target phoneme sequence; The speech data to be evaluated is evaluated with reference to the target source text.

2. The speech evaluation method according to claim 1, wherein: The obtaining of the voice data to be evaluated includes: Real-time acquisition of voice data when users continuously read or recite multiple articles; Processing the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence includes: Processing the real-time acquired speech data into a phoneme sequence to obtain a target phoneme sequence; The step of evaluating the speech data to be evaluated by referring to the target source text includes: Determining whether the user has completed reading or reciting an article based on the target phoneme sequence; If the user has currently completed reading or reciting an article, the reading or reciting voice data of the article completed by the user is evaluated with reference to the target source text.

3. The speech evaluation method according to claim 1 or 2, characterized in that: The determining of the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence includes: Obtaining a similarity between the target phoneme sequence and the phoneme sequence corresponding to the first source text, and a similarity between the target phoneme sequence and the phoneme sequence corresponding to the second source text, to obtain two similarities; The target source text is determined based on the two similarities.

4. The speech evaluation method according to claim 1, wherein: The processing of the target phoneme sequence into a plurality of phoneme subsequences includes: The target phoneme sequence is subjected to phoneme-level n-gram processing to obtain a plurality of first n-gram units, wherein each first n-gram unit includes n consecutive phonemes in the target phoneme sequence, where n is an integer greater than 1.

5. The speech evaluation method according to claim 4, wherein: Determining the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence to obtain the second source text includes: Determining, based on a pre-constructed inverted index table, article identifiers corresponding to the plurality of first n-gram units, wherein the inverted index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and the article identifiers corresponding to the plurality of n-gram units; Counting the number of occurrences of the article identifiers corresponding to the plurality of first n-gram units to obtain a statistical result; Determining a target article identifier based on the statistical results; An article in the article library identified as the target article is determined as the second source text.

6. The speech evaluation method according to claim 5, characterized in that: Determining the target article identifier according to the statistical results includes: Determine the K article identifiers that appear the most times as candidate article identifiers, and obtain K candidate article identifiers, where K is an integer greater than 1; Obtaining the phoneme sequences corresponding to the K candidate article identifiers from a pre-constructed forward index table, wherein the forward index table includes the article identifiers of each article in the article library and the phoneme sequences corresponding to the article identifiers of each article, and the phoneme sequence corresponding to the article identifier of any article is the phoneme sequence corresponding to the article content of the article; Determining the similarity between the target phoneme sequence and the phoneme sequence corresponding to each candidate article identifier; The candidate article identifier corresponding to the phoneme sequence having the highest similarity to the target phoneme sequence is determined as the target article identifier.

7. The speech evaluation method according to claim 3, wherein: Determining the target source text according to the two similarities includes: Determine the maximum similarity of the two similarities as the target similarity; If the target similarity is greater than or equal to a preset similarity threshold, the source text corresponding to the phoneme sequence having the target similarity as the target phoneme sequence is determined as the target source text; If the target similarity is less than a preset similarity threshold, the text corresponding to the target phoneme sequence is predicted based on the large language model, and the predicted text is corrected, and the corrected text is used as the target tracing text.

8. The speech evaluation method according to claim 2, wherein: The step of determining whether the user has completed reading or reciting an article based on the target phoneme sequence includes: Detecting whether the target phoneme sequence has a situation where the time interval between adjacent phonemes is greater than or equal to a preset time length; If the time interval between adjacent phonemes in the target phoneme sequence is greater than or equal to a preset time length, it is determined that the user has currently completed reading or reciting an article.

9. The speech evaluation method according to claim 2, wherein: The step of determining whether the user has completed reading or reciting an article based on the target phoneme sequence includes: Performing phoneme-level n-gram processing on the last M phonemes of the target phoneme sequence to obtain a plurality of second n-gram units, where M is an integer greater than 1; Determining, based on a pre-constructed inverted index table, article identifiers corresponding to the plurality of second n-gram units, wherein the inverted index table includes a plurality of n-gram units obtained by performing n-gram processing on a phoneme sequence corresponding to the article content of each article in the article library, and the article identifiers corresponding to the plurality of n-gram units; Whether the user has currently completed reading aloud an article is determined based on the article identifiers corresponding to the plurality of second n-gram units.

10. The speech evaluation method according to claim 7, wherein: The step of determining whether the user has completed reading or reciting an article based on the target phoneme sequence includes: When the target source text is the first source text or the second source text, determining whether there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text; If there is a phoneme sequence in the target phoneme sequence that matches the phoneme sequence corresponding to the last sentence of the target source text, it is determined that the user has currently completed reading aloud an article.

11. The speech evaluation method according to claim 7, wherein: The step of determining whether the user has completed reading or reciting an article based on the target phoneme sequence includes: When the target source text is the revised text, performing sentence-level N-gram processing on the target source text to obtain a plurality of sentence-level N-gram units, wherein each sentence-level N-gram unit includes N consecutive sentences in the target source text, where N is an integer greater than 1; Based on the pre-trained classification model, each sentence-level N-gram unit is classified to obtain a classification result for each sentence-level N-gram unit. The classification result of any sentence-level N-gram unit can indicate whether the sentence-level N-gram unit has experienced article switching. Based on the classification results of N-gram units at each sentence level, it is determined whether the user has completed reading aloud an article.

12. The speech evaluation method according to claim 1, wherein: The step of evaluating the speech data to be evaluated by referring to the target source text includes: According to the target tracing text, the general decoding network is tailored to obtain a limited decoding network; Extracting speech features from the speech data to be evaluated, and decoding the speech features based on the limited decoding network to obtain a decoding result; Processing the decoding result through a language model to obtain a phoneme sequence; The target source text and the phoneme sequence obtained by the language model are input into a scoring model for scoring to obtain an evaluation result.

13. A speech evaluation device, characterized in that: include: Speech data acquisition module, speech data processing module, first tracing module, second tracing module, target tracing text determination module and speech evaluation module; The voice data acquisition module is used to acquire the voice data to be evaluated; The speech data processing module is used to process the speech data to be tested into a phoneme sequence to obtain a target phoneme sequence; The first source tracing module is configured to obtain a source text of a text corresponding to the target phoneme sequence based on a large language model and the target phoneme sequence, thereby obtaining a first source tracing text; The second source tracing module is configured to process the target phoneme sequence into a plurality of phoneme subsequences, and determine the source text of the text corresponding to the target phoneme sequence by determining the source text of the text corresponding to each phoneme subsequence, thereby obtaining a second source text; The target source text determination module is configured to determine the target source text according to the relevance of the first source text and the second source text to the target phoneme sequence respectively; The speech evaluation module is used to evaluate the speech data to be evaluated with reference to the target source text.

14. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the speech evaluation method according to any one of claims 1 to 12.

15. A computer storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech evaluation method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the steps of the speech evaluation method according to any one of claims 1 to 12.

Images