ASR variable-length text conversion method and device, computer equipment and storage medium

By receiving the initial variable-length text output from the ASR system, obtaining its character sequence and pinyin sequence, performing feature encoding and decoding, and generating a corrected target sequence, the efficiency and accuracy of the ASR system in variable-length text error processing is solved. Improve the accuracy and service experience of voice recognition.

CN119005125APending Publication Date: 2024-11-22PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411257416.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing ASR system cannot achieve fast and accurate effective correction and conversion when facing variable-length text errors, which affects the execution of subsequent downstream tasks such as voice search and voice translation, and affects the customer's service experience.

Method used

By receiving the initial variable-length text output from the ASR system, obtaining its corresponding initial character sequence and initial pinyin sequence, using a preset encoder for feature encoding, generating character sequence vectors and pinyin sequence vectors, and inputting them into the CSC decoder and CPP decoding respectively , generates the corrected target character sequence and target pinyin sequence, and finally integrates the converted target variable-length text.

Benefits of technology

It effectively improves the efficiency and accuracy of ASR variable-length text error correction conversion, improves the accuracy and service experience of speech recognition, and is suitable for financial services and other fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005125A_ABST
    Figure CN119005125A_ABST
Patent Text Reader

Abstract

The invention discloses an ASR variable-length text conversion method and device, computer equipment and a storage medium, relates to the technical field of speech recognition, and is applied to an ASR variable-length text conversion scene related to financial services, and the method comprises the following steps: receiving an initial variable-length text output by an ASR system, obtaining an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text; performing feature coding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and a corresponding initial pinyin sequence vector; respectively inputting the initial character sequence vector and the initial pinyin sequence vector into a corresponding preset CSC decoder and a CPP decoder to generate a corresponding target character sequence and a target pinyin sequence; and based on the target character sequence and the target pinyin sequence, generating a target variable-length text converted from the initial variable-length text. By means of the method, the efficiency and accuracy of error correction and conversion of the ASR variable-length text can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and is applied to ASR variable-length text conversion scenarios related to financial services, and specifically to an ASR variable-length text conversion method, device, computer equipment, and non-volatile computer-readable storage medium. Background Art

[0002] Automatic Speech Recognition (ASR) is a technology that converts human speech into written text. This technology enables computers to "understand" human language and is widely used in various scenarios and devices. With the advancement of technology, ASR systems are becoming more and more accurate and intelligent, and they play an important role in improving the naturalness and convenience of human-machine interaction. However, the current existing ASR systems are unable to quickly and accurately correct and convert errors in variable-length text, resulting in errors introduced in the automatic speech recognition (ASR) process that affect the execution of subsequent downstream tasks such as voice search and voice translation, affecting the customer's service experience.

[0003] Based on this, how to provide an ASR variable-length text conversion method, device, computer equipment and non-volatile computer-readable storage medium that can effectively improve the efficiency and accuracy of ASR variable-length text error correction conversion is a problem that urgently needs to be solved by technical personnel in this field. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide an ASR variable-length text conversion method, device, computer equipment and non-volatile computer-readable storage medium that can be used in financial technology or other related fields, which can effectively improve the efficiency and accuracy of ASR variable-length text error correction conversion.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] An ASR variable-length text conversion method, comprising:

[0007] Receiving an initial variable-length text output by the ASR system, and acquiring an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text;

[0008] Performing feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector;

[0009] Input the initial character sequence vector and the initial pinyin sequence vector into the corresponding preset CSC decoder and CPP decoder respectively to generate the corresponding target character sequence and target pinyin sequence;

[0010] Based on the target character sequence and the target pinyin sequence, a target variable-length text is generated after the initial variable-length text is converted.

[0011] In a further technical solution, the ASR variable-length text conversion method, wherein the receiving of the initial variable-length text output by the ASR system, and obtaining the initial character sequence and the initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text, comprises:

[0012] Receiving the initial variable-length text output by the ASR system;

[0013] Based on the initial variable-length text, extract the corresponding initial character sequence;

[0014] Based on the initial character sequence, its corresponding initial pinyin sequence is generated through PyPinyin.

[0015] In a further technical solution, the ASR variable-length text conversion method, wherein the initial character sequence and the initial pinyin sequence are feature encoded by a preset encoder to generate corresponding initial character sequence vectors and initial pinyin sequence vectors, comprises:

[0016] Pre-load the pre-trained ChineseBERT model;

[0017] The initial character sequence and the initial pinyin sequence are input into the ChineseBERT model for feature encoding to generate corresponding initial character sequence vectors and initial pinyin sequence vectors.

[0018] In a further technical solution, the ASR variable-length text conversion method, wherein the initial character sequence vector and the initial pinyin sequence vector are respectively input into corresponding preset CSC decoders and CPP decoders to generate corresponding target character sequences and target pinyin sequences, comprises:

[0019] Pre-built CSC decoder and CPP decoder;

[0020] Inputting the initial character sequence vector into the CSC decoder to generate a decoded character sequence, and performing post-processing on the decoded character sequence to obtain a target character sequence;

[0021] The initial pinyin sequence vector is input into the CPP decoder to generate a decoded pinyin sequence, and the decoded pinyin sequence is post-processed to obtain a target pinyin sequence.

[0022] In a further technical solution, the ASR variable-length text conversion method, wherein the step of generating a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence, comprises:

[0023] Integrate the target character sequence with the target pinyin sequence to generate a target variable-length text after the initial variable-length text is converted;

[0024] Send the target variable-length text to a specified location.

[0025] In a further technical solution, the ASR variable-length text conversion method, wherein the initial character sequence vector is input into the CSC decoder to generate a decoded character sequence, and the decoded character sequence is post-processed to obtain a target character sequence, further includes:

[0026] The unchanged tokens in the initial character sequence are copied to the target character sequence using a copy enhancement architecture.

[0027] In a further technical solution, the ASR variable-length text conversion method, wherein the initial character sequence vector and the initial pinyin sequence vector are respectively input into corresponding preset CSC decoders and CPP decoders to generate corresponding target character sequences and target pinyin sequences, further comprises:

[0028] The weight of the CPP decoder in the decoding task is adaptively adjusted according to the pinyin edit distance between the initial pinyin sequence and the target pinyin sequence.

[0029] An ASR variable-length text conversion device, comprising:

[0030] A receiving module, used for receiving an initial variable-length text output by the ASR system, and obtaining an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text;

[0031] An encoding module, used for performing feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector;

[0032] A decoding module, used for inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively, to generate corresponding target character sequence and target pinyin sequence;

[0033] A generating module is used to generate a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence.

[0034] A computer device, wherein the computer device comprises at least one processor; and

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores a computer program that can be executed by the at least one processor. When the computer program is executed by the at least one processor, the ASR variable-length text conversion method as described in any one of the above items can be implemented.

[0037] A non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the ASR variable-length text conversion method as described in any one of the above items can be implemented.

[0038] Compared with the prior art, the present invention provides an ASR variable-length text conversion method, device, computer equipment and non-volatile computer-readable storage medium, wherein the method comprises: receiving an initial variable-length text output by an ASR system, and obtaining its corresponding initial character sequence and initial pinyin sequence based on the initial variable-length text; performing feature encoding on the initial character sequence and the initial pinyin sequence by a preset encoder to generate corresponding initial character sequence vectors and initial pinyin sequence vectors; inputting the initial character sequence vectors and the initial pinyin sequence vectors into corresponding preset CSC decoders and CPP decoders respectively to generate corresponding target character sequences and target pinyin sequences; and generating target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence. In this way, the method of the present invention first obtains the corresponding initial character sequence and initial pinyin sequence through the initial variable-length text output by the ASR system, then obtains the corresponding initial character sequence vector and initial pinyin sequence vector through a preset encoder, and then obtains the corresponding corrected, accurate target character sequence and target pinyin sequence through the preset CSC decoder and CPP decoder, and finally generates the target variable-length text after the initial variable-length text is converted through the target character sequence and the target pinyin sequence, so that the method of the present invention can effectively improve the efficiency and accuracy of the ASR variable-length text error correction conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic flow chart of an ASR variable-length text conversion method provided in an embodiment of the present invention.

[0041] Figure 2 A schematic diagram of functional modules of an ASR variable-length text conversion device provided in an embodiment of the present invention.

[0042] Figure 3 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] In the description of the present invention, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps therein is not limited and can be appropriately adjusted as needed.

[0045] Various non-limiting embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0046] Automatic Speech Recognition (ASR) is a technology that converts human speech into written text. This technology enables computers to "understand" human language and is widely used in various scenarios and devices. With the advancement of technology, ASR systems are becoming more and more accurate and intelligent, and they play an important role in improving the naturalness and convenience of human-machine interaction. However, the current existing ASR systems are unable to quickly and accurately correct and convert errors in variable-length text, resulting in errors introduced in the automatic speech recognition (ASR) process that affect the execution of subsequent downstream tasks such as voice search and voice translation, affecting the customer's service experience.

[0047] Based on this, how to provide an ASR variable-length text conversion method, device, computer equipment and non-volatile computer-readable storage medium that can effectively improve the efficiency and accuracy of ASR variable-length text error correction conversion is a problem that urgently needs to be solved by technical personnel in this field.

[0048] Therefore, to solve the above problem, please refer to Figure 1 The embodiment of the present invention provides an ASR variable-length text conversion method, wherein the method comprises the steps of:

[0049] S100, receiving an initial variable-length text output by an ASR system, and acquiring an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text;

[0050] S200, feature encoding the initial character sequence and the initial pinyin sequence by a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector;

[0051] S300, inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively to generate corresponding target character sequence and target pinyin sequence;

[0052] S400: Generate a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence.

[0053] Furthermore, the ASR variable-length text conversion method, wherein the step S100, receiving the initial variable-length text output by the ASR system, and obtaining the initial character sequence and initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text, specifically includes:

[0054] Receiving the initial variable-length text output by the ASR system;

[0055] Based on the initial variable-length text, extract the corresponding initial character sequence;

[0056] Based on the initial character sequence, its corresponding initial pinyin sequence is generated through PyPinyin.

[0057] In the specific implementation, in this embodiment, the initial variable-length text output by the ASR system is received; based on the initial variable-length text, the corresponding initial character sequence is extracted; based on the initial character sequence, the corresponding initial pinyin sequence is generated through PyPinyin; wherein the specific implementation process is roughly as follows:

[0058] 1) Receive the initial variable-length text output by the ASR system

[0059] First, the ASR (Automatic Speech Recognition) system converts the input speech signal into text form through speech recognition technology to generate initial variable-length text; this usually involves the joint application of acoustic models and language models to recognize words or phrases in speech; among them, in the field of automatic speech recognition (ASR), "variable-length text" refers to the text generated by the ASR system, whose length can vary according to the length of the input speech signal;

[0060] 2) Extract the initial character sequence

[0061] Processing the initial variable-length text output by the ASR system to extract the character sequence that constitutes the text; this step may involve removing extra spaces, punctuation marks, and non-text information to obtain a pure initial character sequence;

[0062] 3) Using the PyPinyin library

[0063] The extracted initial character sequence is converted to pinyin using the PyPinyin library in Python; PyPinyin is a library that converts Chinese characters to pinyin, which supports the processing of polyphones and the representation of pinyin tones;

[0064] 4) Generate initial pinyin sequence

[0065] For each Chinese character, PyPinyin generates the corresponding pinyin according to its context; if it is a polyphonic character, multiple pinyin options may be generated; in this step, you can choose the default pinyin, or choose the most appropriate pinyin according to the context; finally, get the corresponding initial pinyin sequence;

[0066] Through the above process, the ASR system can convert speech signals into text, and further convert text into pinyin sequences, providing a basis for subsequent language processing tasks;

[0067] In the specific implementation, first, an initial variable-length text with errors output during the automatic speech recognition process of the ASR system is received, and then, based on the initial variable-length text, an initial character sequence X corresponding to the initial variable-length text is extracted. w = (x1, x2, x3, ..., xn), the goal is to convert it into a target character sequence Y of m characters w =(y1, y2, y3, ..., ym), where n and m can be equal or unequal; then, based on the initial character sequence, the corresponding initial pinyin sequence X is obtained by the PyPinyin tool p ;

[0068] For example: Wrong initial character sequence X wFor "Don't be sad, all difficulties will be overcome.", use the PyPinyin tool to get the corresponding incorrect initial pinyin sequence X p For "ni3 biao3 nan2guo4, yi1 qie4 nan2guan1 dou1 hui4 bei4 ke4 fu2 de1.".

[0069] Further, the ASR variable-length text conversion method, wherein the step S200, feature encoding the initial character sequence and the initial pinyin sequence by a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector, specifically includes:

[0070] Pre-load the pre-trained ChineseBERT model;

[0071] The initial character sequence and the initial pinyin sequence are input into the ChineseBERT model for feature encoding to generate corresponding initial character sequence vectors and initial pinyin sequence vectors.

[0072] In the specific implementation, in this embodiment, the pre-trained ChineseBERT model is pre-loaded; the initial character sequence and the initial pinyin sequence are input into the ChineseBERT model for feature encoding to generate the corresponding initial character sequence vector and initial pinyin sequence vector; wherein the specific implementation process is roughly as follows:

[0073] 1) Loading the pre-trained ChineseBERT model

[0074] Pre-load a pre-trained ChineseBERT model; this model has been trained on a large-scale Chinese corpus and can capture the characteristics and patterns of the Chinese language;

[0075] Make sure the model file, vocabulary, and necessary configuration parameters have been downloaded and correctly loaded into memory for subsequent processing;

[0076] 2) Input processing of character sequences

[0077] The extracted initial character sequence is appropriately preprocessed to adapt to the input requirements of the ChineseBERT model; this usually includes word segmentation, adding special tags (such as [CLS] and [SEP]), and converting to a numerical representation that the model can understand (i.e., tokenID).

[0078] According to the vocabulary of the ChineseBERT model, convert each character in the character sequence into the corresponding tokenID;

[0079] 3) Input processing of pinyin sequences

[0080] Similar to the character sequence, the initial pinyin sequence is also preprocessed and converted into a form suitable for model input; since the pinyin sequence may not directly correspond to the token in the vocabulary, special processing may be required, such as customizing the vocabulary or using character-level representation;

[0081] Convert the pinyin sequence to a tokenID. If the pinyin is not in the vocabulary of the pre-trained model, some form of mapping or replacement may be required.

[0082] 4) Generate sequence vector

[0083] The tokenIDs of the processed character sequences and pinyin sequences are input into the ChineseBERT model; the model forward propagates these sequences through its neural network architecture to generate corresponding embedding vectors;

[0084] For character sequences and pinyin sequences, the model outputs two different vector representations, capturing the semantic and syntactic information of characters and pinyin respectively;

[0085] 5) Post-processing of vectors

[0086] Post-process the generated initial character sequence vector and initial pinyin sequence vector as needed; this may include operations such as vector averaging, maximum pooling, dimensionality reduction or standardization to meet the needs of subsequent applications;

[0087] Through the above process, the pre-trained ChineseBERT model can be effectively used to process the character and pinyin information of Chinese text, generate rich semantic vector representations, and provide support for various natural language processing tasks.

[0088] Further, the ASR variable-length text conversion method, wherein the step S300, inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively, generating corresponding target character sequence and target pinyin sequence, specifically includes:

[0089] Pre-built CSC decoder and CPP decoder;

[0090] Inputting the initial character sequence vector into the CSC decoder to generate a decoded character sequence, and performing post-processing on the decoded character sequence to obtain a target character sequence;

[0091] The initial pinyin sequence vector is input into the CPP decoder to generate a decoded pinyin sequence, and the decoded pinyin sequence is post-processed to obtain a target pinyin sequence.

[0092] In the specific implementation, in this embodiment, a CSC decoder and a CPP decoder are pre-built; the initial character sequence vector is input into the CSC decoder to generate a decoded character sequence, and the decoded character sequence is post-processed to obtain a target character sequence; the initial pinyin sequence vector is input into the CPP decoder to generate a decoded pinyin sequence, and the decoded pinyin sequence is post-processed to obtain a target pinyin sequence; wherein the specific implementation process is roughly as follows:

[0093] 1) Build CSC decoder and CPP decoder

[0094] Develop or acquire two independent decoder models, namely CSC decoder and CPP decoder; CSC decoder focuses on identifying and correcting spelling errors in character sequences, while CPP decoder focuses on predicting accurate Chinese pinyin;

[0095] That is, in this embodiment, two parallel decoders are pre-built on the basis of the encoder, one of which (CSC decoder) is used to generate the correct target character sequence for the main CSC task; the other (CPP decoder) is used to predict the initial consonant, final vowel and tone of each character pinyin in the target character sequence, that is, the CPP task is used to assist in improving the error correction accuracy;

[0096] Among them, CSC stands for Chinese Spelling Correction, and CPP stands for Chinese Pronunciation Prediction. The CSC task is the main text error correction task, while the CPP task assists the CSC task by predicting the initial consonants, finals and tones of pinyin to improve the accuracy of error correction. This combination of two parallel decoders can handle the error correction problem of Chinese text more comprehensively and improve the overall performance of the ASR system.

[0097] 2) Input and decoding of character sequences

[0098] The initial character sequence vectors are input to the CSC decoder; the decoder analyzes these vectors and generates a decoded character sequence, which may include the corrected character;

[0099] The CSC decoder can use language models, machine learning algorithms, or deep learning models to predict and correct errors;

[0100] 3) Post-processing of character sequences

[0101] Post-process the decoded character sequence output by the CSC decoder; the post-processing steps may include syntax checking, context analysis, elimination of repeated characters, adjustment of character order, etc.;

[0102] Post-processing ensures that the target character sequence is grammatically, semantically, and contextually accurate and appropriate;

[0103] 4) Input and decoding of pinyin sequences

[0104] The initial pinyin sequence vectors are input to the CPP decoder; the decoder analyzes these vectors and generates a decoded pinyin sequence, which may contain the predicted accurate initials, finals and tones;

[0105] The CPP decoder can use an acoustic model, a language model, or a deep learning model to predict pronunciation;

[0106] 5) Post-processing of Pinyin sequences

[0107] Post-processing the decoded pinyin sequence output by the CPP decoder; the post-processing steps may include tone correction, polyphone disambiguation, format adjustment, etc.;

[0108] Post-processing ensures that the target phonetic sequences are pronunciationally accurate and natural;

[0109] 6) Output target sequence

[0110] Output the post-processed target character sequence and target pinyin sequence; these sequences can be used for further text processing tasks, such as speech synthesis, displaying corrected text, etc.;

[0111] For example: The correct target character sequence Y w is "Don't be sad, all difficulties will be overcome.", and the corresponding correct target pinyin sequence Y p For "ni3 bu2 yao4 nan2 guo4, yi1qie4 nan2 guan1 dou1 hui4bei4 ke4 fu2 de1.";

[0112] Through the above process, the initial character sequence vector and the initial pinyin sequence vector can be converted into a corrected, accurate target character sequence and target pinyin sequence, thereby improving the accuracy and usability of Chinese text processing.

[0113] Further, the ASR variable-length text conversion method, wherein the step S300, inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively to generate corresponding target character sequence and target pinyin sequence, specifically also includes:

[0114] The weight of the CPP decoder in the decoding task is adaptively adjusted according to the pinyin edit distance between the initial pinyin sequence and the target pinyin sequence.

[0115] In the specific implementation, in this embodiment, the weight of the CPP decoder in the decoding task is adaptively adjusted according to the phonetic edit distance between the initial phonetic sequence and the target phonetic sequence; wherein the specific implementation process is roughly as follows:

[0116] 1) Calculate the phonetic edit distance

[0117] Pinyin edit distance refers to the minimum number of editing operations required to transform the Pinyin sequence of one sentence into the Pinyin sequence of another sentence. These editing operations usually include inserting, deleting, replacing and exchanging characters in the Pinyin sequence. Pinyin edit distance can quantify the similarity between two Pinyin sequences, thus providing an indicator to measure sentence similarity.

[0118] In this embodiment, first, it is necessary to calculate the phonetic edit distance between the initial phonetic sequence and the target phonetic sequence; a dynamic programming algorithm, such as Levenshtein distance, can be used to calculate the phonetic edit distance between two phonetic sequences;

[0119] 2) Analyze the Pinyin edit distance

[0120] Analyze the calculated phonetic edit distance to determine the degree of difference between the initial phonetic sequence and the target phonetic sequence; a larger phonetic edit distance may mean that a more aggressive decoding strategy is needed;

[0121] 3) Determine the weight adjustment strategy

[0122] Determine the weight adjustment strategy of the CPP decoder in the decoding task according to the size of the phonetic edit distance; this may involve defining a function or a rule set to dynamically adjust the weight according to the phonetic edit distance;

[0123] 4) Adjust CPP decoder weights

[0124] Implement weight adjustments; if the phonetic edit distance is large, you may need to increase the weight of the CPP decoder to rely more on phonetic prediction during decoding; if the phonetic edit distance is small, you may need to reduce the weight to rely more on other information sources;

[0125] The weight adjustment can be achieved by modifying the parameters inside the decoder, the coefficients in the loss function, or by introducing additional control logic in the decoding algorithm;

[0126] 5) Apply adjusted weights

[0127] Apply the adjusted weights to the CPP decoder to influence its behavior in subsequent decoding tasks;

[0128] 6) Monitoring and evaluation

[0129] Monitor the performance of the CPP decoder in actual decoding tasks and evaluate the impact of weight adjustment on decoding performance; if necessary, further fine-tune the weight adjustment strategy based on actual performance;

[0130] Through the above process, the CPP decoder can more flexibly cope with different decoding challenges and improve the accuracy and efficiency of Chinese pronunciation prediction.

[0131] Furthermore, the ASR variable-length text conversion method, wherein the initial character sequence vector is input into the CSC decoder to generate a decoded character sequence, and the decoded character sequence is post-processed to obtain a target character sequence, further comprises:

[0132] The unchanged tokens in the initial character sequence are copied to the target character sequence using a copy enhancement architecture.

[0133] Specifically, in natural language processing, a token refers to the smallest unit in an input sequence, which can be a word, a character, or other language elements. This embodiment also provides a replication enhancement architecture to assist in solving the error correction conversion problem of variable-length texts. The specific implementation process is as follows:

[0134] In this embodiment, by copying the unchanged token in the initial character sequence to the target character sequence, the final probability distribution P of the token at the tth position is t is the generating distribution P t gen and the replication distribution P t copy The balance between replication and generation at each time step t is determined by a balance factor Control; use a balance factor to control the balance between replication and generation, so that this embodiment can better handle the error correction conversion problem of variable-length text; this embodiment can better handle the error correction conversion problem of variable-length text by using a replication enhancement architecture, which can improve accuracy and generalization ability;

[0135]

[0136] The encoder encodes the wrong initial character sequence and obtains the hidden state of the i-th character as The decoder decodes the correct target character sequence to obtain the hidden state of the jth character: Matrix L trg ∈R dx×|V| is the word embedding matrix, where dx is the word embedding dimension and |V| is the size of the vocabulary; for the target hidden state With the word embedding matrix L trgThe inner product of is used for softmax operation to obtain the generation probability distribution of the tth word;

[0137]

[0138] The Copy Score is obtained by calculating the Attention Distribution. The calculation method is the same as the encoder-decoder attention calculation method, as shown in the following formula:

[0139]

[0140] Among them, q t , K and V are the query, key and value required to calculate the attention distribution and copy hidden state respectively. This embodiment uses the normalized attention distribution as the Copy Score and uses the copy hidden state to estimate the balance factor

[0141]

[0142] This embodiment can more accurately process variable-length sequences by using a copy-enhanced architecture, especially in tasks that require accurate replication of the correct portion of the input sequence.

[0143] Furthermore, the ASR variable-length text conversion method, wherein the step S400, generating a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence, specifically includes:

[0144] Integrate the target character sequence with the target pinyin sequence to generate a target variable-length text after the initial variable-length text is converted;

[0145] Send the target variable-length text to a specified location.

[0146] In the specific implementation, in this embodiment, the target character sequence and the target pinyin sequence are integrated to generate the target variable-length text after the initial variable-length text is converted; the target variable-length text is sent to a designated location; wherein the specific implementation process is roughly as follows:

[0147] 1) Integrate the target character sequence and the target pinyin sequence

[0148] Integrate the target character sequence and the target pinyin sequence processed by the CSC decoder and the CPP decoder; this step needs to ensure that the two sequences are consistent in semantics and context;

[0149] According to application requirements, it may be necessary to convert the pinyin sequence into a visual pinyin text or match it with a character sequence for subsequent processing;

[0150] 2) Generate target variable-length text

[0151] Convert the integrated data into target variable-length text; this may include organizing character sequences and pinyin sequences in a certain format to form complete text information;

[0152] Make sure the target variable-length text conforms to the expected output format. For example, if pinyin is used to assist pronunciation, it may need to be annotated in the form of phonetic symbols next to the corresponding Chinese characters.

[0153] 3) Verification and proofreading

[0154] Before sending the target variable-length text, verify and proofread it to ensure that the text content is accurate, the format meets the requirements, and there are no omissions or errors;

[0155] This step can be done by manual review or automated quality checking tools;

[0156] 4) Determination of designated location

[0157] Identify the “designated location” where the target variable-length text needs to be sent, such as the execution location of downstream tasks such as subsequent voice search and voice translation in financial services; this can be a file system, database, specific application programming interface (API), or any other system that needs to receive the processed text;

[0158] 5) Send target variable length text

[0159] Send the target variable-length text to the specified location through a programming interface, network request or other communication protocol;

[0160] Ensure that appropriate security measures are taken during the transmission process, such as data encryption, identity authentication, etc., to protect the security and integrity of the data;

[0161] Through the above process, it can be ensured that the target character sequence and the target pinyin sequence generated from the CSC decoder and the CPP decoder are effectively integrated and accurately sent to the required location for further use or storage.

[0162] It can be seen from the above method embodiments that the ASR variable-length text conversion method provided by the present invention includes: receiving the initial variable-length text output by the ASR system, and obtaining its corresponding initial character sequence and initial pinyin sequence based on the initial variable-length text; performing feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate corresponding initial character sequence vectors and initial pinyin sequence vectors; inputting the initial character sequence vectors and the initial pinyin sequence vectors into the corresponding preset CSC decoder and CPP decoder respectively to generate corresponding target character sequence and target pinyin sequence; based on the target character sequence and the target pinyin sequence, generating a target variable-length text after the initial variable-length text is converted. In this way, the method of the present invention first obtains the corresponding initial character sequence and initial pinyin sequence through the initial variable-length text output by the ASR system, then obtains the corresponding initial character sequence vector and initial pinyin sequence vector through a preset encoder, and then obtains the corresponding corrected, accurate target character sequence and target pinyin sequence through the preset CSC decoder and CPP decoder, and finally generates the target variable-length text after the initial variable-length text is converted through the target character sequence and the target pinyin sequence. Therefore, the method of the present invention can effectively improve the efficiency and accuracy of ASR variable-length text error correction conversion, and can contribute to the development of voice assistants, intelligent customer service and other fields related to financial services.

[0163] Specifically, for example: the method of the present invention can be applied to telephone sales scenarios in the financial insurance field to improve the speech recognition accuracy of the ASR model to improve the accuracy of downstream task scenario models. For example, the NLP model needs to use text recognized by ASR, and the corrected text can be used to better understand the customer's intentions and provide customers with a better service experience.

[0164] It should be understood that, although the present application provides method operation steps as described in the embodiments or flowcharts, more or less operation steps may be included based on conventional or non-creative labor, and these operation steps are not necessarily performed in sequence according to the embodiment or flowchart. The order of steps listed in the embodiment or flowchart is only one way of executing the order of many steps, and does not represent the only execution order. It should be noted that there is not necessarily a certain order between the above steps. A person of ordinary skill in the art can understand from the description of the embodiment of the present invention that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or they may be executed in exchange, etc. Moreover, at least a part of the steps in the embodiment or flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but may be executed in turn, alternately or synchronously with other steps or at least a part of the sub-steps or stages of other steps.

[0165] Based on the above method embodiment, please refer to Figure 2 Another embodiment of the present invention further provides an ASR variable-length text conversion device, wherein the device comprises:

[0166] The receiving module 11 is used to receive the initial variable-length text output by the ASR system, and obtain the initial character sequence and initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text;

[0167] The encoding module 12 is used to perform feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector;

[0168] A decoding module 13, used for inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively, to generate corresponding target character sequence and target pinyin sequence;

[0169] The generating module 14 is used to generate a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence.

[0170] Furthermore, in the ASR variable-length text conversion device, the receiving module 11 specifically comprises:

[0171] A variable-length text receiving unit, used for receiving an initial variable-length text output by the ASR system;

[0172] A character sequence extraction unit, used for extracting the initial character sequence corresponding to the initial variable-length text based on the initial variable-length text;

[0173] The pinyin sequence generating unit is used to generate the corresponding initial pinyin sequence through PyPinyin based on the initial character sequence.

[0174] Furthermore, in the ASR variable-length text conversion device, the encoding module 12 specifically includes:

[0175] Model loading unit, used to pre-load the pre-trained ChineseBERT model;

[0176] The feature encoding unit is used to input the initial character sequence and the initial pinyin sequence into the ChineseBERT model for feature encoding, and generate a corresponding initial character sequence vector and an initial pinyin sequence vector.

[0177] Furthermore, in the ASR variable-length text conversion device, the decoding module 13 specifically includes:

[0178] A decoder construction unit, used for pre-constructing a CSC decoder and a CPP decoder;

[0179] A first decoding unit, configured to input the initial character sequence vector into the CSC decoder, generate a decoded character sequence, and perform post-processing on the decoded character sequence to obtain a target character sequence;

[0180] The second decoding unit is used to input the initial pinyin sequence vector into the CPP decoder to generate a decoded pinyin sequence, and post-process the decoded pinyin sequence to obtain a target pinyin sequence.

[0181] Furthermore, in the ASR variable-length text conversion device, the decoding module 13 specifically includes:

[0182] A weight adjustment unit is used to adaptively adjust the weight of the CPP decoder in the decoding task according to the pinyin edit distance between the initial pinyin sequence and the target pinyin sequence.

[0183] Furthermore, the ASR variable-length text conversion device, wherein the initial character sequence vector is input into the CSC decoder to generate a decoded character sequence, and the decoded character sequence is post-processed to obtain a target character sequence, further comprises:

[0184] The unchanged tokens in the initial character sequence are copied to the target character sequence using a copy enhancement architecture.

[0185] Furthermore, in the ASR variable-length text conversion device, the generation module 14 specifically includes:

[0186] A sequence integration unit, used for integrating the target character sequence with the target pinyin sequence to generate a target variable-length text after the initial variable-length text is converted;

[0187] The text sending unit is used to send the target variable-length text to a designated location.

[0188] The specific implementation process and effects of the device embodiment of the present invention can be found in the above method embodiment, which will not be described in detail here.

[0189] It can be seen from the above device embodiments that the ASR variable-length text conversion device provided by the present invention includes: a receiving module, used to receive the initial variable-length text output by the ASR system, and obtain its corresponding initial character sequence and initial pinyin sequence based on the initial variable-length text; an encoding module, used to perform feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder, and generate corresponding initial character sequence vectors and initial pinyin sequence vectors; a decoding module, used to input the initial character sequence vector and the initial pinyin sequence vector into the corresponding preset CSC decoder and CPP decoder respectively, and generate corresponding target character sequence and target pinyin sequence; a generating module, used to generate a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence. In this way, the device of the present invention first obtains the corresponding initial character sequence and initial pinyin sequence through the initial variable-length text output by the ASR system, then obtains the corresponding initial character sequence vector and initial pinyin sequence vector through a preset encoder, and then obtains the corresponding corrected, accurate target character sequence and target pinyin sequence through the preset CSC decoder and CPP decoder, and finally generates the target variable-length text after the initial variable-length text is converted through the target character sequence and the target pinyin sequence. Therefore, the efficiency and accuracy of the ASR variable-length text error correction conversion can be effectively improved through the device of the present invention, and can contribute to the development of voice assistants, intelligent customer service and other fields related to financial services.

[0190] Specifically, for example: the device of the present invention can be applied to telephone sales scenarios in the financial insurance field to improve the speech recognition accuracy of the ASR model to improve the accuracy of downstream task scenario models. For example, the NLP model needs to use text recognized by ASR, and the corrected text can be used to better understand the customer's intentions and provide customers with a better service experience.

[0191] Based on the above method embodiment, please refer to Figure 3 Another embodiment of the present invention further provides a computer device, wherein the computer device 10 includes:

[0192] memory 120 and one or more processors 110, Figure 3A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a communication bus or other means. Figure 3 The example of connecting via a communication bus is taken as an example.

[0193] The processor 110 is used to complete various control logics of the computer device 10. It can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP, or any other such configuration.

[0194] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the computer program corresponding to the ASR variable-length text conversion method in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the computer device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the ASR variable-length text conversion method in the above method embodiment.

[0195] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an application required for operating the device or at least one function; the data storage area may store data created according to the use of the computer device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the computer device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0196] One or more units are stored in the memory 120, and when executed by one or more processors 110, the ASR variable-length text conversion method in any of the above method embodiments can be implemented, for example, the above described Figure 1 The method comprises steps S100 to S400.

[0197] Those skilled in the art will understand that Figure 3The hardware structure diagram shown in the figure is only a schematic diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0198] Based on the above method embodiments, another embodiment of the present invention further provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the ASR variable-length text conversion method in any of the above method embodiments can be implemented, for example, the above-described Figure 1 The method comprises steps S100 to S400.

[0199] As an example, the non-volatile storage medium can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable ROM (EEPROM), or a flash memory. Volatile memory can include a random access memory (RAM) as an external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0200] Based on the above method embodiments, another embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by a processor, the ASR variable-length text conversion method in any of the above method embodiments can be implemented, for example, the above described Figure 1 The method comprises steps S100 to S400.

[0201] The embodiments described above are merely illustrative, in which the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can exist in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0203] Conditional language such as "can," "can," "might," or "may," unless specifically stated otherwise or otherwise understood within the context as used, is generally intended to convey, among other things, that a particular embodiment can include (while other embodiments do not) a particular feature, element, and / or operation. Thus, such conditional language is also generally intended to imply that a feature, element, and / or operation is required for one or more embodiments anyway or that one or more embodiments must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or will be performed in any particular embodiment.

[0204] What has been described herein, in this specification and in the drawings includes examples that can provide an ASR variable-length text conversion method, apparatus, computer equipment, and non-volatile computer-readable storage medium. Of course, it is not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of the present disclosure, but it can be recognized that many other combinations and permutations of the disclosed features are possible. Therefore, it is obvious that various modifications can be made to the present disclosure without departing from the scope or spirit of the present disclosure, but all these various modifications should belong to the scope of protection of the claims attached to the present invention. In addition, or in an alternative, other embodiments of the present disclosure may be obvious from consideration of the present specification and the drawings and from the practice of the present disclosure as presented herein. It is intended that the examples presented in the present specification and the drawings are considered to be illustrative and not restrictive in all respects. Although specific terms are used herein, they are used in a general and descriptive sense and are not used for limiting purposes.

Claims

1. A method for converting ASR variable-length text. It is characterized in that include: Receiving an initial variable-length text output by the ASR system, and acquiring an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text; Performing feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector; Input the initial character sequence vector and the initial pinyin sequence vector into the corresponding preset CSC decoder and CPP decoder respectively to generate the corresponding target character sequence and target pinyin sequence; Based on the target character sequence and the target pinyin sequence, a target variable-length text is generated after the initial variable-length text is converted.

2. The ASR variable length text conversion method according to claim 1, It is characterized in that The receiving of the initial variable-length text output by the ASR system and obtaining the initial character sequence and the initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text include: Receiving the initial variable-length text output by the ASR system; Based on the initial variable-length text, extract the corresponding initial character sequence; Based on the initial character sequence, its corresponding initial pinyin sequence is generated through PyPinyin.

3. The ASR variable length text conversion method according to claim 2, It is characterized in that The method of encoding the initial character sequence and the initial pinyin sequence by a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector includes: Pre-load the pre-trained ChineseBERT model; The initial character sequence and the initial pinyin sequence are input into the ChineseBERT model for feature encoding to generate corresponding initial character sequence vectors and initial pinyin sequence vectors.

4. The ASR variable length text conversion method according to claim 3, It is characterized in that The step of inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoders and CPP decoders respectively to generate corresponding target character sequences and target pinyin sequences includes: Pre-built CSC decoder and CPP decoder; Inputting the initial character sequence vector into the CSC decoder to generate a decoded character sequence, and performing post-processing on the decoded character sequence to obtain a target character sequence; The initial pinyin sequence vector is input into the CPP decoder to generate a decoded pinyin sequence, and the decoded pinyin sequence is post-processed to obtain a target pinyin sequence.

5. The ASR variable length text conversion method according to claim 4, It is characterized in that The step of generating a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence includes: Integrate the target character sequence with the target pinyin sequence to generate a target variable-length text after the initial variable-length text is converted; Send the target variable-length text to a specified location.

6. The ASR variable length text conversion method according to claim 5, It is characterized in that The step of inputting the initial character sequence vector into the CSC decoder to generate a decoded character sequence, and performing post-processing on the decoded character sequence to obtain a target character sequence further includes: The unchanged tokens in the initial character sequence are copied to the target character sequence using a copy enhancement architecture.

7. The ASR variable length text conversion method according to claim 6, It is characterized in that The step of inputting the initial character sequence vector and the initial pinyin sequence vector into a corresponding preset CSC decoder and a CPP decoder to generate a corresponding target character sequence and a target pinyin sequence further includes: The weight of the CPP decoder in the decoding task is adaptively adjusted according to the pinyin edit distance between the initial pinyin sequence and the target pinyin sequence.

8. An ASR variable length text conversion device, It is characterized in that include: A receiving module, used for receiving an initial variable-length text output by the ASR system, and obtaining an initial character sequence and an initial pinyin sequence corresponding to the initial variable-length text based on the initial variable-length text; An encoding module, used for performing feature encoding on the initial character sequence and the initial pinyin sequence through a preset encoder to generate a corresponding initial character sequence vector and an initial pinyin sequence vector; A decoding module, used for inputting the initial character sequence vector and the initial pinyin sequence vector into corresponding preset CSC decoder and CPP decoder respectively, to generate corresponding target character sequence and target pinyin sequence; A generating module is used to generate a target variable-length text after the initial variable-length text is converted based on the target character sequence and the target pinyin sequence.

9. A computer device, It is characterized in that The computer device comprises at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. When the computer program is executed by the at least one processor, the ASR variable-length text conversion method according to any one of claims 1 to 7 can be implemented.

10. A non-volatile computer-readable storage medium, It is characterized in that The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the ASR variable-length text conversion method according to any one of claims 1 to 7 can be implemented.