Data processing method, device and terminal equipment

By training the text restoration model, using voice roles to associate text and time information, the problem that speech recognition cannot form written text is solved, and the efficient conversion of speech conversion into concise written text is realized, and the clerk's work efficiency is improved.

CN112466286BActive Publication Date: 2024-11-05ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910764575.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-08-19
Publication Date
2024-11-05
Estimated Expiration
2039-08-19

AI Technical Summary

Technical Problem

The prior art cannot be effectively converted into written text during speech recognition, resulting in the need of clerks to make large-scale modifications and reduce work efficiency.

Method used

By obtaining training data, including text to be trained, text associated with voice roles and time information, the text restore model is trained, and the model is used to break sentences and symbols of the voice text, and converted into written text.

Benefits of technology

The number of modifications by the clerk has been reduced, work efficiency has been improved, and the written texts generated are more concise and conform to the written record format.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112466286B_ABST
    Figure CN112466286B_ABST
Patent Text Reader

Abstract

The present invention discloses a data processing method and device, and a terminal device. The method comprises: obtaining training data, wherein the training data comprises a text to be trained, associated texts corresponding to each voice character in the text to be trained, and time information during the dialogue process of the voice characters; obtaining a text restoration model by training the training data; and restoring the text to be analyzed using the text restoration model to obtain a target written text. The present invention solves the technical problem in the related art that a written voice text cannot be formed when converting voice, so that the user needs to make a lot of modifications later, which reduces the work efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method and device, and terminal equipment. Background Art

[0002] In the related art, when processing voice information, attention is paid to the specific content of the voice, that is, for each voice information, the voice content is converted into spoken text. The texts formed by this conversion method are all spoken texts, which are prone to text accumulation and cannot form written texts. For example, in the legal field, as people's legal awareness increases, there are more and more court cases, the traditional judicial trial model is inefficient, and more and more cases need to be heard, and the workload of the clerks has increased greatly. At this time, recording and video equipment began to enter the court in large quantities, and voice recognition equipment began to gradually assist the work of the clerks. However, there is a natural gap between the voice recognition text of the trial and the version recorded by the clerk. The purpose of speech recognition (ASR) is to correctly identify the original speech content of the speaker, that is, spoken language, while the transcript text that the clerk wants to record on file is required to be concise and standardized, and is written language; that is, the voice text recognized by the voice recognition method is not the text of the final record of the trial. The trial process is mostly spoken language, while the text filed by the clerk is written, and there are obvious defects in the specific format of the text (such as no obvious sentence breaks and punctuation intervals) and the recorded content. The clerk needs to do a lot of text sorting later, which reduces the efficiency of the trial voice conversion.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a data processing method and apparatus, and a terminal device, so as to at least solve the technical problem in the related art that a written voice text cannot be formed when converting voice, so that the user needs to make a lot of modifications later, which reduces the work efficiency.

[0005] According to one aspect of an embodiment of the present invention, there is provided a data processing method, comprising: obtaining training data, wherein the training data comprises a text to be trained, associated texts corresponding to each voice role in the text to be trained, and time information during a conversation between the voice roles; obtaining a text restoration model by training the training data; and restoring the text to be analyzed using the text restoration model to obtain a target written text.

[0006] According to another aspect of an embodiment of the present invention, a data processing device is provided, comprising: an acquisition unit, used to acquire training data, wherein the training data includes a text to be trained, associated texts corresponding to each voice role in the text to be trained, and time information during the conversation between the voice roles; a training unit, used to obtain a text restoration model by training the training data; and a restoration unit, used to restore the text to be analyzed using the text restoration model to obtain a target written text.

[0007] According to another aspect of an embodiment of the present invention, a storage medium is further provided, the storage medium including a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned data processing methods.

[0008] According to another aspect of an embodiment of the present invention, a processor is further provided, wherein the processor is used to run a program, wherein the program executes any one of the above-mentioned data processing methods when running.

[0009] According to another aspect of an embodiment of the present invention, a terminal device is also provided, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: Step 1, obtaining training data, wherein the training data comprises a text to be trained, associated texts corresponding to each voice role in the text to be trained, and time information during the conversation between the voice roles; Step 2, obtaining a text restoration model by training the training data; Step 3, using the text restoration model to restore the text to be analyzed to obtain a target written text.

[0010] In the embodiment of the present invention, the speech recognition text can be segmented and annotated with symbols, and applied to various speech conversion scenarios. For example, in the trial scenario, the speech of multiple voice characters in the trial can be recognized, and the recognized speech text can be converted into written text through text preprocessing, case entity error recognition, colloquial error type recognition, punctuation restoration, etc. The written text is more concise and conforms to the written record format, reducing the workload of the clerk.

[0011] In an embodiment of the present invention, training data is obtained, wherein the training data includes the text to be trained, the associated text corresponding to each voice role in the text to be trained, and the time information during the voice role dialogue process. By training the training data, a text restoration model is obtained, and the text to be analyzed is restored using the text restoration model to obtain a target written text. That is, the associated text corresponding to each voice role in the text to be trained (i.e., the context information of the current sentence / word to be analyzed) and the time information during the voice dialogue process can be used to segment and organize the text to be trained after voice conversion is completed, and the innovative text restoration model (mainly the BERT model) is used to restore the text to be analyzed to written text, thereby reducing the workload of subsequent text organization of users (such as clerks), reducing the amount of modification by users, and improving work efficiency, thereby solving the technical problem in the related art that a written voice text cannot be formed when converting voice, so that users need to make a lot of modifications later, reducing work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0013] Figure 1 A hardware structure block diagram of a computer terminal for implementing a data processing method is shown;

[0014] Figure 2 A schematic diagram of a data processing method network terminal is shown;

[0015] Figure 3 is a flow chart of a data processing method according to Embodiment 1 of the present invention;

[0016] Figure 4 is an architecture diagram of an optional text restoration model according to an embodiment of the present invention;

[0017] Figure 5 is a schematic diagram of an optional restoration to obtain a written text according to an embodiment of the present invention;

[0018] Figure 6 is a schematic diagram of a data processing device according to an embodiment of the present invention;

[0019] Figure 7 is a structural block diagram of a terminal device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0023] ASR, Automatic Speech Recognition, is a technology used to convert human speech into text.

[0024] LSTM, Long Short-Term Memory, referred to as long short-term memory network, is a time recurrent neural network (RNN).

[0025] NLP, Natural Language Processing, natural language processing.

[0026] The BERT model, Bidirectional Encoder Representations from Transformers, is a processing model in NLP that can predict the current word through the text on the left, right, top, or bottom sides and predict the next sentence through the current sentence.

[0027] Softmax can be understood as a normalization function, which maps some inputs to real numbers between 0 and 1. Normalization ensures that the sum is 1, determines the probabilities of multiple classifications, and the sum of the probabilities of multiple classifications is 1.

[0028] Word embedding refers to converting words into a distributed representation, which represents words as a continuous dense vector of fixed length.

[0029] Example 1

[0030] According to an embodiment of the present invention, a data processing method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0032] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0036] Figure 1 The hardware structure block diagram shown can be used not only as an exemplary block diagram of the above-mentioned computer terminal 10 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Figure 2 A schematic diagram of a data processing method network terminal is shown, such as Figure 2 As shown, the computer terminal 10 (or mobile device) can be connected to one or more servers (e.g., security servers, resource servers, game servers, etc.) via a data network connection or electronic connection. In an optional embodiment, the above-mentioned computer terminal 10 (or mobile device) can be any mobile computing device, etc. The data network connection can be a local area network connection, a wide area network connection, an Internet connection, or other types of data network connections. The computer terminal 10 (or mobile device) can be executed to connect to a network service performed by a server (e.g., a security server) or a group of servers. A network server is a network-based user service, such as a social network, cloud resources, email, online payment or other online applications.

[0037] The present invention can be applied to various speech processing environments. The embodiment of the present invention is exemplified by converting court trial speech into written text, mainly converting the text after the court trial speech is converted from spoken language to written language. However, the environment in which the method is used is not limited to this, and it can also be applied to other speech processing environments.

[0038] Optionally, the data processing method in the embodiment of the present invention is applied to at least one of the following scenarios: a conference scenario, a trial scenario. Among them, the conference scenario may refer to a meeting attended by multiple people. During the multi-person conversation, the data processing method in the embodiment of the present invention may form a written text of the meeting; and the trial scenario may refer to restoring the voice of the multi-person conversation during the trial to obtain a written text of the trial. The trial scenario includes but is not limited to: an on-site trial scene and a trial restoration scene. In the embodiment of the present invention, taking the trial restoration scene as an example, after the trial, the dialogues of the various characters in the trial are restored to obtain the target written text.

[0039] Meanwhile, the embodiments of the present invention focus on the speech recognition technology ASR, and focus on restoring symbols in text (exemplified by punctuation restoration) to form written text. The punctuation marks in the following embodiments of the present invention include but are not limited to: colon, comma, period, question mark.

[0040] Under the above operating environment, this application provides Figure 3 Flowchart of the data processing method shown. Figure 3 is a flow chart of a data processing method according to Embodiment 1 of the present invention. Figure 3 As shown, the method comprises the following steps:

[0041] Step S302, obtaining training data, wherein the training data includes the text to be trained, associated text corresponding to each voice character in the text to be trained, and time information during the dialogue process of the voice characters;

[0042] Step S304, obtaining a text restoration model by training the training data;

[0043] Step S306: Use a text restoration model to restore the text to be analyzed to obtain a target written text.

[0044] Through the above steps, training data can be obtained, wherein the training data includes the text to be trained, the associated text corresponding to each voice role in the text to be trained, and the time information during the voice role dialogue process. By training the training data, a text restoration model is obtained, and the text to be analyzed is restored using the text restoration model to obtain the target written text. In this embodiment, the associated text corresponding to each voice role in the text to be trained (i.e., the context information of the current sentence / word to be analyzed) and the time information during the voice dialogue process can be used to segment and organize the text to be trained after the voice conversion is completed, and the innovative text restoration model (mainly the BERT model) is used to restore the text to be analyzed to written text, reducing the workload of the user (such as the clerk) for subsequent text organization, reducing the amount of modification by the user, and improving work efficiency, thereby solving the technical problem in the related technology that the written voice text cannot be formed when converting the voice, so that the user needs to make a lot of modifications later, which reduces the work efficiency.

[0045] The embodiments of the present invention can be applied to terminal devices using speech recognition technology, and are particularly suitable for devices that convert court trial speech into written record text. There is no specific limitation on the number of voice roles that make speech during the trial, and the voice roles include but are not limited to: the presiding judge, the plaintiff, the defendant, the defendant's attorney, etc. The speech made by each voice role during a trial can be converted to obtain a written record text. The recorded voice roles are diversified, and the recorded content is more in line with the standards for the clerk's record, reducing the clerk's modification rate, and can also provide better data quality for subsequent analysis of trial summaries and element recognition. Compared with the related art that only converts speech into spoken text, the clerk needs to make a lot of text modifications later. The embodiments of the present invention can obviously reduce the amount of modification by the clerk, is faster in text conversion, and can adapt to a variety of voice environments.

[0046] The embodiment of the present invention is described below in conjunction with various steps.

[0047] In the embodiment of the present invention, it is first necessary to convert the speech into a text to be trained (obtained after processing the initial recognition text), and the text to be trained can correspond to various speech dialogue scenarios, including but not limited to: court trial scenario, oral trial scenario, meeting scenario, shopping scenario, etc. The embodiment of the present invention is schematically described using a court trial scenario.

[0048] Step S302, obtaining training data, wherein the training data includes the text to be trained, associated text corresponding to each voice character in the text to be trained, and time information during the dialogue process of the voice characters.

[0049] As an optional implementation of the present invention, the text to be trained includes at least one of the following: speech analysis text and historical dialogue text, wherein the speech analysis text is the text obtained after speech recognition of the voices of each voice character at the scene of the speech dialogue, and the historical dialogue text is the dialogue text of each voice character recorded in the historical process. That is, the text to be trained can not only train the speech analysis text that completes the speech conversion during the trial scene dialogue process, but also process the historical trial records.

[0050] In each voice conversation process, multiple voice roles are included. In the embodiment of the present invention, sentence segmentation is completed by recording the current voice in the voice conversation process, as well as context information and time information (such as pause information) in the conversation process.

[0051] In an embodiment of the present invention, the step of obtaining training data includes: obtaining the text to be trained and the voice dialogue scenario; determining multiple voice characters that conduct dialogues in the voice dialogue scenario; extracting associated text associated with each voice character, wherein the associated text includes at least two dialogue sentences of the voice character corresponding to the current sentence to be analyzed in the historical process; extracting time information in the voice dialogue process; and using the text to be trained, the voice dialogue scenario, the associated text and the time information as training data.

[0052] The above-mentioned steps of obtaining the text to be trained and the voice dialogue scenario include: recognizing the conversational voice content of the voice dialogue scenario to obtain an initial recognition text; preprocessing the initial recognition text; after the preprocessing is completed, identifying entity error data in the initial recognition text; identifying colloquial error data in the initial recognition text to obtain the text to be trained.

[0053] The initial recognition text can be understood as the spoken text after speech conversion.

[0054] If the initial recognition text is processed to obtain the text to be trained, three operations need to be performed, including: text preprocessing (i.e., preprocessing the initial recognition text as mentioned above), entity error recognition (i.e., recognizing entity error data in the initial recognition text as mentioned above), and colloquial error type recognition (i.e., recognizing colloquial error data in the initial recognition text as mentioned above). The following describes these three situations in detail.

[0055] The first operation, text preprocessing

[0056] That is, the initial recognition text can be preprocessed to facilitate subsequent use. Taking the court trial scene as an example, the conversation voice content of the voice dialogue scene is recognized to obtain the initial recognition text, including: recognizing the conversation voice content according to the time sequence of the conversation and each voice role to obtain the initial recognition text. Among them, there can be multiple voice roles, and the words of multiple voice roles are input in chronological order; and each voice role is set to correspond to a voice window, and the voice roles include but are not limited to: the presiding judge, plaintiff 1, defendant 1, plaintiff's attorney, etc.

[0057] When preprocessing the initial recognition text, it may include: segmenting each sentence in the initial recognition text; performing word segmentation processing on each sentence to obtain multiple words; and performing part-of-speech tagging on each word.

[0058] Optionally, the segmentation of each sentence in the initial recognition text refers to sentence segmentation of the training text, i.e., segmentation. When segmenting, the sentence segmentation can be performed according to the dialogue between each voice role. For example, the presiding judge, plaintiff 1, and defendant 1 each said a sentence in succession, and this part of the training text can be divided into three sentences. When segmenting words, word segmentation can be performed based on the meaning and part of speech of various languages. In addition, when performing part of speech tagging, each word is marked, such as verb, noun, conjunction, auxiliary word, etc. In an embodiment of the present invention, entity recognition can also be performed on the training text during preprocessing, for example, identifying the names of people, places, and institutions that appear in the training text.

[0059] The second operation, entity error recognition

[0060] This operation is used to identify erroneous data of case entity names in the initial recognition text. The input is the initial recognition text and the case entity list.

[0061] When the voice dialogue scenario is a court trial scenario, the step of identifying entity error data in the initial recognition text includes: converting the case entity list of the target case corresponding to the initial recognition text and the court trial scenario into a pinyin sequence; after the conversion is completed, searching the case entity name of the target case in the initial recognition text to obtain entity error data. That is, the initial recognition text and the case entity list can be converted into a pinyin sequence first, and the pinyin sequence can be used to find entity errors; and when searching for the case entity name of the target case in the initial recognition text, a heuristic search method (such as maximum forward matching) can be used to find the candidate most similar to the case entity in the initial recognition text, thereby obtaining entity error data.

[0062] The third operation is to identify the types of spoken language errors.

[0063] Taking the court trial scenario as an example, after performing court trial speech recognition, an initial recognition text (i.e., colloquial text) will be obtained. This initial recognition text often contains modal particles, colloquial expressions, filled pauses, and repaired words (repeated, corrected, restarted), etc., which are colloquial jargon words. Through this operation, these error types and their locations can be accurately identified.

[0064] In an alternative embodiment of the present invention, identifying colloquial error data in the initial recognition text includes: identifying at least one of the following colloquial words in the initial recognition text: modal particles, filled pauses, repaired words; setting type tags for each colloquial word; determining the error types and location data of the colloquial words that appear in the initial recognition text; constructing a colloquial error recognition model based on the type tags, error types, and location data of the colloquial words; and identifying colloquial error data through the colloquial error recognition model.

[0065] Embodiments of the present invention can exemplarily illustrate 6 types of colloquial word types:

[0066] First, modal particles (modal)

[0067] Definition: Modal particles often appear at the end of a sentence (declarative sentence, imperative sentence, interrogative sentence, etc.) to express the speaker's mood and emotion. Common modal particles are: uh, ah, um.

[0068] Examples:

[0069] It means <uh / modal>;

[0070] Also take out the household register <um / modal>

[0071] <At / restart> What company <ah / modal>

[0072] Second, colloquial expression words (colloquial)

[0073] Definition: Colloquial expression words often appear in colloquial language. After deleting this colloquial expression word, the sentence is still smooth and the semantics remains complete. Common colloquial expression words are: here, then, should say.

[0074] Example: <Ah / fp><Like this ah / fp><That is / fp> They are involved in two aspects of property <Ah / fp> One is stocks <Then / colloquial> For stocks <Well / fp> Currently <Well / fp> Because the court has taken preservation measures, we don't have the data on hand.

[0075] Third, filled pauses (fp)

[0076] Definition: Pause words are words that are used by speakers to pause while they are thinking about what to say next, in order to reserve the right to continue speaking in a multi-person conversation. Common FPs include: Oh, This, Ah, Um, That's it.

[0077] Example: Wait a moment. I'd like to add something to the reviewer's request for a written document. Do we need to provide a paper version of the document we provided before the court?

[0078] The fourth type is repetition

[0079] Definition: repeated word, matching pattern: ABC{filled pause}ABC. ABC is repeated, and there may be a pause word (fp) in the middle.

[0080] Example: On September 10, 2016, I told him about this incident and he came back at midnight on the 11th. September 11th and November 11th.

[0081] The fifth type is the corrected word correction

[0082] Definition: The corrected word, its matching pattern: ABC{filled pause}XYZ. XYZ corrected ABC's statement. There may be some literal overlap between XYZ and ABC; there may be a pause word (fp) between ABC and XYZ.

[0083] Example: <ah / fp><are there any wounds / correction> Were you injured? Did you have any suggestions for what to do afterwards?

[0084] The sixth type is the word restart

[0085] Definition: The word to be restarted, its matching pattern is: ABC{filled pause}. ABC is ignored, and a new sentence is restarted afterwards.

[0086] Example: <in / restart>what company is <ah / modal>;

[0087] <Let's go / restart> I want to sleep.

[0088] Above are 6 common types of spoken errors.

[0089] In the embodiment of the present invention, the spoken error data can be identified by the spoken error recognition model. When constructing the spoken error recognition model, the type label of the spoken word (which can be understood as a type identifier, each word position corresponds to a type identifier), the error type (the error type is identified at the corresponding word position in the initial recognition text) and the position data (indicating the position of the spoken word in the initial recognition text, which can be identified in the form of rows and columns) are used. The words are sequenced by the type label of the spoken word, the error type and the position data. Optionally, in the embodiment of the present invention, each error type can be labeled with four labels B, E, I, and S, where B represents the starting word of the type, E represents the ending word of the type, I represents the middle word of the type, and S represents the single word of the type. In addition, O can also be used to represent other labels, that is, O represents other labels that do not belong to the above-mentioned 6 general types, for a total of 25 labels.

[0090] Alternatively, in an embodiment of the present invention, the colloquial error recognition model can use BILSTM+CRF, which is a neural network-based architecture, which is divided into 4 layers. The first layer is the input layer, which takes the vector of a single word as input. The word vector is obtained by training the skip-gram model using a large number of court transcripts; the second layer is the bidirectional LSTM layer, which models the input text from both the positive and negative directions, and splices the positive and negative neuron outputs representing each word together, and inputs them to the third layer, namely the CRF (conditional random field) layer, to model the label relationship; the fourth layer is the output layer, and the final label and position are obtained.

[0091] Using the above implementation, the initial recognition text can be identified and errors can be corrected, so that part of the spoken language can be converted into written language. The symbol restoration work can be implemented next. The symbol restoration work can be done through the BERT model in NLP and the context information of each sentence. It can effectively solve the situation where the conversation is interrupted, and the symbols of the text (such as punctuation) can be restored.

[0092] During the restoration process, it is necessary to first train a text restoration model. In order to train the text restoration model, it is necessary to obtain training data, wherein the training data includes the text to be trained obtained through text preprocessing, entity error recognition, and spoken error type recognition, the associated text corresponding to each voice role in the text to be trained, and the time information during the voice dialogue.

[0093] In an embodiment of the present invention, the associated text corresponding to each voice character in the text to be trained may refer to other voice texts issued by the voice character corresponding to the current sentence to be analyzed. For example, voice character A issues: "It is impossible to prove the time of the case". At this time, it is necessary to analyze other words said by the voice character before or after, that is, to obtain the context information of the voice character. In the embodiment of the present invention, not only the information of the sentence to be analyzed is used, but also the words said by the same character before, that is, the context information. For example, when obtaining training data, for each sentence in the text to be trained, the sentences of the first 5 windows of the voice character that issued the sentence are searched.

[0094] As an optional implementation of the present invention, after acquiring the training data, the method further includes: determining the symbol label corresponding to the current sentence to be analyzed in the text to be trained, wherein the symbol label includes at least: punctuation label and letter label.

[0095] In the embodiment of the present invention, the punctuation restoration technology is mainly used for explanation, and the selected symbol label is preferably a punctuation label. By outputting the punctuation label of the sentence to be analyzed, it can be known at which time point the voice character paused when speaking the sentence to be analyzed and the duration of the pause.

[0096] In the embodiment of the present invention, the label of each token (sentence to be analyzed) is the punctuation label before the current token.

[0097] As for the time information, it includes at least one of the following: the duration of the voice pause between each sentence; the start time point and the end time point of the voice pause between each sentence; the start time and the end time of the dialogue. That is, by selecting the pause duration of the voice character, for example, when the user pauses for 0.5 seconds when speaking a sentence, a comma can be added at this time.

[0098] Through the above implementation, the corresponding training data is obtained, and the text restoration model obtained by training with the training data is described below.

[0099] Step S304, obtaining a text restoration model by training the training data.

[0100] In an embodiment of the present invention, when training a text restoration model, it is mainly divided into two stages. Optionally, the step of obtaining a text restoration model by training the training data includes: performing a first-stage training on the feature information of the training text and the associated text to obtain a restoration output layer, wherein each unit of the restoration output layer represents a text feature information of each sentence to be analyzed in the training text; performing a second-stage training on the time information to obtain a time-continuous embedding layer; splicing the restoration output layer, the time-continuous embedding layer and the preset normalization layer to obtain a text restoration model, wherein the normalization layer is used to determine the selection probability of each symbol label to classify the symbol labels.

[0101] The first stage of training mentioned above can be understood as training purely on the training text, while the second stage of training can be understood as training on time information, that is, training on speech pause information.

[0102] In an embodiment of the present invention, the first stage of training is performed on the feature information of the training text and the related text to obtain the step of restoring the output layer, including: using the formatted documents and voice transcript information corresponding to the voice dialogue scenario to initially pre-train the text restoration model to obtain the restoration layer of the text restoration model; using the restoration layer to extract the feature information of the training text and the related text to obtain the restoration output layer.

[0103] A court trial scenario is used as a voice dialogue scenario and the BERT model is used as a text restoration model for example. The above-mentioned format document can be understood as a legal document, and the voice transcript information can be the previously obtained transcript content. The architecture of the BERT model is selected as the architecture of the text restoration model, so that the BERT model can be pre-trained with legal documents and voice transcript information to obtain a restoration layer. In an embodiment of the present invention, the restoration layer can be represented by a BERT layer. The restoration layer is used to extract feature information of each sentence and related text in the text to be trained to obtain a restoration output layer. In an embodiment of the present invention, the restoration output layer can be represented by a BERT output layer.

[0104] In the embodiment of the present invention, each layer in the text restoration model needs to be connected to a normalization layer (Softmax layer) for label classification. The output obtained in the first training stage is the BERT output layer, and each unit represents the text feature information of the current token.

[0105] In an embodiment of the present invention, the step of performing a second-stage training on the time information to obtain a time-continuous embedding layer includes: discretizing the speech pause duration in the time information to obtain an initial embedding layer of the text restoration model; splicing the initial embedding layer to the restoration output layer, and connecting the normalization layer after the splicing is completed; after the connection is completed, updating the initial embedding layer to obtain the text restoration model.

[0106] Optionally, the time information is in milliseconds to determine the start, pause, end, and other information of each voice role in each sentence. First, the voice pause duration in the time information is discretized to obtain an initial embedding layer. In the embodiment of the present invention, the initial embedding layer can be represented by duration embedding, and the duration embedding is spliced ​​in the BERT output layer; at the same time, a fully connected layer (represented by FC layer in the embodiment of the present invention) and a normalization layer (represented by Softmax in the embodiment of the present invention) can be connected to the back. After completion, the text restoration model is trained in the second stage, and the initial embedding layer is updated to complete the training to obtain the text restoration model. In this second training stage, the BERT output layer is kept unchanged.

[0107] Figure 4 is an architecture diagram of an optional text restoration model according to an embodiment of the present invention, such as Figure 4 As shown in the figure, this text restoration model is mainly used for punctuation restoration, and can also be called a punctuation restoration model; below it can be various speech sentences (represented by W1, W2...Wn respectively), and a restoration layer (BERT layer) is established by analyzing the speech sentences; the feature information of each sentence is extracted using the restoration layer BERT layer to obtain the restoration output layer BERT output, and after discretizing the pause duration in the time information, the initial embedding layer duration embedding can be obtained, and the initial embedding layer duration embedding is spliced ​​to the restoration output layer BERT output. After the splicing is completed, it can be connected to the fully connected layer FC layer and a normalized layer Softmax.

[0108] Step S306: Use a text restoration model to restore the text to be analyzed to obtain a target written text.

[0109] Through the above-mentioned embodiments, the spoken text converted from speech can be restored to written text through preprocessing, entity error recognition, spoken error type recognition, symbol restoration, etc. The written text obtained in this way can reduce the modification rate of users (such as clerks), and at the same time provide better data quality for subsequent text extraction and element recognition analysis. The restored text can realize the symbol restoration in each sentence, and effectively restore the situation where the multi-voice characters are interrupted during the conversation. The restored written text is more concise and standardized, effectively realizing the short sentences of the text, restoring the dialogues of each voice character, and the symbols in the written text (with an emphasis on punctuation) are effectively restored.

[0110] Figure 5 is a schematic diagram of an optional restoration to obtain a written text according to an embodiment of the present invention, such as Figure 5 As shown, corresponding to the trial scenario, there are three voice roles. Figure 5 In the figure, the trial recording 1, trial recording 2, and trial recording 3 are respectively represented. The lower left side is the text transcribed by voice recognition, and the lower right side is the written text restored by this application (i.e., the clerk version generated by NLP). It can be seen that the text on the left has completed voice recognition, but the punctuation has not been effectively restored. The clerk version on the right is more concise, and the sentence is broken by punctuation marks such as colons, commas, periods, and question marks, making the recorded content more written.

[0111] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the data processing method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] Example 2

[0114] According to an embodiment of the present invention, a data processing device for implementing the above data processing method is also provided. Figure 6 is a schematic diagram of a data processing device according to an embodiment of the present invention. Figure 6 As shown, the device includes: an acquisition unit 61, a training unit 63, and a restoration unit 65, wherein:

[0115] An acquisition unit 61 is used to acquire training data, wherein the training data includes a text to be trained, associated texts corresponding to each voice character in the text to be trained, and time information during a conversation between the voice characters;

[0116] A training unit 63, configured to obtain a text restoration model by training the training data;

[0117] The restoration unit 65 is used to restore the text to be analyzed by using a text restoration model to obtain a target written text.

[0118] The above-mentioned data processing device can obtain training data through the acquisition unit 61, wherein the training data includes the text to be trained, the associated text corresponding to each voice role in the text to be trained, and the time information during the voice role dialogue process, and the training unit 63 is used to train the training data to obtain a text restoration model, and the restoration unit 65 uses the text restoration model to restore the text to be analyzed to obtain the target written text. In this embodiment, the associated text corresponding to each voice role in the text to be trained (i.e., the context information of the current sentence / word to be analyzed) and the time information during the voice dialogue process can be used to segment and organize the text to be trained after the voice conversion is completed, and the innovative text restoration model is used to restore the text to be analyzed to a written text, thereby reducing the workload of the user (such as a clerk) for subsequent text organization, reducing the amount of modification by the user, and improving work efficiency, thereby solving the technical problem in the related art that the written voice text cannot be formed when converting the voice, so that the user needs to make a lot of modifications later, which reduces the work efficiency.

[0119] In an optional embodiment, the acquisition unit 61 includes: a first acquisition module, used to acquire the text to be trained and the voice dialogue scenario; a first determination module, used to determine multiple voice roles that conduct dialogue in the voice dialogue scenario; a first extraction module, used to extract associated text associated with each voice role, wherein the associated text includes at least two dialogue sentences of the voice role corresponding to the current sentence to be analyzed in the historical process; a second extraction module, used to extract time information in the voice dialogue process; the first determination module, used to use the text to be trained, the voice dialogue scenario, the associated text and the time information as training data.

[0120] Optionally, the time information includes at least one of the following: the duration of the speech pause between each sentence; the start time point and the end time point of the speech pause between each sentence; the start time and the end time of the conversation.

[0121] In an optional embodiment, the data processing device further includes: a determination unit, which is used to determine the symbol label corresponding to the current sentence to be analyzed in the text to be trained after acquiring the training data, wherein the symbol label includes at least: a punctuation label and a letter label.

[0122] In an embodiment of the present invention, the training unit 63 includes: a first training module, which is used to perform a first-stage training on the feature information of the training text and the associated text to obtain a restored output layer, wherein each unit of the restored output layer represents a text feature information of each sentence to be analyzed in the training text; a second training module, which is used to perform a second-stage training on the time information to obtain a time-continuous embedding layer; a first splicing module, which is used to splice the restored output layer, the time-continuous embedding layer and the preset normalization layer to obtain a text restoration model, wherein the normalization layer is used to determine the selection probability of each symbol label to classify the symbol labels.

[0123] In another optional embodiment of the present invention, the first training module includes: a first training sub-module, which is used to perform initial pre-training on the text restoration model using formatted documents and voice transcript information corresponding to the voice dialogue scenario to obtain a restoration layer of the text restoration model; and a first extraction sub-module, which is used to extract feature information of the text to be trained and the related text using the restoration layer to obtain a restoration output layer.

[0124] In another optional embodiment of the present invention, the second training module includes: a discretization submodule, which is used to discretize the speech pause duration in the time information to obtain an initial embedding layer of the text restoration model; a first splicing submodule, which is used to splice the initial embedding layer to the restoration output layer, and connect the normalization layer after the splicing is completed; an update submodule, which is used to update the initial embedding layer after the connection is completed to obtain the text restoration model.

[0125] In an optional embodiment, the above-mentioned text to be trained includes at least one of the following: a speech analysis text and a historical dialogue text, wherein the speech analysis text is a text obtained after speech recognition of the voices of each voice character at the scene of the speech dialogue, and the historical dialogue text is a dialogue text of each voice character recorded in the historical process.

[0126] In an optional embodiment, the first acquisition module includes: a first recognition submodule, which is used to recognize the conversational speech content of the voice dialogue scenario to obtain an initial recognition text; a preprocessing submodule, which is used to preprocess the initial recognition text; a second recognition submodule, which is used to recognize entity error data in the initial recognition text after the preprocessing is completed; and a third recognition submodule, which is used to recognize colloquial error data in the initial recognition text to obtain a text to be trained.

[0127] Optionally, the first recognition submodule recognizes the speech content of the conversation according to the time sequence of the conversation and each voice role to obtain an initial recognition text.

[0128] In another optional embodiment of the present invention, the preprocessing submodule includes: a segmentation submodule for segmenting each sentence in the initial recognition text; a word segmentation submodule for segmenting each sentence to obtain multiple words; and a tagging submodule for tagging the part of speech of each word.

[0129] In another optional embodiment of the present invention, the second recognition submodule includes: a conversion submodule, which is used to convert the case entity list of the target case corresponding to the initial recognition text and the trial scene into a pinyin sequence; and a search submodule, which is used to search the case entity name of the target case in the initial recognition text after the conversion is completed to obtain entity error data.

[0130] In another optional embodiment of the present invention, the third recognition submodule includes: a fourth recognition submodule, used to recognize at least one of the following colloquial words in the initial recognition text: modal particles, pause words, and repaired words; a setting submodule, used to set a type label for each colloquial word; a determination submodule, used to determine the error type and position data of the colloquial words appearing in the initial recognition text; a construction submodule, used to construct a colloquial error recognition model based on the type label, error type and position data of the colloquial words; and a fifth recognition submodule, used to recognize the colloquial error data through the colloquial error recognition model.

[0131] Optionally, the text restoration model is a BERT model.

[0132] It should be noted that the acquisition unit 61, training unit 63, and restoration unit 65 correspond to step S302, step S304, and step S306 in Example 1, respectively. The three modules and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0133] Example 3

[0134] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0135] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0136] Optionally, the above-mentioned computer terminal includes: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: obtaining training data, wherein the training data includes the text to be trained, the associated text corresponding to each voice role in the text to be trained, and the time information during the voice role dialogue process; obtaining a text restoration model by training the training data; and using the text restoration model to restore the text to be analyzed to obtain a target written text.

[0137] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: obtaining the text to be trained and the voice dialogue scenario; determining multiple voice roles that conduct dialogues in the voice dialogue scenario; extracting the associated text associated with each voice role, wherein the associated text includes at least two dialogue sentences of the voice role corresponding to the current sentence to be analyzed in the historical process; extracting the time information in the voice dialogue process; using the text to be trained, the voice dialogue scenario, the associated text and the time information as training data.

[0138] Optionally, the time information includes at least one of the following: the duration of the speech pause between each sentence; the start time point and the end time point of the speech pause between each sentence; the start time and the end time of the conversation.

[0139] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: after obtaining the training data, determine the symbol label corresponding to the current sentence to be analyzed in the text to be trained, wherein the symbol label includes at least: punctuation label and letter label.

[0140] In this embodiment, the above-mentioned text to be trained includes at least one of the following: speech analysis text and historical dialogue text, wherein the speech analysis text is the text obtained after speech recognition of the voices of each voice character at the scene of the speech dialogue, and the historical dialogue text is the dialogue text of each voice character recorded in the historical process.

[0141] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: performing the first stage training on the feature information of the training text and the associated text to obtain a restored output layer, wherein each unit of the restored output layer represents a text feature information of each sentence to be analyzed in the training text; performing the second stage training on the time information to obtain a time-continuous embedding layer; splicing the restored output layer, the time-continuous embedding layer and the preset normalization layer to obtain a text restoration model, wherein the normalization layer is used to determine the selection probability of each symbol label to classify the symbol labels.

[0142] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: using the formatted documents and voice transcript information corresponding to the voice dialogue scenario to initially pre-train the text restoration model to obtain the restoration layer of the text restoration model; using the restoration layer to extract the feature information of the text to be trained and the related text to obtain the restoration output layer.

[0143] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: discretize the speech pause duration in the time information to obtain the initial embedding layer of the text restoration model; splice the initial embedding layer to the restoration output layer, and connect the normalization layer after the splicing is completed; after the connection is completed, update the initial embedding layer to obtain the text restoration model.

[0144] In this embodiment, the text restoration model is a BERT model.

[0145] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: recognizing the conversational speech content of the voice dialogue scenario to obtain an initial recognition text; preprocessing the initial recognition text; after the preprocessing is completed, identifying entity error data in the initial recognition text; identifying colloquial error data in the initial recognition text to obtain the text to be trained.

[0146] In this embodiment, the computer terminal can also execute the program code of the following steps in the data processing method: recognize the conversation voice content according to the time sequence of the conversation and each voice role to obtain an initial recognition text.

[0147] In this embodiment, the computer terminal can also execute the program code of the following steps in the data processing method: segmenting each sentence in the initial recognition text; performing word segmentation on each sentence to obtain multiple words; and performing part-of-speech tagging on each word.

[0148] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: converting the case entity list of the target case corresponding to the initial recognition text and the trial scene into a pinyin sequence; after the conversion is completed, searching the initial recognition text for the case entity name of the target case to obtain entity error data.

[0149] In this embodiment, the above-mentioned computer terminal can also execute the program code of the following steps in the data processing method: identifying at least one of the following colloquial words in the initial recognition text: modal particles, pause words, and repaired words; setting a type label for each colloquial word; determining the error type and position data of the colloquial words appearing in the initial recognition text; constructing a colloquial error recognition model based on the type label, error type and position data of the colloquial words; and identifying the colloquial error data through the colloquial error recognition model.

[0150] Optionally, Figure 7 is a structural block diagram of a terminal device according to an embodiment of the present invention. Figure 7 As shown, the terminal device A may include: one or more processors, memory (including program instructions and data storage devices), as well as a network interface, an input / output interface, a keyboard, and a display.

[0151] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0152] The memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: Step 1, obtaining training data, wherein the training data includes the text to be trained, the associated text corresponding to each voice role in the text to be trained, and the time information during the dialogue process of the voice role; Step 2, obtaining a text restoration model by training the training data; Step 3, using the text restoration model to restore the text to be analyzed to obtain a target written text.

[0153] According to an embodiment of the present invention, a data processing scheme is provided. By acquiring training data, wherein the training data includes a text to be trained, associated texts corresponding to each voice character in the text to be trained, and time information during the dialogue process of the voice characters, a text restoration model is obtained by training the training data, and then the text to be analyzed is restored by using the text restoration model to obtain a target written text, thereby achieving the goal of restoring the voice text by using the text restoration model, implementing sentence segmentation in the restored written text, and marking voice pauses by using punctuation marks and other symbols, effectively restoring the interrupted scenes during the voice dialogue process, and the restored written text is more concise, meets the requirements of the written text, reduces the amount of modification by the user, and thus solves the technical problem in the related art that the written voice text cannot be formed when converting the voice, so that the user needs to make a lot of modifications later, which reduces the work efficiency.

[0154] It can be understood by those skilled in the art that Figure 7 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 7 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Figure 7 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 7 Different configurations are shown.

[0155] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0156] Example 4

[0157] The embodiment of the present invention further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided in the first embodiment.

[0158] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0159] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: obtaining training data, wherein the training data includes the text to be trained, associated text corresponding to each voice role in the text to be trained, and time information during the voice role dialogue; obtaining a text restoration model by training the training data; and restoring the text to be analyzed using the text restoration model to obtain a target written text.

[0160] The embodiment of the present invention further provides a processor. Optionally, in this embodiment, the processor can be used to process the program code executed by the data processing method provided in the first embodiment above.

[0161] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0162] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0167] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data processing method, It is characterized in that include: Acquire training data, wherein the training data includes a text to be trained, associated text corresponding to each voice character in the text to be trained, and time information during a conversation between the voice characters; Obtaining a text restoration model by training the training data; The text restoration model is used to restore the text to be analyzed to obtain a target written text; After acquiring the training data, the method further includes: Determine the symbol label corresponding to the current sentence to be analyzed in the text to be trained, wherein the symbol label at least includes: a punctuation label and a letter label; The step of obtaining a text restoration model by training the training data includes: Performing a first-stage training on the feature information of the text to be trained and the associated text to obtain a restored output layer, wherein each unit of the restored output layer represents a text feature information of each sentence to be analyzed in the text to be trained; Performing a second stage of training on the time information to obtain a time-continuous embedding layer; The restoration output layer, the time-continuous embedding layer and the preset normalization layer are spliced ​​to obtain the text restoration model; wherein the normalization layer is used to determine the selection probability of each of the symbol labels to classify the symbol labels.

2. The method according to claim 1, It is characterized in that The steps to obtain training data include: Obtaining the text to be trained and the voice dialogue scenario; Determining a plurality of voice roles for conducting a conversation in the voice conversation scenario; Extracting associated text associated with each of the voice roles, wherein the associated text includes at least two dialogue sentences of the voice role corresponding to the current sentence to be analyzed in the historical process; Extracting time information during voice conversation; The text to be trained, the voice dialogue scenario, the associated text and the time information are used as the training data.

3. The method according to claim 2, It is characterized in that The time information includes at least one of the following: The length of the speech pause between each sentence; The start and end time points of the speech pause between each sentence; The time the conversation started and the time the conversation ended.

4. The method according to any one of claims 1 to 3, It is characterized in that The text to be trained includes at least one of the following: a speech analysis text and a historical dialogue text, wherein the speech analysis text is a text obtained after speech recognition of the voices of each voice character at the scene of the speech dialogue, and the historical dialogue text is a dialogue text of each voice character recorded in the historical process.

5. The method according to claim 2, It is characterized in that The step of performing a first-stage training on the feature information of the text to be trained and the associated text to obtain a restored output layer includes: Performing initial pre-training on the text restoration model using format documents and voice transcript information corresponding to the voice dialogue scenario to obtain a restoration layer of the text restoration model; The restoration layer is used to extract feature information of the text to be trained and the associated text to obtain a restoration output layer.

6. The method according to claim 1, It is characterized in that The step of performing a second stage training on the time information to obtain a time-continuous embedding layer comprises: Discretizing the speech pause duration in the time information to obtain an initial embedding layer of the text restoration model; Splicing the initial embedding layer to the restored output layer, and connecting the normalization layer after the splicing is completed; After the connection is completed, the initial embedding layer is updated to obtain a text restoration model.

7. The method according to any one of claims 5 to 6, It is characterized in that The text restoration model is a BERT model.

8. The method according to claim 2, It is characterized in that The step of obtaining the text to be trained and the voice dialogue scenario includes: Recognizing the conversational speech content of the voice conversation scenario to obtain an initial recognition text; Preprocessing the initial recognition text; After the preprocessing is completed, identifying entity error data in the initial recognition text; The colloquial error data in the initial recognition text is identified to obtain the text to be trained.

9. The method according to claim 8, It is characterized in that The step of recognizing the conversation speech content of the voice conversation scenario to obtain an initial recognition text includes: According to the time sequence of the dialogue and each voice role, the voice content of the dialogue is recognized to obtain an initial recognition text.

10. The method according to claim 8, It is characterized in that The step of preprocessing the initial recognition text includes: Segmenting each sentence in the initial recognition text; Perform word segmentation processing on each of the sentences to obtain multiple words; Part-of-speech tagging is performed on each of the words.

11. The method according to claim 8, It is characterized in that When the voice dialogue scenario is a court trial scenario, the step of identifying entity error data in the initial recognition text includes: Converting the initial recognition text and the case entity list of the target case corresponding to the trial scene into a pinyin sequence; After the conversion is completed, the case entity name corresponding to the target case is searched in the initial recognition text to obtain the entity error data.

12. The method according to claim 8, It is characterized in that Identifying colloquial error data in the initial recognition text includes: Recognize at least one of the following colloquial words in the initial recognition text: modal particles, pause words, and repaired words; Setting a type label for each of the colloquial words; Determining error types and location data of the colloquial words in the initial recognition text; constructing a spoken error recognition model according to the type label of the spoken word, the error type and the position data; The spoken language error data is recognized by the spoken language error recognition model.

13. The method according to claim 1, It is characterized in that The data processing method is applied to at least one of the following scenarios: a conference scenario and a court trial scenario.

14. A data processing device, It is characterized in that include: An acquisition unit, configured to acquire training data, wherein the training data includes a text to be trained, associated texts corresponding to each voice character in the text to be trained, and time information during a conversation between the voice characters; A training unit, used for obtaining a text restoration model by training the training data; A restoration unit, used for restoring the text to be analyzed by using the text restoration model to obtain a target written text; The data processing device further comprises: a determination unit, which is used to determine, after acquiring the training data, a symbol label corresponding to the current sentence to be analyzed in the text to be trained, wherein the symbol label at least comprises: a punctuation label and a letter label; The training unit includes: a first training module, which is used to perform the first stage training on the feature information of the training text and the associated text to obtain a restored output layer, wherein each unit of the restored output layer represents a text feature information of each sentence to be analyzed in the training text; a second training module, which is used to perform the second stage training on the time information to obtain a time-continuous embedding layer; a first splicing module, which is used to splice the restored output layer, the time-continuous embedding layer and the preset normalization layer to obtain a text restoration model; wherein the normalization layer is used to determine the selection probability of each of the symbol labels to classify the symbol labels.

15. A storage medium, It is characterized in that The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 13.

16. A processor, It is characterized in that The processor is used to run a program, wherein the program executes the data processing method according to any one of claims 1 to 13 when running.

17. A terminal device, It is characterized in that include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Step 1, obtaining training data, wherein the training data includes a text to be trained, associated texts corresponding to each voice character in the text to be trained, and time information during the conversation of the voice characters; Step 2, obtaining a text restoration model by training the training data; Step 3, using the text restoration model to restore the text to be analyzed to obtain the target written text; After obtaining the training data, it also includes: Determine the symbol label corresponding to the current sentence to be analyzed in the text to be trained, wherein the symbol label at least includes: a punctuation label and a letter label; The step of obtaining a text restoration model by training the training data includes: Performing a first-stage training on the feature information of the text to be trained and the associated text to obtain a restored output layer, wherein each unit of the restored output layer represents a text feature information of each sentence to be analyzed in the text to be trained; Performing a second stage of training on the time information to obtain a time-continuous embedding layer; The restoration output layer, the time-continuous embedding layer and the preset normalization layer are spliced ​​to obtain the text restoration model; wherein the normalization layer is used to determine the selection probability of each of the symbol labels to classify the symbol labels.