Model training, text classification method and device, equipment and storage medium

Abstract

The present disclosure relates to a model training method and device, a text classification method and device, equipment and a storage medium. The present disclosure determines first information and second information through a sentence and context information of the sentence, so that the first information and the second information both include the sentence, and previous information or subsequent information in the context information appears in the first information or the second information with a preset probability. Due to the high flexibility of the difference between the first information and the second information, the contrast between the first prediction probability distribution and the second prediction probability distribution obtained after the to-be-trained model processes the first information and the second information respectively is also relatively flexible. This flexibility can make the trained machine learning model pay more attention to the sentence itself, alleviate the problem that the model is disturbed by some irrelevant information in the context, thereby improving the robustness. At the same time, this flexibility can also fully utilize the auxiliary information in the context information to improve the recognition accuracy of the preset key information in the sentence.

Description

Technical Field

[0001] This disclosure relates to the field of information technology, and in particular to a model training, text classification method, apparatus, device and storage medium. Background Technology

[0002] In daily work, Automatic Speech Recognition (ASR) technology can automatically recognize audio and video meeting content as text records, i.e., meeting minutes. Furthermore, the meeting minutes are categorized to identify key information such as topics, conclusions, questions, and actions, which can then be used to generate meeting summaries.

[0003] However, the existing technology is not accurate enough in classifying the text of this meeting record. Summary of the Invention

[0004] To address the aforementioned technical problems, or at least partially address them, this disclosure provides a model training method, text classification method, apparatus, device, and storage medium to improve the accuracy of recognizing preset key information in sentences.

[0005] In a first aspect, embodiments of this disclosure provide a model training method, including:

[0006] For each sentence in the sample text, obtain the context information of the sentence from the sample text;

[0007] Based on the sentence and the context information, first information and second information are determined, where both the first information and the second information include the sentence, and the preceding or following information in the context information appears in the first information or the second information with a preset probability.

[0008] The first information is input into the model to be trained, so that the model to be trained outputs a first prediction probability distribution of whether the sentence includes preset key information;

[0009] The second information is input into the model to be trained, so that the model to be trained outputs a second prediction probability distribution of whether the sentence includes preset key information;

[0010] The model to be trained is trained based on the first predicted probability distribution and the second predicted probability distribution to obtain the trained machine learning model.

[0011] Secondly, embodiments of this disclosure provide a text classification method, including:

[0012] Obtain the target text to be classified;

[0013] For each sentence in the target text, obtain the context information of the sentence from the target text;

[0014] The sentence and the context information are input into a machine learning model, which outputs a prediction result of whether the sentence includes preset key information. The machine learning model is trained using the method described in the first aspect.

[0015] Thirdly, embodiments of this disclosure provide a model training apparatus, comprising:

[0016] The acquisition module is used to acquire the context information of each sentence in the sample text.

[0017] The determining module is used to determine first information and second information based on the sentence and the context information, wherein both the first information and the second information include the sentence, and the preceding or following information in the context information appears in the first information or the second information with a preset probability;

[0018] The first input module is used to input the first information into the model to be trained, so that the model to be trained outputs a first prediction probability distribution of whether the sentence includes preset key information.

[0019] The second input module is used to input the second information into the model to be trained, so that the model to be trained outputs a second prediction probability distribution of whether the sentence includes preset key information.

[0020] The training module is used to train the model to be trained based on the first predicted probability distribution and the second predicted probability distribution to obtain the trained machine learning model.

[0021] Fourthly, embodiments of this disclosure provide a text classification device, including:

[0022] The first acquisition module is used to acquire the target text to be classified.

[0023] The second acquisition module is used to acquire the context information of each sentence in the target text.

[0024] The classification module is used to input the sentence and the context information into the machine learning model, so that the machine learning model outputs a prediction result of whether the sentence includes preset key information. The machine learning model is trained using the method described in the first aspect.

[0025] Fifthly, embodiments of this disclosure provide an electronic device, including:

[0026] Memory;

[0027] Processor; and

[0028] Computer programs;

[0029] The computer program is stored in the memory and configured to be executed by the processor to implement the method as described in the first or second aspect.

[0030] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method described in the first or second aspect.

[0031] The model training, text classification method, apparatus, device, and storage medium provided in this disclosure, for each sentence in the sample text, determine first information and second information through the sentence and its context information, such that both the first and second information include the sentence. The preceding or following information in the context information appears in the first or second information with a preset probability, improving the flexibility of the difference between the first and second information. Furthermore, the model to be trained processes the first information to obtain a first predicted probability distribution and processes the second information to obtain a second predicted probability distribution. Due to the high flexibility of the difference between the first and second information, the contrast between the first and second predicted probability distributions is also relatively flexible. This flexibility allows the trained machine learning model to focus more on the sentence itself, mitigating the problem of the model being interfered with by irrelevant information in the context, thereby improving the robustness of the machine learning model. Simultaneously, this flexibility also allows the trained machine learning model to fully utilize the auxiliary information in the context information to improve the recognition accuracy of preset key information in the sentence, thereby improving the accuracy of sample text classification. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0033] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0034] Figure 1 A flowchart of the model training method provided in this embodiment of the disclosure;

[0035] Figure 2 A schematic diagram illustrating an application scenario provided by an embodiment of this disclosure;

[0036] Figure 3 A schematic diagram illustrating model training provided in an embodiment of this disclosure;

[0037] Figure 4 A schematic diagram illustrating model training provided in an embodiment of this disclosure;

[0038] Figure 5 A schematic diagram illustrating model training provided for another embodiment of this disclosure;

[0039] Figure 6 A schematic diagram illustrating model training provided for another embodiment of this disclosure;

[0040] Figure 7 A schematic diagram illustrating model training provided for another embodiment of this disclosure;

[0041] Figure 8 A schematic diagram illustrating model training provided for another embodiment of this disclosure;

[0042] Figure 9 A schematic diagram illustrating model training provided in an embodiment of this disclosure;

[0043] Figure 10 A flowchart of a text classification method provided in another embodiment of this disclosure;

[0044] Figure 11 This is a schematic diagram of the structure of the model training device provided in the embodiments of this disclosure;

[0045] Figure 12 A schematic diagram of the structure of the text classification device provided in the embodiments of this disclosure;

[0046] Figure 13 A schematic diagram of the structure of an electronic device embodiment provided in this disclosure. Detailed Implementation

[0047] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0048] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0049] It should be noted that the sample texts and target texts involved in this application (including but not limited to text converted from audio in a meeting scenario, pre-stored text, etc.) are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0050] In addition, the model training and text classification methods provided in this application involve the following explanations of terms, detailed below:

[0051] Automatic Speech Recognition (ASR): Through various methods and technologies, computers can convert human speech into text. It is an interdisciplinary field that combines computer science, computational linguistics, and other disciplines.

[0052] Action Item Detection: Computers automatically determine whether text contains action items.

[0053] Text classification: Using computers to automatically classify and label text sets (or other entities or objects) according to a certain classification system or standard.

[0054] In daily work, meetings, such as audio and video conferences, are frequently required. To improve work efficiency, with the support of Automatic Speech Recognition (ASR) technology, the audio in the meeting can be automatically recognized into corresponding text, which can be recorded as meeting minutes. Furthermore, key information such as the meeting agenda, conclusions, questions, and action items can be extracted from these minutes, and this extracted key information can be organized to obtain meeting summaries. Action items, in terms of time, begin after the meeting; in terms of responsibility, they are to be executed by the relevant parties; and in terms of content, they are specific actions. Meeting action items often need to be compiled in the meeting minutes, recorded under items such as next steps, follow-up actions, and subsequent tasks, or compiled into the to-do list of the corresponding responsible person for execution, follow-up, and feedback after the meeting. For example, "I'll do some statistics when I get back tonight," or "We need to produce a report next." On the one hand, action items are an essential part of work meeting minutes. Work meetings often involve information sharing, problem-solving, planning, and task assignment, with action items implicitly included, such as implementing suggestions and solutions, and executing plans and tasks. Furthermore, action items are key to improving post-meeting execution efficiency. Once action items are identified, we can streamline the entire process of creating, scheduling, notifying, synchronizing, and reviewing them. By focusing on subsequent actions, meeting minutes become more than just written records; they transform into a comprehensive post-meeting management platform, significantly boosting post-meeting execution efficiency.

[0055] However, before extracting key information such as the meeting agenda, conclusions, questions, and action items from the meeting minutes, it is necessary to first identify whether each sentence in the meeting minutes includes key information such as agenda, conclusions, questions, or action items. Failure to accurately identify whether each sentence includes key information may affect the quality and accuracy of subsequent meeting minutes. Therefore, it is urgent to improve the accuracy of text classification of these meeting minutes. To address this problem, this disclosure provides a model training and text classification method, which will be described below with reference to specific embodiments.

[0056] Figure 1 This is a flowchart illustrating a model training method provided in an embodiment of this disclosure. The method can be executed by a model training device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server or terminal, where the terminal specifically includes a mobile phone, computer, or tablet computer. Furthermore, the method described in this embodiment is applicable to... Figure 2The application scenario shown includes a terminal 21 and a server 22. The server 22 can train the model to be trained, resulting in a trained machine learning model. This trained model can classify text, for example, identifying whether each sentence in the text contains key information such as issues, conclusions, questions, or action items. The training method for this machine learning model is described first, followed by the method of applying the trained model to implement text classification. Figure 1 As shown, the specific steps of this training method are as follows:

[0057] S101. For each sentence in the sample text, obtain the context information of the sentence from the sample text.

[0058] For example, terminal 21 can be an audio acquisition device, such as a microphone, in a conference setting. Specifically, terminal 21 sends the acquired conference audio to server 22, which uses ASR technology to recognize the conference audio as text, which can be used as sample text. Alternatively, terminal 21 can use ASR technology to recognize the acquired conference audio as text and send this text as sample text to server 22. Or, the sample text can be pre-stored text in server 22, and this text is not limited to text obtained after ASR of the conference audio; for example, the text could be text from the internet. Furthermore, the number of sample texts is not limited to one; for example, it can be multiple. Server 22 can perform one iteration of training on the model to be trained based on each sample text until the parameters of the model converge to a preset level, or the number of iterations is greater than or equal to the preset number. This embodiment uses one iteration of training as an example to illustrate how server 22 can perform one iteration of training on the model to be trained based on a specific sample text.

[0059] Specifically, the sample text includes multiple sentences, each of which can end with any punctuation mark or with a specific punctuation mark (e.g., period, question mark, exclamation mark, etc.). For each sentence in the sample text, the server 22 needs to use the model to be trained to determine whether the sentence includes preset key information. This preset key information can be key information such as the meeting's agenda, conclusion, questions, or action items, as described above. Since action items have the aforementioned importance, this embodiment can take action items as an example to determine whether each sentence includes an action item. That is, in this embodiment, the identification of action items is treated as a binary classification task, i.e., the input to the model to be trained is a sentence in the sample text, and the output of the model to be trained is the judgment result of whether the sentence includes an action item. In addition, in some other embodiments, the identification of action items can also be treated as a multi-classification task. In a multi-classification task, the categories include the description of the action item, the person in charge, the time limit, and whether the action is confirmed, etc.

[0060] Specifically, for each sentence in the sample text, server 22 can obtain the context information of that sentence from the sample text. Specifically, the context information includes the preceding and following information of the sentence. The preceding information can be several sentences preceding the sentence, and the following information can be several sentences following the sentence.

[0061] S102. Based on the sentence and the context information, determine the first information and the second information, where both the first information and the second information include the sentence, and the preceding or following information in the context information appears in the first information or the second information with a preset probability.

[0062] Optionally, the preset probability of the preceding information appearing in the first information and the preset probability of the preceding information not appearing in the first information are the same or different; the preset probability of the preceding information appearing in the second information and the preset probability of the preceding information not appearing in the second information are the same or different; the preset probability of the following information appearing in the first information and the preset probability of the following information not appearing in the first information are the same or different; the preset probability of the following information appearing in the second information and the preset probability of the following information not appearing in the second information are the same or different.

[0063] For example, server 22 can determine first information and second information based on the sentence, its preceding information, and its following information. The first and second information are inputs to the model to be trained. Specifically, both the first and second information include the sentence; that is, the sentence appears not only in the first information but also in the second information. However, the preceding or following information of the sentence may appear in the first information and may appear in the second information. In other words, the preceding or following information of the sentence may appear in the first or second information with a preset probability, and it may also not appear in the first or second information with a preset probability. The preset probabilities of appearance and non-appearance can be the same or different. For example, if the preset probabilities of appearance and non-appearance are the same... Figure 3 As shown, the sentence in the first piece of information corresponds to a solid-line box, indicating that the sentence will appear in the first piece of information. The preceding and following information in the first piece of information corresponds to dashed-line boxes, indicating that the preceding information may or may not appear in the first piece of information. Similarly, the following information may or may not appear in the first piece of information. For example, the preceding information has a 50% probability of appearing in the first piece of information and a 50% probability of not appearing in it. Similarly, the following information has a 50% probability of appearing in the first piece of information and a 50% probability of not appearing in it. Furthermore, the preceding and following information are independent of each other; that is, the appearance or absence of the preceding information will not affect the appearance or absence of the following information.

[0064] Similarly, the sentence in the second piece of information is the same as the sentence in the first piece of information, and the sentence in the second piece of information also corresponds to a solid-line box. The meaning of this solid-line box is the same as that of the solid-line box described above. The preceding information in the second piece of information is the same as the preceding information in the first piece of information, and the following information in the second piece of information is the same as that of the following information in the first piece of information. The preceding and following information in the second piece of information respectively correspond to dashed-line boxes, and the meaning of this dashed-line box is the same as that of the dashed-line box described above, so it will not be repeated here. It can be understood that the appearance or absence of the preceding information in the first piece of information will not affect the appearance or absence of the preceding information in the second piece of information, nor will it affect the appearance or absence of the following information in the second piece of information. Similarly, the appearance or absence of the following information in the first piece of information will not affect the appearance or absence of the preceding information in the second piece of information, nor will it affect the appearance or absence of the following information in the second piece of information. That is, the preceding information, the following information in the first piece of information, the preceding information in the second piece of information, and the following information in the second piece of information are independent of each other.

[0065] S103. Input the first information into the model to be trained, so that the model to be trained outputs a first prediction probability distribution of whether the sentence includes preset key information.

[0066] like Figure 3 As shown, the model to be trained includes an encoder and a classifier. After the first information is input into the encoder, the encoder's output can be a representation vector. This representation vector can then be used as input to the classifier, causing the classifier to output a first predicted probability distribution indicating whether the sentence includes preset key information, such as an action item. This first predicted probability distribution is denoted as P1(y|x). Specifically, P1(y|x) can be considered a vector containing a first probability that the sentence includes an action item and a second probability that the sentence does not include an action item. Further, if the first probability is greater than the second probability, it indicates that the sentence includes an action item, and the sentence's label is set to 1. If the first probability is less than the second probability, it indicates that the sentence does not include an action item, and the sentence's label is set to 0.

[0067] For sentence-level action item recognition, contextual information plays a crucial role. In meeting scenarios, each sentence typically contains a lot of omitted or referential information, which can be supplemented by contextual information. Furthermore, adjacent sentences often belong to the same category because actions frequently appear in discussions related to task assignment.

[0068] S104. Input the second information into the model to be trained, so that the model to be trained outputs a second prediction probability distribution of whether the sentence includes preset key information.

[0069] like Figure 3 As shown, after the second information is input into the encoder, the encoder's output can be a representation vector. This representation vector can then be used as input to the classifier, causing the classifier to output a second predicted probability distribution indicating whether the sentence includes preset key information, such as an action item. This second predicted probability distribution is denoted as P2(y|x). Specifically, P2(y|x) can be considered a vector containing a third probability that the sentence includes an action item and a fourth probability that the sentence does not include an action item. Further, if the third probability is greater than the fourth probability, it indicates that the sentence includes an action item, and the sentence's label is set to 1. If the third probability is less than the fourth probability, it indicates that the sentence does not include an action item, and the sentence's label is set to 0.

[0070] S105. The model to be trained is trained according to the first predicted probability distribution and the second predicted probability distribution to obtain the trained machine learning model.

[0071] For example, the first and second prediction probability distributions are predictions for the same sentence. Typically, auxiliary information in the context does not change the sentence's label. That is, regardless of whether the first or second information includes preceding or following context, the label obtained from the first and second prediction probability distributions is theoretically consistent. Therefore, for the same sentence, we want the first and second prediction probability distributions to be as close as possible to enable training the model and obtaining the trained machine learning model.

[0072] Optionally, the first predicted probability distribution includes a first probability and a second probability, and the second predicted probability distribution includes a third probability and a fourth probability; the first probability and the third probability respectively represent the probability that the sentence includes the preset key information, and the second probability and the fourth probability respectively represent the probability that the sentence does not include the preset key information.

[0073] Optionally, training the model to be trained based on the first predicted probability distribution and the second predicted probability distribution to obtain a trained machine learning model includes: calculating the distance between the predicted probability distributions of the first and second predicted probability distributions; and training the model to be trained based on the distance between the predicted probability distributions and the loss function for text classification to obtain a trained machine learning model.

[0074] Since the first information and the second information may contain preceding or following information, they may be the same or different. When the first and second information are different, P1(|) and P2(|) may be different, meaning the first probability and the third probability are different, and the second probability and the fourth probability are different. Furthermore, during the training of the model, even when the first and second information are the same, the different selection strategies employed by the model for different internal characteristics such as nodes, layers, or parameters can lead to different P1(|) and P2(|). Therefore, during the training of the model, the distance between the predicted probability distributions of P1(|) and P2(|) can be calculated using KL divergence, denoted as D. KL (1||2). The larger this distance, the less close P1(|) and P2(|) are; the smaller the distance, the closer P1(|) and P2(|) are. Therefore, for the same sentence, the training model can be obtained by shortening this distance.

[0075] In addition, this embodiment will be as follows Figure 3 The method shown, which comprehensively utilizes contextual information, is denoted as the Context-Drop (Dynamic) method. This means that for the same sentence, it can dynamically select either preceding or following contextual information. When neither the first nor the second piece of information contains preceding or following contextual information, this form is equivalent to the single-sentence R-Drop method, for example... Figure 4 The R-Drop (Sentence) example illustrates this. When both the first and second messages contain preceding and following information, this format is equivalent to the context-based R-Drop method. Figure 5 The R-Drop(Context) shown is equivalent to the following format when the first message contains no preceding or following context, and when the second message contains preceding and following context. Figure 6 The Context-Drop(Fixed) method is shown below. In other words, in the Context-Drop(Fixed) method, for the same sentence, we want the first predicted probability distribution obtained after inputting the sentence into the model to be as close as possible to the second predicted probability distribution obtained after inputting the sentence with context information into the model to be trained.

[0076] Understandably, since the first piece of information may or may not contain preceding or following information, similarly, the second piece of information may or may not contain preceding or following information. Therefore, for the same sentence, the first and second pieces of information can be combined in multiple different ways. These combinations include not only... Figure 4 , Figure 5 , Figure 6 The combinations shown include those not illustrated in the diagram. However, for this sentence, one or more of these different combinations can be used to train the model. Furthermore, this embodiment is not limited to... Figure 3 The order of the first and second pieces of information shown is optional. For example, the order could be sentence, preceding information, and following information, or it could be preceding information, sentence, and following information, or any other order. However, usually, the order of the first and second pieces of information is consistent.

[0077] This embodiment of the disclosure determines first information and second information for each sentence in the sample text using the sentence and its context information, such that both the first and second information include the sentence. The preceding or following information in the context information appears in the first or second information with a preset probability, increasing the flexibility of the difference between the first and second information. Furthermore, the model to be trained processes the first information to obtain a first predicted probability distribution and processes the second information to obtain a second predicted probability distribution. Because the difference between the first and second information is highly flexible, the contrast between the first and second predicted probability distributions is also relatively flexible. This flexibility allows the trained machine learning model to focus more on the sentence itself, mitigating the problem of the model being interfered with by irrelevant information in the context, thereby improving the robustness of the machine learning model. Simultaneously, this flexibility also allows the trained machine learning model to fully utilize the auxiliary information in the context to improve the recognition accuracy of preset key information in the sentence, thereby improving the accuracy of sample text classification.

[0078] Based on the above embodiments, the preceding context information includes local preceding context information and / or global preceding context information, wherein the distance between the local preceding context information and the sentence is less than or equal to a first preset value, and the semantic similarity between the global preceding context information and the sentence is greater than or equal to a second preset value; the following context information includes local following context information and / or global following context information, wherein the distance between the local following context information and the sentence is less than or equal to the first preset value, and the semantic similarity between the global following context information and the sentence is greater than or equal to the second preset value.

[0079] For example, the preceding information of a sentence can be several sentences preceding it, and the following information can be several sentences following it. The preceding information includes local preceding information and / or global preceding information. The distance between the local preceding information and the sentence is less than or equal to a first preset value; for example, the local preceding information consists of several sentences preceding and adjacent to the sentence. The semantic similarity between the global preceding information and the sentence is greater than or equal to a second preset value; for example, the global preceding information may not be adjacent to the sentence but is relatively far away, yet the semantic similarity between the global preceding information and the sentence is high. For example, in a meeting scenario, sometimes the lack of topic organization leads to tasks related to a previously mentioned topic not being assigned immediately, but rather being assigned near the end of the meeting. In this case, although the distance between the topic and the task may be far, the topic can serve as the global preceding information for the task due to their semantic connection or similarity. In other words, since the local context information is adjacent to the sentence, it is generally related to the sentence. However, in addition to the local context information, the global context information is also related to the sentence. Therefore, this embodiment can expand the context information related to the sentence.

[0080] Additionally, it is understood that in some embodiments, for the same sentence, the local context information and the global context information may be the same or partially overlapping.

[0081] Similarly, regarding the context information of a sentence, this context information includes local context information and / or global context information. The local context information can refer to the local preceding context information as described above, and the global context information can refer to the global preceding context information as described above, which will not be repeated here. Similarly, this embodiment can expand the context information related to the sentence. By expanding the preceding and following context information related to the sentence, the accuracy of identifying whether the sentence includes preset key information can be further improved.

[0082] Furthermore, when searching for global context or global background information of a sentence, every other sentence in the sample text can be searched to determine the semantic similarity between each other sentence and the given sentence. The sentence with the highest similarity is then used as the global context or global background information of the given sentence. Here, cosine similarity of n-grams can be used to calculate the similarity between sentences, or other methods can be used.

[0083] For example, global context information and global context information can be referred to as global context information, while local context information and local context information can be referred to as local context information. Considering global and local context information, the Context-Drop(Dynamic) method is specifically as follows: Figure 7 As shown, local or global context information may appear in either the first or second piece of information. It's understandable that although local context information is enclosed in a dashed box, the local preceding and following context information are independent of each other; that is, the presence or absence of local preceding context information does not affect the presence or absence of local following context information. Similarly, although global context information is enclosed in a dashed box, global preceding and following context information are independent of each other; that is, the presence or absence of global preceding context information does not affect the presence or absence of global following context information. Likewise, considering both global and local context information, the R-Drop(Context) method is specifically as follows: Figure 8 As shown, the Context-Drop(Fixed) method is specifically as follows: Figure 9 As shown.

[0084] Furthermore, this embodiment is not limited to... Figure 7 The first and second pieces of information shown are arranged in the order of sentence, local context information, and global context information. When considering local and global context information, the possible orders become even more numerous. For example, the order could be sentence, local preceding information, local following information, global preceding information, global following information, or other orders, which will not be elaborated here. These orders are all optional. However, in general, the order of the first and second pieces of information is consistent.

[0085] Optionally, the model to be trained includes an encoder and a classifier, wherein the initial parameters of the encoder are the initial parameters of the encoder in the first pre-trained model, and the initial parameters of the classifier are the initial parameters of the classifier in the second pre-trained model.

[0086] like Figures 3-9As shown, the model to be trained includes an encoder (Transformer Encoder) and a classifier. During training, the encoder's initial parameters are the same as those of the encoder in the first pre-trained model, and the classifier's initial parameters are the same as those of the classifier in the second pre-trained model. In other words, training begins when the encoder's initial parameters are the same as those of the encoder in the first pre-trained model, and the classifier's initial parameters are the same as those of the classifier in the second pre-trained model. Compared to initializing the classifier and encoder using the same pre-trained model, this embodiment initializes the classifier and encoder using different pre-trained models. This allows the trained machine learning model to achieve better performance and improve its ability to recognize preset key information without changing the number of parameters or computational complexity of the model to be trained.

[0087] Understandably, the trained machine learning model can be stored on server 22, or server 22 can deploy the trained machine learning model to terminal 21.

[0088] Figure 10 A flowchart of a text classification method provided in another embodiment of this disclosure. The specific steps of the method are as follows:

[0089] S1001. Obtain the target text to be classified.

[0090] For example, the target text to be classified could be a meeting transcript obtained by recognizing meeting audio using ASR technology.

[0091] S1002. For each sentence in the target text, obtain the context information of the sentence from the target text.

[0092] For example, when the server 22 obtains the target text, it can extract the context information of each sentence in the target text. The context information may include the local context information and / or global context information as described above.

[0093] S1003. Input the sentence and the context information into the machine learning model, so that the machine learning model outputs a prediction result of whether the sentence includes preset key information. The machine learning model is trained using the model training method described above.

[0094] For example, server 22 can input the sentence and its context information into a machine learning model, which can be trained using the model training method described above. Furthermore, the machine learning model can output a prediction of whether the sentence includes preset key information. This prediction can include the probability that the sentence includes the preset key information and the probability that the sentence does not include the preset key information. Based on these two probabilities, it is determined whether the sentence includes the preset key information. This classifies each sentence in the target text, that is, it classifies the sentences in the target text according to the preset key information, thereby determining which sentences in the target text include the preset key information and which do not. Additionally, the sentence classification process can be viewed as a process of identifying preset key information.

[0095] Understandably, if server 22 deploys the trained machine learning model to terminal 21, then when terminal 21 collects the meeting audio and converts it into target text, it can directly determine whether each sentence in the target text contains preset key information using the trained machine learning model. In the meeting minutes, the trained machine learning model can automatically identify whether each sentence contains preset key information, such as action items. If a sentence contains preset key information, it can be highlighted. Furthermore, since different sentences may contain different preset key information, the color of the highlighted sentence can correspond to the type of preset key information it contains. For example, if a sentence contains action items, it is highlighted in yellow. If a sentence contains topics, it is highlighted in red. After each sentence in the target text has been categorized, meeting minutes can be compiled based on all highlighted sentences or sentences highlighted in different colors, thereby improving post-meeting work efficiency.

[0096] This embodiment determines first and second information for each sentence in the sample text based on the sentence and its context information, ensuring that both the first and second information include the sentence. The preceding or following information in the context information appears in the first or second information with a preset probability, increasing the flexibility of the difference between the first and second information. Furthermore, the model to be trained processes the first information to obtain a first predicted probability distribution and processes the second information to obtain a second predicted probability distribution. Due to the high flexibility of the difference between the first and second information, the contrast between the first and second predicted probability distributions is also relatively flexible. This flexibility allows the trained machine learning model to focus more on the sentence itself, mitigating the problem of interference from irrelevant information in the context, thereby improving the robustness of the machine learning model. Simultaneously, this flexibility also allows the trained machine learning model to fully utilize auxiliary information in the context to improve the recognition accuracy of preset key information in the sentence. Therefore, when classifying target text using the trained machine learning model, the accuracy of recognizing whether each sentence in the target text includes preset key information can be effectively improved.

[0097] Figure 11 This is a schematic diagram of the structure of a model training apparatus provided in an embodiment of this disclosure. The model training apparatus provided in this embodiment of the disclosure can execute the processing flow provided in the model training method embodiment, such as... Figure 11 As shown, the model training device 110 includes:

[0098] The acquisition module 111 is used to acquire the context information of each sentence in the sample text from the sample text;

[0099] The determining module 112 is used to determine first information and second information based on the sentence and the context information, wherein both the first information and the second information include the sentence, and the preceding or following information in the context information appears in the first information or the second information with a preset probability;

[0100] The first input module 113 is used to input the first information into the model to be trained, so that the model to be trained outputs a first prediction probability distribution of whether the sentence includes preset key information.

[0101] The second input module 114 is used to input the second information into the model to be trained, so that the model to be trained outputs a second prediction probability distribution of whether the sentence includes preset key information.

[0102] The training module 115 is used to train the model to be trained based on the first predicted probability distribution and the second predicted probability distribution to obtain the trained machine learning model.

[0103] Optionally, the preceding information includes local preceding information and / or global preceding information, wherein the distance between the local preceding information and the sentence is less than or equal to a first preset value, and the semantic similarity between the global preceding information and the sentence is greater than or equal to a second preset value;

[0104] The context information includes local context information and / or global context information. The distance between the local context information and the sentence is less than or equal to a first preset value, and the semantic similarity between the global context information and the sentence is greater than or equal to a second preset value.

[0105] Optionally, the model to be trained includes an encoder and a classifier, wherein the initial parameters of the encoder are the initial parameters of the encoder in the first pre-trained model, and the initial parameters of the classifier are the initial parameters of the classifier in the second pre-trained model.

[0106] Optionally, the first predicted probability distribution includes a first probability and a second probability, and the second predicted probability distribution includes a third probability and a fourth probability;

[0107] The first probability and the third probability represent the probability that the sentence includes the preset key information, and the second probability and the fourth probability represent the probability that the sentence does not include the preset key information.

[0108] Optionally, when the training module 115 trains the model to be trained based on the first prediction probability distribution and the second prediction probability distribution to obtain the trained machine learning model, it is specifically used for:

[0109] Calculate the distance between the first predicted probability distribution and the second predicted probability distribution;

[0110] The model to be trained is trained based on the distance of the predicted probability distribution and the loss function of text classification to obtain the trained machine learning model.

[0111] Optionally, the preset probability that the preceding information appears in the first information and the preset probability that the preceding information does not appear in the first information are the same or different;

[0112] The preset probability that the preceding information appears in the second information and the preset probability that the preceding information does not appear in the second information are the same or different;

[0113] The preset probability that the following information appears in the first information and the preset probability that the following information does not appear in the first information are the same or different;

[0114] The preset probability that the following information appears in the second information and the preset probability that the following information does not appear in the second information are the same or different.

[0115] Figure 11 The model training apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0116] Figure 12 This is a schematic diagram of the structure of a text classification device provided in an embodiment of the present disclosure. The text classification device provided in this embodiment can execute the processing flow provided in the text classification method embodiment, such as... Figure 12 As shown, the text classification device 120 includes:

[0117] The first acquisition module 121 is used to acquire the target text to be classified.

[0118] The second acquisition module 122 is used to acquire the context information of each sentence in the target text from the target text.

[0119] The classification module 123 is used to input the sentence and the context information into the machine learning model, so that the machine learning model outputs a prediction result of whether the sentence includes preset key information. The machine learning model is trained using the model training method described above.

[0120] Figure 12 The text classification device shown in the embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0121] The above describes the internal functions and structure of a model training device or text classification device, which can be implemented as an electronic device. Figure 13 A schematic diagram illustrating the structure of an electronic device embodiment provided in this disclosure. (See attached diagram.) Figure 13 As shown, the electronic device includes a memory 131 and a processor 132.

[0122] Memory 131 is used to store programs. In addition to the programs described above, memory 131 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0123] The memory 131 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0124] The processor 132 is coupled to the memory 131 and executes the program stored in the memory 131 for:

[0125] For each sentence in the sample text, obtain the context information of the sentence from the sample text;

[0126] Based on the sentence and the context information, first information and second information are determined, where both the first information and the second information include the sentence, and the preceding or following information in the context information appears in the first information or the second information with a preset probability.

[0127] The first information is input into the model to be trained, so that the model to be trained outputs a first prediction probability distribution of whether the sentence includes preset key information;

[0128] The second information is input into the model to be trained, so that the model to be trained outputs a second prediction probability distribution of whether the sentence includes preset key information;

[0129] The model to be trained is trained based on the first predicted probability distribution and the second predicted probability distribution to obtain the trained machine learning model.

[0130] Alternatively, processor 132 can also be used for:

[0131] Obtain the target text to be classified;

[0132] For each sentence in the target text, obtain the context information of the sentence from the target text;

[0133] The sentence and the context information are input into a machine learning model, which then outputs a prediction of whether the sentence includes preset key information. The machine learning model is trained using the model training method described above.

[0134] Furthermore, such as Figure 13 As shown, the electronic device may also include other components such as a communication component 133, a power supply component 134, an audio component 135, and a display 136. Figure 13 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 13 The components shown.

[0135] Communication component 133 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 133 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 133 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0136] Power supply component 134 provides power to various components of an electronic device. Power supply component 134 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0137] Audio component 135 is configured to output and / or input audio signals. For example, audio component 135 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 131 or transmitted via communication component 133. In some embodiments, audio component 135 also includes a speaker for outputting audio signals.

[0138] Display 136 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0139] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the model training method or text classification method described in the above embodiments.

[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0141] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model training method, wherein, The method comprises: For each sentence in the sample text, obtaining context information of the sentence from the sample text; According to the sentence and the context information, determine the first information and the second information, the first information and the second information both include the sentence, the context information, the context information or the context information in the first information or the second information appears in the first information or the second information with a preset probability; The first information is input into the to-be-trained model, so that the to-be-trained model outputs the first prediction probability distribution of whether the sentence includes the preset key information; The second information is input into the to-be-trained model, so that the to-be-trained model outputs the second prediction probability distribution of whether the sentence includes the preset key information; According to the first prediction probability distribution and the second prediction probability distribution, the to-be-trained model is trained to obtain a trained machine learning model; Wherein, according to the first prediction probability distribution and the second prediction probability distribution, the to-be-trained model is trained to obtain a trained machine learning model, comprising: calculating the distance of the prediction probability distribution between the first prediction probability distribution and the second prediction probability distribution; According to the distance of the prediction probability distribution and the loss function of text classification, the to-be-trained model is trained to obtain the trained machine learning model.

2. The method of claim 1, wherein, The context information includes local context information and / or global context information, the distance between the local context information and the sentence is less than or equal to a first preset value, and the semantic similarity between the global context information and the sentence is greater than or equal to a second preset value. The context information includes local context information and / or global context information, the distance between the local context information and the sentence is less than or equal to a first preset value, and the semantic similarity between the global context information and the sentence is greater than or equal to a second preset value.

3. The method of claim 1, wherein, The to-be-trained model includes an encoder and a classifier, the initial parameters of the encoder are the initial parameters of the encoder in the first pre-trained model, and the initial parameters of the classifier are the initial parameters of the classifier in the second pre-trained model.

4. The method of claim 1, wherein, The first prediction probability distribution includes a first probability and a second probability, and the second prediction probability distribution includes a third probability and a fourth probability; The first probability and the third probability respectively represent the probability that the sentence includes the preset key information, and the second probability and the fourth probability respectively represent the probability that the sentence does not include the preset key information.

5. The method of claim 1, wherein, The preset probability of the context information appearing in the first information and the preset probability of the context information not appearing in the first information are the same or different; The preset probability of the context information appearing in the second information and the preset probability of the context information not appearing in the second information are the same or different; The preset probability of the context information appearing in the first information and the preset probability of the context information not appearing in the first information are the same or different; The preset probability of the context information appearing in the second information and the preset probability of the context information not appearing in the second information are the same or different.

6. A method of text classification, wherein, The method comprises: obtaining a target text to be classified; for each sentence in the target text, obtaining context information of the sentence from the target text; inputting the sentence and the context information into a machine learning model, so that the machine learning model outputs a prediction result of whether the sentence includes preset key information, the machine learning model being trained by the method in any one of claims 1-5.

7. A model training apparatus, wherein, comprising: an obtaining module, configured to, for each sentence in sample text, obtain context information of the sentence from the sample text; a determining module, configured to determine first information and second information according to the sentence and the context information, the first information and the second information both including the sentence, and context information in the context information appearing in the first information or the second information with a preset probability a first input module, configured to input the first information into a to-be-trained model, so that the to-be-trained model outputs a first prediction probability distribution of whether the sentence includes preset key information; a second input module, configured to input the second information into the to-be-trained model, so that the to-be-trained model outputs a second prediction probability distribution of whether the sentence includes preset key information; a training module, configured to train the to-be-trained model according to the first prediction probability distribution and the second prediction probability distribution, to obtain a trained machine learning model; wherein the training module is configured to perform the following steps to train the to-be-trained model according to the first prediction probability distribution and the second prediction probability distribution, to obtain the trained machine learning model: calculating a distance of prediction probability distributions between the first prediction probability distribution and the second prediction probability distribution; training the to-be-trained model according to the distance of prediction probability distributions and a loss function of text classification, to obtain the trained machine learning model.

8. A text classification apparatus, wherein, comprising: a first obtaining module, configured to obtain a target text to be classified; a second obtaining module, configured to, for each sentence in the target text, obtain context information of the sentence from the target text; a classification module, configured to input the sentence and the context information into a machine learning model, so that the machine learning model outputs a prediction result of whether the sentence includes preset key information, the machine learning model being trained by the method in any one of claims 1-5.

9. An electronic device, comprising: comprising: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method in any one of claims 1-6.

10. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method in any one of claims 1-6.

Images