Information extraction method and device, electronic equipment and computer program product
By dividing text types according to text length and using corresponding language models for information extraction, the problem of traditional methods reducing prediction accuracy in long text processing is solved, and a more efficient and accurate information extraction effect is achieved.
Patent Information
- Application Number
- CN202411640346.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-03-11
AI Technical Summary
When traditional news timeline generation methods process long texts, the prediction accuracy is reduced and the information extraction efficiency is low, which cannot meet the needs of improving information extraction efficiency and accuracy.
By getting the text length of the target text, it is divided into the first text type (short text) and the second text type (long text). For the first text type, the target time nodes and events are extracted using the first language model; for the second text type, the text is divided into multiple text blocks, and the target time nodes and events in each text block are extracted using the second language model.
The accuracy and efficiency of information extraction are improved, especially when processing long texts, the accuracy and efficiency of event extraction are significantly improved through text segmentation and language model distillation technology.
Smart Images
Figure CN119597924A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of natural language processing, and in particular to an information extraction method, an information extraction device, an electronic device, and a computer program product. Background Art
[0002] In recent years, with the development of the Internet and the increase in people's channels for obtaining information, traditional news reporting methods can no longer meet people's needs, so news timeline generation technology has become increasingly important.
[0003] When using traditional statistical models, such as HMM (Hidden Markov Model) to generate timelines, the prediction accuracy gradually decreases as the length of the input text increases. In addition, traditional statistical models take a long time to extract event timelines from long texts.
[0004] In view of this, there is an urgent need in this field for an information extraction method that can improve the efficiency and accuracy of information extraction during timeline generation.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] The purpose of the present disclosure is to provide an information extraction method, an information extraction device, an electronic device and a computer program product, thereby at least to a certain extent being able to improve the efficiency and accuracy of information extraction during timeline generation.
[0007] According to a first aspect of the present disclosure, there is provided an information extraction method, comprising:
[0008] Acquire the text length of the target text, and classify the target text into a first text type or a second text type according to the text length;
[0009] If the target text is of the first text type, extracting a target time node in the target text and a target event corresponding to the target time node through a first language model;
[0010] If the target text is of the second text type, the target text is segmented into a plurality of text blocks, and a target time node in each of the text blocks and a target event corresponding to the target time node are extracted through a second language model.
[0011] In an exemplary embodiment of the present disclosure, the method further includes:
[0012] According to the target time node in the target text and the target event corresponding to the target time node, the timeline information corresponding to the target text is generated.
[0013] In an exemplary embodiment of the present disclosure, dividing the target text into a plurality of text blocks includes:
[0014] The target text is segmented into a plurality of text segments according to a preset text segmentation length;
[0015] Splicing the text segment with other text segments before the text segment until the length of the spliced text reaches a preset text length threshold;
[0016] The text block is obtained according to the spliced text segments.
[0017] In an exemplary embodiment of the present disclosure, the method further includes:
[0018] Acquire training corpus, segment the training corpus into training text blocks, and obtain a first training data set based on the training text blocks;
[0019] Inputting a training text block in the first training data set into a third language model to obtain a training label corresponding to the training text block;
[0020] A second training data set is obtained according to the training text blocks and the corresponding training labels, and a preset language model is distilled using the second training data set to obtain the second language model.
[0021] In an exemplary embodiment of the present disclosure, the step of inputting the training text block in the first training data set into the third language model to obtain a training label corresponding to the training text block includes:
[0022] Inputting the training text block and the preset prompt words in the first training data set into the third language model to obtain the target time node in the training text block and the target event corresponding to the target time node;
[0023] The target time node in the training text block and the target event corresponding to the target time node are used as training labels corresponding to the training text block.
[0024] In an exemplary embodiment of the present disclosure, performing model distillation on a preset language model through the second training data set to obtain the second language model includes:
[0025] Inputting the training text blocks in the second training data set into the preset language model and the second language model respectively;
[0026] Obtaining a first model loss according to an output result of the preset language model and an output result of the second language model;
[0027] Obtaining a second model loss according to an output result of the second language model and training labels in the second training data set;
[0028] An overall loss is obtained according to the first model loss and the second model loss, and model parameters in the second language model are updated according to the overall loss to train the second language model.
[0029] In an exemplary embodiment of the present disclosure, segmenting the training corpus to obtain training text blocks includes:
[0030] The training corpus is divided into a plurality of training text segments according to a preset text segmentation length;
[0031] Splicing the training text segment with other training text segments before the training text segment until the length of the spliced text reaches a preset text length threshold;
[0032] The training text block is obtained according to the spliced training text segments.
[0033] According to a second aspect of the present disclosure, there is provided an information extraction device, comprising:
[0034] A text type determination module, used for obtaining the text length of the target text, and classifying the target text into a first text type or a second text type according to the text length;
[0035] A first text extraction module, configured to extract a target time node in the target text and a target event corresponding to the target time node through a first language model if the target text is of the first text type;
[0036] The second text extraction module is used to divide the target text into multiple text blocks if the target text is of the second text type, and extract the target time node in each of the text blocks and the target event corresponding to the target time node through the second language model.
[0037] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned information extraction methods by executing the executable instructions.
[0038] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned information extraction methods.
[0039] The exemplary embodiments of the present disclosure may have the following beneficial effects:
[0040] In the information extraction method of the example implementation mode of the present disclosure, the text length of the target text is obtained, and the target text is divided into the first text type or the second text type according to the text length. For the first text type, the target time node in the target text and the target event corresponding to the target time node are extracted through the first language model; for the second text type, the target text is segmented into multiple text blocks, and the target time node in each text block and the target event corresponding to the target time node are extracted through the second language model. The information extraction method in the example implementation mode of the present disclosure, on the one hand, uses different methods and language models for event extraction according to the different lengths of the input text, which can improve the accuracy of event extraction; on the other hand, by performing text segmentation on the second text type with a longer text length before performing event extraction, the efficiency and accuracy of event extraction can be further improved.
[0041] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0043] Figure 1 A schematic diagram showing a flow chart of an information extraction method according to an exemplary embodiment of the present disclosure;
[0044] Figure 2 A schematic diagram showing a process of dividing a target text into a plurality of text blocks according to an exemplary embodiment of the present disclosure is shown;
[0045] Figure 3 A schematic diagram of text segment splicing according to a specific embodiment of the present disclosure is schematically shown;
[0046] Figure 4 A schematic diagram showing a flow chart of a method for training a second language model according to an exemplary embodiment of the present disclosure;
[0047] Figure 5A schematic diagram of a process of performing model distillation on a preset language model according to an exemplary embodiment of the present disclosure is shown;
[0048] Figure 6 A schematic diagram showing a flow chart of a method for training a second language model according to a specific embodiment of the present disclosure is shown;
[0049] Figure 7 A schematic diagram showing a flow chart of a method for generating a news timeline according to a specific implementation of the present disclosure;
[0050] Figure 8 A block diagram showing an information extraction apparatus according to an exemplary embodiment of the present disclosure;
[0051] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0053] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0054] News timeline generation is a news reporting method based on artificial intelligence technology. Its emergence has brought revolutionary changes to traditional news reporting methods. With the popularization of the Internet and mobile devices, people's channels for obtaining news are becoming more and more diverse, and traditional paper media and television media can no longer meet people's needs. News timeline generation uses artificial intelligence technology and big data analysis to organize news events into coherent timelines in chronological order, so that people can understand the development process of events more intuitively. Compared with traditional news reporting methods, news timeline generation can present event information more quickly and accurately, avoid information fragmentation and redundancy, and allow people to quickly understand the full picture of events in a short time. In addition, news timeline generation is also interactive, and users can choose events and time periods of interest according to their interests, so as to obtain personalized news content. In the future, with the continuous development of artificial intelligence technology, news timeline generation is expected to become a mainstream news reporting method, providing people with a more convenient and intuitive news reading experience. This invention is quite forward-looking in related fields and has important research value and research significance.
[0055] This example implementation first provides an information extraction method, which can be applied to the generation process of a timeline, and can also be applied to other similar application scenarios. Figure 1 As shown, the above information extraction method may include the following steps:
[0056] Step S110: Obtain the text length of the target text, and classify the target text into the first text type or the second text type according to the text length.
[0057] Step S120: If the target text is of the first text type, extract the target time node in the target text and the target event corresponding to the target time node through the first language model.
[0058] Step S130: If the target text is of the second text type, the target text is segmented into a plurality of text blocks, and a target time node in each text block and a target event corresponding to the target time node are extracted through the second language model.
[0059] In the information extraction method of the example implementation mode of the present disclosure, the text length of the target text is obtained, and the target text is divided into the first text type or the second text type according to the text length. For the first text type, the target time node in the target text and the target event corresponding to the target time node are extracted through the first language model; for the second text type, the target text is segmented into multiple text blocks, and the target time node in each text block and the target event corresponding to the target time node are extracted through the second language model. The information extraction method in the example implementation mode of the present disclosure, on the one hand, uses different methods and language models for event extraction according to the different lengths of the input text, which can improve the accuracy of event extraction; on the other hand, by performing text segmentation on the second text type with a longer text length before performing event extraction, the efficiency and accuracy of event extraction can be further improved.
[0060] Next, combine Figures 2 to 5 The above steps of this exemplary embodiment are described in more detail.
[0061] In step S110 , the text length of the target text is obtained, and the target text is divided into the first text type or the second text type according to the text length.
[0062] In this example implementation, the target text refers to the text to be extracted, such as a news document, a legal document, etc. The text length of the target text may be the number of characters in the target text, or the number of tokens in the target text.
[0063] According to the text length of the target text, the target text may be divided into a first text type or a second text type, wherein the first text type is a short text type and the second text type is a long text type.
[0064] In this example implementation, a total token length judgment module can be used to judge the total number of tokens in the input target text, and the input of this module is a quantity threshold. If the total number of tokens in the target text is less than the given input threshold, it is determined as the first text type, that is, the short text type; if the total number of tokens in the target text is greater than the given input threshold, it is determined as the second text type, that is, the long text type.
[0065] In step S120, if the target text is of the first text type, a target time node in the target text and a target event corresponding to the target time node are extracted using the first language model.
[0066] In this example implementation, if the target text is of the first text type, the target time node in the target text and the target event corresponding to the target time node are directly extracted through a lightweight information extraction language model, such as a UIE (Universal Information Extraction) model.
[0067] In step S130, if the target text is of the second text type, the target text is segmented into a plurality of text blocks, and a target time node in each text block and a target event corresponding to the target time node are extracted through the second language model.
[0068] In this example implementation, if the target text is of the second text type, the target text is segmented before content extraction.
[0069] In this example implementation, Figure 2 As shown, the target text is divided into multiple text blocks, which may include the following steps:
[0070] Step S210: Segment the target text into multiple text segments according to a preset text segmentation length.
[0071] Since the large model has a clear limit on the input length, the target text can be first segmented into multiple text segments according to the appropriate text segmentation length.
[0072] Step S220: Splice the text segment with other text segments before the text segment until the length of the spliced text reaches a preset text length threshold.
[0073] Step S230: Obtain a text block based on the spliced text segments.
[0074] Since the semantic information in each segmented text segment may also be segmented, resulting in semantic loss, the text segmentation backtracking method can be used to splice the current text segment with other text segments before the text segment until the length of the spliced text reaches the maximum token length limit of the model input.
[0075] Figure 3 The following schematic diagram shows a text segment splicing method according to a specific embodiment of the present disclosure. Figure 3 As shown in the figure, the target text is divided into four text segments C1, C2, C3, and C4. For each text segment, it is concatenated with the previous adjacent text segment until it is concatenated with the first text segment in the target text or the maximum token length limit of the model input is reached.
[0076] Event chunking and extraction technology divides long text into shorter text chunks, and identifies and extracts information related to specific events from the text, usually involving multiple participants or things, and can structure this information for subsequent analysis and utilization.
[0077] By using a chunk-based backtracking splicing method, multiple texts can be processed in parallel, increasing the speed and reducing latency, which also means that longer texts can be processed.
[0078] In this example implementation, after the information extraction is completed, the timeline information corresponding to the target text can be generated according to the target time node in the target text and the target event corresponding to the target time node. For the segmented text blocks, the extraction results corresponding to each text block are integrated and spliced to obtain the final result.
[0079] In this example implementation, the second language model is a student model obtained by model compression and model distillation of a preset large model. Figure 4 As shown, the training method of the second language model may specifically include the following steps:
[0080] Step S410: Acquire training corpus, segment the training corpus into training text blocks, and obtain a first training data set based on the training text blocks.
[0081] In order to make the model more generalizable, it is necessary to ensure data diversity during the data collection stage. More complete training corpus can be obtained from multiple fields. For example, corpus can be collected from the fields of location development and changes, historical conflicts and events, biographies of people, real-time hot news, issues, origins of items, itineraries, political changes, political events, and career development.
[0082] In this example implementation, the training corpus can be segmented into multiple training text segments according to a preset text segmentation length, and then the training text segment is spliced with other training text segments before the training text segment until the spliced text length reaches a preset text length threshold, and a training text block is obtained according to the spliced training text segments. The specific segmentation method is similar to the segmentation method of the target text, and will not be repeated here.
[0083] Step S420: Input the training text block in the first training data set into the third language model to obtain a training label corresponding to the training text block.
[0084] In this example implementation, the training text block and the preset prompt words in the first training data set can be input into the third language model to obtain the target time node in the training text block and the target event corresponding to the target time node, and the target time node in the training text block and the target event corresponding to the target time node are used as the training label corresponding to the training text block.
[0085] The third language model is a language model used to generate training data. You can use a large model to evaluate the authoritative expert model in the field, such as the gpt4 (Generative Pre-trained Transformer) model. By designing the prompt, gpt4 can generate all the timelines contained in the current training text block and the events corresponding to each time point on the timeline from each input training text block.
[0086] Prompt words are instructions or questions that users input into the large model to prompt the model and let the model answer specific content. In large models, prompt words are a key technology for interacting with the model. Prompt words that are more suitable for large models can significantly improve the output quality of the model.
[0087] Step S430. Obtain a second training data set according to the training text blocks and the corresponding training labels, and perform model distillation on the preset language model through the second training data set to obtain a second language model.
[0088] Each training text block is used as a feature, and its corresponding timeline and the events corresponding to each time point on the timeline are used as training labels to obtain a second training data set. The preset language model is distilled through the second training data set to obtain a trained second language model. The preset language model is a teacher model, for example, the qwen110b large model can be used.
[0089] In this example implementation, Figure 5 As shown, the preset language model is distilled through the second training data set to obtain a second language model, which may specifically include the following steps:
[0090] Step S510: Input the training text blocks in the second training data set into the preset language model and the second language model respectively.
[0091] During model compression and distillation, a training method may be used to first input the training text blocks in the second training data set into the preset language model and the second language model respectively to obtain corresponding output results.
[0092] Step S520: Obtain a first model loss according to an output result of the preset language model and an output result of the second language model.
[0093] The first model loss is the mean square error loss function between the student model and the teacher model, which is used to measure the degree of fit between the student model and the teacher model. The first model loss is calculated as follows:
[0094]
[0095] Among them, n is the total number of training corpus, is the student model, i.e., the second language model, ρ is the teacher model, i.e., the preset language model, x i is the i-th training text block.
[0096] Step S530: Obtain a second model loss according to the output result of the second language model and the training labels in the second training data set.
[0097] The second model loss is the loss function between the student model and the real timeline and its various time points. Since it is a multi-classification problem, cross entropy loss can be used. The calculation method of the second model loss is as follows:
[0098]
[0099] Among them, m is the total number of label categories, It represents the probability that the i-th training text block belongs to the j-th category.
[0100] Step S540: Obtain an overall loss according to the first model loss and the second model loss, and update the model parameters in the second language model according to the overall loss to train the second language model.
[0101] The overall loss function can be composed of the loss function of the student model itself and the compression distillation loss function. The calculation formula of the overall loss is as follows:
[0102] L=(1-α)L st +αL student
[0103] Among them, α is the weight hyperparameter, L st is the mean square error loss function between the student model and the teacher model, which is used to measure the degree of fit between the student model and the teacher model. student It is the loss function between the student model and the real timeline and its various time points. By updating the model parameters in the second language model through the overall loss, the second language model after model distillation and compression can be obtained.
[0104] In this example implementation, a model distillation and compression method is used to effectively present the effect of a large model on a small model, thereby saving resources and achieving better prediction results.
[0105] like Figure 6 FIG. 1 is a flow chart of a method for training a second language model in a specific embodiment of the present disclosure. The specific steps of the flow chart are as follows:
[0106] Step S610: Output the labels of the training corpus through gpt4.
[0107] By designing prompt words, gpt4 can output all timelines contained in the current training expectation and the events corresponding to each time point on the timeline from each training expectation input.
[0108] Step S620: Obtain a training set based on the training corpus and labels.
[0109] Take each training forecast as a feature, and use its corresponding timeline and the events corresponding to each time point on the timeline as labels to create a training set.
[0110] Step S630. Perform model compression and model distillation on the qwen110b large model through the training set to obtain the student model.
[0111] like Figure 7 The figure is a flowchart of a method for generating a news timeline in a specific implementation of the present disclosure, which is an example of the above steps in this example implementation. The specific steps of the flowchart are as follows:
[0112] Step S710: Determine the total number of tokens in the input news document through the token total length determination module.
[0113] The total token length judgment module is used to judge the total number of tokens in the input news document. The input of this module is a quantity threshold.
[0114] Step S720: For short token types, extract the timeline and the content of each time point on the timeline through UIE.
[0115] If the total number of news documents is less than a given input threshold, UIE is used to extract the content of the timeline and each time point on the timeline.
[0116] Step S730: For long token types, divide the token into chunks.
[0117] If the total number of news documents is greater than the given input threshold, the tokens need to be chunked.
[0118] Step S740. Input each chunk into the trained student model in sequence to output the timeline and the content of each time point on the timeline.
[0119] Finally, the output results corresponding to each chunk are integrated and spliced to get the final result.
[0120] In this example implementation, the prompt word technology and event block extraction technology are used to effectively extract and generate the timeline and the event information corresponding to each time node in the timeline from the news. By using gpt4 to generate training data, the large model is distilled and compressed to obtain a small model with better performance and time extraction accuracy similar to the original large model; adaptive prompt words are designed to facilitate more accurate output of the small model. In addition, different methods are used to extract the news timeline according to the different lengths of the input news text. For short news texts, the UIE model with better performance is used for extraction, and for long news texts, the model after distillation and compression is used for extraction, thereby reducing the delay in the generation of the news timeline. In addition, the text block backtracking method is used to make the extracted events more complete and the time more accurate.
[0121] The information extraction method in this example implementation can be applied to search engines, and the method can be used to generate a news timeline, allowing users to better view the development process of events. For special scenarios, it can also allow users to synchronize and update new developments of events more promptly. Compared with traditional news reporting methods, news timeline generation can present event information more quickly and accurately, avoiding information fragmentation and redundancy, allowing people to quickly understand the full picture of an event in a short period of time.
[0122] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0123] Furthermore, the present disclosure also provides an information extraction device. Figure 8 As shown, the information extraction device may include a text type determination module 810, a first text extraction module 820, and a second text extraction module 830. Among them:
[0124] The text type determination module 810 may be used to obtain the text length of the target text, and classify the target text into the first text type or the second text type according to the text length;
[0125] The first text extraction module 820 may be used to extract a target time node in the target text and a target event corresponding to the target time node through a first language model if the target text is of the first text type;
[0126] The second text extraction module 830 can be used to divide the target text into multiple text blocks if the target text is of the second text type, and extract the target time node in each text block and the target event corresponding to the target time node through the second language model.
[0127] In some exemplary embodiments of the present disclosure, an information extraction device provided by the present disclosure may also include a timeline generation module, which can be used to generate timeline information corresponding to the target text based on the target time nodes in the target text and the target events corresponding to the target time nodes.
[0128] In some exemplary embodiments of the present disclosure, the second text extraction module 830 may include a text segmentation unit, a text segment splicing unit, and a text block generation unit. Among them:
[0129] The text segmentation unit can be used to segment the target text into multiple text segments according to a preset text segmentation length;
[0130] The text segment concatenation unit may be used to concatenate the text segment with other text segments before the text segment until the length of the concatenated text reaches a preset text length threshold;
[0131] The text block generation unit can be used to obtain a text block according to the spliced text segments.
[0132] In some exemplary embodiments of the present disclosure, an information extraction device provided by the present disclosure may further include a second language model training module, which may include a first data set determination unit, a training label determination unit, and a model distillation unit. Wherein:
[0133] The first data set determination unit may be used to obtain training corpus, segment the training corpus to obtain training text blocks, and obtain a first training data set according to the training text blocks;
[0134] The training label determination unit may be used to input the training text block in the first training data set into the third language model to obtain a training label corresponding to the training text block;
[0135] The model distillation unit can be used to obtain a second training data set according to the training text blocks and the corresponding training labels, and perform model distillation on the preset language model through the second training data set to obtain a second language model.
[0136] In some exemplary embodiments of the present disclosure, the training label determination unit may include a prompt word input unit and a training label output unit.
[0137] The prompt word input unit can be used to input the training text block and the preset prompt word in the first training data set into the third language model to obtain the target time node in the training text block and the target event corresponding to the target time node;
[0138] The training label output unit can be used to use the target time node in the training text block and the target event corresponding to the target time node as the training label corresponding to the training text block.
[0139] In some exemplary embodiments of the present disclosure, the model distillation unit may include a training text block input unit, a first model loss determination unit, a second model loss determination unit, and an overall loss determination unit.
[0140] The training text block input unit may be used to input the training text blocks in the second training data set into the preset language model and the second language model respectively;
[0141] The first model loss determination unit may be used to obtain the first model loss according to the output result of the preset language model and the output result of the second language model;
[0142] The second model loss determination unit may be used to obtain a second model loss according to an output result of the second language model and training labels in the second training data set;
[0143] The overall loss determination unit can be used to obtain the overall loss according to the first model loss and the second model loss, and update the model parameters in the second language model according to the overall loss to train the second language model.
[0144] In some exemplary embodiments of the present disclosure, the first data set determination unit may include a training text segment segmentation unit, a training text segment splicing unit, and a training text block generation unit.
[0145] The training text segmentation unit can be used to segment the training corpus into multiple training text segments according to a preset text segmentation length;
[0146] The training text segment concatenation unit may be used to concatenate the training text segment with other training text segments preceding the training text segment until the length of the concatenated text reaches a preset text length threshold;
[0147] The training text block generation unit can be used to obtain a training text block according to the spliced training text segments.
[0148] The specific details of each module / unit in the above-mentioned information extraction device have been described in detail in the corresponding method embodiment part and will not be repeated here.
[0149] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0150] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.
[0151] It should be noted that Fig. 9 The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0152] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0153] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.
[0154] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present disclosure are executed.
[0155] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned information extraction method is implemented.
[0156] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.
[0157] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.
[0158] The code of the computer program can be written in one or more programming languages. Programming languages such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).
[0159] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned information extraction method.
[0160] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0161] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be concretized in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be concretized.
[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.
[0163] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An information extraction method, characterized in that: include: Acquire the text length of the target text, and classify the target text into a first text type or a second text type according to the text length; If the target text is of the first text type, extracting a target time node in the target text and a target event corresponding to the target time node through a first language model; If the target text is of the second text type, the target text is segmented into a plurality of text blocks, and a target time node in each of the text blocks and a target event corresponding to the target time node are extracted through a second language model.
2. The information extraction method according to claim 1, characterized in that: The method further comprises: According to the target time node in the target text and the target event corresponding to the target time node, the timeline information corresponding to the target text is generated.
3. The information extraction method according to claim 1, characterized in that: The step of dividing the target text into a plurality of text blocks comprises: The target text is segmented into a plurality of text segments according to a preset text segmentation length; Splicing the text segment with other text segments before the text segment until the length of the spliced text reaches a preset text length threshold; The text block is obtained according to the spliced text segments.
4. The information extraction method according to claim 1, characterized in that: The method further comprises: Acquire training corpus, segment the training corpus into training text blocks, and obtain a first training data set based on the training text blocks; Inputting a training text block in the first training data set into a third language model to obtain a training label corresponding to the training text block; A second training data set is obtained according to the training text blocks and the corresponding training labels, and a preset language model is distilled using the second training data set to obtain the second language model.
5. The information extraction method according to claim 4, characterized in that: The step of inputting the training text block in the first training data set into the third language model to obtain a training label corresponding to the training text block includes: Inputting the training text block and the preset prompt words in the first training data set into the third language model to obtain the target time node in the training text block and the target event corresponding to the target time node; The target time node in the training text block and the target event corresponding to the target time node are used as training labels corresponding to the training text block.
6. The information extraction method according to claim 4, characterized in that: The performing model distillation on the preset language model through the second training data set to obtain the second language model includes: Inputting the training text blocks in the second training data set into the preset language model and the second language model respectively; Obtaining a first model loss according to an output result of the preset language model and an output result of the second language model; Obtaining a second model loss according to an output result of the second language model and training labels in the second training data set; An overall loss is obtained according to the first model loss and the second model loss, and model parameters in the second language model are updated according to the overall loss to train the second language model.
7. The information extraction method according to claim 4, characterized in that: The step of segmenting the training corpus to obtain training text blocks includes: The training corpus is divided into a plurality of training text segments according to a preset text segmentation length; Splicing the training text segment with other training text segments before the training text segment until the length of the spliced text reaches a preset text length threshold; The training text block is obtained according to the spliced training text segments.
8. An information extraction device, characterized in that: include: A text type determination module, used for obtaining the text length of the target text, and classifying the target text into a first text type or a second text type according to the text length; A first text extraction module, configured to extract a target time node in the target text and a target event corresponding to the target time node through a first language model if the target text is of the first text type; The second text extraction module is used to divide the target text into multiple text blocks if the target text is of the second text type, and extract the target time node in each of the text blocks and the target event corresponding to the target time node through the second language model.
9. An electronic device, characterized in that: include: processor; as well as A memory for storing one or more programs, which, when executed by the processor, enables the processor to implement the information extraction method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the information extraction method according to any one of claims 1 to 7 is implemented.