A long text event extraction method for cultivated land protection

Abstract

The present application is applicable to the field of natural language processing technology, and provides a long text event extraction method for farmland protection, including: dividing the long text to be extracted into blocks, obtaining the first part-of-speech feature of each sentence after the block; encoding using the Bert model to obtain the first embedding vector of each sentence after the block; extracting sentence-level features for each first embedding vector to obtain the first sentence feature; determining the event trigger word and entity according to the first embedding vector, the first sentence feature and the first part-of-speech feature; determining the event type of the long text to be extracted according to the event trigger word, the entity and the first sentence feature; determining the prior text with the most similarity to the long text to be extracted; determining the corresponding event prompt template according to the event type; filling the event prompt template, and processing the filled event prompt template using a large language model to obtain the event extraction result. The present application can improve the accuracy of long text event extraction in the field of farmland protection.

Description

Technical Field

[0001] The present application belongs to the field of natural language processing technology, and in particular, relates to a long text event extraction method for farmland protection. Background Art

[0002] Although data-driven deep learning technology has achieved remarkable results in many remote sensing studies, it has limited applicability when it comes to reasoning about complex remote sensing scenes. In order to solve the above problems, the coupling of knowledge models and deep learning is an effective means to solve the modeling of complex surface scenes. Therefore, the remote sensing intelligent interpretation method coupled with prior knowledge is an important research trend in current cultivated land monitoring. Therefore, the extraction of prior knowledge of cultivated land protection is particularly important.

[0003] According to the theory of natural language processing, event extraction is the mainstream method to achieve the above-mentioned prior knowledge extraction. Specifically, the purpose of event extraction (EE) is to extract event information that meets the event pattern from unstructured or semi-structured text based on pre-defined event patterns, such as trigger words and event argument entities (including time, people, places, etc.), and express them in a structured form. Event extraction is mainly based on deep learning methods, such as the classic BERT (BERT is a language representation model) and conditional random fields (CRF) combined event extraction model. These deep learning-based event extraction methods often require specific domain data sets for model training to meet the needs of a certain field. Therefore, for specific domain event extraction, building a domain data set is the primary task to achieve domain event extraction. However, compared with other domain data, the text of cultivated land protection events contains multiple time and space information at the same time, has a variety of subject and object arguments, and lacks corresponding annotated data sets, which makes it difficult to directly migrate event extraction data sets and event extraction methods in other fields to the field of cultivated land protection. At the same time, the datasets in the field of cultivated land protection have low-resource phenomena such as scarce data samples and unbalanced data distribution. Conventional event extraction methods based on pre-training such as BERT do not work well in low-resource scenarios. Existing low-resource event extraction methods do not fully consider features such as prior knowledge and part of speech. At the same time, most event extraction methods for long texts have problems such as missing features. Under the impact of ChatGPT (Chat Generative Pre-trained Transformer), a natural language processing tool driven by artificial intelligence technology, large language models (LLM) have become a hot topic in the field of natural language processing (NLP) today, and LLM base models such as GLM, GPT-4, and LLaMA-2 have been born. LLM has strong natural language understanding, text generation and emergence capabilities, so it is also used in the exploration of event extraction. Since the input length of the large language model has significant advantages over pre-training models such as BERT, however, the accuracy of event extraction directly using the large language model is not high, and the domain fine-tuning of the large language model requires a lot of resources.To address the above issues, prompt learning is widely used in event extraction of long texts. The prompt learning framework is a new paradigm that has emerged in the field of pre-trained language models in recent years: the prompt learning paradigm allows downstream tasks to adapt to pre-trained language models. Compared with the use of BERT pre-trained models and fine-tuning to adapt to downstream tasks, the prompt learning framework does not rely on training data for specific tasks and is adaptable to a variety of NLP tasks and low-resource scenarios. However, existing event extraction methods for long texts do not fully consider prior knowledge, and do not fully consider features such as part of speech during encoding, resulting in low accuracy in long text event extraction in the field of farmland protection. Summary of the invention

[0004] The embodiment of the present application provides a long text event extraction method for farmland protection, which can solve the problem of low accuracy in long text event extraction in the field of farmland protection.

[0005] The present application embodiment provides a long text event extraction method for farmland protection, including:

[0006] The long text to be extracted in the field of cultivated land protection is divided into blocks, and the jieba library is used to obtain the part-of-speech representation of each sentence after the block is divided. The part-of-speech representation is vectorized to obtain the first part-of-speech feature, and the trained Bert model is used to encode each sentence obtained by the block to obtain the first embedding vector of each sentence.

[0007] Using a convolutional neural network to extract sentence-level features from each first embedding vector, to obtain a first sentence feature of each first embedding vector;

[0008] Determine the event trigger words and entities of the long text to be extracted according to all first part-of-speech features, first embedding vectors, and first sentence features of each first embedding vector corresponding to the long text to be extracted;

[0009] Determine the event type of the long text to be extracted according to the event trigger word, the entity and the first sentence feature of each first embedding vector; the event type is one or more of a cultivated land change event, a restoration event and a discovery event;

[0010] Determine the target prior text with the highest similarity to the long text to be extracted from the previously obtained event extraction data set in the field of cultivated land protection; the event extraction data set includes multiple prior texts;

[0011] Determine an event prompt template corresponding to the event type; the event prompt template includes: extraction task information, prior knowledge to be filled according to event information of the target prior text, the long text to be extracted, and a conditional instruction for indicating the selection of the target entity from the entities of the long text to be extracted, and the extraction task information is used to indicate the event extraction of the long text to be extracted according to the prior knowledge;

[0012] The event prompt template is filled according to the event information of the target prior text, the long text to be extracted and the entities of the long text to be extracted, and the filled event prompt template is input into the large language model for processing to obtain the event extraction result of the long text to be extracted.

[0013] Optionally, determining the event trigger words and entities of the long text to be extracted according to all first part-of-speech features, the first embedding vectors, and the first sentence features of each first embedding vector corresponding to the long text to be extracted includes:

[0014] For each first embedding vector corresponding to the long text to be extracted, concatenate the first embedding vector, the first sentence feature of the first embedding vector, and the first part-of-speech feature corresponding to the first embedding vector to obtain a first fusion feature;

[0015] All first fused features corresponding to the long text to be extracted are input into the conditional random field for decoding to obtain the event trigger words and entities of the long text to be extracted.

[0016] Optionally, determining the event type of the long text to be extracted according to the event trigger word, the entity, and the first sentence feature of each first embedding vector includes:

[0017] According to the position of the event trigger word in the long text to be extracted, a first target embedding vector corresponding to the event trigger word is determined from all first embedding vectors corresponding to the long text to be extracted, and according to the position of the entity in the long text to be extracted, a second target embedding vector corresponding to the entity is determined from all first embedding vectors corresponding to the long text to be extracted;

[0018] An event type of the long text to be extracted is determined according to the first target embedding vector, the second target embedding vector, and a first sentence feature of each first embedding vector.

[0019] Optionally, determining the event type of the long text to be extracted according to the first target embedding vector, the second target embedding vector, and the first sentence feature of each first embedding vector includes:

[0020] For the first target embedding vector, the second target embedding vector and the first sentence of each first embedding vector

[0021] The features are spliced ​​to obtain the second fusion feature;

[0022] The second fused feature is input into the event type classifier for classification processing to obtain the event type of the long text to be extracted.

[0023] Optionally, the event type classifier is a multi-classifier.

[0024] Optionally, determine the event prompt template corresponding to the event type, including:

[0025] The event prompt template corresponding to the event type is queried from a pre-built prompt library; the prompt library stores multiple event types and the event prompt template corresponding to each event type.

[0026] Optionally, before encoding each sentence obtained by segmentation using the trained Bert model, the long text event extraction method further includes:

[0027] For each of the multiple prior texts, the prior text is divided into blocks, and the jieba library is used to obtain the part-of-speech representation of each sentence after the block, and the part-of-speech representation is vectorized to obtain the second part-of-speech feature, and each sentence obtained by the block is encoded using the original Bert model to obtain the second embedding vector of each sentence;

[0028] A convolutional neural network is used to extract sentence-level features from each second embedding vector to obtain second sentence features of each second embedding vector;

[0029] The original Bert model is trained according to all second embedding vectors corresponding to the event extraction data set, the second sentence features of all second embedding vectors, and the second part-of-speech features corresponding to all second embedding vectors to obtain a trained Bert model.

[0030] Optionally, the original Bert model is trained according to all second embedding vectors corresponding to the event extraction data set, the second sentence features of all second embedding vectors, and the second part-of-speech features corresponding to all second embedding vectors to obtain a trained Bert model, including:

[0031] For each sentence corresponding to the event extraction data set, the second embedding vector, the second sentence feature and the second part-of-speech feature corresponding to the sentence are concatenated to obtain a third fusion feature;

[0032] For each third fusion feature obtained, the third fusion feature and the true label corresponding to the third fusion feature are used as a training sample; the true label is the true entity of the sentence corresponding to the third fusion feature;

[0033] All the obtained training samples are input into the conditional random field to obtain the entity recognition loss, the event trigger word sample and the entity sample of each sentence corresponding to the third fusion feature;

[0034] For each third fusion feature, according to the position of the event trigger word sample of the sentence corresponding to the third fusion feature in the sentence, a third target embedding vector corresponding to the event trigger word sample is determined from the second embedding vector corresponding to the sentence, and according to the position of the entity sample of the sentence corresponding to the third fusion feature in the sentence, a fourth target embedding vector corresponding to the entity sample is determined from the second embedding vector corresponding to the sentence, and the third target embedding vector, the fourth target embedding vector, and the second sentence feature corresponding to the third fusion feature are fused to obtain a fourth fusion feature;

[0035] Input all the fourth fusion features obtained into the event type classifier for classification processing, and obtain the event type classification result of the priori text corresponding to each fourth fusion feature;

[0036] Determine the event detection loss according to the event type classification result of each fourth fused feature and the actual event type of the priori text corresponding to each fourth fused feature;

[0037] According to the sum of the entity recognition loss and the event detection loss, the parameters of the original BERT model are adjusted, and the adjusted BERT model is used as the trained BERT model.

[0038] Optionally, according to the sum of the entity recognition loss and the event detection loss, the parameters of the original BERT model are adjusted, and the adjusted BERT model is used as the trained BERT model, including:

[0039] Determine whether the sum of entity recognition loss and event detection loss is less than or equal to a preset loss value;

[0040] If the sum of the entity recognition loss and the event detection loss is greater than the preset loss value, the parameters of the original Bert model are adjusted, and the Bert model after the parameter adjustment is used as the original Bert model, and the step of encoding each sentence obtained by the block using the original Bert model is returned to obtain the second embedding vector of each sentence;

[0041] If the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value, the original BERT model is used as the trained BERT model.

[0042] Optionally, a target priori text having the highest similarity to the long text to be extracted is determined from a pre-acquired event extraction dataset in the field of cultivated land protection, including:

[0043] For each prior text in the event extraction data set, calculate the similarity value between the long text to be extracted and the prior text according to all the first embedding vectors corresponding to the long text to be extracted and all the second embedding vectors corresponding to the prior text;

[0044] The priori text corresponding to the largest similarity value among all the calculated similarity values ​​is used as the target priori text with the highest similarity to the long text to be extracted.

[0045] The above solution of the present application has the following beneficial effects:

[0046] In an embodiment of the present application, the Bert model is used to encode each sentence after the long text to be extracted is divided into blocks, the jieba library is used to obtain the part-of-speech representation of each sentence, the part-of-speech representation is vectorized to obtain the first part-of-speech feature, and the convolutional neural network is used to extract sentence-level features from the encoded embedding vector, and then the encoded embedding vector and the extracted sentence features and part-of-speech features are used to determine the event trigger words and entities of the long text to be extracted, and the event trigger words, entities and sentence features are used to determine the event type of the long text to be extracted, and then an event prompt template is determined based on the event type, and the prior text most similar to the long text to be extracted is determined from the prior event extraction data set, and then the event prompt template is filled based on the prior text, the long text to be extracted and its entities, and the filled event prompt template is input into the large language model for processing to obtain the event extraction result of the long text to be extracted. Among them, the event trigger words, entities, and event types of the long text to be extracted are determined based on information from three dimensions: embedding vectors, sentence features, and part-of-speech features, which improves the accuracy of event trigger words, entities, and event types. At the same time, the designed event prompt templates are filled based on prior knowledge in high-accuracy entities and prior event extraction datasets, which improves the accuracy of event prompt templates. As a result, the large language model can make full use of existing prior knowledge when extracting events from event prompt templates based on its own context understanding ability, thereby improving the event extraction ability of the large language model and improving the accuracy of long text event extraction in the field of farmland protection.

[0047] In addition, since there is no need to fine-tune the large language model when using it for event extraction, the problem of large resource consumption in fine-tuning the large language model is solved.

[0048] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1A flowchart of a long text event extraction method for farmland protection provided in an embodiment of the present application;

[0051] Figure 2 A training flowchart of the Bert model provided in one embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0053] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0054] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0055] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0056] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0057] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0058] In view of the low accuracy of long text event extraction in the field of farmland protection, an embodiment of the present application provides a long text event extraction method for farmland protection. The method uses the Bert model to encode each sentence after the long text to be extracted is divided into blocks, and the jieba library is used to obtain the part-of-speech representation of each sentence block. The part-of-speech representation is vectorized to obtain the first part-of-speech feature, and a convolutional neural network is used to extract sentence-level features from the encoded embedding vector. Then, the encoded embedding vector and the extracted sentence features and part-of-speech features are used to determine the event trigger words and entities of the long text to be extracted, and the event trigger words, entities and sentence features are used to determine the event type of the long text to be extracted. Then, an event prompt template is determined based on the event type, and the prior text most similar to the long text to be extracted is determined from the prior event extraction data set. Then, the event prompt template is filled based on the prior text, the long text to be extracted and its entities, and the filled event prompt template is input into the large language model for processing to obtain the event extraction result of the long text to be extracted. Among them, the event trigger words, entities, and event types of the long text to be extracted are determined based on information from three dimensions: embedding vectors, sentence features, and part-of-speech features, which improves the accuracy of event trigger words, entities, and event types. At the same time, the designed event prompt templates are filled based on prior knowledge in high-accuracy entities and prior event extraction datasets, which improves the accuracy of event prompt templates. As a result, the large language model can make full use of existing prior knowledge when extracting events from event prompt templates based on its own context understanding ability, thereby improving the event extraction ability of the large language model and improving the accuracy of long text event extraction in the field of farmland protection.

[0059] In addition, since there is no need to fine-tune the large language model when using it for event extraction, the problem of large resource consumption in fine-tuning the large language model is solved.

[0060] The following is an illustrative description of the long text event extraction method for farmland protection provided in the embodiments of the present application in conjunction with specific embodiments.

[0061] like Figure 1As shown in the figure, the long text event extraction method for cultivated land protection provided by the embodiments of the present application includes the following steps:

[0062] Step 11: Perform chunking on the long text to be extracted in the field of cultivated land protection. Use the jieba library to obtain the part-of-speech representation of each sentence after chunking. After vectorizing the part-of-speech representation, obtain the first part-of-speech feature, and use the trained Bert model to encode each sentence obtained by chunking to obtain the first embedding vector of each sentence.

[0063] The above long text to be extracted is the long text that needs to be subjected to event extraction in the field of cultivated land protection. To solve the problem that the Bert model is difficult to process long texts, before encoding using the Bert model, it is necessary to perform chunking on the long text to be extracted, and convert the long text to be extracted into the BIO form of named entity recognition acceptable to the Bert model. The BIO annotation form is a commonly used sequence annotation method. Among them, B: Begin, indicating the start of a named entity. I: Inside, indicating the inside of a named entity. O: Outside, indicating not belonging to any named entity. For example, B-PER: Start a person name entity (PER is the entity type, indicating a person), I-PER: Inside the person name entity.

[0064] It can be understood that since the relevant notification texts for cultivated land protection have strong norms, in order to improve the accuracy of event extraction, before performing chunking, the long text can be simply preprocessed. The following preprocessing can be specifically performed: Convert English characters to Chinese characters; Remove common stop words (such as "de", "le", "zai"); Remove extra spaces to ensure the neatness of the text; Remove or replace special characters in the text, such as line breaks, tab characters, etc.

[0065] In some embodiments of the present application, the long text to be extracted can be chunked by a common text segmentation method, and the long text to be extracted can be segmented into multiple sentences. Exemplarily, the target characters (such as full stops) in the long text to be extracted can be first identified, and then, taking the target characters as the segmentation points, the long text to be extracted can be segmented to obtain multiple segmented sentences.

[0066] Among them, the above trained Bert model is mainly used to encode the segmented sentences to obtain the encoded vectors (i.e., the above first embedding vectors). It should be noted that the training process of the Bert model will be elaborated in detail later.

[0067] Step 12: Use a convolutional neural network to perform sentence-level feature extraction on each first embedding vector to obtain the first sentence feature of each first embedding vector.

[0068] The sentence-level feature extraction is to extract the basic features of the sentence, that is, the first sentence feature is used to characterize the basic features of the corresponding sentence;

[0069] It can be understood that the above convolutional neural network is obtained after training with a large amount of training data (each training data includes a sentence and the sentence features of the sentence).

[0070] Step 13, determining the event trigger words and entities of the long text to be extracted based on all first part-of-speech features, all first embedding vectors, and the first sentence features of each first embedding vector corresponding to the long text to be extracted.

[0071] The event trigger words are the core words of the event, such as the event trigger word of the fire event is fire; the entities can be understood as event elements, and the types of entities include time, person, organization, region, farmland, quantity, description, feature, type, content, status, position, etc. For example, the time entity can be the time of the illegal event and the time of the farmland change event.

[0072] In some embodiments of the present application, a specific implementation method for determining the event trigger words and entities of the long text to be extracted according to all first part-of-speech features, first embedding vectors, and first sentence features of each first embedding vector corresponding to the long text to be extracted includes the following steps:

[0073] Step 13.1, for each first embedding vector corresponding to the long text to be extracted, concatenate the first embedding vector, the first sentence feature of the first embedding vector, and the first part-of-speech feature corresponding to the first embedding vector to obtain a first fusion feature.

[0074] In some embodiments of the present application, the data representation of the above-mentioned first sentence feature and first part-of-speech feature can be a vector. To facilitate data processing, after concatenating the first embedding vector, the first sentence feature of the first embedding vector, and the corresponding first part-of-speech feature, the concatenated result can be sent to the fully connected layer to obtain the first fusion feature.

[0075] Step 13.2, input all the first fusion features corresponding to the long text to be extracted into the conditional random field for decoding, and obtain the event trigger words and entities of the long text to be extracted.

[0076] In some embodiments of the present application, all first fused features are decoded by a conditional random field (CRF) to obtain event trigger words and entities of the long text to be extracted.

[0077] It can be understood that in order to enable CRF to decode event trigger words and entities, the above CRF is obtained after training with a large amount of training data (each training data includes a fusion feature obtained based on the sentence embedding vector, sentence features and part-of-speech features (the fusion feature is obtained in the same way as the first fusion feature mentioned above), as well as the real event trigger words and real entities of the sentence).

[0078] Step 14, determining the event type of the long text to be extracted based on the event trigger word, the entity and the first sentence feature of each first embedding vector.

[0079] Event types in the field of farmland protection include farmland change events, restoration events, discovery events, etc. The event type of the long text to be extracted can be one or more of these types.

[0080] In some embodiments of the present application, the event type of the long text to be extracted can be determined by the following steps:

[0081] Step 14.1, according to the position of the event trigger word in the long text to be extracted, determine the first target embedding vector corresponding to the event trigger word from all the first embedding vectors corresponding to the long text to be extracted, and according to the position of the entity in the long text to be extracted, determine the second target embedding vector corresponding to the entity from all the first embedding vectors corresponding to the long text to be extracted.

[0082] Step 14.2, determining the event type of the long text to be extracted based on the first target embedding vector, the second target embedding vector and the first sentence feature of each first embedding vector.

[0083] Specifically, the first target embedding vector, the second target embedding vector and the first sentence feature of each first embedding vector can be concatenated to obtain a second fused feature; then the second fused feature is input into an event type classifier for classification processing to obtain the event type of the long text to be extracted.

[0084] It can be understood that the data representation of the above-mentioned first sentence feature can be a vector. After concatenating the first target embedding vector, the second target embedding vector and the first sentence feature, the concatenated result can be sent to the fully connected layer to obtain the second fusion feature.

[0085] In order to enable the event type classifier to be used to determine the event type, the above event type classifier is obtained after training with a large amount of training data (each training data includes a fusion feature obtained based on the sentence embedding vector, event trigger words, entities and sentence features (the fusion feature is obtained in the same way as the second fusion feature mentioned above), and the actual event type of the sentence).

[0086] The above event type classifier may be a multi-classifier. It is understandable that the output of the multi-classification is whether the event type is included.

[0087] Step 15, determining the target priori text with the highest similarity to the long text to be extracted from the pre-acquired event extraction data set in the field of farmland protection.

[0088] The above-mentioned event extraction data set includes multiple prior texts. The prior texts are long texts that record the existing prior knowledge in the field of farmland protection and can be obtained through public data platforms.

[0089] Step 16: Determine the event prompt template corresponding to the event type.

[0090] In some embodiments of the present application, an event prompt template corresponding to an event type can be obtained by querying a pre-built prompt library. The prompt library stores multiple event types and an event prompt template corresponding to each event type.

[0091] The above-mentioned event prompt template includes: extraction task information, prior knowledge that needs to be filled in according to the event information of the target prior text, the long text to be extracted, and conditional instructions for instructing the screening of target entities from the entities of the long text to be extracted. The extraction task information is used to instruct event extraction of the long text to be extracted according to the prior knowledge.

[0092] Step 17, fill the event prompt template according to the event information of the target prior text, the long text to be extracted and the entities of the long text to be extracted, and input the filled event prompt template into the large language model for processing to obtain the event extraction result of the long text to be extracted.

[0093] In some embodiments of the present application, the above event information is used to describe relevant information of the event, such as the event type contained in the prior text and the event role information corresponding to the event type, such as the time of the illegal event, etc. After the event prompt template is filled with the event information of the target prior text, the long text to be extracted and its entities, and the filled event prompt template is obtained, the large language model (LLM) can extract events according to the extraction task information based on its own context understanding ability, make full use of the existing prior knowledge, improve the event extraction ability of the large language model, and improve the accuracy of long text event extraction in the field of farmland protection.

[0094] Exemplarily, after the event prompt template is filled based on the event information of the target prior text and the entity of the long text to be extracted, the filled event prompt template includes: prior text, prior text event information, text to be extracted, text entity information to be extracted, extraction target information, etc. The large language model extracts events from the filled event prompt template, and the event extraction results obtained include: illegal event time, etc.

[0095] The training process of the Bert model is exemplarily described below in conjunction with specific embodiments.

[0096] Specifically, before encoding each sentence obtained by the block using the trained Bert model, such as Figure 2 As shown, the above long text event extraction method also includes the following training steps:

[0097] Step 21, for each of the multiple prior texts, the prior text is divided into blocks, the jieba library is used to obtain the part-of-speech representation of each sentence after the block, the part-of-speech representation is vectorized to obtain the second part-of-speech feature, and each sentence obtained by the block is encoded using the original Bert model to obtain the second embedding vector of each sentence.

[0098] It is understandable that the method of dividing and encoding the prior text is the same as the method of dividing and encoding the extracted long text in the above step 11. Therefore, the method of dividing and encoding the prior text is not described in detail here.

[0099] In some embodiments of the present application, in order to facilitate the storage and retrieval of data, the above-mentioned second embedded vector can be stored and indexed using the FAISS vector database, which supports efficient similarity search and large-scale vector retrieval, making it easy to quickly find the prior text that is most similar to the long text to be extracted.

[0100] Step 22: Use a convolutional neural network to extract sentence-level features from each second embedding vector to obtain second sentence features of each second embedding vector.

[0101] It can be understood that the method of extracting sentence-level and part-of-speech-level features from the second embedding vector is the same as the method of extracting sentence-level features from the first embedding vector in the above step 12. Therefore, the method of extracting sentence-level features from the second embedding vector will not be described in detail here.

[0102] Step 23, training the original Bert model according to all second embedding vectors corresponding to the event extraction data set, the second sentence features of all second embedding vectors, and the second part-of-speech features corresponding to all second embedding vectors to obtain a trained Bert model.

[0103] In some embodiments of the present application, the above step 23, training the original Bert model according to all the second embedding vectors corresponding to the event extraction data set, the second sentence features of all the second embedding vectors, and the second part-of-speech features corresponding to all the second embedding vectors, to obtain the trained Bert model includes the following steps:

[0104] Step 23.1, for each sentence corresponding to the event extraction data set, concatenate the second embedding vector, the second sentence feature, and the second part-of-speech feature corresponding to the sentence to obtain a third fusion feature.

[0105] In some embodiments of the present application, the data representation of the second sentence feature and the second part-of-speech feature may be a vector. To facilitate data processing, after concatenating the second embedding vector, the second sentence feature and the second part-of-speech feature of the second embedding vector, the concatenated result may be sent to a fully connected layer to obtain a third fusion feature.

[0106] Step 23.2, for each third fusion feature obtained, the third fusion feature and the true label corresponding to the third fusion feature are used as a training sample; the true label is the true entity of the sentence corresponding to the third fusion feature.

[0107] Exemplarily, the above-mentioned real label can be a time entity, a person entity, an organization entity, a cultivated land entity, a quantity entity, etc.

[0108] In step 23.3, all the obtained training samples are input into the conditional random field to obtain the entity recognition loss, the event trigger word samples and entity samples of the sentences corresponding to each third fusion feature.

[0109] The above event trigger word samples and entity samples refer to the event trigger words and entity samples obtained by inputting the third fusion feature into the conditional random field and decoding the third fusion feature by the conditional random field. It can be understood that after all training samples are input into the conditional random field, the conditional random field can obtain the entity recognition loss through the negative log-likelihood loss while decoding.

[0110] Step 23.4, for each third fusion feature respectively, according to the position of the event trigger word sample of the sentence corresponding to the third fusion feature in the sentence, determine the third target embedding vector corresponding to the event trigger word sample from the second embedding vector corresponding to the sentence, and according to the position of the entity sample of the sentence corresponding to the third fusion feature in the sentence, determine the fourth target embedding vector corresponding to the entity sample from the second embedding vector corresponding to the sentence, and fuse the third target embedding vector, the fourth target embedding vector, and the second sentence feature corresponding to the third fusion feature to obtain the fourth fusion feature.

[0111] It can be understood that the data representation of the above-mentioned second sentence feature can be a vector. After concatenating the third target embedding vector, the fourth target embedding vector and the second sentence feature, the concatenated result can be sent to the fully connected layer to obtain the fourth fusion feature.

[0112] Step 23.5: input all the fourth fusion features obtained into the event type classifier for classification processing to obtain the event type classification result of the priori text corresponding to each fourth fusion feature.

[0113] Step 23.6, determining the event detection loss according to the event type classification result of each fourth fusion feature and the actual event type of the prior text corresponding to each fourth fusion feature.

[0114] The event type classification result of the above-mentioned fourth fusion feature is actually the event type classification result of the prior text corresponding to the fourth fusion feature, which may be specifically illegal events, farmland change events, restoration events, etc.; the real event type of the prior text refers to the actual event type of the prior text, which is prior knowledge, which may be specifically illegal events, farmland change events, restoration events, etc.

[0115] In some embodiments of the present application, after the event type classifier outputs the event type classification results of each prior text, the cross-entropy function can be used to calculate the event detection loss based on the actual event type of each prior text and the event type classification results output by the event type classifier. The calculation method of the event detection loss is the same as the calculation method of the loss value of the event type classifier, and its specific calculation process will not be repeated here.

[0116] In step 23.7, the parameters of the original BERT model are adjusted according to the sum of the entity recognition loss and the event detection loss, and the adjusted BERT model is used as the trained BERT model.

[0117] Specifically, determine whether the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value. If the sum of the entity recognition loss and the event detection loss is greater than the preset loss value, adjust the parameters of the original Bert model, and use the Bert model after parameter adjustment as the original Bert model, and return to execute the step of encoding each sentence obtained by block using the original Bert model to obtain the second embedding vector of each sentence; if the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value, use the original Bert model as the trained Bert model.

[0118] That is, if the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value (the preset loss value can be set according to the actual situation, for example, set to 0.1), the training is completed and the trained Bert model is obtained; otherwise, the parameters of the original Bert model are adjusted, and the adjusted original Bert model is used to return to step 21 to encode each sentence obtained by block using the original Bert model to obtain the second embedding vector of each sentence, until the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value, the training is completed and the trained Bert model is obtained.

[0119] It should be noted that during the training process of the Bert model, only the parameters of the Bert model are adjusted, and the convolutional neural network, conditional random field, event type classifier, etc. involved in the process are all trained model structures.

[0120] The following is an exemplary description of the method for determining the target priori text.

[0121] Specifically, the specific implementation method of the above step 15, determining the target priori text with the highest similarity to the long text to be extracted from the pre-acquired event extraction data set in the field of cultivated land protection, can be:

[0122] First, for each prior text in the event extraction dataset, the similarity value between the long text to be extracted and the prior text is calculated based on all the first embedding vectors corresponding to the long text to be extracted and all the second embedding vectors corresponding to the prior text; then the prior text corresponding to the largest similarity value among all the calculated similarity values ​​is taken as the target prior text with the highest similarity to the long text to be extracted.

[0123] In some embodiments of the present application, the similarity value between the long text to be extracted and each priori text may be calculated specifically by means of a cosine similarity function.

[0124] In summary, the long text event extraction method for farmland protection provided in the embodiment of the present application has the following advantages:

[0125] First, we use pre-trained CRF for entity recognition and large language model context understanding ability for role classification event extraction paradigm. This paradigm not only solves the problem of large language model fine-tuning consuming large resources, but also makes full use of the excellent performance of pre-trained CRF in event extraction and the ability of large language model, thus improving the performance of low-resource long text event extraction in the field of cultivated land protection.

[0126] Second, by designing prior knowledge types in the field of cultivated land protection, including event description information, event role description information, and event information of existing data sets that are most similar to the input text, the large language model can make full use of existing knowledge and improve the event extraction ability of the large language model;

[0127] Third, by designing an event prompt template in the field of farmland protection, the template can fully integrate various types of prior knowledge, so that the large language model can make full use of existing knowledge and improve the accuracy of long text event extraction in the field of farmland protection.

[0128] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A long text event extraction method for farmland protection, characterized in that: include: The long text to be extracted in the field of cultivated land protection is divided into blocks, and the jieba library is used to obtain the part-of-speech representation of each sentence after the block is divided. The part-of-speech representation is vectorized to obtain the first part-of-speech feature, and the trained Bert model is used to encode each sentence obtained by the block to obtain the first embedding vector of each sentence. Using a convolutional neural network to extract sentence-level features from each of the first embedding vectors, to obtain a first sentence feature of each of the first embedding vectors; Determine the event trigger words and entities of the long text to be extracted according to all first part-of-speech features, first embedding vectors, and first sentence features of each of the first embedding vectors corresponding to the long text to be extracted; Determine the event type of the long text to be extracted according to the event trigger word, the entity and the first sentence feature of each of the first embedding vectors; the event type is one or more of a cultivated land change event, a restoration event and a discovery event; Determine the target priori text with the highest similarity to the long text to be extracted from the previously obtained event extraction data set in the field of cultivated land protection; The event extraction data set includes a plurality of prior texts; Determine an event prompt template corresponding to the event type; The event prompt template includes: extraction task information, prior knowledge to be filled according to the event information of the target prior text, a long text to be extracted, and a conditional instruction for indicating the selection of a target entity from the entities of the long text to be extracted, wherein the extraction task information is used to indicate the extraction of events from the long text to be extracted according to the prior knowledge; Filling the event prompt template according to the event information of the target priori text, the long text to be extracted and the entities of the long text to be extracted, and inputting the filled event prompt template into the large language model for processing to obtain the event extraction result of the long text to be extracted; The step of determining the event trigger words and entities of the long text to be extracted according to all first part-of-speech features, first embedding vectors, and first sentence features of each of the first embedding vectors corresponding to the long text to be extracted includes: For each first embedding vector corresponding to the long text to be extracted, concatenate the first embedding vector, the first sentence feature of the first embedding vector, and the first part-of-speech feature corresponding to the first embedding vector to obtain a first fusion feature; Input all first fused features corresponding to the long text to be extracted into a conditional random field for decoding to obtain event trigger words and entities of the long text to be extracted; The step of determining the event type of the long text to be extracted according to the event trigger word, the entity, and the first sentence feature of each of the first embedding vectors includes: According to the position of the event trigger word in the long text to be extracted, a first target embedding vector corresponding to the event trigger word is determined from all first embedding vectors corresponding to the long text to be extracted, and according to the position of the entity in the long text to be extracted, a second target embedding vector corresponding to the entity is determined from all first embedding vectors corresponding to the long text to be extracted; Determining the event type of the long text to be extracted according to the first target embedding vector, the second target embedding vector, and the first sentence feature of each of the first embedding vectors; The determining the event type of the long text to be extracted according to the first target embedding vector, the second target embedding vector, and the first sentence feature of each of the first embedding vectors includes: The first target embedding vector, the second target embedding vector and each of the first embedding vectors The first sentence features of the quantity are concatenated to obtain the second fusion feature; The second fused feature is input into an event type classifier for classification processing to obtain the event type of the long text to be extracted.

2. The long text event extraction method according to claim 1, characterized in that: The event type classifier is a multi-classifier.

3. The long text event extraction method according to claim 1, characterized in that: The determining of the event prompt template corresponding to the event type includes: The event prompt template corresponding to the event type is obtained by querying from a pre-built prompt library; the prompt library stores multiple event types and the event prompt template corresponding to each event type.

4. The long text event extraction method according to claim 1, characterized in that: Before encoding each sentence obtained by segmentation using the trained Bert model, the long text event extraction method further includes: For each of the multiple prior texts, the prior text is divided into blocks, the part-of-speech representation of each sentence after the block is obtained by using the jieba library, the part-of-speech representation is vectorized to obtain a second part-of-speech feature, and each sentence obtained by the block is encoded by using the original Bert model to obtain a second embedding vector for each sentence; Using a convolutional neural network to extract sentence-level features from each of the second embedding vectors, to obtain second sentence features of each of the second embedding vectors; The original Bert model is trained according to all second embedding vectors corresponding to the event extraction data set, the second sentence features of all second embedding vectors, and the second part-of-speech features corresponding to all second embedding vectors to obtain a trained Bert model.

5. The long text event extraction method according to claim 4, characterized in that: The method of training the original Bert model according to all second embedding vectors corresponding to the event extraction data set, the second sentence features of all second embedding vectors, and the second part-of-speech features corresponding to all second embedding vectors to obtain a trained Bert model includes: For each sentence corresponding to the event extraction data set, concatenate the second embedding vector, the second sentence feature, and the second part-of-speech feature corresponding to the sentence to obtain a third fusion feature; For each obtained third fusion feature, the third fusion feature and the real label corresponding to the third fusion feature are used as a training sample; the real label is the real entity of the sentence corresponding to the third fusion feature; Input all obtained training samples into the conditional random field to obtain entity recognition loss, event trigger word samples and entity samples of each sentence corresponding to the third fusion feature; For each third fusion feature, according to the position of the event trigger word sample of the sentence corresponding to the third fusion feature in the sentence, determine the third target embedding vector corresponding to the event trigger word sample from the second embedding vector corresponding to the sentence, and according to the position of the entity sample of the sentence corresponding to the third fusion feature in the sentence, determine the fourth target embedding vector corresponding to the entity sample from the second embedding vector corresponding to the sentence, and fuse the third target embedding vector, the fourth target embedding vector, and the second sentence feature corresponding to the third fusion feature to obtain a fourth fusion feature; Inputting all the fourth fusion features obtained into the event type classifier for classification processing to obtain the event type classification result of the priori text corresponding to each of the fourth fusion features; Determining an event detection loss according to an event type classification result of each of the fourth fused features and a real event type of a priori text corresponding to each of the fourth fused features; According to the sum of the entity recognition loss and the event detection loss, the parameters of the original Bert model are adjusted, and the adjusted Bert model is used as the trained Bert model.

6. The long text event extraction method according to claim 5, characterized in that: The adjusting the parameters of the original Bert model according to the sum of the entity recognition loss and the event detection loss, and using the adjusted Bert model as the trained Bert model, comprises: Determining whether the sum of the entity identification loss and the event detection loss is less than or equal to a preset loss value; If the sum of the entity recognition loss and the event detection loss is greater than the preset loss value, the parameters of the original Bert model are adjusted, and the Bert model after the parameter adjustment is used as the original Bert model, and the step of encoding each sentence obtained by the block using the original Bert model to obtain a second embedding vector for each sentence is returned to be executed; If the sum of the entity recognition loss and the event detection loss is less than or equal to the preset loss value, the original Bert model is used as the trained Bert model.

7. The long text event extraction method according to claim 4, characterized in that: The step of determining the target priori text having the highest similarity to the long text to be extracted from the pre-acquired event extraction data set in the field of cultivated land protection includes: For each priori text in the event extraction data set, respectively, according to all first embedding vectors corresponding to the long text to be extracted and all second embedding vectors corresponding to the priori text, calculate the similarity value between the long text to be extracted and the priori text; The priori text corresponding to the similarity value with the largest value among all the calculated similarity values ​​is used as the target priori text with the highest similarity to the long text to be extracted.

Images