Semi-automatic requirement extraction method based on pre-trained language fine-tuning and dependency features

Abstract

The present invention discloses a semi-automatic requirement extraction method based on pre-trained language fine-tuning and dependency features, comprising the following steps: pre-processing, entity extraction, entity fusion confirmation, intent extraction, intent fusion confirmation, subject relationship post-processing, and output modeling. The semi-automatic requirement extraction method provided by the present invention combines the advantages of the pre-trained language fine-tuning model and the dependency analysis feature: on the one hand, rules are designed for the domain problems of software requirement modeling, and the interpretability and reliability of the system are improved through domain knowledge; on the other hand, the convenience of generalization of the pre-trained language model is used for appropriate fine-tuning, without paying for additional large-scale data set annotation training for the accuracy premium.

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction in natural language processing, and in particular to a semi-automatic requirement extraction method based on a pre-trained language fine-tuning model and dependency syntactic feature analysis, which is suitable for the analysis process of iStar target modeling of English software requirement documents described in natural language. Background Art

[0002] In the current software engineering field, software requirements are the focus of software project stakeholders. With the continuous development of the information technology industry, the size of software projects is increasing, and complex and huge requirements are systematically collected, analyzed and managed, and software requirements engineering came into being. As one of the most important and complex links in software requirements engineering, requirements modeling should be able to clearly define the primary and secondary requirements information in the software project from a high-level perspective, so that stakeholders of the software project can easily communicate and verify the requirements information, so that developers can accurately and efficiently develop software projects.

[0003] Among many modeling methods, the i* (iStar) target modeling framework, as an agent-oriented target modeling framework, has attracted attention in the field of demand modeling for its high efficiency and practicality. iStar is an ontology-based feature modeling framework, which contains two views, namely, the policy dependency view and the policy interpretation view. In the policy dependency model, the basic content is the relationship between actors, which is mainly used to describe the dependency relationship formed between different actors on the intention elements in a specific organizational context, expressing dependency, generalization and subordination. The concept of subject can be further extended to agent and role in the iStar framework: the former represents a participant with specific physical manifestations, such as a specific human individual or organization; the latter represents an abstract description of a social actor in a specific context, such as a certain profession. In the policy interpretation view, the basic content is the internal elements in the policy dependency view, which mainly describes the internal intentions of the participants and the related situations in which they are constrained by the system and environment. The intention element represents what the participant wants, such as goal, task, quality, resource, etc.

[0004] In the process of building a model based on the requirement text described in natural language, the requirement analyst needs to have relevant domain knowledge and be able to manually extract requirements from the text and convert them into the form specified by the model. In the target modeling framework iStar, the modeler needs to extract the corresponding subject and the intention elements representing the task and resource from the natural language requirement text, and extract the dependency, generalization and subordination between the subjects, and finally generate the model. With the increase of software scale, due to the complexity of the requirements and modeling domain tasks themselves, the corresponding model will also become large and complex. Complete manual modeling is not only difficult but also very time-consuming. However, the relationship between the models is very close, and one move affects the whole body; the fully automatic process extracts all model elements at the same time, and cannot timely consider the feedback of the requirement analyst during the modeling process, and the effect is not satisfactory. Therefore, a semi-automatic modeling method that can assist the requirement modeler in extracting requirements for a specific modeling framework is particularly important.

[0005] Information extraction from natural language text is an important research and application field in natural language processing technology. Its main purpose is to convert semi-structured or unstructured natural language text into structured data for subsequent processing. With the continuous improvement of information technology, information extraction from natural language text is also an important means to improve field efficiency. Information extraction in natural language processing technology usually adopts template-based information extraction methods, statistical machine learning-based information extraction methods, and deep learning-based information extraction methods. Template-based methods can be divided into keyword-based information extraction methods and dependency syntax analysis-based information extraction methods. Keyword-based information extraction methods require the definition of regular expressions or keyword sequences, and use algorithms to search for given texts to extract sentences or keyword information that meet the requirements. Dependency syntax-based information extraction processes texts through a dependency syntax analyzer as an infrastructure to generate a corresponding dependency syntax tree, and matches the corresponding rules on the tree to find the information to be extracted that meets the requirements. Dependency analysis is to traverse the syntactic structure by analyzing the dependency relationship between components within a language unit; the template-based method has a good accuracy rate, can be better customized for specific fields, and can also be implemented on small-scale data sets; the disadvantage is that it has poor portability and rule customization usually requires certain domain knowledge. Information extraction methods based on statistical machine learning are usually built based on classification models. It is necessary to pre-define the data label content to be extracted, manually annotate a large amount of information, and divide the annotated data into training sets, validation sets, and test sets, and select and train classifiers. After training, the trained classifier is used to perform relevant reasoning on information extraction on unknown data sets. The advantage is that it has relatively good generalization and relatively good accuracy; the disadvantage is that it requires a large amount of data annotation and training adjustments.

[0006] In recent years, with the rapid development of hardware, deep learning technology has achieved good results in many fields, especially in the field of natural language. The more typical methods in the early days were based on recurrent neural networks and text convolutional neural networks. The method that has attracted widespread attention recently is the deep learning technology based on pre-trained language models. Among them, the most famous BERT model uses MaskLM and context prediction joint training to learn language features as pre-training weights in a large amount of unsupervised corpus through a Transformer decoder with 12 layers, 12 heads and 768 hidden elements, and has achieved excellent accuracy. However, the shortcomings of the BERT model in engineering research are also prominent. The training of the BERT model is time-consuming and requires a lot of hardware computing power support; the interpretability of deep learning is poor, resulting in the relatively opaque rules captured by BERT; in order to improve training performance, a large amount of corpus data needs to be pre-annotated for fine-tuning training. Therefore, for domain-specific tasks with fewer data sets, the full use of pre-trained language models has cost and efficiency concerns. Summary of the invention

[0007] Aiming at the natural language extraction method required for requirement modeling in software engineering, especially the requirement extraction for iStar target model modeling, in order to solve the above-mentioned problems existing in the prior art, the present invention provides a semi-automatic requirement extraction method for English documents based on the fusion of a pre-trained language fine-tuning model and dependency syntactic features, which is suitable for the analysis process of English software requirement documents described in natural language for iStar target model modeling.

[0008] The present invention comprises the following steps:

[0009] Preprocessing step: preprocess the document input by the user to generate a natural language demand statement as the output of this step for use in subsequent steps;

[0010] The entity extraction step uses the processed demand statement output in the previous step as input, and aims to extract the subject and resources of the iStar target modeling framework: the demand statement obtained after the input preprocessing step is used. On the one hand, the entity information is sequenced and labeled based on the pre-trained language model. On the other hand, the heuristic rule matching method is applied to the dependency syntactic features to extract nominal entities, and the respective preliminary pending entity information sequences are generated as the output of this step.

[0011] Entity fusion confirmation step, this step takes the two preliminary pending entity information sequences output in the previous step as input, the purpose is to fuse the two results, and confirm the final entity through user feedback. This step uses the heuristic rule matching method to input the preliminary pending reasoning results of the pre-trained model output in the entity extraction step and the preliminary pending extraction results of the dependent features, fuses the reasoning results, and outputs the final pending entity information sequence. The user adds, deletes, modifies, and confirms the pending results to form a confirmed entity extraction result as the output of this step.

[0012] The intention extraction step takes the processed requirement statement output by the preprocessing step as input, and aims to extract the dependencies between tasks and subjects in the iStar target modeling framework: input the processed requirement statement output by the preprocessing step, perform sequence labeling reasoning on the intent information based on the pre-trained language model, and generate a preliminary pending intent information sequence as the output of this step.

[0013] Intention fusion confirmation step, this step takes the preliminary pending reasoning results of the pre-trained model output in the previous step as input, with the purpose of integrating the pre-training results with the dependency analysis features, and confirming the final entity through user feedback. This step applies the heuristic rule matching method to perform dependency feature analysis on the preliminary pending reasoning results of the pre-trained model output in the intention extraction step, prunes the abnormal boundary features of the output sequence and matches the entity information sequence determined in the entity fusion confirmation step, and outputs the final pending intention information sequence. The user adds, deletes, modifies, and confirms the pending results to form a confirmed intention extraction result containing the agent, patient, and instrumental entity arguments as the output of this step.

[0014] The subject relationship post-processing step includes two steps: reference resolution and subject relationship. This step takes the information sequence determined in the entity fusion confirmation step and the intention fusion confirmation step as input, and its purpose is to extract the subject relationship of the iStar target modeling framework. Input the entity and intention information sequence determined in the entity fusion confirmation step and the intention fusion confirmation step. In the reference resolution stage, dependency analysis is applied to the entities that are included in the intention extraction results and whose part of speech is a pronoun to extract the corresponding referred information. The user adds, deletes, modifies, confirms the pending results, and replaces the corresponding pronouns. In the subject relationship confirmation stage, the keyword matching method is applied to the dependency feature analysis to retrieve the entities that are not included in the intention extraction results. The user adds, deletes, modifies, and confirms the pending results, and determines the generalization and inclusion relationship between the subjects as the output of this step.

[0015] The purpose of output modeling is to combine the products of the previous steps to generate the final result. The result of reference resolution obtained in the subject relationship post-processing step is input and applied to the intention extraction result obtained in the intention fusion confirmation step; the inter-subject relationship obtained in the subject relationship post-processing step is input and applied to the entity extraction result obtained in the entity fusion confirmation step. The final sequence result containing subjects, relationships, and detailed intention information is output for demand modeling.

[0016] Compared with the prior art, the semi-automatic requirement extraction method provided by the present invention combines the advantages of the pre-trained language fine-tuning model and the dependency analysis feature: on the one hand, rules are designed for the domain problems of software requirement modeling, and the interpretability and reliability of the system are improved through domain knowledge; on the other hand, the convenience of generalization of the pre-trained language model is utilized for appropriate fine-tuning without paying additional large-scale data set annotation training for the accuracy premium. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention.

[0018] Figure 2 It is a schematic diagram of the dependency analysis tree of the present invention.

[0019] Figure 3 Schematic diagram of the BERT pre-trained language model structure of the present invention.

[0020] Figure 4 Flow chart of the pretreatment steps of the present invention.

[0021] Figure 5 This is a flowchart of fine-tuning the pre-trained model of the present invention.

[0022] Figure 6 It is a flow chart of the entity extraction steps of the present invention.

[0023] Figure 7 It is a flow chart of the entity fusion confirmation steps of the present invention.

[0024] Figure 8 A flow chart of steps is extracted for the purpose of the present invention.

[0025] Fig. 9 A flow chart of the steps for merging the confirmation steps for the purposes of the present invention.

[0026] Fig.10 It is a flow chart of the main relationship post-processing steps of the present invention. DETAILED DESCRIPTION

[0027] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0028] like Figure 1 As shown in the figure, the semi-automatic requirement extraction method includes the following steps: preprocessing step, entity extraction step, entity fusion confirmation step, intention extraction step, intention fusion confirmation step, subject relationship post-processing step and output modeling. According to the components required by the iStar target model, the subject (Actor) and the resource (Resource) are processed in the entity extraction, the intention elements such as the task (Task) and the dependency relationship between the subject are processed in the intention extraction, and the generalization and subordination relationship between the subjects are processed in the entity post-processing step.

[0029] The dependency analysis and pre-trained language model used in the present invention are as follows: Figure 2 , Figure 3 shown.

[0030] In dependency analysis, the core verb of a sentence is the central component that controls other language components. The core verb itself is not controlled by any other component: all the controlled components are subordinate to the core verb in a certain dependency relationship. Figure 2 Take “Shopping system shall display the shopping cart during purchase.” as an example. The core verb is “display”, and its subject and object are both child nodes of the core verb.

[0031] BERT is the most famous pre-trained language model architecture. It uses a 12-layer, 12-head, 768-hidden-element Transformer decoder, and adopts Mask LM and context prediction joint training to learn language features as pre-training weights in a large amount of unsupervised corpus, achieving excellent accuracy. In this invention, since content extraction is required, the sequence labeling subtask setting is adopted, and a fully connected layer is added after the output layer of BERT. The output size is the number of categories required for BIO tags, see Figure 3 Indicates the content.

[0032] In the preprocessing step, the overall process is as follows Figure 4This step takes the English natural language requirement document as input and the preprocessed single sentence information as output. The user inputs the natural language requirement document in text format, and uses the spaCy natural language processing toolkit to perform the pipeline steps of sentence segmentation, word segmentation, part-of-speech tagging, and dependency analysis on the input user document, and outputs the preprocessed single sentence information set for subsequent processing.

[0033] After the preprocessing step, the entity extraction step is performed. This step takes the processed requirement sentence outputted in the previous step as input and generates the preliminary pending entity information sequence of the pre-trained model and the dependent features as the output of this step. However, it should be noted that the BERT model used in the entity extraction step has been fine-tuned in advance. The fine-tuning method of the pre-trained model is as follows: Figure 5 shown.

[0034] First, collect and annotate the dataset. Collect a small sample of software requirement specification documents and perform manual BIO sequence annotation in the form of sentences and words. Among them, B represents the beginning word of the entity or intent target; I represents the middle word or ending word of the entity or intent target; O represents a non-entity or intent target word. Generate entity corpus files and intent corpus files in CoNLL format for fine-tuning training.

[0035] Next, we use the annotated dataset to fine-tune the BERT model to make it suitable for domain tasks. For each sentence of input information, we add a [CLS] tag to the beginning of each sentence and a [SEP] tag to the end of each sentence. We input the annotated content into the BERT model with pre-trained weights, obtain the feature vectors H1, H2, ..., Hm of each word in each corpus, and output the corresponding label probability vector through the multi-classification fully connected layer and the Softmax layer. We calculate the loss based on the annotated file and perform back propagation to complete the fine-tuning.

[0036] Finally, record the weights of each layer and classifier of the fine-tuned BERT and save the corresponding model. Since the tasks of entity extraction and intent extraction are different, they are required to be fine-tuned separately, and finally two different fine-tuned BERT models are obtained. The entity extraction step is divided into two submodules, namely pre-trained entity extraction and dependency analysis entity extraction. The specific process is as follows Figure 6 shown.

[0037] The pre-trained model entity extraction module uses BERT reasoning in a similar way to the fine-tuning model steps of the pre-trained model BERT. The only difference is that the fine-tuning weights of the entity extraction task are used, and the gradient error is not back-propagated, so it will not be repeated. The dependency analysis entity extraction module uses the dependency analysis information generated in the preprocessing step for iterative processing. First, for the input content, search for the noun dependency types nsubj (noun subject), nsubjpass (noun passive subject), iobj (indirect object), dobj (direct slogan), and pobj (prepositional object) word by word to obtain the noun sequence.

[0038] For each noun in the sequence, loop through the dependency traversal rules:

[0039] If its child node is prep (preposition) or poss (possessive case), and the preposition refers to another noun, then the current target is expanded to the noun phrase containing the prepositional phrase;

[0040] If its child node is conj (conjunction) or appos (appositive), and there is a series of parallel nouns after the conjunction, then add the parallel nouns to the noun sequence;

[0041] If its child node is amod (adjective) or advmod (adverb), the current target is expanded to a noun phrase containing an adjective or adverb;

[0042] If its child node is a compound, the current noun target is expanded to a compound noun.

[0043] The loop rule is executed until the loop condition is not met, all nouns in the sequence complete the algorithm, and all target objects are output.

[0044] Take “Shopping system shall display the shopping cart during purchase.” as an example. The noun subject is “system”, the object is “cart”, the prepositional phrase is “purchase”; the compound word “shopping” is expanded; and finally there are entities “shopping system”, “shopping cart” and “purchase”.

[0045] In the entity fusion confirmation step, the entity extraction results of the pre-trained model and the entity extraction results of the dependency analysis are used as the input of the entity fusion confirmation step. The process is as follows: Figure 7As shown. This step uses the entity extraction results of the pre-trained model output in the previous step and the entity extraction results of the dependency analysis as inputs. The purpose is to fuse the two results and confirm the final entity through user feedback. For the two inputs, the same phrase may appear but the word boundaries are different. By setting the word threshold to 2, the difference in the index boundaries of the phrases is detected to determine whether there are the same phrases; in a more rigorous deployment method, the phrase structure tree analysis is used to determine whether the two are in the same clause, and then determine whether the phrases are the same.

[0046] For the same phrases (i.e., phrases shared by both), the word boundaries of dependency analysis are used as the results, and the classification output by the pre-trained model is used to distinguish whether the entity is an Actor or a Resource, and the results are marked as pre-selected options. For entity extraction results that have pre-trained but not dependent entities, or have dependencies but not pre-trained entities, they are sorted according to their corresponding probability, and the results with a probability greater than 20% are marked as alternative options.

[0047] Output the results of the pre-selected and alternative options, and feed them back to the user for confirmation through the front end. The user can add entities, delete entities, modify the boundaries and types of entities, and submit for confirmation.

[0048] Taking "Shopping system shall display the shopping cart during purchase." as an example, BERT outputs the Actor entity "shopping system" and the Resource entity "the shopping cart". By integrating the dependency analysis results, we get the Actor information "shopping system", the Resource entity "shopping cart", and the candidate entity "purchase". The user confirms and saves the entity information.

[0049] Perform the intention extraction step, the process is as follows Figure 8 As shown. This step takes the processed demand sentence output from the preprocessing step as input and generates a preliminary sequence of pending intent information of the pre-trained model as the output of this step. Intent extraction uses BERT reasoning in a similar way to the fine-tuning model step of the pre-trained model BERT. The only difference is that the fine-tuning weights of the intent extraction task are used, and the gradient error is not back-propagated, so it will not be repeated.

[0050] The intent extraction result is used as the input of the intent fusion confirmation step. The process is as follows Fig. 9 This step uses the preliminary pending inference results of the pre-trained model output in the previous step as input, with the goal of integrating the pre-training results with the dependency analysis features and confirming the final entity through user feedback.

[0051] Iterate the dependency analysis information of the input sentence. First, find the nsubj (noun subject) and nsubjpass (passive noun subject) dependency type subjects corresponding to the sentence and record them as the agent of the clause. Extract its head node as the core verb and traverse the child nodes of the core verb.

[0052] If its child node has prep (preposition) and is adjacent to the core verb, it means that the current verb is a verb phrase with a preposition, and the current core verb is expanded to a verb phrase containing a preposition.

[0053] If its child node has conj (conjunction), it means that there is more than one core verb, and the parallel verbs are also added to the circular queue.

[0054] If its child node has advmod (adverb) modifying the core verb, the related adverbial phrase is expanded as the target.

[0055] If its child nodes have dobj (direct object), iobj (indirect object) and pobj (prepositional object), the object is processed in the same way as the noun phrase described above, and is marked as the object part of the target or the instrumental argument.

[0056] If there are advcl (adverbial clause modification), acl (noun clause modification), pcomp (prepositional complement), ccomp (clause complement), and xcomp (open complement clause), it is declared that there is a clause after it, and the clause verb it points to is used as the new core verb for recursive search.

[0057] Finally, when the loop condition is not met, the process ends and a series of sentence components are output; the sentence components obtained by dependency analysis are integrated with the intent extraction results of the pre-trained model to query their existence.

[0058] If the intent extracted by the pre-trained model is included in the dependency analysis sentence components except the subject (except the object pointed to by the agent dependency relationship for passive subjects), and the patient and agent are both previously marked subjects, then mark the current item as a pre-option, align the boundary of the target word through the dependency component, and exclude the complement component. If the intent extracted by the pre-trained model is not included in the dependency component, or the patient and agent are not the subject, sort them according to their corresponding probability, and mark the results with a probability greater than 20% as alternatives. Output the results of the pre-options and alternatives and feedback them to the user through the front end. Users can add intents, delete intents, modify the boundaries of intents, and submit for confirmation.

[0059] The post-processing steps of the subject relationship are as follows: Fig.10As shown. This step takes the information sequence determined in the entity fusion confirmation step and the intention fusion confirmation step as input, with the purpose of extracting the subject relationship of the iStar target modeling framework. Input the entity fusion result to capture the appos (appositive) that appears in the dependency analysis; if one of the parties exists in the intention sequence result and is a pronoun, it is confirmed that there is a reference relationship, and the reference resolution is completed by text replacement. If it does not meet the requirements, there is a generalization relationship between the two subjects. For references using user story sentences, the keyword matching method is used to obtain the reference items, and the reference resolution is completed by text replacement. For generalization and inclusion relationships, the subject relationship is also extracted by keyword matching. Finally, the output is for the user to add and modify and confirm, and the reference resolution information and subject and relationship information are obtained.

[0060] Finally, the modeling is output and combined with the products of the previous steps to generate the final result. The reference resolution result obtained in the subject relationship post-processing step is applied to the intent extraction result obtained in the intent fusion confirmation step, and the subject relationship obtained in the subject relationship post-processing step is applied to the entity extraction result obtained in the entity fusion confirmation step. The final sequence result containing the subject and relationship and detailed intent information is output for demand modeling of the iStar target model.

Claims

1. A semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features, characterized in that: include: (1) Preprocessing step: preprocessing the document input by the user to generate a natural language demand statement with clauses; (2) Entity extraction step: extract the subject and resource intention of the iStar target modeling framework: input the demand statement to be processed, perform sequence annotation reasoning on the entity information based on the pre-trained language model, and apply the heuristic rule matching method to the dependency syntactic features to extract the nominal entity, and output the preliminary entity information sequence to be determined; (3) Entity fusion confirmation step: The preliminary pending reasoning results of the pre-trained model are fused with the preliminary pending extraction results of the dependent features through a heuristic rule matching method, and the final pending entity information sequence is output. The user adds, deletes, modifies, and confirms the pending results to form a confirmed entity extraction result; (4) Intent extraction step: extracting tasks and dependencies of the iStar target modeling framework: inputting the requirement statement to be processed, performing sequence labeling reasoning on the intent information based on the pre-trained language model, and outputting a preliminary sequence of intent information to be determined; (5) Intention fusion confirmation step: by applying the heuristic rule matching method to the preliminary pending reasoning results of the pre-trained model to perform dependency feature analysis, the abnormal boundary features of the output sequence are pruned and matched with the entity information sequence determined in the previous step, and the final pending intention information sequence is output. The user adds, deletes, modifies, and confirms the pending results to form a confirmed intention extraction result containing the agent, patient, and instrument entity arguments; (6) Subject relationship post-processing step, including reference resolution and subject relationship, is used to extract the subject relationship of the iStar target modeling framework; In the reference resolution stage, dependency analysis is applied to entities that are included in the intent extraction results and are pronouns to extract the corresponding referred information. The user adds, deletes, modifies, and confirms the pending results, and replaces the corresponding pronouns. In the subject relationship confirmation stage, the keyword matching method is applied to the dependency feature analysis to retrieve entities that are not included in the intent extraction results. The user adds, deletes, modifies, and confirms the pending results to determine the generalization and inclusion relationship between the subjects. (7) Output modeling: Apply the reference resolution result obtained in (6) to the intent extraction result obtained in (5), apply the subject relationship obtained in (6) to the entity extraction result obtained in (3), and output the final sequence result containing the subject relationship and detailed intent information for demand modeling.

2. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In step (1), the spaCy natural language processing toolkit is used to perform sentence segmentation, word segmentation, part-of-speech tagging, and dependency analysis on the input user document, and a collection of preprocessed single sentence information is output.

3. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In step (2) and step (4), the pre-trained language models used for inference are each pre-trained and fine-tuned; Training fine-tuning is divided into several steps: Marking stage; Collect software requirement specification documents, and manually annotate BIO sequences in the form of sentences and words; where B represents the start word of an entity or intended target; I represents the middle word or end word of an entity or intended target; O represents a non-entity or intended target word; generate a corpus file in CoNLL format for subsequent fine-tuning training; In the fine-tuning phase, the Transformer architecture pre-trained model BERT and pre-trained weights are used to add a [CLS] tag to the beginning of each sentence and a [SEP] tag to the end of the sentence. The annotated content is input into the pre-trained BERT model to obtain the embedding vectors of each word in the output sentence. The corresponding label probabilities are output through a multi-classification fully connected layer, and the loss is calculated and then back-propagated to complete the fine-tuning. In the saving stage, the weights of each layer of BERT and the classifier weights after fine-tuning are recorded and the corresponding model is saved. Due to the different directions of the training tasks in steps (2) and (4), the resulting model weights and classifier matrix sizes are different, and they are two similar but different pre-trained language models.

4. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In the heuristic rule matching method of step (2), by matching the noun subject and the direct object, indirect object, and noun component after the preposition in the dependency analysis, and iteratively querying its adjacent conjunctions, compound words, and adjective components, a complete noun entity information is constructed as a preliminary pending entity information sequence.

5. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In step (3), the fusion method compares the entity pending results of the pre-trained model with the entity pending results of the dependency analysis, and adopts the boundary of the dependency analysis for the common items between the two.

6. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In step (6), the generalized relationship between the two subjects in the relationship is determined by capturing the appositives that appear in the dependency analysis; if one of the parties exists in the intent sequence result and is a pronoun, then it is confirmed that there is a reference relationship, and the reference resolution is completed by text replacement; if it does not meet the requirements, the two subjects have a generalized relationship; for references using user story sentences, the keyword matching method is used to obtain the reference items, and the reference resolution is completed by text replacement; for generalization and inclusion relationships, the subject relationship is also extracted by keyword matching; finally, the output is provided for the user to add, modify and confirm, and the reference resolution information and subject relationship information are obtained.

7. The semi-automatic demand extraction method based on pre-trained language fine-tuning and dependency features as claimed in claim 1, characterized in that: In step (7), the coreference resolution result obtained in (6) is applied to the intent extraction result obtained in (5), and the subject relationship obtained in (6) is applied to the entity extraction result obtained in (3) to obtain the final result, which is output in the form of structured text for use in subsequent modeling processes.

Images