Method and system for automatically compiling user manual based on neural network model

Abstract

The invention relates to the technical field of generative large models, in particular to an automatic user manual compiling method and system based on a neural network model, and the method comprises the steps: setting a user manual template and a semantic extraction model based on a BERT model and a CRF neural network, carrying out the training and fine tuning of the semantic extraction model, receiving development document data through the semantic extraction model, and storing the development document data in a database; and extracting entity information related to the user manual in the development document, and filling the entity information into the user manual template to generate a complete user manual. A phrase replacement position and a text replacement position are set in a user manual template, a semantic extraction model based on a BERT model and a CRF neural network is utilized, entity information in one-to-one matching with the corresponding positions in a development document is combined with original auxiliary fields, complete natural language paragraphs corresponding to all content modules of the user manual are automatically generated, and then the user manual is completed. The time of developers is saved, and the compilation efficiency of the user manual is improved.

Description

Technical Field

[0001] This application belongs to the technical field of generative large models, and specifically relates to a method and system for automatically compiling user manuals based on neural network models. Background Art

[0002] In the highly competitive software development industry, efficiency and quality are the keys for enterprises to survive. As an important bridge connecting products and users, the optimization of the compilation process of software user manuals is crucial for software development enterprises. The writing of software user manuals requires a certain degree of professionalism and is difficult for non-professionals. Professional personnel need to be involved in writing word by word. From function descriptions to operation steps, the process is cumbersome and error-prone, and it also consumes a lot of manpower and time. Improving the efficiency and accuracy of writing software user manuals and saving the time of developers are technical problems that the industry urgently needs to solve. Summary of the Invention

[0003] This application provides a method and system for automatically compiling user manuals based on neural network models to solve or partially solve the problems raised in the above background art.

[0004] This application provides a method for automatically compiling user manuals based on neural network models, including the following steps:

[0005] S1: Set a user manual template and a semantic extraction model based on the BERT model plus the CRF neural network, and perform training and fine-tuning on the semantic extraction model;

[0006] S2: The semantic extraction model receives development document data, extracts entity information related to the user manual in the development document, and fills the entity information into the user manual template to generate a complete user manual;

[0007] Set phrase replacement positions and text replacement positions in the user manual template, and use the semantic extraction model to extract entity information that matches the corresponding positions one by one.

[0008] Preferably, in the step S1, the specific method for setting the user manual template is as follows:

[0009] Mark the phrases to be replaced in each phrase replacement position and the continuous text to be replaced in the text replacement position as fields to be filled with examples, and the remaining content in each paragraph as auxiliary fields.

[0010] Set semantic discrimination labels for each field to be filled with examples.

[0011] Preferably, in the step S1, the training and fine-tuning method of the semantic extraction model includes:

[0012] Define entity types customarily based on the fields to be filled with examples in the user manual template, specifically as follows:

[0013] Define the phrase feature entity type and the paragraph feature entity type, corresponding to the phrase content and text content required by the user manual respectively;

[0014] For the phrase feature entity type, add qualified feature words to its actual entity type;

[0015] For the paragraph feature entity type, set a tag summary with a limited number of words as the entity type name, and split the paragraph feature entity with a word count greater than the preset word count M into multiple paragraph sub-entities and define the relationship between the sub-entities, where M is the preset word count threshold.

[0016] Preferably, in step S2, the specific method for the semantic extraction model to extract entity information related to the user manual in the development document is as follows:

[0017] Preprocess the input text, construct a complete input sequence, and input the input sequence into the BERT model;

[0018] Through the multi-layer Transformer structure of the BERT model, perform deep feature extraction on the input sequence to obtain the semantic representation vector of each group of Tokens;

[0019] Use the CRF layer to perform label prediction on the semantic representation vector of the Tokens, calculate the probability distribution of each Token belonging to different entity categories, and select the entity category with the highest probability;

[0020] S205: Screen the sub-entities of the paragraph feature entity, calculate the relationship score between the sub-entities, extract the entity pairs with the sub-entity spacing ≤ N in the context, perform vector splicing and MLP processing on the entity pairs, and calculate the score of each relationship between the entities:

[0021]

[0022] In the formula, r represents the relationship type, ei and ej are entity representation vectors, Context is the BERT encoding of the text between the two entities, represents vector splicing;

[0023] For the score Perform Softmax normalization and take the relationship type with the highest probability;

[0024] Merge the paragraph sub-entities according to the recognized relationships, and finally output the paragraph feature entity and the phrase feature entity, where N is the preset sub-entity quantity threshold.

[0025] Preferably, an entity auxiliary retrieval model parallel to the CRF neural network is set in the semantic extraction model to cooperate with the CRF neural network to identify the paragraph feature entity and its sub-entities. The specific method for entity recognition by the entity auxiliary retrieval model is as follows:

[0026] Establish a mapping dictionary D between the semantic vectors transformed by BERT in the training set and the custom entity types.

[0027]

[0028] Among them, Si represents the semantic vector, yi represents the entity type, and N represents the total number of entity types.

[0029] Input the semantic representation vector into the entity auxiliary retrieval model, and use the cosine similarity to screen the Top-K nearest neighbors of the vector.

[0030] ,

[0031] Among them, Nt represents the nearest neighbor set, K represents the number of nearest neighbors, rank is the sorting function from high to low, St is the input vector, and (Sj, yj) is the key-value pair in D.

[0032] Use the radial basis function RBF to assign similarity weights to the nearest neighbor set to generate the entity type probability distribution Pknn.

[0033] ,

[0034] Among them, Xt is the input sample, C is the entity type, σ is the bandwidth parameter in RBF, its default value is 0.5, and its function is to control the influence intensity of the weights of neighboring samples on the final vote. The smaller σ is, the faster the weights decay with the increase of distance. Adjust the value of σ through pre-training. I is the indicator function, taking 1 when the condition is satisfied, otherwise 0.

[0035] Preferably, input the semantic representation vector output by the BERT model into the CRF neural network and the entity auxiliary retrieval model at the same time, obtain the entity type probability distribution Pcrf based on CRF and the entity type probability distribution Pknn based on the entity auxiliary retrieval model respectively, and combine the two probability distributions through weighted voting to generate the final probability distribution. The specific formula is as follows:

[0036] ,

[0037] Among them, yt represents the entity type, λt represents the weight coefficient, and select the entity type with the highest probability.

[0038] Preferably, in step S2, after obtaining the paragraph feature entity and the phrase feature entity, the specific method of filling each entity into the user manual template is as follows:

[0039] For the phrase feature entity, perform semantic matching on the entity type of the phrase feature entity and the semantic discrimination labels of each phrase-to-be-filled example field, and fill and cover the phrase feature entity to the corresponding position.

[0040] For the paragraph feature entity, semantically match its tag summary with the semantic discrimination tags of each text-to-be-filled example field, and fill the paragraph feature entity into the corresponding position.

[0041] Preferably, when multiple paragraph feature entities of the same type are recognized, compare the similarity between the text content of the to-be-filled example field corresponding to the semantic discrimination tag and the multiple feature entities, and select the feature entity with the highest similarity to match the corresponding filling position.

[0042] Preferably, the content modules of the user manual template include a cover page, a preface, installation and configuration, function use cases, interface introduction, and data management.

[0043] This application also provides a user manual automatic compilation system based on a neural network model, including: a model construction module, an entity extraction module, and a matching and insertion module;

[0044] The model construction module generates a semantic extraction model based on the BERT model plus the CRF neural network, sets an entity auxiliary retrieval model parallel to the CRF neural network, receives software document data for training, performs BIO annotation on the software document data using an open-source annotation tool, divides the annotated data into a training set, a validation set, and a test set, and completes the construction and training of the semantic extraction model;

[0045] The semantic extraction module uses the semantic extraction model to receive target development document data and extract entity information related to the user manual in the development document;

[0046] The matching and insertion module matches and fills the recognized entity information with the to-be-filled example fields in the user manual template one by one to generate a complete user manual.

[0047] Compared with the prior art, the beneficial effects of this application are as follows:

[0048] (1) This application sets phrase replacement positions and text replacement positions in the user manual template, uses a semantic extraction model based on the BERT model plus the CRF neural network to match entity information corresponding to the corresponding positions in the development document, and combines it with the original auxiliary fields to automatically generate complete natural language paragraphs corresponding to each content module of the user manual, saving time for developers and improving the compilation efficiency of the user manual.

[0049] (2) This application divides two types of custom entities based on the word count to fill the user manual template, adapts to the setting structure of the auxiliary content plus professional content of the user manual, decomposes the entity extraction task, and reduces the difficulty of entity extraction.

[0050] (3) This application sets up an entity-assisted retrieval model parallel to CRF based on a clustering algorithm, which helps to overcome the training data density problem caused by different software specificities and improves the recognition accuracy of paragraph feature entities. Description of the Drawings

[0051] The following further describes this application with reference to the drawings and embodiments.

[0052] Figure 1 It is a schematic diagram of the system composition of this application.

[0053] Figure 2 It is a schematic diagram of the method flow of this application. Specific Embodiments

[0054] As used in the specification and claims, certain terms are used to refer to specific components. Those skilled in the art should understand that hardware manufacturers may use different terms to refer to the same component. The specification and claims do not use the difference in names as a way to distinguish components, but use the difference in functions of components as the criterion for distinction. As mentioned throughout the specification and claims, "comprising" is an open-ended term and should be interpreted as "including but not limited to". "Substantially" means within an acceptable error range. Those skilled in the art can solve the technical problem within a certain error range and basically achieve the technical effect.

[0055] In the description of this application, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "horizontal", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0056] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] Embodiment 1

[0058] As Figures 1 to 2 shown, this application provides an automatic user manual compilation method based on a neural network model, and the specific steps are as follows:

[0059] S1: Set the user manual template and the semantic extraction model based on the BERT model plus the CRF neural network, and train and fine-tune the semantic extraction model.

[0060] S2: The semantic extraction model receives the development document data, extracts the entity information related to the user manual in the development document, and fills the entity information into the user manual template to generate a complete user manual.

[0061] Specifically, the development documents are various types of document data in the development, testing, and practical processes.

[0062] Specifically, the BERT model, that is, the Bidirectional Encoder Representations from Transformers, is a pre-trained language model that can capture bidirectional semantic information of text and performs well in natural language processing tasks such as question answering, text classification, and semantic similarity comparison. The CRF, that is, the Conditional Random Field, is an undirected graph model commonly used in sequence labeling tasks. It can consider context information and improve the labeling accuracy in part-of-speech tagging, named entity recognition, etc.

[0063] In practice, a software user manual generally includes the following content:

[0064] Cover: It contains information such as the software name, version number, manual title, release date, copyright statement, etc., such as "XX Office Software V1.0 User Manual", giving users a preliminary overall understanding of the software and the manual.

[0065] Preface: It includes the writing purpose, applicable objects, and software overview, briefly introducing the applicable population of the manual and the software functions, features, and application scenarios, etc.

[0066] Installation and Configuration: The software and hardware environments required for software installation and the installation and configuration instructions.

[0067] Function Use Cases: It includes function module division, function description, and operation process.

[0068] Interface Introduction: Introduce the overall interface layout of the software, such as the menu bar, toolbar, work area, status bar, etc.

[0069] Data Management: Data input, storage, backup, and recovery.

[0070] Some user manuals also include troubleshooting, shortcut key lists, etc.

[0071] Based on the content modules (cover, preface, function use cases, etc.) of the above software user manual, form and set the user manual template, and adjust the content modules of the template according to the software type to be generated. The specific setting method is as follows:

[0072] S101: Set the phrase replacement positions and text replacement positions in the natural language paragraphs corresponding to each content module.

[0073] S102: Mark the phrases to be replaced in each phrase replacement position and the continuous text to be replaced in the text replacement position as fields to be filled with examples, and the remaining content in each paragraph as auxiliary fields;

[0074] S103: Set semantic discrimination labels for each field to be filled with examples.

[0075] The phrase replacement positions and text replacement positions are classified and set based on the number of characters of the fields to be filled with examples to be replaced. Different placeholders can be added at the beginning and end of the phrase replacement positions and text replacement positions for distinction, and the placeholders are removed after filling.

[0076] In the user manual, the importance, professionalism, and generality of each content module are different. For example, each content paragraph in the cover and preface has a relatively high consistency in description in different user manuals, and its role in the user manual is also auxiliary. Only some phrases (such as the customer company name, software name, etc.) need to be simply replaced to achieve the reuse of the content paragraph. On the contrary, for content paragraphs with higher professionalism, such as function use cases, data storage, etc., the differences in user manuals of different software are relatively large, and the generalizable description content is relatively small. Text replacement positions need to be set to achieve accurate description. Obviously, for each auxiliary paragraph in the cover and preface, several phrase replacement positions are set, and almost no text replacement positions are set, and the proportion of auxiliary fields in the paragraph is relatively high. For professional paragraphs such as function use cases, text replacement positions need to be set, and the proportion of auxiliary fields in the paragraph is relatively low.

[0077] Each phrase replacement position and text replacement position is the insertion position where the entity information to be extracted by the semantic extraction model is located, and its specific content is the example description at the corresponding position of the same type of software. For example, at the field to be filled with an example corresponding to the customer company name phrase, it is "xxxx Company". The setting of the semantic discrimination label is for the one-to-one matching of the entity information and the field to be filled with an example, and its content is the semantic feature description of the entity at the replacement position.

[0078] Use the software document data of relevant types to pre-train the semantic extraction model based on the neural network, so that the module can better understand the professional terms and context information in the software field. The specific method for training and fine-tuning the semantic extraction model is as follows:

[0079] S111: Customize entity types based on the content to be filled (fields to be filled with examples) in the user manual template, specifically as follows:

[0080] Define the phrase feature entity type and the paragraph feature entity type, corresponding to the phrase content and text content required by the user manual respectively;

[0081] For the phrase feature entity type, add a qualifying feature word to its actual entity type;

[0082] For the paragraph feature entity type, set a tag summary with a limited number of words as the entity type name, and split the paragraph feature entity with a word count greater than the preset word count M into multiple paragraph sub-entities and define the relationship between the sub-entities;

[0083] S112: Obtain a software document dataset, perform BIO annotation on the software document data using an open-source annotation tool, divide the annotated data into a training set, a validation set, and a test set, and set a validation accuracy threshold and a prediction accuracy threshold;

[0084] S113: Use the training set data to train the semantic extraction model;

[0085] S114: Use the validation set data to evaluate the recognition accuracy of entities and relationships by the document parsing model. If the recognition accuracy is greater than the validation accuracy, execute step S115; otherwise, return to execute step S113;

[0086] S115: Use the test set data to test the semantic extraction model. If the recognition accuracy of entities and relationships is less than the prediction accuracy threshold, return to step S113 and adjust the hyperparameters of the semantic extraction model; otherwise, end the training.

[0087] In step S111, the phrase feature entity type refers to various phrase vocabularies that need to be filled in the user manual template, such as customer company names, developers, shortcut key combinations, software names, special function module names, development software or platform names, various open-source algorithm modules, and interface names, etc. Adding a qualifying feature word to the phrase feature entity means adding a modifier to the general phrase entity type name to narrow its type range. For example, the format of the "developer name" entity is "developer + name", and the format of the "customer company" entity is "customer + company name"; the paragraph feature entity type refers to the various text filling contents required in the user manual template, such as the login process paragraph in the function use case, etc. Extracting a title summary for it is to name the paragraph entity, which is convenient for filling the recognized entities into the user manual template. Performing sub-entity division on it is to reduce the recognition difficulty of complex entities and adapt to the input length limit of the BERT model (usually 512 Tokens, equivalent to a content length of 300 to 500 Chinese characters). Setting the relationship between sub-entities is to clarify the nested relationship between sub-entities and paragraph entities and the sorting of sub-entities combined into paragraph entities, which is convenient for combining sub-entities into paragraph feature entities after recognizing sub-entities.

[0088] Specifically, in step S2, the specific method for the semantic extraction model to extract entity information related to the user manual in the development document is as follows:

[0089] S201: Preprocess the input text, split the software development document into sentences or paragraphs to ensure that each segment fits the input length limit of the BERT model (usually 512 tokens), tokenize the segmented text using the BERT Tokenizer, and convert the text into a token sequence acceptable to the model;

[0090] S202: Model input. First, construct the input sequence. Add the special token [CLS] (classification token) at the beginning of the token sequence and the [SEP] (separator) token between sentences to construct a complete input sequence, and then send the constructed input sequence into the pre-trained BERT model;

[0091] S203: Obtain semantic representation vectors. First, perform feature extraction. Through the multi-layer Transformer structure of the BERT model, perform in-depth feature extraction on the input sequence to obtain the semantic representation vectors of each group of tokens. Utilize the bidirectional encoding ability of BERT so that the semantic representation vectors of each group of tokens incorporate their context information;

[0092] S204: Conditional random field (CRF) decoding. Use the CRF layer to predict the labels of the semantic representation vectors of the tokens, considering the dependencies between the labels to obtain the globally optimal label sequence. According to the CRF decoding result, calculate the probability distribution of each token belonging to different entity categories and select the entity category with the highest probability;

[0093] S205: Screen the sub-entities of the paragraph feature entities, calculate the relationship scores between the sub-entities, extract entity pairs with a distance ≤ N between sub-entities in the context, perform vector concatenation and MLP processing on the entity pairs, and calculate the scores of each relationship between the entities:

[0094]

[0095] In the formula, r represents the relationship type, ei and ej are entity representation vectors, and Context is the BERT encoding of the text between the two entities. represents vector concatenation;

[0096] S206: Perform Softmax normalization on the scores and select the relationship type with the highest probability;

[0097] S207: Merge the paragraph sub-entities according to the identified relationships, and finally output the paragraph feature entities and phrase feature entities.

[0098] M and N are preset thresholds.

[0099] Specifically, after obtaining the paragraph feature entities and phrase feature entities, each entity is filled into the user manual template. For the phrase feature entities, the entity type of the phrase feature entities (including the defining feature words) is semantically matched with the semantic discrimination labels of each phrase-to-be-filled example field, and the phrase feature entities are filled and overwritten to the corresponding positions; for the paragraph feature entities, their label summaries are semantically matched with the semantic discrimination labels of each text-to-be-filled example field, and the paragraph feature entities are filled into the corresponding positions.

[0100] Through the setting of semantic tags in this application, the ideal state is to achieve a one-to-one correspondence between the entities and each to-be-filled example field. To achieve this purpose, it is only necessary to set the defining feature words of the phrase feature entities and the label summaries of the paragraph feature entities to be consistent with the corresponding semantic discrimination labels. However, due to the complexity of the paragraph feature entities, multiple paragraph feature entities of the same type may be recognized. In this regard, as a preferred technical solution, when this situation occurs, the text content of the to-be-filled example field corresponding to the semantic discrimination label and multiple feature entities is compared for similarity, and the feature entity with the highest similarity is selected to match the corresponding filling position.

[0101] The specific algorithm for similarity comparison can utilize existing algorithm modules. Multiple algorithm modules including the BERT model can perform similarity comparison operations, which will not be elaborated here.

[0102] Embodiment 2

[0103] This embodiment is based on Embodiment 1. In Embodiment 1, this application uses software development document data to train and fine-tune the semantic extraction model based on the BERT model and the CRF neural network, enabling it to recognize the paragraph feature entities and their sub-entities in the user manual template. Due to the high complexity of the paragraph feature entities and the specificity of the functions between different software, there is less data for the specific software function paragraphs, and the semantic extraction model is prone to the long-tail problem, that is, the recognition performance for high-frequency common entities (head data) is good, but the recognition ability for low-frequency and diverse entities (tail data) significantly decreases.

[0104] Based on the above problems, this application sets up an entity auxiliary retrieval model parallel to the CRF neural network in the semantic extraction model to cooperate with the CRF neural network to identify the paragraph feature entities and their sub-entities. The specific method is as follows:

[0105] Establish a mapping dictionary D between the semantic vectors transformed by BERT in the training set and the custom entity types.

[0106]

[0107] Among them, Si represents the semantic vector, yi represents the entity type, and N represents the total number of entity types.

[0108] When extracting using the semantic extraction model, in step S204, the semantic representation vectors of each group of Tokens are simultaneously input into the CRF neural network and the entity-assisted retrieval model to obtain their entity type probability distributions P crf and the entity type probability distribution P knn based on the entity-assisted retrieval model. The two probability distributions are combined to generate the final probability distribution based on weighted voting. The specific formula is as follows:

[0109] ,

[0110] where y t represents the entity type, λ t represents the weight coefficient, and the entity type with the highest probability is selected;

[0111] Input the semantic representation vector into the entity-assisted retrieval model, and use cosine similarity to screen the Top-K nearest neighbors of the vector,

[0112] ,

[0113] where N t represents the nearest neighbor set, K represents the number of nearest neighbors, rank is the sorting function from high to low, S t is the input vector, and (S j ,y j ) is the key-value pair in D;

[0114] Use the radial basis function RBF to assign similarity weights to the nearest neighbor set to generate the entity type probability distribution P knn ,

[0115] ,

[0116] where Xt is the input sample, C is the entity type, σ is the bandwidth parameter in RBF, and its default value is 0.5. Its function is to control the influence intensity of the weights of neighboring samples on the final vote. The smaller σ is, the faster the weights decay with the increase of distance. The value of σ is adjusted through pre-training. I is the indicator function, taking 1 when the condition is satisfied and 0 otherwise.

[0117] Example 3

[0118] Based on Example 1 and Example 2, this example provides a user manual automatic compilation system based on a neural network model, which can implement the user manual automatic compilation systems in Example 1 and Example 2, including a model construction module, an entity extraction module, and a matching and insertion module.

[0119] A model construction module generates a semantic extraction model based on a BERT model plus a CRF neural network, sets up an entity-assisted retrieval model parallel to the CRF neural network, receives software document data for training, performs BIO annotation on the software document data using an open-source annotation tool, divides the annotated data into a training set, a validation set, and a test set, and completes the construction and training of the semantic extraction model;

[0120] A semantic extraction module uses the semantic extraction model to receive target development document data and extracts entity information related to the user manual in the development document;

[0121] A matching and insertion module matches and fills the identified entity information with the to-be-filled example fields in the user manual template one by one to generate a complete user manual.

[0122] The embodiments of the present application have been described in detail above with reference to the accompanying drawings. However, the present application is not limited to the above embodiments. Within the knowledge scope of those of ordinary skill in the art, various changes can be made without departing from the purpose of the present application.

Claims

1. A method for automatically compiling a user manual based on a neural network model, characterized in that: The steps include: S1: Set up a user manual template and a semantic extraction model based on the BERT model plus the CRF neural network, and train and fine-tune the semantic extraction model; S2: The semantic extraction model receives the development document data, extracts the entity information related to the user manual in the development document, and fills the entity information into the user manual template to generate a complete user manual; The phrase replacement positions and text replacement positions are set in the user manual template, and the semantic extraction model is used to extract entity information that matches the corresponding positions one by one.

2. The method for automatically compiling a user manual based on a neural network model according to claim 1, characterized in that: In step S1, the specific method of setting the user manual template is as follows: Mark the phrases to be replaced in each phrase replacement position and the continuous text to be replaced in the text replacement position as sample fields to be filled, and the remaining contents in each paragraph as auxiliary fields; Sets a semantic identification label for each example field to be filled.

3. The method for automatically compiling a user manual based on a neural network model according to claim 2, characterized in that: In step S1, the training and fine-tuning method of the semantic extraction model includes: Customize the entity type based on the example fields to be filled in the user manual template, as follows: Define phrase feature entity types and paragraph feature entity types, corresponding to the phrase content and text content required by the user manual respectively; For phrase feature entity types, add qualifying feature words to their actual entity types; For paragraph feature entity types, a label summary with a limited number of words is set as the entity type name, and paragraph feature entities with a word count greater than a preset word count M are split into multiple paragraph sub-entities and the relationship between the sub-entities is defined, where M is a preset word count threshold.

4. The method for automatically compiling a user manual based on a neural network model according to claim 3, characterized in that: In step S2, the specific method of extracting entity information related to the user manual in the development document by the semantic extraction model is as follows: Preprocess the input text, construct a complete input sequence, and input the input sequence into the BERT model; Through the multi-layer Transformer structure of the BERT model, deep feature extraction is performed on the input sequence to obtain the semantic representation vector of each group of tokens; Use the CRF layer to predict the label of the semantic representation vector of the token, calculate the probability distribution of each token belonging to different entity categories, and select the entity category with the highest probability; S205: Filter the sub-entities of the paragraph feature entity, calculate the relationship scores between the sub-entities, extract entity pairs with a sub-entity distance ≤ N in the context, perform vector concatenation and MLP processing on the entity pairs, and calculate the score of each relationship between the entities: In the formula, r represents the relationship type, ei and ej are entity representation vectors, and Context is the BERT encoding of the text between two entities. Represents vector concatenation; Score Perform Softmax normalization and take the highest probability relationship type; The paragraph sub-entities are merged according to the identified relationships, and finally the paragraph feature entities and phrase feature entities are output, where N is the preset sub-entity number threshold.

5. The method for automatically compiling a user manual based on a neural network model according to claim 4, characterized in that: In the semantic extraction model, an entity-assisted retrieval model is set in parallel with the CRF neural network to cooperate with the CRF neural network to identify paragraph feature entities and their sub-entities. The specific method of entity recognition of the entity-assisted retrieval model is as follows: A mapping dictionary D is established between the semantic vectors transformed by BERT in the training set and the custom entity type. Among them, Si represents the semantic vector, yi represents the entity type, and N represents the total number of entity types; The semantic representation vector is input into the entity-assisted retrieval model, and the top-K neighbors of the vector are screened using cosine similarity. ， Where Nt represents the neighbor set, K represents the number of neighbors, rank is the sorting function from high to low, St is the input vector, (Sj, yj) is the key-value pair in D; Use radial basis function RBF to assign similarity weights to neighbor sets to generate entity type probability distribution Pknn. ， Among them, Xt is the input sample, C is the entity type, ‌σ is the bandwidth parameter in RBF, and its default value is 0.

5. Its function is to control the influence of the weight of neighboring samples on the final vote. The smaller σ is, the faster the weight decays with increasing distance. The value of σ is adjusted through pre-training. I is the indicator function, which takes 1 when the condition is met and 0 otherwise.

6. The method for automatically compiling a user manual based on a neural network model according to claim 5, characterized in that: The semantic representation vector output by the BERT model is input into the CRF neural network and the entity-assisted retrieval model at the same time, and its entity type probability distribution Pcrf based on CRF and entity type probability distribution Pknn based on the entity-assisted retrieval model are obtained respectively. The two probability distributions are combined based on weighted voting to generate the final probability distribution. The specific formula is as follows: ， Among them, y t Represents the entity type, λt represents the weight coefficient, and the entity type with the highest probability is selected.

7. The method for automatically compiling a user manual based on a neural network model according to claim 4, characterized in that: In step S2, after obtaining the paragraph feature entity and the phrase feature entity, the specific method of filling each entity into the user manual template is: For phrase feature entities, semantic matching is performed between the entity type of the phrase feature entity and the semantic identification label of each phrase-to-be-filled example field, and the phrase feature entity is filled and covered to the corresponding position; For paragraph feature entities, semantic matching is performed between their label summaries and the semantic identification labels of each text-to-be-filled example field, and the paragraph feature entities are filled into the corresponding positions.

8. The method for automatically compiling a user manual based on a neural network model according to claim 7, characterized in that: When multiple paragraph feature entities of the same type are identified, a similarity comparison is performed between the to-be-filled example field corresponding to the semantic identification tag and the text contents of the multiple feature entities, and the feature entity with the highest similarity is selected to match the corresponding filling position.

9. The method for automatically compiling a user manual based on a neural network model according to any one of claims 1 or 2, characterized in that: The content modules of the user manual template include cover, preface, installation configuration, functional use cases, interface introduction, and data management.

10. The user manual automatic compilation system based on the neural network model is characterized by: include: Model building module, entity extraction module, matching and insertion module; The model building module generates a semantic extraction model based on the BERT model plus the CRF neural network, and sets up an entity-assisted retrieval model in parallel with the CRF neural network. It receives software document data for training, uses open source annotation tools to perform BIO annotation on the software document data, divides the annotated data into training sets, validation sets, and test sets, and completes the construction and training of the semantic extraction model. A semantic extraction module receives target development document data using a semantic extraction model and extracts entity information related to the user manual in the development document; The matching and insertion module matches and fills the identified entity information with the example fields to be filled in the user manual template one by one to generate a complete user manual.

Images