Document structure learning and generation method and device based on exponential probability model
Through the document structure learning and generation method based on the exponential probability model, the corpus analysis and training model is used to solve the problem that the document structure in specific fields cannot be automatically learned in the existing technology, and efficient document structure generation is achieved, suitable for documents in different scenarios Structure generation.
Patent Information
- Application Number
- CN202011331607.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-11-24
AI Technical Summary
The prior art lacks efficient methods to automatically learn specific domain document structures through corpus, and it is impossible to obtain a suitable document structure directly based on the target scenario.
Using the document structure learning and generation method based on the exponential probability model, the exponential probability model of the document structure is analyzed and constructed, and the model is trained to adapt to the target scenario and generate document structures in a specific field.
The method of automatically learning document structures in specific fields is realized, and the document structure of target scenarios can be efficiently generated based on the corpus, and local coherent information of the article is captured, providing an effective way to automatically learn document structures.
Smart Images

Figure CN112612869B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of automatic learning technology, and in particular to a document structure learning and generation method and device based on an exponential probability model. Background Art
[0002] Different scenarios and fields have different requirements for document structure. How to directly obtain the document structure of a specific field through the target scenario is an urgent problem to be solved in this field. In the related technology, there is no efficient method to automatically learn the document structure of a specific field through a corpus. Summary of the invention
[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.
[0004] To this end, an object of the present invention is to propose a document structure learning and generation method based on an exponential probability model, which regards the document structure as the order structure of topics in the ontology, uses the topic order in the scene description to capture the local coherence information of the article, and provides a method for automatically learning document structure, which can efficiently and automatically learn the document structure of a specific domain based on the corpus.
[0005] Another object of the present invention is to provide a document structure learning and generating device based on an exponential probability model.
[0006] To achieve the above object, an embodiment of the present invention proposes a document structure learning and generation method based on an exponential probability model, comprising:
[0007] Obtaining a database with annotated corpus, analyzing the database, and obtaining feature information related to the structure and feature information related to the nodes and their values;
[0008] Constructing an exponential probability model of the document structure, and training the exponential probability model of the document structure by using the feature information related to the structure and the feature information related to the nodes and their values;
[0009] The target scene is processed by using the trained exponential probability model of the document structure to obtain a document structure corresponding to the target scene.
[0010] The document structure learning and generation method based on the exponential probability model of the embodiment of the present invention obtains a database with annotated corpus, analyzes the database, obtains feature information related to the structure and feature information related to the nodes and their values; constructs an exponential probability model of the document structure, trains the exponential probability model of the document structure through the feature information related to the structure and the feature information related to the nodes and their values; processes the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene. Thus, a method for automatically learning document structure in a specific field is provided by automatically learning document structure through a corpus.
[0011] In addition, the document structure learning and generation method based on the exponential probability model according to the above embodiment of the present invention may also have the following additional technical features:
[0012] Furthermore, in one embodiment of the present invention, the database with annotated corpus is composed of a sequence of multiple topics.
[0013] Furthermore, in one embodiment of the present invention, the characteristic information related to the structure includes:
[0014]
[0015]
[0016]
[0017] Among them, x is the current topic, y is the current topic type, t i is the i-th topic in the sequence, u, v, w are topic types, t i .type is the topic type of the i-th topic, feature F1 is used to capture the relevant information between the current topic and the previous topic, feature F2 is used to capture the relevant information between the current topic and the previous two topics, and feature F3 is used to describe the relationship between the current topic and the previous topic types.
[0018] Furthermore, in one embodiment of the present invention, the characteristic information related to the node and its value includes:
[0019]
[0020]
[0021]
[0022]
[0023] Among them, feature F4 is used to describe the importance of the topic itself, feature F5 is used to describe the importance of the type of the topic itself, feature F6 is used to describe the types of attributes and relationships contained in the topic, and feature F7 is used to describe the values of the attributes and relationships contained in the topic.
[0024] Furthermore, in one embodiment of the present invention, the exponential probability model of the document structure is:
[0025]
[0026] Among them, f(x i ,y i ) is the feature information, and λ is the parameter learned from the corpus.
[0027] To achieve the above object, another embodiment of the present invention proposes a document structure learning and generating device based on an exponential probability model, comprising:
[0028] A processing module, used to obtain a database with annotated corpus, analyze the database, and obtain feature information related to the structure and feature information related to the nodes and their values;
[0029] A training module, used for constructing an exponential probability model of a document structure, and training the exponential probability model of the document structure through the feature information related to the structure and the feature information related to the nodes and their values;
[0030] The generation module is used to process the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene.
[0031] The document structure learning and generation device based on the exponential probability model of the embodiment of the present invention obtains a database with annotated corpus, analyzes the database, obtains feature information related to the structure and feature information related to the nodes and their values; constructs an exponential probability model of the document structure, trains the exponential probability model of the document structure through the feature information related to the structure and the feature information related to the nodes and their values; processes the target scene through the trained exponential probability model of the document structure, and obtains the document structure corresponding to the target scene. Thus, a method for automatically learning document structure in a specific field is provided by automatically learning document structure through a corpus.
[0032] In addition, the document structure learning and generating device based on the exponential probability model according to the above embodiment of the present invention may also have the following additional technical features:
[0033] Furthermore, in one embodiment of the present invention, the database with annotated corpus is composed of a sequence of multiple topics.
[0034] Furthermore, in one embodiment of the present invention, the characteristic information related to the structure includes:
[0035]
[0036]
[0037]
[0038] Among them, x is the current topic, y is the current topic type, t i is the i-th topic in the sequence, u, v, w are topic types, t i .type is the topic type of the i-th topic, feature F1 is used to capture the relevant information between the current topic and the previous topic, feature F2 is used to capture the relevant information between the current topic and the previous two topics, and feature F3 is used to describe the relationship between the current topic and the previous topic types.
[0039] Furthermore, in one embodiment of the present invention, the characteristic information related to the node and its value includes:
[0040]
[0041]
[0042]
[0043]
[0044] Among them, feature F4 is used to describe the importance of the topic itself, feature F5 is used to describe the importance of the type of the topic itself, feature F6 is used to describe the types of attributes and relationships contained in the topic, and feature F7 is used to describe the values of the attributes and relationships contained in the topic.
[0045] Furthermore, in one embodiment of the present invention, the exponential probability model of the document structure is:
[0046]
[0047] Among them, f(x i ,y i ) is the feature information, λ is the parameter learned from the corpus
[0048] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0050] Figure 1 A flow chart of a document structure learning and generation method based on an exponential probability model according to an embodiment of the present invention;
[0051] Figure 2 A framework diagram of a document structure learning algorithm based on a probability model according to an embodiment of the present invention;
[0052] Figure 3 A schematic diagram of an example of a document structure based on order output according to an embodiment of the present invention;
[0053] Figure 4 The figure is a schematic diagram of the structure of a document structure learning and generating device based on an exponential probability model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0054] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0055] The following describes a document structure learning and generation method and device based on an exponential probability model according to an embodiment of the present invention with reference to the accompanying drawings.
[0056] First, a document structure learning and generation method based on an exponential probability model proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0057] Figure 1 The present invention is a flowchart of a document structure learning and generation method based on an exponential probability model according to an embodiment of the present invention.
[0058] like Figure 1 As shown, the document structure learning and generation method based on the exponential probability model includes the following steps:
[0059] Step S1, obtain a database with annotated corpus, analyze the database, and obtain feature information related to the structure and feature information related to the nodes and their values.
[0060] Combination Figure 2 As shown, the main idea of the method of the present invention is: first, the knowledge graph structure is mapped according to the data set, a scene description based on the knowledge graph description is obtained, the structural features on the knowledge graph are calculated based on the scene description, and the corresponding classifier is trained. Finally, for a specific scene, the final result can be obtained through the classifier.
[0061] The corpus is analyzed and the selected features are divided into the following two categories: feature information related to the structure and feature information related to the nodes and their values.
[0062] First, the feature information related to the structure is similar to the language model and is defined as follows:
[0063]
[0064]
[0065]
[0066] Among them, x is the current topic, y is the current topic type, t i is the i-th topic in the sequence, u, v, w are topic types, t i .type is the topic type of the i-th topic. Feature F1 is actually the representation under the binary model exponential probability model, capturing the relevant information between the current topic and the previous topic; feature F2 is the representation under the ternary model exponential probability model, capturing the relevant information between the current topic and the previous two topics. Feature F3 describes the relationship between the current topic and the topic types that have appeared before.
[0067] Secondly, the characteristics of the nodes and their values are defined as follows:
[0068]
[0069]
[0070]
[0071]
[0072] Among them, feature F4 describes the importance of the topic itself, which can be considered as the expression of the univariate model under the exponential model; feature F5 describes the importance of the type of topic itself; feature F6 describes the types of attributes and relationships contained in the topic; feature F7 describes the values of the attributes and relationships contained in the topic.
[0073] Step S2, constructing an exponential probability model of the document structure, and training the exponential probability model of the document structure through feature information related to the structure and feature information related to the nodes and their values.
[0074] like Figure 3 As shown in the figure, an example of a document structure based on order output is shown. Figure 3(a) is the set of topics selected in the scene description, and the conceptual entities that appear include temperature, wind speed, wind direction, sky obscuration, and gusts; Figure 3 (b) is the output content of the order-based document structure, indicating that the generation order of concept entities should be sky cover, temperature, wind speed, wind direction, wind speed, and gusts. Figure 3 (c) is the text generated by the final natural language generation system according to the document structure. The method of the present invention does not temporarily study the process from the document structure to the final text.
[0075] Specifically, in the embodiment of the present invention, it is assumed that there is a labeled corpus C, which consists of a series of topic sequences, such as Figure 3 (b). Its language model is defined as follows.
[0076] Define T as all topics in an ontology in a certain field. Assuming T is a finite set, the sequential description of the document structure can be expressed as:
[0077] t0t1t2...t n
[0078] n is an integer, for n ≥ 1, we have t i ∈T,{1...(n-1)}, and assume that t0 and t n Is a special symbol. t0 represents the symbol START, indicating the beginning of the document. n Represents the symbol STOP, indicating the end of the document.
[0079] but Figure 3 (b) can be expressed as: START, 4, 1, 2, 3, 5, STOP.
[0080] Define T* as a sequence set composed of elements in T.
[0081] Then the document structure model can be defined as a finite set V and a function p(t1,t2,...t n ),have:
[0082] For any <t1,t2...t n >∈V * , p(t1,t2,...t n )≥0
[0083]
[0084] p(t1,t2,...t n ) is the probability distribution of documents on V*.
[0085] Then, the order of generating the document structure can be regarded as finding t1, t2, ... tn The order is such that:
[0086] argmaxp(t1,t2,...t n )
[0087] To further solve p(t1,t2,...t n ), make the following assumptions: Assuming that the document structure obeys an m-order Markov process, we have:
[0088]
[0089] Here, m means that the current topic is affected by the previous m-1 topics.
[0090] And define: T is the set of all topics;
[0091] X is t i-1 ,t i-2 ,...t i-m+1 The set of all possible topic sequences, t j ∈T, j∈{i-1,i-2,...,i-m+1};
[0092] Y is t i The set of all possible topics, Y is equivalent to T;
[0093] definition Maps the (x, y) pair to the feature vector f(x, y), that is, according to t i-1 ,t i-2 ,...t i-m+1 Various features calculated.
[0094] is the parameter of the model, which needs to be estimated based on corpus training.
[0095] Then for x∈X, y∈Y, we have
[0096]
[0097] in, is the normalized denominator, exp(x) = e x , It is the inner product of the parameter vector v and the feature vector f(x,y).
[0098] The exponential probability model of the document structure is defined as follows:
[0099]
[0100] Among them, f(x i ,y i) is the feature information, λ is the parameter learned from the corpus
[0101] Compared with the model of lapata, the model of the present invention can further integrate multiple features into the model by defining a feature vector f(x, y). The feature set used has been introduced in detail above and will not be repeated here.
[0102] Step S3, processing the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene.
[0103] Through the trained model, it is possible to automatically learn the document structure of a specific field through the corpus. The document structure generation is learned on an ideal dataset. The training set marks the order N of topics appearing in a certain scene. N∈{1,2,...n} is a finite set. The mark defines the order in which the topics appear in the final text.
[0104] According to the document structure learning and generation method based on the exponential probability model proposed in the embodiment of the present invention, a database with annotated corpus is obtained, the database is analyzed, and feature information related to the structure and feature information related to the nodes and their values are obtained; an exponential probability model of the document structure is constructed, and the exponential probability model of the document structure is trained through the feature information related to the structure and the feature information related to the nodes and their values; the target scene is processed through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene. Thus, a method for automatically learning document structure in a specific field is provided through a corpus.
[0105] Next, a document structure learning and generating device based on an exponential probability model according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0106] Figure 4 The figure is a schematic diagram of the structure of a document structure learning and generating device based on an exponential probability model according to an embodiment of the present invention.
[0107] like Figure 4 As shown, the document structure learning and generating device based on the exponential probability model includes: a processing module 401, a training module 402 and a generating module 403.
[0108] The processing module 401 is used to obtain a database with annotated corpus, analyze the database, and obtain feature information related to the structure and feature information related to the nodes and their values. The training module 402 is used to construct an exponential probability model of the document structure, and train the exponential probability model of the document structure through the feature information related to the structure and the feature information related to the nodes and their values. The generation module 403 is used to process the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene. The document structure of the device is regarded as the order structure of the topics in the ontology, and the topic order in the scene description is used to capture the local coherence information of the article, providing a method for automatically learning the document structure.
[0109] Furthermore, in one embodiment of the present invention, the database with annotated corpus is composed of a sequence of multiple topics.
[0110] Furthermore, in one embodiment of the present invention, the characteristic information related to the structure includes:
[0111]
[0112]
[0113]
[0114] Among them, feature F1 is used to capture the relevant information between the current topic and the previous topic, feature F2 is used to capture the relevant information between the current topic and the previous two topics, and feature F3 is used to describe the relationship between the current topic and the topic types that have appeared before.
[0115] Furthermore, in one embodiment of the present invention, the characteristic information related to the node and its value includes:
[0116]
[0117]
[0118]
[0119]
[0120] Among them, x is the current topic, y is the current topic type, t i is the i-th topic in the sequence, u, v, w are topic types, t i.type is the topic type of the i-th topic, feature F4 is used to describe the importance of the topic itself, feature F5 is used to describe the importance of the type of the topic itself, feature F6 is used to describe the types of attributes and relationships contained in the topic, and feature F7 is used to describe the values of the attributes and relationships contained in the topic.
[0121] Further, in one embodiment of the present invention, the exponential probability model of the document structure is:
[0122]
[0123] Among them, f(x i ,y i ) is the feature information, λ is the parameter learned from the corpus
[0124] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and will not be repeated here.
[0125] According to the document structure learning and generation device based on the exponential probability model proposed in the embodiment of the present invention, a database with annotated corpus is obtained, the database is analyzed to obtain feature information related to the structure and feature information related to the nodes and their values; an exponential probability model of the document structure is constructed, and the exponential probability model of the document structure is trained through the feature information related to the structure and the feature information related to the nodes and their values; the target scene is processed through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene. Thus, a method for automatically learning document structure in a specific field is provided by automatically learning document structure through a corpus.
[0126] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0128] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A document structure learning and generation method based on an exponential probability model, characterized in that: The following steps are involved: Obtaining a database with annotated corpus, analyzing the database to obtain feature information related to structure and feature information related to nodes and their values, wherein the structure is the order of topics in the database, and the nodes are topics in the database; Constructing an exponential probability model of the document structure, and training the exponential probability model of the document structure by using the feature information related to the structure and the feature information related to the nodes and their values; Processing the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene; The characteristic information related to the structure includes: in, x For the current topic, y is the current topic type, t i For the sequence i Topic, u , v , w For topic type, t i . type For the i Topic type, characteristics of topics F 1 is used to capture relevant information about the current topic and the previous topic, feature F2 Used to capture relevant information between the current topic and the previous two topics, feature F 3 is used to describe the relationship between the current topic and the previous topic types; The characteristic information related to the node and its value includes: Among them, the characteristics F 4 is used to describe the importance and characteristics of the topic itself F 5 is used to describe the importance and characteristics of the topic type itself F 6 is used to describe the types and characteristics of the attributes and relationships contained in the topic F 7 is used to describe the values of the attributes and relationships contained in the topic.
2. The method according to claim 1, characterized in that The database with annotated corpus is composed of a sequence of multiple topics.
3. The method according to claim 1, characterized in that: The exponential probability model of the document structure is: in, f ( x i ,y i ) is the feature information, and λ is the parameter learned from the corpus.
4. A document structure learning and generation device based on an exponential probability model, characterized in that: include: A processing module, used to obtain a database with annotated corpus, analyze the database, and obtain feature information related to structure and feature information related to nodes and their values, wherein the structure is the order of topics in the database, and the nodes are topics in the database; A training module, used for constructing an exponential probability model of a document structure, and training the exponential probability model of the document structure through the feature information related to the structure and the feature information related to the nodes and their values; A generation module, used to process the target scene through the trained exponential probability model of the document structure to obtain the document structure corresponding to the target scene; The characteristic information related to the structure includes: in, x For the current topic, y is the current topic type, t i For the sequence i Topic, u , v , w For topic type, t i . type For the i Topic type, characteristics of topics F 1 is used to capture relevant information about the current topic and the previous topic, feature F2 Used to capture relevant information between the current topic and the previous two topics, feature F 3 is used to describe the relationship between the current topic and the previous topic types; The characteristic information related to the node and its value includes: Among them, the characteristics F 4 is used to describe the importance and characteristics of the topic itself F 5 is used to describe the importance and characteristics of the topic type itself F 6 is used to describe the types and characteristics of the attributes and relationships contained in the topic F 7 is used to describe the values of the attributes and relationships contained in the topic.
5. The device according to claim 4, characterized in that The database with annotated corpus is composed of a sequence of multiple topics.
6. The device according to claim 4, characterized in that The exponential probability model of the document structure is: in, f ( x i ,y i ) is the feature information, and λ is the parameter learned from the corpus.
