Automatic recognition method and system of paragraph semantics based on sentence association

Abstract

The invention discloses a method and system for automatic recognition of chapter semantics based on sentence and sub-association. The method comprises: constructing a correspondence between original sentences and chapter semantics in annotated corpus according to the chapter; contracting and merging the original sentences to obtain new sentences, constructing a correspondence between the new sentences and chapter semantics, saving the new sentences and corresponding semantic labels as one column in the form of a dictionary, and generating a semantic dictionary; dictionary annotation of the semantic dictionary, outputting two columns of sentences and semantic labels by word for the sequences in the annotated dictionary, training a CRF model according to the two columns of sentences and semantic labels output by word, and constructing the CRF model; predicting semantic labels for input documents by using the CRF model, predicting semantic labels for input documents by using the semantic dictionary, giving priority to semantic labels determined by the semantic dictionary, fusing semantic labels queried by the semantic dictionary and semantic labels predicted by the CRF model, and obtaining final semantic labels for the input documents.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for automatic recognition of paragraph semantics based on sentence association. Background Art

[0002] Natural language is the expression of human thoughts. Natural language processing studies human thinking patterns. The granularity of natural language processing represents the level of human thinking. In terms of semantic technology research, the more mature research is sentence-level semantic processing technology, such as frame semantics, role semantics, dependency semantics, etc. The sentence level shows that people's research on natural language is still at the most basic functional level, just like a screw. The chapter semantics is equivalent to a device; the lag in chapter semantics research has led to the inability of natural language processing technology to conduct effective research on large-scale engineering scenarios, including the definition, annotation and automatic recognition of chapter semantics. For word docx format documents, the analysis of chapter semantics can generally be obtained by parsing their hierarchical structure, but for many doc, txt, pdf and other format files, they do not have a hierarchical structure, and it is unrealistic to manually annotate them. Therefore, it is necessary to develop a method for automatic recognition of chapter semantics.

[0003] At present, the existing context-independent 0-type sentence semantic recognition technology cannot recognize the text semantics with strict hierarchy and logical relationship. The existing sentence-level processing technology treats the sentence as an independent 0-type semantic unit, without considering the context of the sentence and the position of the sentence in the article. This is OK for simple semantics such as word segmentation and part-of-speech tagging. However, in the case of text semantics, the entire text-chapter-section-paragraph-sentence is organized as a whole to fully express the scene. The text-chapter-section-paragraph is composed of multiple sentences. The order between sentences determines the meaning of the sentence. The sentence obtains semantics in the corresponding hierarchical structure. Without this structure, the meaning of the sentence is unclear. In engineering, the text-chapter-section-paragraph-sentence corresponds to scenes of different granularity, such as business domain-business theme-research object-business activity in petrochemical business, or corresponds to the hierarchical structure of country-headquarters-branch-research room. Since a sentence is only the smallest semantic unit representing the smallest activity or function in engineering, it cannot exist in isolation from the framework of the entire activity, which means that the sentence must be analyzed together with its context to determine the text semantics of the sentence.

[0004] In addition, existing algorithms, whether statistical learning or deep learning, cannot handle long sentences. Single sentences in engineering are generally very long, and multiple sentences connected together are even longer, such as the common two sentences "The SL223 logging team of the Second Geological Logging Branch Company undertook the geological logging task for this well. According to the design requirements, drilling time, comprehensive logging, rock cuttings, well wall coring, fluorescence, drilling fluid, rock pyrolysis geochemistry, two-dimensional quantitative fluorescence, rock and mineral identification, tank top gas light hydrocarbon gas chromatography analysis, thermal evaporation hydrocarbon chromatography, nuclear magnetic resonance, paleontological analysis, fluorescent thin sections and other logging work were carried out from the second opening to the bottom of the well. The statistics of the workload of the whole well are shown in Table 1. During the construction process, the late time and mud (shale) density were also measured. The rock samples were collected in time during the drilling process. During the logging process, the SL-ADVANTAGE integrated logging instrument worked normally, giving full play to the various acquisition functions of the integrated logging instrument, collecting all geological data, gas logging data and engineering parameters in a complete and accurate manner, and successfully completing the design task. "The first sentence contains 180 characters. In professional literature, sentences are generally very long, with an average of more than 80 characters. One sentence exceeds the limit of less than 50 characters for the input sentence width of statistical learning or CRF, and it is even more impossible to handle paragraphs of several consecutive sentences connected together. Since CRF can only process the association between the nearest +-4 characters, the algorithm mechanism has weak processing capabilities for long-range associations; when the number of input symbols for deep learning is greater than 100, the amount of calculation for calculating the second-order gradient matrix is ​​very large, which cannot meet the requirements of system speed performance. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for automatic recognition of paragraph semantics based on sentence association, aiming to solve the above-mentioned problems in the prior art.

[0006] The present invention provides a method for automatic recognition of paragraph semantics based on sentence association, comprising:

[0007] Constructing the correspondence between the original sentence and the text semantics in the annotated corpus according to the text, wherein the text semantics is stored in the hierarchical form of text-chapter-section-paragraph-sentence-slot;

[0008] The original sentences are contracted to obtain new sentences, a corresponding relationship between the new sentences and the text semantics is constructed, the new sentences and the corresponding semantic labels are saved as a column in the form of a dictionary, and a semantic dictionary is generated;

[0009] Perform dictionary annotation on the semantic dictionary, output the sequences in the annotated dictionary as sentences and two columns of semantic labels by word, train a CRF model based on the two columns of sentences and semantic labels output by word, and construct a CRF model;

[0010] The CRF model is used to predict semantic labels for the input document, and the semantic dictionary is used to predict semantic labels for the input document, with the semantic labels determined according to the semantic dictionary being given priority. The semantic labels queried through the semantic dictionary and the semantic labels predicted by the CRF model are fused to obtain the final semantic labels for the input document.

[0011] The present invention provides a sentence association-based automatic text semantics recognition system, comprising:

[0012] The data layer is used to store raw annotated corpus, semantic dictionaries, text semantic documents, and annotated corpus after annotation;

[0013] The data processing layer is used to construct the correspondence between the original sentence and the semantics of the chapter in the annotated corpus according to the chapter, wherein the chapter semantics is saved in the hierarchical form of chapter-chapter-section-paragraph-sentence-slot; the original sentence is contracted to obtain a new sentence, the correspondence between the new sentence and the chapter semantics is constructed, the new sentence and the corresponding semantic label are saved as a column in the form of a dictionary, and a semantic dictionary is generated; the semantic dictionary is dictionary-annotated, the sequence in the annotated dictionary is output as two columns of sentences and semantic labels by word, and a CRF model is trained based on the two columns of sentences and semantic labels output by word to construct a CRF model; the semantic label of the input document is predicted by the CRF model, and the semantic label of the input document is predicted by the semantic dictionary, with the semantic label determined by the semantic dictionary being given priority, and the semantic label queried by the semantic dictionary and the semantic label predicted by the CRF model are merged to obtain the final semantic label of the input document;

[0014] The application layer is used to implement interactive operations with users.

[0015] By adopting the embodiment of the present invention, a complex hierarchical structure is equivalent to a single-layer sequence for processing, which can realize the chapter semantic recognition of unlabeled documents, restore the hierarchical structure of pure text documents, re-endow the text chapter semantics, improve people's ability to understand documents, and expand people's horizons in understanding the world.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flow chart of a method for automatic recognition of text semantics based on sentence association according to an embodiment of the present invention;

[0019] Figure 2 Schematic diagram of the system structure of the method for automatic recognition of text semantics based on sentence association according to an embodiment of the present invention;

[0020] Figure 3a is a schematic diagram of a document with a hierarchical structure according to an embodiment of the present invention;

[0021] Figure 3b is a schematic diagram of a document without a hierarchical structure according to an embodiment of the present invention;

[0022] Figure 4a is a schematic diagram of a semantic dictionary structure according to an embodiment of the present invention;

[0023] Figure 4b is a schematic diagram of an example of a semantic dictionary structure according to an embodiment of the present invention;

[0024] Figure 5 It is a schematic diagram of the composition structure of the text semantics of an embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of the annotated corpus format of an embodiment of the present invention;

[0026] Figure 7a is a schematic diagram of annotated corpus according to an embodiment of the present invention;

[0027] Figure 7b It is a schematic diagram of the 2-column text format required for the CRF model prediction of an embodiment of the present invention;

[0028] Figure 8 is a schematic diagram of a CRF++ model construction interface of an embodiment of the present invention;

[0029] Fig. 9 is a schematic diagram of a post-processing result of an embodiment of the present invention;

[0030] Fig.10 is a detailed flow chart of a method for automatic recognition of text semantics based on sentence association according to an embodiment of the present invention;

[0031] Fig.11is a schematic diagram of a notation dictionary format according to an embodiment of the present invention;

[0032] Fig.12a Schematic diagram of contraction of original sentences in an embodiment of the present invention;

[0033] Figure 12b is a schematic diagram of a single-column input file according to an embodiment of the present invention;

[0034] Fig.12c It is a schematic diagram of the prediction results of 2 columns according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0037] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0038] Method Embodiment

[0039] According to an embodiment of the present invention, a method for automatic recognition of paragraph semantics based on sentence association is provided. Figure 1is a flow chart of a method for automatic recognition of text semantics based on sentence association according to an embodiment of the present invention. Figure 1 As shown, the automatic paragraph semantics recognition method based on sentence association according to an embodiment of the present invention specifically includes:

[0040] Step 101, construct the correspondence between the original sentence and the chapter semantics in the annotated corpus according to the chapter, wherein the chapter semantics is saved in the hierarchical form of chapter-chapter-section-paragraph-sentence-slot; in step 101, specifically: read the annotated corpus, integrate the chapter name and the sentence into one column based on the chapter, call the chapter semantic dictionary, determine the semantic tag corresponding to the chapter name, and obtain the sentence semantics from the annotated corpus, and superimpose the chapter semantics and the sentence semantics into one column.

[0041] Step 102, abbreviate the original sentence to obtain a new sentence, construct a correspondence between the new sentence and the text semantics, save the new sentence and the corresponding semantic label in a dictionary form as one column, and generate a semantic dictionary; specifically: take the N words before and after the original sentence to form a reduced new sentence, add the M sentences before and after the sentence to form a compound new sentence, and process the special symbols in the new sentence; save the new sentence and the semantic label in a dictionary form as one column, and generate a semantic dictionary.

[0042] Step 103, dictionary annotation is performed on the semantic dictionary, and the sequence in the annotated dictionary is outputted into two columns of sentences and semantic labels by word, and a CRF model is trained based on the two columns of sentences and semantic labels outputted by word, and a CRF model is constructed; specifically, the semantic dictionary is sorted according to long words first, a word sequence of a sentence and a corresponding semantic replacement sequence are established, and the sequence is outputted into two columns by word for storage, wherein the two columns are separated by a tab; crf_learn.exe template CRF_in.txt CRF_model is run under cmd, and a CRF model is trained by operating the input CRF_in.txt file, and a CRF model named CRF_model is constructed.

[0043] Step 104, predicting the semantic label of the input document through the CRF model, predicting the semantic label of the input document through the semantic dictionary, giving priority to the semantic label determined by the semantic dictionary, fusing the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model, and obtaining the final semantic label of the input document. Specifically, read the input document, process the input document to sentence granularity, and shrink and merge to obtain a new sentence, transform the new sentence into 1 column, execute crf_test-m CRF_model CRF_test.txt>CRF_result.txt under cmd, calculate the output result by applying the CRF model, and use the semantic label that appears most in the sentence as the semantic label of the sentence according to the output result; use the original sentence, query the semantic label of the original sentence from the chapter semantic dictionary and the sentence semantic dictionary; give priority to the semantic label determined by the semantic dictionary, fuse the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model, and obtain the final semantic label of the input document.

[0044] In an embodiment of the present invention, the method further comprises: obtaining the modified final semantic tag and saving it as annotated corpus.

[0045] The technical solution of the embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0046] The technical solution of the embodiment of the present invention proposes a method for automatic recognition of text semantics based on sentence association, which processes a complex hierarchical structure equivalent to a single-layer sequence, as follows:

[0047] First, establish the chapter semantics of the document in a hierarchical structure arranged in the order of chapter-section-paragraph-sentence-slot. The chapter semantics adopts a hierarchical structure instead of the definition of interrelated graph semantics, because the tree structure can be equivalent to a sequence structure, and the processing of the tree hierarchy is equivalent to the processing of the sequence of leaf nodes. The hierarchical tree is the structural feature of complex systems, representing the thinking mode of people, and is also a common expression of knowledge systems. Preprocess the entire input annotated corpus and establish a sequence table of all sentences and their corresponding semantics;

[0048] Secondly, for the logical relationship between sentences, the sentences before and after (+-1 sentence) are merged into a long sentence, and the CRF modeling is performed on this long sentence. For a single sentence, the sentence cannot be moved at will, which reflects that the sentence must meet the requirements of the text. This does not meet the assumption of CRF that the sentence can be moved at will; the long sentence composed of three sentences can be moved at will, regardless of the context, which meets the requirements of CRF. At the same time, the upper and lower sentences carry framework information, so the fused long sentence meets the requirements of the CRF algorithm and the requirements of the text semantics for sentences. The number of sentences related to each other can be adjusted between (-4, +4) according to the actual situation. Since the order or association between sentences is the most important factor in text semantics, and the content of the sentence itself or the length of the sentence does not affect the connection between sentences, therefore, the length of the sentence is modified, and only N = 5 to 10 characters before and after the sentence are taken to form a short sentence, so that the three sentences are concatenated together without exceeding 60 characters, which is the length that any algorithm can apply. The combined sentence can meet the requirements of the CRF model for sentence length;

[0049] Finally, two methods are used to post-process the semantics of the entire document. First, dictionary query is used to inherit people's prior knowledge and the query is independent of the sample size. Second, for sentences whose semantic labels cannot be found, the CRF model is used to identify the sentence's chapter semantic label and assign a semantic label with confidence.

[0050] like Figure 2 As shown, the network structure of a method for automatic recognition of text semantics based on sentence association according to an embodiment of the present invention includes: data layer 1, data processing layer 2 and application layer 3. The function of data layer 1 is to realize the storage, reading, writing and modification of files, including raw corpus documents, semantic dictionaries, text semantic documents and documents and annotated familiar corpus 4 parts; data processing layer 2 realizes the establishment and application of CRF semantic model according to familiar corpus, and realizes semantic prediction of raw corpus, including data preprocessing, establishment of CRF semantic model, CRF semantic model prediction, and prediction result post-processing 4 parts; application layer 3 realizes interactive operations with users, including reading documents, editing Word documents and saving annotated documents 3 parts, among which multi-level editing and visualization of documents are carried out using Word.

[0051] The data layer 1 is composed of raw corpus document 1-1, semantic dictionary 1-2, chapter semantic document 1-3 and document and annotated familiar corpus 1-4. The format of the raw corpus document 1-1 is as follows: Figure 3a and Figure 3b As shown, Figure 3a It is a word document with a hierarchical structure. Figure 3bIt is a txt document without any hierarchical structure. The goal of automatic text semantic recognition is to Figure 3b Documentation plus Figure 3a The format of the semantic dictionary 1-2 is as follows Figure 4a and Figure 4b As shown, Figure 4a and Figure 4b The dictionary names and their corresponding fields are listed. In the article-chapter-section dictionary, the corresponding levels of article-chapter-section need to be included; in the paragraph semantic dictionary, a list of sentence semantic combinations needs to be included. The recognition of paragraph semantics also involves two methods: recognition based on the original text and recognition based on a combination of sentences. Since paragraphs are sometimes very long, it is unrealistic to recognize them based on the original text in most cases, and recognition based on sentence semantic combinations has implementation advantages. The article semantic documents 1-3 are the common document templates for all the final documents, and are also the hierarchical labels for each sentence in the final document, such as Figure 5 The chapter annotation idioms 1-4 store all the results for each document, and their structure is as follows: Figure 6 As shown. The format of the annotated corpus is stored in a table, including word text-table sorting, title, title level, body paragraph, table number, paragraph text, number of pictures, Heading n, title template, Heading n_chapter semantics, paragraph semantics, sentence split text, sentence semantics, NER-mode and object semantics, object, sentence-picture-table semantics and other fields. These fields are full information tables obtained after the word document is split and annotated, through which different document formats can be converted. The CRF semantic model construction corpus is composed of Figure 5 The relevant fields are integrated.

[0052] The data processing layer 2 is composed of four parts: data preprocessing 2-1, building a CRF semantic model 2-2, CRF semantic model prediction 2-3, and prediction result post-processing 2-4. The data preprocessing 2-1 expands and contracts the familiar corpus into Figure 7a The corpus is then automatically annotated to establish the input sample data required by the CRF model; the input word document is indexed at the sentence granularity and processed into the 2-column mode required by the CRF model, such as Figure 7b The task of establishing the CRF semantic model 2-2 is to establish the CRF semantic model according to the selected template, optimize the CRF parameters to meet the requirements of the algorithm capacity and system capacity, and ensure that the model can be successfully operated. The CRF semantic model prediction 2-3 refers to the application of the established CRF model to an article Figure 3b The unstructured document shown in the figure is used for semantic prediction to obtain a predicted semantic label. The interface for normal operation of CRF is as follows Figure 8The prediction result post-processing 2-4 uses two methods to decide on the document semantic label. The first is through the dictionary matching method. When the dictionary cannot be found, the result predicted by the CRF semantic model is used. This can prevent the mediocre solution of the model, such as Fig. 9 shown.

[0053] The application layer 3 includes three parts: reading documents 3-1, editing Word documents 3-2, and saving annotated documents 3-3.

[0054] The module reads the document 3-1 to implement the document selection function and reads the word document into the software platform. The editing of the Word document 3-2 is to use the editing function of Word to visualize the predicted structure. The saving of the annotated document 3-3 is to save the document modified in the Word interface and update the Figure 6 Columns related to .

[0055] refer to Fig.10 The method and system flow chart of automatic recognition of text semantics based on sentence association include two related processes: model building and prediction results. The specific process steps are as follows:

[0056] Step 1: Sentence-semantic processing

[0057] The purpose of sentence semantic processing is to construct the correspondence between the original sentence (also called entity or object) and the text semantics in the annotated corpus according to the text. The text annotated corpus is mainly saved in the hierarchical form of text-chapter-section-paragraph-sentence-slot, rather than in the two-column format required by the CRF model, so format conversion is required.

[0058] Step 1-1: Read the annotated corpus document

[0059] Read the chapter annotated corpus recorded in format 5.

[0060] Step 1-2: Combine chapter names and sentences into one column per article

[0061] Add up each Heading column and merge it with the sentence column into one column.

[0062] Step 1-3: Define chapter semantics from the chapter semantic dictionary

[0063] Call the chapter semantic dictionary to add semantic tags to the chapter name.

[0064] Step 1-4: Overlay chapter semantics and sentence semantics into one column

[0065] The chapter semantics and sentence semantics are integrated into one column. For the figure name and table name, their semantic labels are defined as "figure" and "table" respectively. In practice, figures and tables are stored separately and independently.

[0066] Step 2: Combine the previous and next sentences

[0067] Take N=5-10 words from the front and back of a long sentence to form a new sentence, which can reduce the algorithm's memory requirements; in addition, splice the front and back sentences together to form a new sentence, which can ensure that the sentence is coherent. The position of the new sentence can be changed, but the new sentence carries the chapter information of the original text, so the new sentence can realize the recognition of the sentence chapter semantics. Generally, the sentence semantics is just the name of the sentence. The recognition of chapter semantics requires not only the recognition of the sentence name, but also the recognition of the hierarchical position of the sentence.

[0068] Step 2-1: Take the N words before and after to form a new sentence

[0069] Because the focus of paragraph semantic recognition is the connection between sentences rather than the content of the sentence itself, the middle part of the sentence can be ignored, so there are only short sentences with 10 characters. N = 5 to 10, and the specific parameter value is determined by the memory required for calculation.

[0070] Step 2-2: Add the two sentences together to form a compound sentence

[0071] Integrate the two sentences into a matching sentence, such as adding "Jimusar Ao##1-1)." to the two sentences before and after to become "1.1 Oil## and environmental conditions@@Jimusar Ao##1-1).@@The surface is grassland##660m."

[0072] It should be noted that the sentences are integrated into one sentence, but the semantic tags of the paragraph are not integrated, and the corresponding semantic tags "Jimusar Ao##1-1)." are still used.

[0073] Step 2-3: Processing special symbols in sentences

[0074] CRF has some special requirements for data, such as spaces cannot appear. To this end, we replace all punctuation marks and keep only plain text, which are mainly regular symbols and Chinese punctuation marks, such as reg = '.\^$*+? {}[]|(). -(),:? 、 / ;_~“”'. After replacement, "1.1Oil##and environmental conditions@@Jimusar Ao##1-1).@@The surface is grassland##660m." becomes "11Oil##and environmental conditions@@Jimusar Ao##11@@The surface is grassland##660m".

[0075] In the chapter semantic tags, a format such as "1.2 Well Location Deployment" with punctuation and spaces is generally used, which is completely understandable to humans. However, in order to meet the requirements of the CRF algorithm, the special characters in these semantic tags must also be replaced accordingly.

[0076] Step 2-4: Build an annotation dictionary

[0077] Save sentences and semantic tags in the form of a dictionary, such as "11 oil## and environmental conditions@@Jimusar Ao##11@@The surface is grassland##660m / Oil reservoir location and environmental conditions", where using " / " to tag is a common way in natural language. Many methods can draw on existing method modules.

[0078] Step 3: Dictionary annotation

[0079] Dictionary annotation is a basic method of natural language annotation, in which dictionary entries are required to be unique, which is generally achieved by increasing the length of words. The result of dictionary annotation is to label sentences with semantic tags.

[0080] Step 3-1: Sort the dictionary by long words first

[0081] Long words generally have more accurate semantics and can more accurately describe people's thoughts. Therefore, giving priority to long words has an epistemological basis. The sorting is implemented using the sort_values ​​method of DataFrame in pandas. The format after sorting is as follows: Fig.11 shown.

[0082] Step 3-2: Establish the word sequence of the sentence and the corresponding semantic replacement sequence:

[0083] Sentence sequences and label sequences are established according to the longest word priority. For example, "11 oil##and environmental conditions@@Jimusar Ao##11@@the surface is grassland##660m&@@directory@@1basic situation of drilling and completion&Tarim Basin" is transformed into two sequences, word:[1 oil##and environmental conditions@@Jimusar Ao##11@@the surface is grassland##660m,&,@@directory@@1basic situation of drilling and completion,&Tarim Basin], label:[BMMMMMMMMMMMMMMMMMMMMMMMMMMMMMME,N,BMMMMMMMMMMMME,NNNNNN].

[0084] Step 3-3: Output the sequence word by word into two columns and save:

[0085] Output the above two columns as Figure 7b The CRF model shown requires an input format with two columns separated by a tab.

[0086] Step 4: Build the CRF model

[0087] Run crf_learn.exe template CRF_in.txt CRF_model in cmd, and perform operations on the input CRF_in.txt file. The correct operation interface is as follows Figure 8As shown, the obtained model is named CRF_model.

[0088] Step 5: Model prediction

[0089] Apply the established CRF model to predict semantic labels for input documents.

[0090] Step 5-1: Read the original corpus document

[0091] Read an article Figure 3b The document shown has no semantic hierarchy and is processed to sentence level.

[0092] Step 5-2: Process by consolidation of the sentences before and after

[0093] This part of the processing is the same as 2-1 and 2-2, and is also to reduce the amount of calculation, such as Fig.12a shown.

[0094] One option is to not process it and directly use the original sentence for prediction, and the actual effect is basically the same.

[0095] Step 5-3: Transform into a single column

[0096] Figure 7b The corpus needs to be organized into 2 columns because the semantic tags are known.

[0097] In this step, semantic labels need to be predicted, so there is only one column of the original document, such as Figure 12b shown.

[0098] Step 5-4: CRF model calculates semantic labels

[0099] Run crf_test -m CRF_model CRF_test.txt>CRF_result.txt in cmd, and use the CRF model to calculate the output results of the transformed documents, such as Fig.12c shown.

[0100] Step 5-5: Take the most semantic tags as the sentence tag

[0101] Unlike the traditional BMEO method of marking the position of words, here we only need to determine that the label that appears most frequently in the sentence is the label of the entire sentence, and merge the results into Fig. 9 One option is to confirm the most common label in the middle of the sentence as the label of the sentence.

[0102] Step 6: Post-Processing

[0103] The sentence labels matched by the dictionary and the calculated labels are processed uniformly.

[0104] Step 6-1: Assign semantics based on the semantic dictionary

[0105] Use the original sentence instead of the contracted sentence. Look up the semantics of the sentence from the text semantic dictionary and sentence semantic dictionary. The matching results are shown in Fig. 9 The "Dictionary Matching Results" column in the . Dictionary matching has nothing to do with the sample size and represents people's prior understanding of things, which is the most accurate knowledge.

[0106] Step 6-2: Dictionary and prediction fusion

[0107] The dictionary results are prioritized and fused with the CRF prediction results. See the results in Fig. 9 The "Final Semantic" column of .

[0108] Step 7: Add to corpus file

[0109] Check the calculation results in Word and make changes, and save the modified document as corpus in Figure 6 The document shown in the figure is used as a new annotated corpus. By continuously accumulating the corpus, the accuracy of automatic text semantic recognition is continuously improved.

[0110] This step is an indispensable part of the entire cycle of automatic semantic recognition of the passage. Natural language processing ultimately relies on the size of the sample to improve recognition accuracy.

[0111] To sum up, with the help of the technical solution of the embodiment of the present invention, the complex hierarchical structure is equivalent to a single-layer sequence for processing, which can realize the chapter semantic recognition of unlabeled documents, restore the hierarchical structure of pure text documents, re-endow the text chapter semantics, improve people's ability to understand documents, and broaden people's horizons in understanding the world.

[0112] System Example

[0113] According to an embodiment of the present invention, a system for automatic recognition of text semantics based on sentence association is provided. Figure 2 As shown, the automatic paragraph semantics recognition system based on sentence association according to an embodiment of the present invention specifically includes:

[0114] Data layer 1 is used to store raw annotated corpus, semantic dictionaries, text semantic documents, and annotated corpus after annotation;

[0115] The data processing layer 2 is used to construct the correspondence between the original sentence and the chapter semantics in the annotated corpus according to the chapter, wherein the chapter semantics is saved in the hierarchical form of chapter-chapter-section-paragraph-sentence-slot; the original sentence is contracted to obtain a new sentence, the correspondence between the new sentence and the chapter semantics is constructed, the new sentence and the corresponding semantic label are saved as a column in the form of a dictionary, and a semantic dictionary is generated; the semantic dictionary is dictionary-annotated, the sequence in the annotated dictionary is outputted as two columns of sentences and semantic labels by word, and a CRF model is trained based on the two columns of sentences and semantic labels outputted by word to construct a CRF model; the input document is predicted for semantic labels by the CRF model, and the input document is predicted for semantic labels by the semantic dictionary, with the semantic label determined by the semantic dictionary being given priority, and the semantic label queried by the semantic dictionary and the semantic label predicted by the CRF model are merged to obtain the final semantic label of the input document; the data processing layer 2 is specifically used for:

[0116] Read the annotated corpus, integrate the chapter names and sentences into one column in units of chapters, call the chapter semantic dictionary, determine the semantic label corresponding to the chapter name, and obtain the sentence semantics from the annotated corpus, and superimpose the chapter semantics and the sentence semantics into one column;

[0117] Taking the N words before and after the original sentence to form a reduced new sentence, adding the M words before and after the sentence to form a composite new sentence, and processing the special symbols in the new sentence;

[0118] Save the new sentences and semantic labels in a dictionary format into one column to generate a semantic dictionary;

[0119] Sorting the semantic dictionary according to long words first, establishing a word sequence of a sentence and a corresponding semantic replacement sequence, outputting the sequence word by word into two columns for storage, wherein the two columns are separated by a tab character;

[0120] Run crf_learn.exe template CRF_in.txt CRF_model in cmd, train the CRF model by operating the input CRF_in.txt file, and build a CRF model named CRF_model;

[0121] Read the input document, process the input document to the sentence granularity, shrink and merge to obtain a new sentence, transform the new sentence into 1 column, execute crf_test -m CRF_model CRF_test.txt>CRF_result.txt in cmd, apply the CRF model to calculate the output result, and take the semantic label that appears most in the sentence as the semantic label of the sentence according to the output result;

[0122] Using an original sentence, querying a semantic tag of the original sentence from a text semantic dictionary and a sentence semantic dictionary;

[0123] The semantic label determined according to the semantic dictionary is given priority, and the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model are fused to obtain the final semantic label of the input document.

[0124] Application layer 3 is used to implement interactive operations with users. The application layer 3 is specifically used for:

[0125] The modified final semantic tag is obtained and saved as annotated corpus in the data layer.

[0126] Read input documents and edit Word documents.

[0127] The embodiment of the present invention is a system embodiment corresponding to the above-mentioned method embodiment. The specific operations of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0128] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0129] In the 1930s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0130] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0131] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0132] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0133] It should be understood by those skilled in the art that one or more embodiments of this specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0138] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0139] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0141] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0142] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] The above description is only an embodiment of this document and is not intended to limit this document. For those skilled in the art, this document may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document should be included in the scope of the claims of this document.

Claims

1. A method for automatic recognition of text semantics based on sentence association, characterized in that: include: Constructing the correspondence between the original sentence and the text semantics in the annotated corpus according to the text, wherein the text semantics is stored in the hierarchical form of text-chapter-section-paragraph-sentence-slot; The original sentences are contracted to obtain new sentences, a corresponding relationship between the new sentences and the text semantics is constructed, the new sentences and the corresponding semantic labels are saved as a column in the form of a dictionary, and a semantic dictionary is generated; Perform dictionary annotation on the semantic dictionary, output the sequences in the annotated dictionary as sentences and two columns of semantic labels by word, train a CRF model based on the two columns of sentences and semantic labels output by word, and construct a CRF model; The CRF model is used to predict semantic labels for the input document, and the semantic dictionary is used to predict semantic labels for the input document, with the semantic labels determined according to the semantic dictionary being given priority. The semantic labels queried through the semantic dictionary and the semantic labels predicted by the CRF model are fused to obtain the final semantic labels for the input document.

2. The method according to claim 1, characterized in that The method further comprises: The modified final semantic label is obtained and saved as annotated corpus.

3. The method according to claim 1, characterized in that The correspondence between the original sentence and the text semantics is constructed in the annotated corpus according to the text, including: Read the annotated corpus, integrate the chapter names and sentences into one column in units of chapters, call the chapter semantic dictionary, determine the semantic tag corresponding to the chapter name, and obtain the sentence semantics from the annotated corpus, and superimpose the chapter semantics and the sentence semantics into one column.

4. The method according to claim 1, characterized in that: The original sentences are contracted to obtain new sentences, a corresponding relationship between the new sentences and the text semantics is constructed, and the new sentences and the corresponding semantic labels are saved as a column in the form of a dictionary. Generating a semantic dictionary specifically includes: Taking the N words before and after the original sentence to form a reduced new sentence, adding the M words before and after the sentence to form a composite new sentence, and processing the special symbols in the new sentence; Save the new sentences and semantic labels in a dictionary format as one column to generate a semantic dictionary.

5. The method according to claim 1, characterized in that The semantic dictionary is annotated, and the sequences in the annotated dictionary are outputted as sentences and two columns of semantic labels by word. The CRF model is trained according to the two columns of sentences and semantic labels outputted by word. The construction of the CRF model specifically includes: Sorting the semantic dictionary according to long words first, establishing a word sequence and a corresponding semantic replacement sequence of the sentence, outputting the word sequence and the corresponding semantic replacement sequence in two columns by word, wherein the two columns are separated by a tab character; Run crf_learn.exe template CRF_in.txt CRF_model in cmd, train the CRF model by operating the input CRF_in.txt file, and build a CRF model named CRF_model.

6. The method according to claim 1, characterized in that Predicting the semantic label of the input document through the CRF model, predicting the semantic label of the input document through the semantic dictionary, giving priority to the semantic label determined according to the semantic dictionary, fusing the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model, and obtaining the final semantic label of the input document specifically includes: Read the input document, process the input document to the sentence granularity, shrink and merge to obtain a new sentence, transform the new sentence into 1 column, execute crf_test -m CRF_model CRF_test.txt>CRF_result.txt in cmd, apply the CRF model to calculate the output result, and take the semantic label that appears most in the sentence as the semantic label of the sentence according to the output result; Using an original sentence, querying a semantic tag of the original sentence from a text semantic dictionary and a sentence semantic dictionary; The semantic label determined according to the semantic dictionary is given priority, and the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model are fused to obtain the final semantic label of the input document.

7. A sentence association-based automatic text semantics recognition system, characterized in that: include: The data layer is used to store raw annotated corpus, semantic dictionaries, text semantic documents, and annotated corpus after annotation; The data processing layer is used to construct the correspondence between the original sentence and the semantics of the chapter in the annotated corpus according to the chapter, wherein the chapter semantics is saved in the hierarchical form of chapter-chapter-section-paragraph-sentence-slot; the original sentence is contracted to obtain a new sentence, the correspondence between the new sentence and the semantics of the chapter is constructed, the new sentence and the corresponding semantic label are saved as one column in the form of a dictionary, and a semantic dictionary is generated; the semantic dictionary is dictionary-annotated, the sequence in the annotated dictionary is outputted as two columns of sentences and semantic labels by word, a CRF model is trained based on the two columns of sentences and semantic labels outputted by word, and a CRF model is constructed; the semantic label of the input document is predicted by the CRF model, and the semantic label of the input document is predicted by the semantic dictionary, with the semantic label determined by the semantic dictionary being given priority, and the semantic label queried by the semantic dictionary and the semantic label predicted by the CRF model are merged to obtain the final semantic label of the input document; The application layer is used to implement interactive operations with users.

8. The system according to claim 7, characterized in that The application layer is specifically used for: The modified final semantic tag is obtained and saved as annotated corpus in the data layer.

9. The system according to claim 7, characterized in that The application layer is specifically used for: reading input documents and editing Word documents.

10. The system according to claim 7, characterized in that The data processing layer is specifically used for: Read the annotated corpus, integrate the chapter names and sentences into one column in units of chapters, call the chapter semantic dictionary, determine the semantic tag corresponding to the chapter name, and obtain the sentence semantics from the annotated corpus, and superimpose the chapter semantics and the sentence semantics into one column; Taking the N words before and after the original sentence to form a reduced new sentence, adding the M words before and after the sentence to form a composite new sentence, and processing the special symbols in the new sentence; Save the new sentences and semantic labels in a dictionary format into one column to generate a semantic dictionary; Sorting the semantic dictionary according to long words first, establishing a word sequence and a corresponding semantic replacement sequence of the sentence, outputting the word sequence and the corresponding semantic replacement sequence in two columns by word, wherein the two columns are separated by a tab character; Run crf_learn.exe template CRF_in.txt CRF_model in cmd, train the CRF model by operating the input CRF_in.txt file, and build a CRF model named CRF_model; Read the input document, process the input document to the sentence granularity, shrink and merge to obtain a new sentence, transform the new sentence into 1 column, execute crf_test -m CRF_model CRF_test.txt>CRF_result.txt in cmd, apply the CRF model to calculate the output result, and take the semantic label that appears most in the sentence as the semantic label of the sentence according to the output result; Using an original sentence, querying a semantic tag of the original sentence from a text semantic dictionary and a sentence semantic dictionary; The semantic label determined according to the semantic dictionary is given priority, and the semantic label queried through the semantic dictionary and the semantic label predicted by the CRF model are fused to obtain the final semantic label of the input document.

Images