Text summary extraction method and system based on NLP technology

Abstract

The present invention relates to the field of natural language processing, and discloses a method and system for realizing text summary extraction based on NLP technology, wherein the method comprises: creating pre-trained multicultural terminology text, integrating a multilingual dictionary knowledge base of the multicultural terminology text, and constructing a multilingual summary model of the multilingual terminology text; obtaining the text to be extracted, calculating the complexity coefficient of the text to be extracted, stratifying the text to be extracted, and obtaining stratified text to be extracted; performing sentence segmentation on the stratified text to be extracted, obtaining sentence text to be extracted, performing effective value extraction on the sentence text to be extracted, obtaining effective sentence text, and marking the part of speech of the text word corresponding to the effective sentence text; identifying the dependency relationship between the text word corresponding to the effective sentence text, analyzing the sentence structure of the effective sentence text, and extracting the sentence features of the effective sentence text; and outputting the reconstructed summary of the text to be extracted using the multilingual summary model. The present invention can improve the accuracy of text content summary extraction.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a method and system for extracting text summaries based on NLP technology. Background Art

[0002] Text summarization is the process of automatically generating concise and coherent summaries from long articles, documents, reports or other forms of text. A summary is a shortened version of a text that contains the main ideas, information or conclusions of the original text, but omits details and redundant content. The purpose of text summarization is to reduce the amount of information that readers need to read and understand while retaining the key information.

[0003] Currently, text summarization mainly extracts important sentences by semantic analysis of the text and combines the sentences to form a summary. This method is difficult to accurately understand and summarize texts containing complex logic, metaphors or professional terms, thereby reducing the accuracy of text summary extraction. Summary of the invention

[0004] The present invention provides a method and system for extracting text summaries based on NLP technology, the main purpose of which is to improve the accuracy of text content summary extraction.

[0005] To achieve the above object, the present invention provides a method for extracting text summaries based on NLP technology, comprising:

[0006] Creating a pre-trained multicultural terminology text, integrating a multilingual dictionary knowledge base of the multicultural terminology text, and constructing a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base;

[0007] Acquire the text to be extracted, calculate the complexity coefficient of the text to be extracted, and stratify the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted;

[0008] Sentence the layered text to be extracted to obtain sentence text to be extracted, extract valid values ​​from the sentence text to be extracted to obtain valid sentence text, and mark the part of speech of the text word corresponding to the valid sentence text;

[0009] Based on the part of speech, identifying the dependency relationship between the text words corresponding to the effective sentence text, analyzing the sentence structure of the effective sentence text based on the dependency relationship, and extracting the sentence features of the effective sentence text based on the sentence structure;

[0010] Based on the sentence features, the multilingual summary model is used to output a reconstructed summary of the text to be extracted.

[0011] Optionally, the multilingual dictionary knowledge base integrating the multicultural terminology texts includes:

[0012] Performing data cleaning on the multicultural terminology text to obtain a cleaned terminology text;

[0013] Standardizing the cleaning terminology text to obtain a standardized terminology text;

[0014] Constructing a multilingual mapping relationship of the standardized terminology text;

[0015] Based on the multilingual mapping relationship, a multilingual dictionary knowledge base is integrated with the multicultural terminology texts.

[0016] Optionally, constructing a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base includes:

[0017] selecting a summary model architecture for the multicultural terminology text;

[0018] extracting terminology text features of the multicultural terminology text;

[0019] Based on the terminology text features and the multicultural terminology text, training the summary model architecture to obtain a trained multilingual summary model;

[0020] Calculating the ROUGE-L metric of the trained multilingual summarization model;

[0021] When the ROUGE-L metric meets the preset standard metric, a multilingual summary model of the multicultural terminology text is constructed.

[0022] Optionally, calculating the ROUGE-L metric of the trained multilingual summarization model includes:

[0023] Constructing a preset text for training the multilingual summary model;

[0024] Extracting a generated summary of the preset text based on the trained multilingual summary model;

[0025] Based on the generated summary, the ROUGE-L metric of the trained multilingual summary model is calculated using the following formula:

[0026]

[0027] in, represents the ROUGE-L metric for training a multilingual summarization model, indicates a reference summary, Indicates the generation of a summary. Generate summary and reference summary The length of the longest common subsequence of Indicates the length of the reference summary, Represents a collection of reference summaries.

[0028] Optionally, as an embodiment of the present invention, the calculating the complexity coefficient of the text to be extracted includes:

[0029] Identifying the complexity factor of the text to be extracted;

[0030] Analyzing the complex correlation between the complex factors and the text to be extracted;

[0031] Based on the text to be extracted, identifying the factor state value of the complex factor;

[0032] Based on the factor state value and the complex association relationship, the complexity coefficient of the text to be extracted is calculated.

[0033] Optionally, the calculating the complexity coefficient of the text to be extracted based on the factor state value and the complex association relationship includes:

[0034] Based on the factor state value and the complex association relationship, calculating the complex factor weight of the complex factor corresponding to the text to be extracted;

[0035] Based on the complexity factor weight and the factor state value, the complexity coefficient of the text to be extracted is calculated using the following formula:

[0036]

[0037] in, Represents the complexity coefficient of the text to be extracted, Indicates the number of complex factors corresponding to the text to be extracted, Indicates the text to be extracted The complex factor weights of complex factors, The text to be extracted The factor state value of a complex factor, Represents the correlation coefficient between the complex factors corresponding to the text to be extracted, Indicates A complex factor.

[0038] Optionally, the step of segmenting the layered text to be extracted into sentences to obtain segmented text to be extracted includes:

[0039] Constructing a sentence model of the hierarchical text to be extracted;

[0040] Based on the sentence model, identifying sentence boundaries of the hierarchical text to be extracted;

[0041] Based on the sentence boundaries, the layered text to be extracted is divided into sentences to obtain the sentence-divided text to be extracted.

[0042] Optionally, the identifying the dependency relationship between the valid sentence text and the corresponding text words based on the part of speech includes:

[0043] identifying part-of-speech features of the part-of-speech;

[0044] Based on the part of speech, marking the semantic role of the text word;

[0045] Based on the semantic roles, the text words are associated with roles;

[0046] Based on the role association, the dependency relationship between the text words corresponding to the valid sentence text is identified.

[0047] Optionally, outputting a reconstructed summary of the text to be extracted by using the multilingual summary model based on the sentence features includes:

[0048] Calculating feature weights of the sentence features;

[0049] Based on the feature weight, calculating the sentence weight of the valid sentence text;

[0050] Based on the sentence weights, selecting target sentences of the valid sentence text;

[0051] The multilingual summary model is used to perform sentence reconstruction on the target sentence to obtain a reconstructed summary of the text to be extracted.

[0052] In order to solve the above problems, the present invention also provides a text summary extraction system based on NLP technology, the system comprising:

[0053] A summary model building module, used to create a pre-trained multicultural terminology text, integrate a multilingual dictionary knowledge base of the multicultural terminology text, and build a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base;

[0054] A text stratification module, used for obtaining text to be extracted, calculating the complexity coefficient of the text to be extracted, and stratifying the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted;

[0055] A layered text processing module is used to perform sentence segmentation on the layered text to be extracted to obtain a sentence-based text to be extracted, perform effective value extraction on the sentence-based text to be extracted to obtain an effective sentence-based text, and mark the part of speech of the text word corresponding to the effective sentence-based text;

[0056] A sentence feature extraction module is used to identify the dependency relationship between the text words corresponding to the valid sentence text based on the part of speech, analyze the sentence structure of the valid sentence text based on the dependency relationship, and extract the sentence features of the valid sentence text based on the sentence structure;

[0057] A text summary extraction module is used to output a reconstructed summary of the text to be extracted based on the sentence features and using the multilingual summary model.

[0058] The embodiment of the present invention constructs a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base, which can use the multilingual dictionary knowledge base to improve the understanding of professional and cultural specific texts, thereby generating high-quality summaries; the embodiment of the present invention stratifies the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted, which can reduce the complexity of text extraction, thereby improving the effect of summary extraction; further, the embodiment of the present invention obtains sentence text to be extracted for further text processing and analysis by sentence segmentation of the stratified text to be extracted, and further, the embodiment of the present invention recognizes the dependency relationship of the effective sentence text based on the part of speech, which can better analyze text information, and finally, the embodiment of the present invention analyzes the sentence structure of the effective sentence text based on the dependency relationship, which can improve the accuracy of later sentence analysis, and the embodiment of the present invention outputs the reconstructed summary of the text to be extracted based on the sentence feature, which can realize the accurate extraction of the summary of the text to be extracted. Therefore, the text summary extraction method and system based on NLP technology proposed in the present invention can improve the accuracy of text content summary extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flowchart of a text summary extraction method based on NLP technology provided by an embodiment of the present invention;

[0060] Figure 2 A functional module diagram of a text summary extraction system based on NLP technology provided in one embodiment of the present invention;

[0061] Figure 3 A schematic diagram of the structure of an electronic device for implementing a text summary extraction system based on NLP technology provided by an embodiment of the present invention;

[0062] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0064] The embodiment of the present application provides a method for extracting a text summary based on NLP technology. The execution subject of the method for extracting a text summary based on NLP technology includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for extracting a text summary based on NLP technology can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0065] Reference Figure 1 FIG. 1 is a flow chart of a method for extracting a text summary based on NLP technology according to an embodiment of the present invention. In this embodiment, the method for extracting a text summary based on NLP technology includes:

[0066] S1. Create a pre-trained multicultural terminology text, integrate a multilingual dictionary knowledge base of the multicultural terminology text, and build a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base.

[0067] In the embodiment of the present invention, the multicultural terminology text refers to texts containing multiple cultures and languages ​​in the training data.

[0068] Optionally, the embodiment of the present invention can more accurately understand and process cross-language and cross-cultural texts by integrating the multilingual dictionary knowledge base of the multicultural terminology text, thereby improving the quality of abstract extraction. The multilingual dictionary knowledge base refers to a library of professional terms and culture-specific expressions collected from various sources.

[0069] As an embodiment of the present invention, the multilingual dictionary knowledge base integrating the multicultural terminology texts includes: performing data cleaning on the multicultural terminology texts to obtain cleaned terminology texts; performing standardization processing on the cleaned terminology texts to obtain standardized terminology texts; constructing a multilingual mapping relationship for the standardized terminology texts; and integrating the multilingual dictionary knowledge base for the multicultural terminology texts based on the multilingual mapping relationship.

[0070] Among them, the cleaned term text refers to the data after removing errors and redundant information from the multicultural term text, the standardized term text refers to the text after the cleaned term text is standardized in meaning, ensuring that synonyms and near-synonyms are correctly identified and associated, and the multilingual mapping relationship refers to establishing a cross-language mapping relationship for each term, ensuring that the expressions of the same concept in different languages ​​are correctly associated.

[0071] Furthermore, in the embodiment of the present invention, based on the multilingual dictionary knowledge base, a multilingual summary model for the multicultural terminology text is constructed, which can utilize the multilingual dictionary knowledge base to improve the understanding of professional and cultural specific texts, thereby generating high-quality summaries. The multilingual summary model refers to a model that is trained to extract summaries from parameter texts.

[0072] Further, as an embodiment of the present invention, constructing a multilingual summary model for the multicultural terminology text based on the multilingual dictionary knowledge base includes: selecting a summary model architecture for the multicultural terminology text; extracting terminology text features of the multicultural terminology text; training the summary model architecture based on the terminology text features and the multicultural terminology text to obtain a trained multilingual summary model; calculating the ROUGE-L metric of the trained multilingual summary model; and constructing a multilingual summary model for the multicultural terminology text when the ROUGE-L metric meets a preset standard metric.

[0073] Among them, the summary model architecture refers to the structure and design of the machine learning model used to generate text summaries, the term text features refer to the characteristic attributes of the multicultural term text, such as word frequency, TF-IDF, word embedding and other features, the trained multilingual summary model refers to the model architecture after text training, and the ROUGE-L metric refers to the longest common subsequence between the generated summary and the reference summary.

[0074] Further, in an optional embodiment of the present invention, the calculating of the ROUGE-L metric of the training multilingual summary model includes: constructing a preset text for the training multilingual summary model; extracting a generated summary of the preset text based on the training multilingual summary model; and calculating the ROUGE-L metric of the training multilingual summary model based on the generated summary using the following formula:

[0075]

[0076] in, represents the ROUGE-L metric for training a multilingual summarization model, indicates a reference summary, Indicates the generation of a summary. Generate summary and reference summary The length of the longest common subsequence of Indicates the length of the reference summary, Represents a collection of reference summaries.

[0077] in, Represents a set of high-quality, human-written summaries that are often considered the gold standard for evaluating the performance of automatic summarization systems.

[0078] S2. Obtain the text to be extracted, calculate the complexity coefficient of the text to be extracted, and stratify the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted.

[0079] In the embodiment of the present invention, the text to be extracted refers to the text that needs to be abstracted.

[0080] Optionally, the calculation of the complexity coefficient of the text to be extracted may be used as a basis for the text stratification order, wherein the complexity coefficient refers to the content complexity of the text to be extracted.

[0081] Optionally, as an embodiment of the present invention, the calculation of the complexity coefficient of the text to be extracted includes: identifying the complexity factors of the text to be extracted; analyzing the complex association relationship between the complex factors and the text to be extracted; based on the text to be extracted, identifying the factor state value of the complex factor; based on the factor state value and the complex association relationship, calculating the complexity coefficient of the text to be extracted.

[0082] Among them, the complexity factor refers to the factors that affect the complexity of the text to be extracted, such as vocabulary diversity, sentence length, grammatical complexity, terminology expertise, reading difficulty, information density and other factors; the complex association relationship refers to the association between the complexity factor and the complexity of the text to be extracted, for example, the higher the vocabulary diversity, the higher the complexity of the text to be extracted.

[0083] Further, in an optional embodiment of the present invention, the calculation of the complexity coefficient of the text to be extracted based on the factor state value and the complex association relationship includes: calculating the complex factor weight of the complex factor corresponding to the text to be extracted based on the factor state value and the complex association relationship; and calculating the complexity coefficient of the text to be extracted based on the complex factor weight and the factor state value using the following formula:

[0084]

[0085] in, Represents the complexity coefficient of the text to be extracted, Indicates the number of complex factors corresponding to the text to be extracted, Indicates the text to be extracted The complex factor weights of complex factors, The text to be extracted The factor state value of a complex factor, Represents the correlation coefficient between the complex factors corresponding to the text to be extracted, Indicates A complex factor.

[0086] The embodiment of the present invention stratifies the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted, which can reduce the complexity of text extraction, thereby improving the effect of summary extraction. The stratified text to be extracted refers to a text set obtained after stratifying the text to be extracted. In detail, the stratified text to be extracted refers to stratification according to the complexity coefficient and a preset stratification standard, for example, 2 layers are divided when the complexity coefficient is 0.5, and 4 layers are divided when the complexity coefficient is 1.

[0087] S3, dividing the layered text to be extracted into sentences to obtain sentence-based text to be extracted, performing effective value extraction on the sentence-based text to be extracted to obtain effective sentence text, and marking the part of speech of the text word corresponding to the effective sentence text.

[0088] The embodiment of the present invention divides the layered text to be extracted into sentences to obtain the sentence-divided text to be extracted for further text processing and analysis. The sentence-divided text to be extracted refers to the text after the layered text to be extracted is divided into sentences.

[0089] As an embodiment of the present invention, the sentence segmentation of the layered text to be extracted to obtain the sentence segmented text to be extracted includes: constructing a sentence segmentation model of the layered text to be extracted; based on the sentence segmentation model, identifying the sentence boundaries of the layered text to be extracted; based on the sentence boundaries, the layered text to be extracted is segmented to obtain the sentence segmented text to be extracted.

[0090] The sentence model refers to a model that can accurately identify text sentence boundaries, such as a sentence model based on LSTM or Transformer, and the sentence boundary refers to the start and end positions of a sentence in the text.

[0091] In the embodiment of the present invention, the valid sentence text refers to the text after the redundant information and invalid values ​​of the sentence text to be extracted are deleted. The part of speech refers to a part of speech tag assigned to each word, and these tags usually include noun (N), verb (V), adjective (A), adverb (R), etc.

[0092] S4. Based on the part of speech, identify the dependency relationship between the text words corresponding to the effective sentence text, analyze the sentence structure of the effective sentence text based on the dependency relationship, and extract the sentence features of the effective sentence text based on the sentence structure.

[0093] The embodiment of the present invention can better analyze text information by identifying the dependency relationship of the effective sentence text based on the part of speech, wherein the dependency relationship refers to the mutual dependency relationship between the text words corresponding to the effective sentence.

[0094] As an embodiment of the present invention, the process of identifying the dependency relationship between the valid sentence text and the corresponding text words based on the part of speech includes: identifying the part of speech features of the part of speech; marking the semantic role of the text words based on the part of speech; associating the text words with roles based on the semantic role; and identifying the dependency relationship between the valid sentence text and the corresponding text words based on the role association.

[0095] Among them, the part-of-speech feature refers to the characteristic attribute of the grammatical category or part of speech assigned to each vocabulary or word in natural language processing, and the semantic role refers to the defined role of each text word of each vocabulary in a sentence, and the semantic roles include agent, patient, instrument, target and other roles.

[0096] Optionally, the embodiment of the present invention analyzes the sentence structure of the valid sentence text based on the dependency relationship to improve the accuracy of subsequent sentence analysis. The sentence structure refers to the grammatical relationship and organization between words in a sentence. In detail, the sentence structure can be analyzed through a dependency graph of the dependency relationship.

[0097] In the embodiment of the present invention, the sentence features refer to features extracted from the grammatical structure of the valid sentence text, such as sentence length, sentence type, subject-predicate-object structure, modification relationship, sentence position and other features.

[0098] S5. Based on the sentence features, use the multilingual summary model to output a reconstructed summary of the text to be extracted.

[0099] In the embodiment of the present invention, based on the sentence features, the multilingual summary model is used to output a reconstructed summary of the text to be extracted, so as to accurately extract the summary of the text to be extracted. The reconstructed summary refers to the summary extracted by the multilingual summary model from the text to be extracted.

[0100] As an embodiment of the present invention, the method of outputting a reconstructed summary of the text to be extracted based on the sentence features and using the multilingual summary model includes: calculating feature weights of the sentence features; calculating sentence weights of the valid sentence text based on the feature weights; screening target sentences of the valid sentence text based on the sentence weights; and performing sentence reconstruction on the target sentence using the multilingual summary model to obtain a reconstructed summary of the text to be extracted.

[0101] Among them, the feature weight refers to the influence of the sentence feature in the text, the sentence weight refers to the importance of the sentence to be extracted in the text, and the target sentence refers to a sentence whose sentence weight value is higher than the preset standard weight.

[0102] The embodiment of the present invention constructs a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base, which can use the multilingual dictionary knowledge base to improve the understanding of professional and cultural specific texts, thereby generating high-quality summaries; the embodiment of the present invention stratifies the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted, which can reduce the complexity of text extraction, thereby improving the effect of summary extraction; further, the embodiment of the present invention obtains sentence text to be extracted for further text processing and analysis by sentence segmentation of the stratified text to be extracted, and further, the embodiment of the present invention recognizes the dependency relationship of the effective sentence text based on the part of speech, which can better analyze text information, and finally, the embodiment of the present invention analyzes the sentence structure of the effective sentence text based on the dependency relationship, which can improve the accuracy of later sentence analysis, and the embodiment of the present invention outputs the reconstructed summary of the text to be extracted based on the sentence feature, which can realize the accurate extraction of the summary of the text to be extracted. Therefore, the text summary extraction method based on NLP technology proposed in the present invention can improve the accuracy of text content summary extraction.

[0103] like Figure 2 , which is a functional module diagram of a text summary extraction system based on NLP technology provided by an embodiment of the present invention.

[0104] The text summary extraction system 200 based on NLP technology of the present invention can be installed in an electronic device. According to the functions implemented, the text summary extraction system 200 based on NLP technology can include a summary model construction module 201, a text stratification module 202, a stratified text processing module 203, a sentence feature extraction module 204 and a text summary extraction module 205. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0105] In this embodiment, the functions of each module / unit are as follows:

[0106] The summary model construction module 201 is used to create a pre-trained multi-cultural terminology text, integrate a multi-lingual dictionary knowledge base of the multi-cultural terminology text, and construct a multi-lingual summary model of the multi-cultural terminology text based on the multi-lingual dictionary knowledge base;

[0107] The text stratification module 202 is used to obtain the text to be extracted, calculate the complexity coefficient of the text to be extracted, and stratify the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted;

[0108] The layered text processing module 203 is used to perform sentence segmentation on the layered text to be extracted to obtain the sentence-based text to be extracted, perform effective value extraction on the sentence-based text to be extracted to obtain effective sentence-based text, and mark the part of speech of the text word corresponding to the effective sentence-based text;

[0109] The sentence feature extraction module 204 is used to identify the dependency relationship between the text words corresponding to the valid sentence text based on the part of speech, analyze the sentence structure of the valid sentence text based on the dependency relationship, and extract the sentence features of the valid sentence text based on the sentence structure;

[0110] The text summary extraction module 205 is used to output a reconstructed summary of the text to be extracted based on the sentence features and using the multilingual summary model.

[0111] In detail, the modules described in the text summary extraction system 200 based on NLP technology described in the embodiment of the present invention adopt the same technical means as the text summary extraction method based on NLP technology described in the accompanying drawings when used, and can produce the same technical effects, which will not be repeated here.

[0112] An embodiment of the present invention provides an electronic device for implementing a text summary extraction method based on NLP technology.

[0113] See also Figure 3 As shown, the electronic device may include a processor 30, a memory 31, a communication bus 32 and a communication interface 33, and may also include a computer program stored in the memory 31 and executable on the processor 30, such as a program for implementing a text summary extraction method based on NLP technology.

[0114] In some embodiments, the processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory (for example, executing a text summary extraction program based on NLP technology, etc.), and calling data stored in the memory.

[0115] The memory includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory may also include both an internal storage unit of the electronic device and an external storage device. The memory may not only be used to store application software and various types of data installed in the electronic device, such as the code for implementing a text summary extraction program based on NLP technology, but may also be used to temporarily store data that has been output or is to be output.

[0116] The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory and at least one processor, etc.

[0117] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0118] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0119] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0120] The text summary extraction program based on NLP technology stored in the memory of the electronic device is a combination of multiple instructions. When running in the processor, it can achieve:

[0121] Creating a pre-trained multicultural terminology text, integrating a multilingual dictionary knowledge base of the multicultural terminology text, and constructing a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base;

[0122] Acquire the text to be extracted, calculate the complexity coefficient of the text to be extracted, and stratify the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted;

[0123] Sentence the layered text to be extracted to obtain sentence text to be extracted, extract valid values ​​from the sentence text to be extracted to obtain valid sentence text, and mark the part of speech of the text word corresponding to the valid sentence text;

[0124] Based on the part of speech, identifying the dependency relationship between the text words corresponding to the effective sentence text, analyzing the sentence structure of the effective sentence text based on the dependency relationship, and extracting the sentence features of the effective sentence text based on the sentence structure;

[0125] Based on the sentence features, the multilingual summary model is used to output a reconstructed summary of the text to be extracted.

[0126] Specifically, the specific implementation method of the processor for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0127] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or system that can carry the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0128] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0129] Creating a pre-trained multicultural terminology text, integrating a multilingual dictionary knowledge base of the multicultural terminology text, and constructing a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base;

[0130] Acquire the text to be extracted, calculate the complexity coefficient of the text to be extracted, and stratify the text to be extracted based on the complexity coefficient to obtain stratified text to be extracted;

[0131] Sentence the layered text to be extracted to obtain sentence text to be extracted, extract valid values ​​from the sentence text to be extracted to obtain valid sentence text, and mark the part of speech of the text word corresponding to the valid sentence text;

[0132] Based on the part of speech, identifying the dependency relationship between the text words corresponding to the effective sentence text, analyzing the sentence structure of the effective sentence text based on the dependency relationship, and extracting the sentence features of the effective sentence text based on the sentence structure;

[0133] Based on the sentence features, the multilingual summary model is used to output a reconstructed summary of the text to be extracted.

[0134] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0135] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0137] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0138] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0139] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A text summary extraction method based on NLP technology, characterized in that: The method comprises: Creating a pre-trained multicultural terminology text, integrating a multilingual dictionary knowledge base of the multicultural terminology text, and constructing a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base; Obtain a text to be extracted, identify the complexity factor of the text to be extracted, wherein the complexity factor refers to a factor that affects the complexity of the text to be extracted, including vocabulary diversity, sentence length, grammatical complexity, terminology expertise, reading difficulty and information density, analyze the complex correlation between the complex factor and the text to be extracted, identify the factor state value of the complex factor based on the text to be extracted, and calculate the complexity coefficient of the text to be extracted based on the factor state value and the complex correlation, wherein the calculation of the complexity coefficient of the text to be extracted based on the factor state value and the complex correlation includes: calculating the complex factor weight of the complex factor corresponding to the text to be extracted based on the factor state value and the complex correlation, and calculating the complexity coefficient of the text to be extracted based on the complex factor weight and the factor state value using the following formula: in, Represents the complexity coefficient of the text to be extracted, Indicates the number of complex factors corresponding to the text to be extracted, Indicates the text to be extracted The complex factor weights of complex factors, The text to be extracted The factor state value of a complex factor, Represents the correlation coefficient between the complex factors corresponding to the text to be extracted, Indicates A complexity factor is obtained, and based on the complexity coefficient, the text to be extracted is layered to obtain layered text to be extracted, wherein the layered text to be extracted refers to a text set obtained after the text to be extracted is layered, and the layered text to be extracted is layered according to the complexity coefficient and a preset layering standard; Sentence the layered text to be extracted to obtain sentence text to be extracted, extract valid values ​​from the sentence text to be extracted to obtain valid sentence text, and mark the part of speech of the text word corresponding to the valid sentence text; Based on the part of speech, identifying the dependency relationship between the text words corresponding to the effective sentence text, analyzing the sentence structure of the effective sentence text based on the dependency relationship, and extracting the sentence features of the effective sentence text based on the sentence structure; Based on the sentence features, the multilingual summary model is used to output a reconstructed summary of the text to be extracted.

2. The method for extracting text summaries based on NLP technology as claimed in claim 1, characterized in that: A multilingual dictionary knowledge base integrating the multicultural terminology texts includes: Performing data cleaning on the multicultural terminology text to obtain a cleaned terminology text; Standardizing the cleaning terminology text to obtain a standardized terminology text; Constructing a multilingual mapping relationship of the standardized terminology text; Based on the multilingual mapping relationship, a multilingual dictionary knowledge base is integrated with the multicultural terminology texts.

3. The method for extracting text summaries based on NLP technology as claimed in claim 1, characterized in that: Based on the multilingual dictionary knowledge base, a multilingual summary model of the multicultural terminology text is constructed, including: selecting a summary model architecture for the multicultural terminology text; extracting terminology text features of the multicultural terminology text; Based on the terminology text features and the multicultural terminology text, training the summary model architecture to obtain a trained multilingual summary model; Calculating the ROUGE-L metric of the trained multilingual summarization model; When the ROUGE-L metric meets the preset standard metric, a multilingual summary model of the multicultural terminology text is constructed.

4. The method for extracting text summaries based on NLP technology as claimed in claim 3, characterized in that: Calculating the ROUGE-L metric of the trained multilingual summarization model includes: Constructing a preset text for training the multilingual summary model; Extracting a generated summary of the preset text based on the trained multilingual summary model; Based on the generated summary, the ROUGE-L metric of the trained multilingual summary model is calculated using the following formula: in, represents the ROUGE-L metric for training a multilingual summarization model, indicates a reference summary, Indicates the generation of a summary. Generate summary and reference summary The length of the longest common subsequence of Indicates the length of the reference summary, Represents a collection of reference summaries.

5. The method for extracting text summaries based on NLP technology as claimed in claim 1, characterized in that: Sentence the layered text to be extracted to obtain the sentence-separated text to be extracted, including: Constructing a sentence model of the hierarchical text to be extracted; Based on the sentence model, identifying sentence boundaries of the hierarchical text to be extracted; Based on the sentence boundaries, the layered text to be extracted is divided into sentences to obtain the sentence-divided text to be extracted.

6. The method for extracting text summaries based on NLP technology as claimed in claim 1, characterized in that: Based on the part of speech, identifying the dependency relationship between the valid sentence text and the text word, including: identifying part-of-speech features of the part-of-speech; Based on the part of speech, marking the semantic role of the text word; Based on the semantic roles, the text words are associated with roles; Based on the role association, the dependency relationship between the text words corresponding to the valid sentence text is identified.

7. The method for extracting text summaries based on NLP technology as claimed in claim 1, characterized in that: Based on the sentence features, outputting a reconstructed summary of the text to be extracted using the multilingual summary model includes: Calculating feature weights of the sentence features; Based on the feature weight, calculating the sentence weight of the valid sentence text; Based on the sentence weights, selecting target sentences of the valid sentence text; The multilingual summary model is used to perform sentence reconstruction on the target sentence to obtain a reconstructed summary of the text to be extracted.

8. A text summary extraction system based on NLP technology, characterized in that: The system is used to implement the text summary extraction method based on NLP technology as described in any one of claims 1 to 7, and comprises: A summary model building module, used to create a pre-trained multicultural terminology text, integrate a multilingual dictionary knowledge base of the multicultural terminology text, and build a multilingual summary model of the multicultural terminology text based on the multilingual dictionary knowledge base; The text stratification module is used to obtain the text to be extracted, identify the complex factors of the text to be extracted, analyze the complex association relationship between the complex factors and the text to be extracted, identify the factor state value of the complex factor based on the text to be extracted, calculate the complex factor weight of the complex factor corresponding to the text to be extracted based on the factor state value and the complex association relationship, and calculate the complex coefficient of the text to be extracted based on the complex factor weight and the factor state value using the following formula: in, Represents the complexity coefficient of the text to be extracted, Indicates the number of complex factors corresponding to the text to be extracted, Indicates the text to be extracted The complex factor weights of complex factors, The text to be extracted The factor state value of a complex factor, Represents the correlation coefficient between the complex factors corresponding to the text to be extracted, Indicates A complexity factor, based on which the text to be extracted is layered to obtain layered text to be extracted; A layered text processing module is used to perform sentence segmentation on the layered text to be extracted to obtain a sentence-based text to be extracted, perform effective value extraction on the sentence-based text to be extracted to obtain an effective sentence-based text, and mark the part of speech of the text word corresponding to the effective sentence-based text; A sentence feature extraction module, used for identifying the dependency relationship between the text words corresponding to the valid sentence text based on the part of speech, analyzing the sentence structure of the valid sentence text based on the dependency relationship, and extracting the sentence features of the valid sentence text based on the sentence structure; A text summary extraction module is used to output a reconstructed summary of the text to be extracted based on the sentence features and using the multilingual summary model.

Images