Text semantic signal automatic extraction method and system

Abstract

The embodiment of the invention discloses a text semantic signal automatic extraction method and system, and belongs to the technical field of natural language processing, and the method comprises the steps: receiving an input article text or paragraph text, carrying out the preprocessing of the text, carrying out the comprehensive semantic signal analysis and extraction of the preprocessed text, extracting a plurality of semantic signals from the text, and carrying out the automatic extraction of the semantic signals. According to the method, the semantic signals extracted from the text are recognized and comprehensively processed, the main semantic purpose of the semantic signals is obtained, a system framework can become a unified whole, extraction and comprehensive analysis of the multi-dimensional semantic signals are completed, and through comprehensive semantic signal extraction, sentence structures and virtual and real semantics are analyzed, so that the semantic effect of the text is improved. The method overcomes the defects in the context understanding aspect in the prior art, so that the main expression intention of the text can be better recognized, the method is suitable for various practical application scenes, and the application value is remarkably improved.

Description

Technical Field

[0001] The present application belongs to the technical field of natural language processing, and specifically, relates to a method and system for automatically extracting text semantic signals. Background Art

[0002] In the field of natural language processing, text semantic analysis technology has been widely developed and applied. Current research mainly focuses on sentiment analysis, topic classification, semantic similarity calculation, etc. These methods can effectively analyze the basic semantic information of the text and extract the text's emotional tendency, topic characteristics and other information.

[0003] Although existing technologies have made significant progress in text semantic analysis, they are still insufficient in extracting fine-grained semantic signals: existing technologies often focus on extracting semantic signals in a single dimension, but lack comprehensive analysis and integration of multi-dimensional semantic signals in the text. This leads to the inability to fully understand and extract the main semantic purpose of the text in practical applications; existing semantic analysis technologies often perform poorly in identifying complex semantic structures and contextual relationships. When faced with texts containing complex expressions such as metaphors and puns, it is difficult to accurately extract key signals, which affects the overall effect of semantic analysis. Existing technologies usually only focus on semantic signals in a single dimension and lack an effective mechanism to integrate multiple semantic dimensions, making it impossible to fully understand the semantic intent of the text. Summary of the invention

[0004] In order to solve the above problems and technical defects, the embodiment of the present application adopts the following technical solution, a method for automatically extracting text semantic signals, comprising the following steps:

[0005] Step 1, receiving the input article text or paragraph text, and preprocessing the text;

[0006] Step 2: Perform comprehensive semantic signal analysis and extraction on the preprocessed text to extract multiple semantic signals from the text;

[0007] Step 3: Identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

[0008] Preferably, the preprocessing includes: first segmenting the text, dividing the text into multiple sentences according to punctuation marks, then performing word segmentation on each sentence, and performing part-of-speech tagging and named entity recognition on each word.

[0009] Preferably, the comprehensive semantic signal analysis and extraction is completed by establishing a unified scheduling framework, using the unified scheduling framework to connect to the operation database, obtaining vocabulary and domain information, using regular expressions to perform module matching based on the vocabulary and domain information, and then performing text processing to complete the analysis and extraction of the comprehensive semantic signal.

[0010] Furthermore, when the text is processed, the OOP object-oriented programming design method is adopted to implement the rewriting of class inheritance methods, expand and reuse functions, and then use the ThreadPoolExecutor class analysis to perform multi-threaded operations.

[0011] Furthermore, the unified scheduling framework includes:

[0012] Template method module, which is used to redefine the steps of the algorithm without changing the algorithm structure, so that each subclass can add or rewrite specific analysis steps as needed while inheriting the general methods of the parent class without modifying the parent class code;

[0013] A combination module is used to combine target objects into a tree-like hierarchical structure that can represent parts and the whole, so that users can use single objects and combined objects in a consistent manner;

[0014] The singleton module is used to provide a global access point when connecting to related classes and methods in the operating database. The global access point can be reused multiple times throughout the analysis process;

[0015] The strategy module is used to regard different signal analysis algorithms as different strategies, define the signal analysis algorithms, and perform strategy encapsulation on each algorithm so that the encapsulated algorithms can be interchangeable.

[0016] Furthermore, the identification and comprehensive processing is to obtain the main semantic purpose of the semantic signal by analyzing the sentence structure and the imaginary and real semantics, and to identify the main expression intention of the text. The specific process is as follows:

[0017] Use regular expressions to detect semantic signals extracted from text to determine whether the sentence contains reference signals;

[0018] If it contains a reference signal, the singleton module is called according to the expression form of the reference signal to perform regular matching of the text, obtain the historical text with the reference signal, and obtain the expression content of the sentence;

[0019] The signal words in the sentence are analyzed according to the expression content of the sentence and the code is recognized. The recognized code is used as the main semantic purpose of each sentence and the result is stored.

[0020] Furthermore, the unified scheduling framework also includes a fault-tolerant mechanism module, in which a fault-tolerant mechanism is provided, so that when processing complex or non-standard texts, semantic signals can still be extracted and reasonable results can be output, and the fault-tolerant mechanism includes:

[0021] Exception handling mechanism, used to capture and handle exceptions, prevent program crashes and record error logs;

[0022] Logging mechanism, used to record operation logs, and can debug and troubleshoot problems based on operation logs;

[0023] Data detection mechanism is used to check data integrity and consistency when processing data to ensure that the data meets expectations;

[0024] Run recovery mechanism, which is used to ensure that the program continues to run using default settings when an abnormality or operation failure is detected;

[0025] Resource protection mechanism, which is used to ensure that data resources can be properly closed at the end of program execution to avoid resource leakage;

[0026] The result return mechanism is used to return a default value when module matching, algorithm matching, and text matching do not match any results;

[0027] The grammatical error correction mechanism is used to make correction judgments on the syntactic structure blocks of each sentence during syntactic analysis. After the syntactic analysis is completed, correction judgments will be made again based on the overall content of the sentence.

[0028] A text semantic signal automatic extraction system, comprising:

[0029] A text acquisition module is used to receive input article text or paragraph text and pre-process the text;

[0030] The signal extraction module is used to perform comprehensive semantic signal analysis and extraction on the preprocessed text and extract multiple semantic signals from the text;

[0031] The semantic acquisition module is used to identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

[0032] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for automatically extracting text semantic signals when executing the program.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for automatically extracting text semantic signals as described above.

[0034] Compared with the prior art, the beneficial effects of the embodiments of the present application are:

[0035] This application takes into account the characteristics of different semantic signals and introduces automatic extraction and integration of multi-dimensional semantic signals to achieve a high cohesion and low coupling state of the code, enhance the maintainability and scalability of the code, make the system framework a unified whole, complete the extraction and comprehensive analysis of multi-dimensional semantic signals, and greatly improve the efficiency of text processing. Through comprehensive semantic signal extraction, analysis of sentence structure and real and imaginary semantics, it overcomes the shortcomings of the prior art in context understanding, thereby being able to better identify the main expression intention of the text, is suitable for a variety of practical application scenarios, and significantly improves the application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In the attached picture:

[0037] Figure 1 Schematic diagram of the method steps of the embodiment of this application

[0038] Figure 2 A schematic diagram of the system structure of an embodiment of the present application;

[0039] Figure 3 This is a schematic diagram of the device structure of an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] Example 1

[0042] like Figure 1 As shown, a method and system for automatically extracting text semantic signals include the following steps:

[0043] Receive input article text or paragraph text and pre-process the text;

[0044] Preprocessing includes: first segmenting the text, dividing the text into multiple sentences according to punctuation marks, then segmenting each sentence, and performing part-of-speech tagging and named entity recognition on each word.

[0045] Perform comprehensive semantic signal analysis and extraction on the preprocessed text to extract multiple semantic signals from the text;

[0046] The comprehensive semantic signal analysis and extraction is completed by establishing a unified scheduling framework, using the unified scheduling framework to connect to the operation database, obtain vocabulary and domain information, use regular expressions to match modules according to the vocabulary and domain information, and then perform text processing to complete the analysis and extraction of comprehensive semantic signals.

[0047] When processing text, the OOP object-oriented programming design method is adopted to rewrite the class inheritance method, expand and reuse the function, and then use the ThreadPoolExecutor class analysis to perform multi-threaded operations to improve the query and processing speed.

[0048] The unified scheduling framework includes:

[0049] Template method module, which is used to redefine the steps of the algorithm without changing the algorithm structure, so that each subclass can add or rewrite specific analysis steps as needed while inheriting the general methods of the parent class without modifying the parent class code;

[0050] For example, the AD class defines some general semantic analysis methods, while the LF class and the P class not only inherit the AD class, but also add more specific analysis methods based on it.

[0051] A combination module is used to combine target objects into a tree-like hierarchical structure that can represent parts and the whole, so that users can use single objects and combined objects in a consistent manner;

[0052] For example, the code for processing a single sentence and the code for processing an entire paragraph are consistent, which facilitates expansion and maintenance.

[0053] The singleton module is used to provide a global access point when connecting to related classes and methods in the operating database. The global access point can be reused multiple times throughout the analysis process;

[0054] Ensures uniqueness and global accessibility of operation database connections.

[0055] The strategy module is used to regard different signal analysis algorithms as different strategies, define the signal analysis algorithms, and perform strategy encapsulation on each algorithm so that the encapsulated algorithms can be interchangeable.

[0056] Identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals;

[0057] Recognition and comprehensive processing is to obtain the main semantic purpose of the semantic signal by analyzing the sentence structure and the real and imaginary semantics, and to identify the main expression intention of the text. The specific process is as follows:

[0058] Use regular expressions to detect semantic signals extracted from text to determine whether the sentence contains reference signals;

[0059] If it contains a reference signal, the singleton module is called according to the expression form of the reference signal to perform regular matching of the text, obtain the historical text with the reference signal, and obtain the expression content of the sentence;

[0060] The signal words in the sentence are analyzed according to the expression content of the sentence and the code is recognized. The recognized code is used as the main semantic purpose of each sentence and the result is stored.

[0061] Each semantic signal has a corresponding identification code, and each code corresponds one-to-one to a semantic signal.

[0062] Semantic signals include:

[0063] Level signals, which identify the signal level of semantic signals by analyzing the specificity of text details;

[0064] Emotional signals: Analyze and extract emotional signals in texts to determine the emotional tendency of texts;

[0065] Expression signals, identifying whether the text contains relevant expressions of the author’s or speaker’s expression signals;

[0066] Lexical signal, extracting the signal words of lexical signal;

[0067] Participle signals, words used to indicate divisions between paragraphs or sentences;

[0068] Content signal, by analyzing the content of the text, determines whether the text falls within the content signal range.

[0069] The unified scheduling framework also includes a fault-tolerant mechanism module. The fault-tolerant mechanism module is equipped with a fault-tolerant mechanism. When processing complex or non-standard texts, it can still extract semantic signals and output reasonable results, improve the high robustness and adaptability of the method, and maintain stable performance in different text types and application scenarios. The fault-tolerant mechanism includes:

[0070] Exception handling mechanism, used to capture and handle exceptions, prevent program crashes and record error logs;

[0071] Logging mechanism, used to record operation logs, and can debug and troubleshoot problems based on operation logs;

[0072] Data detection mechanism, used to perform data integrity and consistency checks when processing data, such as list length consistency checks, to ensure that the data meets expectations;

[0073] Run recovery mechanism, which is used to ensure that the program continues to run using default settings when an abnormality or operation failure is detected;

[0074] Resource protection mechanism, which is used to ensure that data resources, such as database connections, are properly closed at the end of the program to avoid resource leaks;

[0075] The result return mechanism is used to return a default value when module matching, algorithm matching, and text matching do not match any results;

[0076] The grammatical error correction mechanism is used to make correction judgments on the syntactic structure blocks of each sentence during syntactic analysis. After the syntactic analysis is completed, correction judgments will be made again based on the overall content of the sentence. The correction judgment is to determine whether the sentence's syntax is incorrect based on grammatical rules. If the sentence's syntax is incorrect, a preset signal is output.

[0077] The unified scheduling framework also includes an automatic syntax analysis module, which reduces manual intervention and operation, improves processing efficiency, and saves labor costs through automated design and implementation.

[0078] It can extract and comprehensively analyze multi-dimensional semantic signals without human intervention, automatically generate syntactic structures and perform part-of-speech tagging, and improve the efficiency of text processing;

[0079] Automatically extract the semantic information of sentences through semantic signal extraction methods, and conduct multi-dimensional analysis to achieve fully automatic semantic signal extraction and integration;

[0080] At the same time, multi-threaded query is realized through parallel processing to improve query efficiency;

[0081] Based on the extracted semantic signal interpretation analysis results, a user-oriented descriptive natural language text is generated to complete the automatic interpretation;

[0082] By automatically integrating the interpretation of each signal and generating automated reports, the interaction between the system and the user becomes more efficient and user-friendly.

[0083] Through comprehensive semantic signal extraction, it is suitable for a variety of practical application scenarios, including but not limited to automatic text summary generation, sentiment analysis, topic classification, semantic search engine optimization, etc.

[0084] Its multi-dimensional semantic analysis capability enables it to demonstrate higher effectiveness and accuracy in these applications, significantly improving the application value.

[0085] Example 2

[0086] like Figure 2 As shown, a text semantic signal automatic extraction system includes:

[0087] A text acquisition module is used to receive input article text or paragraph text and pre-process the text;

[0088] The signal extraction module is used to perform comprehensive semantic signal analysis and extraction on the preprocessed text and extract multiple semantic signals from the text;

[0089] The semantic acquisition module is used to identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

[0090] Example 3

[0091] like Figure 3 As shown, from the hardware level, the present application provides an embodiment of an electronic device that implements all or part of the content of the text semantic signal automatic extraction method, and the electronic device specifically includes the following content:

[0092] Step 1, receiving the input article text or paragraph text, and preprocessing the text;

[0093] Step 2: Perform comprehensive semantic signal analysis and extraction on the preprocessed text to extract multiple semantic signals from the text;

[0094] Step 3: Identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

[0095] Example 4

[0096] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the method for automatically extracting text semantic signals in the above embodiments, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the method for automatically extracting text semantic signals in the above embodiments are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0097] Step 1, receiving the input article text or paragraph text, and preprocessing the text;

[0098] Step 2: Perform comprehensive semantic signal analysis and extraction on the preprocessed text to extract multiple semantic signals from the text;

[0099] Step 3: Identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

[0100] The above-mentioned embodiments only express the preferred implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications, improvements and substitutions can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for automatically extracting text semantic signals, characterized in that: The following steps are involved: Step 1, receiving the input article text or paragraph text, and preprocessing the text; Step 2: Perform comprehensive semantic signal analysis and extraction on the preprocessed text to extract multiple semantic signals from the text; Step 3: Identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

2. The method for automatically extracting text semantic signals according to claim 1, characterized in that: The preprocessing includes: first segmenting the text, segmenting the text into multiple sentences according to punctuation marks, then performing word segmentation on each sentence, and performing part-of-speech tagging and named entity recognition on each word.

3. The method for automatically extracting text semantic signals according to claim 1, characterized in that: The comprehensive semantic signal analysis and extraction is completed by establishing a unified scheduling framework, using the unified scheduling framework to connect to the operation database, obtaining vocabulary and domain information, using regular expressions to perform module matching based on the vocabulary and domain information, and then performing text processing to complete the analysis and extraction of the comprehensive semantic signal.

4. The method for automatically extracting text semantic signals according to claim 3, characterized in that: When the text is processed, the OOP object-oriented programming design method is adopted to implement the rewriting of class inheritance methods, expand and reuse functions, and then use the ThreadPoolExecutor class analysis to perform multi-threaded operations.

5. The method for automatically extracting text semantic signals according to claim 4, characterized in that: The unified scheduling framework includes: Template method module, which is used to redefine the steps of the algorithm without changing the algorithm structure, so that each subclass can add or rewrite specific analysis steps as needed while inheriting the general methods of the parent class without modifying the parent class code; A combination module is used to combine target objects into a tree-like hierarchical structure that can represent parts and the whole, so that users can use single objects and combined objects in a consistent manner; The singleton module is used to provide a global access point when connecting to related classes and methods in the operating database. The global access point can be reused multiple times throughout the analysis process; The strategy module is used to regard different signal analysis algorithms as different strategies, define the signal analysis algorithms, and perform strategy encapsulation on each algorithm so that the encapsulated algorithms can be interchangeable.

6. The method for automatically extracting text semantic signals according to claim 5, characterized in that: The identification and comprehensive processing is performed by analyzing the sentence structure and the imaginary and real semantics, obtaining the main semantic purpose of the semantic signal, and identifying the main expression intention of the text. The specific process is as follows: Use regular expressions to detect semantic signals extracted from text to determine whether the sentence contains reference signals; If it contains a reference signal, the singleton module is called according to the expression form of the reference signal to perform regular matching of the text, obtain the historical text with the reference signal, and obtain the expression content of the sentence; The signal words in the sentence are analyzed according to the expression content of the sentence and the code is recognized. The recognized code is used as the main semantic purpose of each sentence and the result is stored.

7. The method for automatically extracting text semantic signals according to claim 6, characterized in that: The unified scheduling framework also includes a fault-tolerant mechanism module, which is provided with a fault-tolerant mechanism. When processing complex or non-standard texts, it is still possible to extract semantic signals and output reasonable results. The fault-tolerant mechanism includes: Exception handling mechanism, used to capture and handle exceptions, prevent program crashes and record error logs; Logging mechanism, used to record operation logs, and can debug and troubleshoot problems based on operation logs; Data detection mechanism is used to check data integrity and consistency when processing data to ensure that the data meets expectations; Run recovery mechanism, which is used to ensure that the program continues to run using default settings when an abnormality or operation failure is detected; Resource protection mechanism, which is used to ensure that data resources can be properly closed at the end of program execution to avoid resource leakage; The result return mechanism is used to return a default value when module matching, algorithm matching, and text matching do not match any results; The grammatical error correction mechanism is used to make correction judgments on the syntactic structure blocks of each sentence during syntactic analysis. After the syntactic analysis is completed, correction judgments will be made again based on the overall content of the sentence.

8. A text semantic signal automatic extraction system, characterized in that: include: A text acquisition module is used to receive input article text or paragraph text and pre-process the text; The signal extraction module is used to perform comprehensive semantic signal analysis and extraction on the preprocessed text and extract multiple semantic signals from the text; The semantic acquisition module is used to identify and comprehensively process the semantic signals extracted from the text to obtain the main semantic purpose of the semantic signals.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for automatically extracting text semantic signals according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically extracting text semantic signals according to any one of claims 1 to 7 are implemented.

Images