A text classification method and system based on natural language processing

Abstract

The present invention relates to the field of text processing technology, specifically a text classification method and system based on natural language processing, comprising the following steps: loading a text data set, performing part-of-speech analysis on each paragraph of text, determining the part-of-speech of each word through natural language processing, extracting the basic grammatical features of the text, and obtaining preliminary grammatical analysis results. In the present invention, by deeply applying natural language processing technology, combining detailed syntactic dependency analysis and graphical representation methods, the parsing depth of text structure and semantics is significantly improved, dependency labels are constructed, and the complex semantic network inside the text is effectively revealed, and the understanding of sentence structure is enhanced. Graphical representation and feature aggregation technology strengthen the key information in the text, and by adjusting the weights of key dependencies, the importance assessment of information is optimized. Multi-level text analysis can adapt to the needs of different semantic levels, especially when processing complex texts that require deep semantic understanding, it is more efficient.

Description

Technical Field

[0001] The present invention relates to the technical field of text processing, and in particular to a text classification method and system based on natural language processing. Background Art

[0002] The field of text processing technology includes data input, processing, storage and output, involving algorithms and methods to analyze, understand and generate text data. The core content includes text mining, search engine technology, semantic understanding and language generation. The systematic introduction covers a variety of application scenarios from basic text data processing to complex natural language processing applications, such as sentiment analysis, topic classification and machine translation.

[0003] Among them, text classification methods refer to the technology of using computer programs to identify and classify information from text data, and the technical matters targeted include text feature extraction and classification decision making. Specifically, key features are extracted through text analysis, and then classification algorithms are used to systematically classify texts based on the features, such as rule-based classification or machine learning classification methods.

[0004] Although existing technologies provide basic text processing capabilities, they generally lack in-depth understanding of complex semantics and structures, especially when dealing with complex structured or profound text data. Traditional text classification technology mainly relies on surface feature extraction and simple statistical analysis, and is often unable to cope with the diversity of language expressions and complex syntactic structures. In addition, existing methods are also limited in adaptability and flexibility, and it is difficult to effectively update to adapt to emerging text types and changing language usage. For example, the complexity of legal documents and academic articles often exceeds the processing scope of simple rules or basic machine learning models, resulting in low classification accuracy and efficiency. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a text classification method and system based on natural language processing.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a text classification method based on natural language processing, comprising the following steps:

[0007] S1: Load the text dataset, perform part-of-speech analysis on each text segment, determine the part-of-speech of each word through natural language processing, extract the basic grammatical features of the text, and obtain preliminary grammatical analysis results;

[0008] S2: Based on the preliminary grammatical analysis results, construct dependency labels for each word, analyze the dependencies between words one by one, reveal the interdependence of words in the sentence, and draw a syntactic dependency structure diagram with the connection between each word and other words represented as an edge and the word itself as a node;

[0009] S3: performing feature aggregation on the nodes in the syntactic dependency structure graph, identifying the neighbor nodes of each node and integrating the feature information of the neighbor nodes, updating the node status, refining the mutual connection between words, and outputting the node feature aggregation result;

[0010] S4: Based on the node feature aggregation result, perform weight adjustment of key dependency relationships, selectively strengthen structural information in the text, reveal the semantic structure of the text, mark key semantic connections, and generate dependency feature optimization results;

[0011] S5: Using the dependency feature optimization result, text classification is performed in a multi-level structure, the semantics of the word and phrase level are analyzed at the bottom level, and the semantic structure of the paragraph and the whole text is synthesized at the high level to obtain the text classification result.

[0012] As a further solution of the present invention, the preliminary grammatical analysis results include part-of-speech tags, dependency chains and syntactic types; the syntactic dependency structure graph includes node lists, edge connections and structural attribute records; the node feature aggregation results include node weights, feature vectors and aggregation indexes; the dependency feature optimization results include optimization weights, key connections and semantic enhancement items; the text classification results include classification labels, semantic hierarchies and text structure evaluation results.

[0013] As a further solution of the present invention, the specific steps of loading a text data set, performing part-of-speech analysis on each text segment, determining the part-of-speech of each word through natural language processing, extracting the basic grammatical features of the text, and obtaining a preliminary grammatical analysis result are as follows:

[0014] S101: Load a text dataset, extract each text in the dataset paragraph by paragraph, split each paragraph into sentences, and preliminarily mark the structure in each sentence to obtain text paragraph and sentence division results;

[0015] S102: using the text paragraph and sentence segmentation results, performing word-level segmentation on the text in each sentence, deconstructing the sentence into words and punctuation marks, marking the position of each word, and obtaining a word and position index;

[0016] S103: assigning a part-of-speech tag to each word through the word and position index, determining the part-of-speech using context information and typical usage of the word, establishing a part-of-speech file for each word, and obtaining a preliminary grammatical analysis result.

[0017] As a further solution of the present invention, based on the preliminary grammatical analysis results, a dependency label is constructed for each word, the dependencies between words are analyzed word by word, and the interdependencies of the words in the sentence are revealed. The connection between each word and other words is represented as an edge, and the word itself is a node. The specific steps of drawing a syntactic dependency structure diagram are as follows:

[0018] S201: using the preliminary grammatical analysis result, identifying the function and relative position of each word, assigning dependency labels by comparing the relationship with surrounding words, assigning labels showing dependency to each pair of adjacent words, and obtaining a word dependency list;

[0019] S202: Based on the word dependency list, use the connection lines between nodes to represent the dependency boundary, each word as a node, reveal the structural relationship between words, and obtain the word connection mapping result;

[0020] S203: By combining the vocabulary connection mapping result with the data of nodes and edges, the interdependence of the words in the sentence is displayed, the sentence structure is visualized, and a syntactic dependency structure graph is obtained.

[0021] As a further solution of the present invention, the specific steps of performing feature aggregation on the nodes in the syntactic dependency structure graph, identifying the neighbor nodes of each node and integrating the feature information of the neighbor nodes, updating the node status, refining the mutual connection between words, and outputting the node feature aggregation result are:

[0022] S301: Based on the syntactic dependency structure graph, identify the word represented by each node and its direct neighbors, query the connection information of each node, extract the feature information of the nodes directly connected to each node, and obtain the feature set of the node and its neighbors;

[0023] S302: Aggregate the feature information of each node and its neighbors through the feature set of the node and its neighbors, merge the data of each node and the node data directly associated with it, update the state of each node, and obtain node state update data;

[0024] S303: According to the node status update data, refine the mutual connection between each node and its neighbors, organize and display the updated feature data of each node, calculate the aggregated feature values ​​between the nodes, and obtain the node feature aggregation result.

[0025] As a further solution of the present invention, the aggregated feature value is according to the formula:

[0026] ;

[0027] Calculate, where represents the aggregate feature value, Representative Node The characteristic vector of Representative Node The characteristic vector of represents a learnable weight matrix used to transform node feature vectors to higher or lower dimensions, Representative Node The set of adjacent nodes of Representative Node With Node The absolute value of the difference of the eigenvectors.

[0028] As a further solution of the present invention, based on the node feature aggregation result, the weight adjustment of key dependencies is performed, the structural information in the text is selectively strengthened, the semantic structure of the text is revealed, the key semantic connections are marked, and the specific steps of generating dependency feature optimization results are as follows:

[0029] S401: Based on the node feature aggregation result, the dependency weights between each node and its neighbors are evaluated, and the weights that need to be adjusted are determined by comparing the feature aggregation data between the nodes, highlighting the key structural connections, and obtaining the adjusted weight relationship;

[0030] S402: Using the adjusted weight relationship, screening and marking key structural connections in the text, strengthening the visual and data representation of the structural connections, revealing the semantic structure of the text, and obtaining significant marking results of the key structural connections;

[0031] S403: Based on the significant marking results of the key structural connections, all key dependency features are sorted, the semantic connections of the text are marked, the deep semantic structure of the text is revealed, and the dependency feature optimization results are obtained.

[0032] As a further solution of the present invention, the dependency feature optimization result is used to perform text classification in a multi-level structure, analyze the semantics of the word and phrase level at the bottom level, and synthesize the semantic structure of the paragraph and the full text at the high level. The specific steps to obtain the text classification result are:

[0033] S501: Based on the dependency feature optimization result, the dependency structure of words and phrases is analyzed, the relationship strength index between each word and phrase and its dependent item is calculated, key semantic features are extracted, and a semantic feature set of words and phrases is obtained;

[0034] S502: Using the semantic feature set of the words and phrases, perform semantic structure synthesis on the paragraphs and the entire article in the text, and construct a coherent structure of the entire text by accumulating individual semantic elements to obtain a high-level semantic structure of the text;

[0035] S503: Based on the high-level semantic structure of the text, a classification operation is performed to classify the text into corresponding categories according to the overall semantic content, accurately reveal the theme and structure of each article, and obtain a text classification result.

[0036] As a further embodiment of the present invention, the strength index is according to the formula:

[0037] ;

[0038] Calculate, where represents the strength index value, Representative The distance between the dependent item and the core word is used to measure the position relationship in the dependency structure. The absolute value of the distance reflects the closeness of the relationship. Represents the average value of all dependent item distances and is used to standardize distance data to ensure consistency of analysis. Representative The structural angle of the dependent item indicates the angular position between the dependent item and the core word. A smaller angle indicates direct structural dependence. Represents the average value of all structural angles, helping to adjust the distribution of angle data and balance directional bias in structural analysis.

[0039] A text classification system based on natural language processing, comprising:

[0040] The text preprocessing module loads the text dataset, extracts each text in the dataset segment by segment, splits each segment at the sentence level, and preliminarily marks the structure within each sentence. It also performs word-level segmentation on the text in each sentence, assigns a part-of-speech tag to each word and determines the part of speech, and obtains preliminary grammatical analysis results.

[0041] The syntactic dependency analysis module uses the preliminary grammatical analysis results to identify the function and relative position of each word, assigns a label showing dependency to each pair of adjacent words, reveals the structural relationship between words, displays the interdependence of words in a sentence, and visualizes the sentence structure to obtain a syntactic dependency structure diagram;

[0042] The node aggregation analysis module identifies the words represented by each node and its direct neighbors based on the syntactic dependency structure graph, aggregates the features of each node with the feature information of its neighbors, merges the data of each node and its directly related node data, updates the node status, calculates the aggregate feature values ​​between nodes, and obtains the node feature aggregation result;

[0043] The key structure mining module evaluates the dependency weights between each node and its neighbors based on the node feature aggregation results, highlights key structure connections, screens and marks key structure connections in the text, reveals the semantic structure of the text, marks the semantic connections of the text, reveals the deep semantic structure of the text, and obtains the dependency feature optimization results;

[0044] The text classification execution module analyzes the dependency structure of words and phrases based on the dependency feature optimization result, calculates the relationship strength index between each word and phrase and its dependent item, synthesizes the semantic structure of paragraphs and the entire article in the text, performs classification operations, divides the text into corresponding categories according to the overall semantic content, and obtains the text classification result.

[0045] Compared with the prior art, the advantages and positive effects of the present invention are:

[0046] In the present invention, by deeply applying natural language processing technology, combined with detailed syntactic dependency analysis and graphical representation methods, the parsing depth of text structure and semantics is significantly improved, dependency relationship labels are constructed, the complex semantic network inside the text is effectively revealed, and the understanding of sentence structure is enhanced. Graphical representation and feature aggregation technology enhance the key information in the text, and by adjusting the weights of key dependencies and optimizing the importance assessment of information, multi-level text analysis can adapt to the needs of different semantic levels, especially when processing complex texts that require deep semantic understanding, it is more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the steps of the present invention;

[0048] Figure 2 is a flow chart of the steps of S1 of the present invention;

[0049] Figure 3 is a flow chart of the steps of S2 of the present invention;

[0050] Figure 4 is a flow chart of the steps of S3 of the present invention;

[0051] Figure 5 is a flow chart of the steps of S4 of the present invention;

[0052] Figure 6 is a flow chart of the steps of S5 of the present invention;

[0053] Figure 7 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0056] See also Figure 1 , a text classification method based on natural language processing, comprising the following steps:

[0057] S1: Load the text dataset, perform part-of-speech analysis on each text segment, determine the part-of-speech of each word through natural language processing, extract the basic grammatical features of the text, and obtain preliminary grammatical analysis results;

[0058] S2: Based on the preliminary grammatical analysis results, construct dependency labels for each word, analyze the dependencies between words one by one, reveal the interdependence of words in the sentence, and draw a syntactic dependency structure diagram with the connection between each word and other words represented as an edge and the word itself as a node;

[0059] S3: Perform feature aggregation on the nodes in the syntactic dependency structure graph, identify the neighbor nodes of each node and integrate the feature information of the neighbor nodes, update the node status, refine the mutual connection between words, and output the node feature aggregation result;

[0060] S4: Based on the node feature aggregation results, perform weight adjustment of key dependencies, selectively strengthen the structural information in the text, reveal the semantic structure of the text, mark key semantic connections, and generate dependency feature optimization results;

[0061] S5: Use dependency feature optimization results to perform text classification in a multi-level structure, analyze the semantics at the word and phrase level at the bottom level, and synthesize the semantic structure of paragraphs and the entire text at the high level to obtain the text classification result.

[0062] The preliminary grammatical analysis results include part-of-speech tags, dependency chains and syntactic types; the syntactic dependency structure graph includes node lists, edge connections and structural attribute records; the node feature aggregation results include node weights, feature vectors and aggregation indexes; the dependency feature optimization results include optimization weights, key connections and semantic enhancement items; the text classification results include classification labels, semantic hierarchies and text structure evaluation results.

[0063] See also Figure 2 , the specific steps of S1 are:

[0064] S101: Load a text dataset, extract each text in the dataset paragraph by paragraph, split each paragraph into sentences, and preliminarily mark the structure in each sentence to obtain text paragraph and sentence division results;

[0065] When loading a text dataset, the database interface is first called to retrieve the relevant dataset from the server through SQL query statements. The dataset usually contains a variety of text data, such as articles, reports, and other document forms. The dataset needs to go through a series of preprocessing steps to ensure data consistency and integrity. The preprocessing includes encoding unification and format standardization. This step ensures the accuracy and efficiency of subsequent processing. Next, the system uses a specific text analysis library, such as NLTK or SpaCy in Python, to extract each text paragraph by paragraph. During the process, the script automatically detects the paragraph boundaries of each text, determines the start and end of the paragraph by finding line breaks and paragraph marks, and performs sentence-level splitting after each paragraph. The sentence boundaries are identified according to punctuation and capital letter start rules, and the internal structure of each sentence is preliminarily marked to identify the subject, predicate, and object, and to classify and mark conjunctions, prepositions, and phrases. Finally, the paragraph and sentence division results of the text are obtained.

[0066] S102: Using the text paragraph and sentence segmentation results, perform word-level segmentation on the text in each sentence, deconstruct the sentence into words and punctuation marks, mark the position of each word, and obtain a word and position index;

[0067] Based on the results of text paragraph and sentence segmentation, the text analysis process is further refined to the word level. First, the tokenize module in the text processing library such as NLTK is used to segment each sentence at the word level. In this process, words and punctuation marks are accurately separated and identified. For example, punctuation marks such as commas and periods are separated separately and retained in the sentence structure to maintain semantic integrity. After each word is identified, it is indexed and marked according to its specific position in the sentence. The marking process includes recording the starting and ending positions of the word in the original text. Position information is crucial for subsequent text analysis. For example, when performing semantic analysis or reference detection, the position index helps to accurately track the specific position of each word, thereby performing more in-depth text parsing and obtaining each word and its position index, which is used to establish an index database for words to provide support for subsequent queries and analysis.

[0068] S103: assigning a part-of-speech tag to each word through word and position index, determining the part-of-speech using context information and typical usage of the word, establishing a part-of-speech file for each word, and obtaining a preliminary grammatical analysis result;

[0069] Based on the word and position index, each word will then be assigned a part-of-speech tag. The process involves the comprehensive use of contextual information and the typical usage of words to determine the part-of-speech. First, the part-of-speech tagging model is loaded. The model is trained based on a large number of language data sets and can recognize and annotate the parts of speech of words, such as nouns, verbs, adjectives, etc. The part-of-speech tagging of each word is performed by considering the grammatical structure of the word in the sentence and the parts of speech of adjacent words. For example, if a word is followed by a definite article, the word may be a noun. In addition, the typical usage of the word is also taken into account. For example, "running" can be a verb or a noun. The specific part of speech depends on its usage in the sentence. The system uses composite information to establish a detailed part-of-speech file for each word, which includes all possible parts of speech of the word and its frequency of use in different contexts. The preliminary grammatical analysis results obtained provide the necessary foundation for subsequent more advanced text processing tasks, such as syntactic parsing and semantic analysis.

[0070] See also Figure 3 , the specific steps of S2 are:

[0071] S201: using the preliminary grammatical analysis results, identifying the function and relative position of each word, comparing the relationship with surrounding words to assign dependency labels, assigning labels showing dependency to each pair of adjacent words, and obtaining a word dependency list;

[0072] Using the preliminary grammatical analysis results, we begin to refine the functional identification and relative position of each word in the sentence. This first involves processing each word with a labeling algorithm. By comparing the typical relationships between each word and the surrounding words, such as subject-predicate relationships, attributive and object relationships, etc., appropriate dependency labels are assigned to the relationships between words. The labels reflect the grammatical dependencies between words. The labeling process uses the existing dependency grammar library for reference and verification to ensure the accuracy and consistency of the labels. Each pair of adjacent words is given a label that shows the dependency. This is achieved by analyzing the sentence structure in detail and identifying the dependency dynamics between keywords and functional words. After completing this step, the word dependencies of each sentence are stored by the system in the form of a list, forming a word dependency list, which provides basic data and structural framework for subsequent more complex language processing tasks.

[0073] S202: Based on the word dependency list, use the connection between nodes to represent the dependency boundary, each word as a node, reveal the structural relationship between words, and obtain the word connection mapping result;

[0074] Based on the word dependency list, each word is regarded as a node, and the lines between nodes are used to represent the dependency boundaries between nodes. This process uses a graphical representation method to reveal the structural relationship between words in a graph theory way. Each line not only represents the dependency relationship between words, but also carries information about the dependency type, such as dominance relationship, complementation relationship, etc. In the graphical process, each node and line is generated using automated graphics tools, such as Graphviz or Gephi, which can adjust the style of the line and the position of the node according to the type and strength of the dependency relationship to optimize the visual presentation effect. The resulting vocabulary connection mapping results not only clearly show the relationship between words in the sentence, but also provide intuitive graphical support for understanding and analyzing sentence structure.

[0075] S203: by combining the vocabulary connection mapping results with the data of nodes and edges, the interdependence of the words in the sentence is displayed, the sentence structure is visualized, and a syntactic dependency structure graph is obtained;

[0076] The sentence structure is visualized through the results of vocabulary connection mapping. In this process, the interdependence of words in the sentence is displayed visually by combining the data of nodes and edges. The color, size and thickness of the nodes and the color of the edges can be adjusted according to the dependency strength and type between the nodes, so that the dependency structure of the sentence can be seen at a glance. The visual representation helps to better understand and study the language structure and function. The resulting syntactic dependency structure diagram is widely used in academic research and language teaching, providing an intuitive perspective to observe and analyze the grammatical structure and semantic connection of sentences.

[0077] See also Figure 4 , the specific steps of S3 are:

[0078] S301: Based on the syntactic dependency structure graph, identify the word represented by each node and its direct neighbors, query the connection information of each node, extract the feature information of the nodes directly connected to each node, and obtain the feature set of the node and its neighbors;

[0079] Based on the syntactic dependency structure graph, further in-depth analysis of each node, that is, each word and its direct neighbors, first involves querying the connection information of each node, using graph database technology, such as Neo4j, for efficient node query and connection data extraction. The database stores detailed information on each node, including the node's part of speech, function, and dependency type with other nodes. The directly connected nodes of each node are identified through graph query, and the feature information of the nodes directly connected to each node is further extracted, including the part of speech of the neighboring nodes, the nature of the dependency relationship, and the function in the sentence. The information obtained is used to construct a feature set for each node and its neighbors, integrating all key information related to the word and its context, providing rich data resources for subsequent language processing and analysis tasks.

[0080] S302: Aggregate the feature information of each node and its neighbors through the feature set of the node and its neighbors, merge the data of each node and the node data directly associated with it, update the state of each node, and obtain node state update data;

[0081] Through the feature sets of nodes and their neighbors, the features of each node are then aggregated with the feature information of its neighbors. In the process, feature fusion technology is used to merge the data of each node and its directly related node data. Data fusion takes into account the semantic importance of the node and the strength of the dependency relationship to ensure the integrity of the information and the accurate reflection of the context. The step of updating the state of each node is performed through iterative calculation. Each iteration updates the node state, taking into account the influence of neighboring nodes, thereby obtaining a fully updated node state data. The data reflects the new semantic and structural position of each word in the sentence. The updated data provides a basis for subsequent text analysis and information extraction.

[0082] S303: updating data based on the node status, refining the mutual connection between each node and its neighbors, organizing and displaying the updated feature data of each node, calculating the aggregated feature values ​​between the nodes, and obtaining the node feature aggregation result;

[0083] Aggregate eigenvalues ​​according to the formula:

[0084] ;

[0085] Calculate, where represents the aggregate feature value, Representative Node The characteristic vector of Representative Node The characteristic vector of represents a learnable weight matrix used to transform node feature vectors to higher or lower dimensions, Representative Node The set of adjacent nodes of Representative Node With Node The absolute value of the difference of the eigenvectors.

[0086] Setting Node and its adjacent node set ,in and yes Directly connected nodes. Parameters , node feature vector , and The specific value of needs to be obtained through actual data. In this example, through network data monitoring, we get , , The weight matrix for The matrix, after training, is determined to be .

[0087] calculate and :

[0088] ;

[0089] ;

[0090] Compute the square root of a sum of squares:

[0091] ;

[0092] ;

[0093] Compute the sum of the denominators:

[0094] ;

[0095] ;

[0096] ;

[0097] Final calculation: ;

[0098] The results show that after weighted update through the graph attention network, the node The aggregate eigenvalue of the node is 0.220, which represents The new feature representation in the weighted network provides key information for further network analysis such as classification and clustering.

[0099] See also Figure 5 , the specific steps of S4 are:

[0100] S401: Based on the node feature aggregation results, the dependency weights between each node and its neighbors are evaluated, and the weights that need to be adjusted are determined by comparing the feature aggregation data between the nodes, highlighting the key structural connections, and obtaining the adjusted weight relationship;

[0101] Based on the node feature aggregation results, the dependency weights between each node and its neighbors are further evaluated in detail. Machine learning algorithms are used to analyze and compare the feature aggregation data between nodes. The weights of the dependencies between each pair of nodes can be calculated based on the differences and similarities of the aggregated features. The purpose of weight evaluation is to determine the connections that play a key role in the structure and which weights need to be adjusted to better reflect the actual semantic importance of the text structure. During the adjustment process, the weights are automatically optimized to emphasize or weaken the influence of certain connections, thereby highlighting the key structural connections in the text. The adjusted weight relationship provides an accurate structural basis for the next step of text analysis.

[0102] S402: Using the adjusted weight relationship, screening and marking key structural connections in the text, strengthening the visual and data representation of the structural connections, and obtaining significant marking results of the key structural connections;

[0103] Using the adjusted weight relationship, the key structural connections in the text are then screened and marked, and those structural connections with high weights are displayed with a more significant visual effect. For example, the line thickness and color depth of these connections are enhanced to make them more visually prominent. In addition, the data representation is also enhanced, and the detailed data of each key connection, such as weight value and connection type, is displayed through data labels and prompt information. This process not only enhances the visual and data representation of the structural connections, but also makes the key structural connections easier to identify and operate in text analysis and information extraction. The significant marking results of the key structural connections provide an important tool for further text parsing.

[0104] S403: Based on the significant marking results of key structural connections, all key dependency features are sorted, the semantic connections of the text are marked, the deep semantic structure of the text is revealed, and the dependency feature optimization results are obtained;

[0105] Based on the significant marking results of key structure connections, continue to organize all key dependency relationship features. Focus on carefully marking the semantic connections in the text, mark and annotate each key dependency relationship, and reveal the semantic contributions and functional roles of each connection. In this way, the deep semantic structure of the text is systematically displayed. The marking results not only help understand the basic structure of the text but also reveal the deep semantics and logical relationships of the text. The organized and optimized results of dependency relationship features provide a solid foundation for subsequent advanced semantic analysis and text interpretation. The optimized results of dependency relationship features demonstrate the depth and precision of text analysis and provide users with a richer and more useful text understanding tool.

[0106] Please refer to Figure 6 , and the specific steps of S5 are as follows:

[0107] S501: Based on the optimized results of dependency relationship features, analyze the dependency structure of words and phrases, calculate the relationship strength index between each word and phrase and its dependent item, extract key semantic features, and obtain the semantic feature set of words and phrases;

[0108] The strength index is calculated according to the formula:

[0109] ;

[0110] Perform the calculation, where represents the strength index value, represents the distance between the i-th dependent item and the core word, used to measure the positional relationship in the dependency structure. The absolute value of the distance reflects the closeness of the relationship, represents the average value of the distances of all dependent items, used to standardize the distance data to ensure the consistency of the analysis, represents the structural angle of the i-th dependent item, indicating the angular position between the dependent item and the core word. A smaller angle indicates a direct dependency in the structure, represents the average value of all structural angles, helping to adjust the distribution of angle data and balance the directional bias in the structural analysis.

[0111] is the distance between the i-th dependent item and the core word, characterizing the positional relationship in the dependency structure. The distance is obtained by the number of edges between nodes in the parse tree. In practical applications, this value is directly provided by the dependency parser. Suppose the parsing result shows that the distance between "sit" and "on" in the sentence "He sits on the riverbank" is 3.

[0112] is the average value of the distances of all dependent items. Calculate the average value of all . If there are four dependency distances of 2, 3, 3, and 4 respectively, then .

[0113] It is the structural angle of the i-th dependency, referring to the angular position of the dependency and the head word in the sentence structure. By analyzing the syntactic structure of the sentence, the structural relationship between each word and the head word is quantified. This angle can be calculated from the graphical representation of the syntactic tree. The angle measurement shows that the angle between "sitting" and "river" is 45 degrees.

[0114] It is the average value of all structural angles. Calculate all averages. If the angle values are 30, 45, 60, and 45, then .

[0115] Combining the above, there are four dependencies, and the specific data is as follows:

[0116] Distance , average distance .

[0117] Angle , average angle .

[0118] Substitute into the formula for calculation:

[0119] ;

[0120] ;

[0121] ;

[0122] The results show that the overall strength index of the dependency relationship is 0.094, reflecting that the dependency relationship between words and phrases in the text is relatively weak, meaning that the structure of the text is complex or diverse. Through this calculation, the deep semantic features of the text can be further analyzed and understood.

[0123] S502: Adopt the semantic feature set of words and phrases to perform semantic structure synthesis on the paragraphs and the overall article in the text, construct the coherent structure of the full text by accumulating individual semantic elements, and obtain the high-level semantic structure of the text;

[0124] Using the semantic feature set of words and phrases, the system synthesizes the semantic structures of paragraphs and entire articles in the text. This process builds a coherent structure of the entire text by integrating individual semantic elements, such as keywords, phrases and their dependencies, and reconstructs the text using semantic synthesis tools. It automatically identifies the logical flow and structural hierarchy of the text based on the semantic feature set, and generates a high-level semantic structure of the text by accumulating and integrating semantic elements. It not only displays the main content and internal connections of the article, but also provides a visual and data representation of the logical relationship between paragraphs. The resulting high-level semantic structure of the text provides a solid foundation for in-depth understanding and further analysis of the text.

[0125] S503: Based on the high-level semantic structure of the text, a classification operation is performed to classify the text into corresponding categories according to the overall semantic content, accurately reveal the theme and structure of each article, and obtain a text classification result;

[0126] Based on the high-level semantic structure of the text, classification operations are performed and machine learning classification models, such as support vector machines or deep learning classification networks, are used to accurately divide the text into corresponding categories according to the overall semantic content of the text, such as themes, core concepts and their semantic relationships. During the classification process, model training is carried out using a large amount of classified text data to ensure the accuracy and wide applicability of the classification. In this way, the theme and structure of each article are clearly revealed, and the classification results help users quickly understand the content and purpose of the text. The obtained text classification results provide effective classification information for the further use and research of the text.

[0127] See also Figure 7 , a text classification system based on natural language processing, comprising:

[0128] The text preprocessing module loads the text dataset, extracts each text in the dataset segment by segment, splits each segment at the sentence level, and preliminarily marks the structure within each sentence. It also performs word-level segmentation on the text in each sentence, assigns a part-of-speech tag to each word and determines the part of speech, and obtains preliminary grammatical analysis results.

[0129] The syntactic dependency analysis module uses the preliminary grammatical analysis results to identify the function and relative position of each word, assigns labels showing dependencies to each pair of adjacent words, reveals the structural relationship between words, displays the interdependence of words in a sentence, and visualizes the sentence structure to obtain a syntactic dependency structure diagram;

[0130] The node aggregation analysis module identifies the words represented by each node and its direct neighbors based on the syntactic dependency structure graph, aggregates the features of each node with the feature information of its neighbors, merges the data of each node and its directly related node data, updates the node status, calculates the aggregate feature values ​​between nodes, and obtains the node feature aggregation results;

[0131] The key structure mining module evaluates the dependency weights between each node and its neighbors based on the node feature aggregation results, highlights the key structure connections, screens and marks the key structure connections in the text, reveals the semantic structure of the text, marks the semantic connections of the text, reveals the deep semantic structure of the text, and obtains the dependency feature optimization results;

[0132] The text classification execution module analyzes the dependency structure of words and phrases based on the dependency feature optimization results, calculates the relationship strength index between each word and phrase and its dependent items, synthesizes the semantic structure of paragraphs and the entire article in the text, performs classification operations, and divides the text into corresponding categories according to the overall semantic content to obtain text classification results.

[0133] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A text classification method based on natural language processing, characterized in that: The following steps are involved: S1: Load the text dataset, perform part-of-speech analysis on each text segment, determine the part-of-speech of each word through natural language processing, extract the basic grammatical features of the text, and obtain preliminary grammatical analysis results; S2: Based on the preliminary grammatical analysis results, construct dependency labels for each word, analyze the dependencies between words one by one, reveal the interdependence of words in the sentence, and draw a syntactic dependency structure diagram with the connection between each word and other words represented as an edge and the word itself as a node; S3: performing feature aggregation on the nodes in the syntactic dependency structure graph, identifying the neighbor nodes of each node and integrating the feature information of the neighbor nodes, updating the node status, refining the mutual connection between words, and outputting the node feature aggregation result; S4: Based on the node feature aggregation result, perform weight adjustment of key dependency relationships, selectively strengthen structural information in the text, reveal the semantic structure of the text, mark key semantic connections, and generate dependency feature optimization results; S5: Using the dependency feature optimization result, text classification is performed in a multi-level structure, the semantics of the word and phrase level are analyzed at the bottom level, and the semantic structure of the paragraph and the whole text is synthesized at the high level to obtain the text classification result.

2. A text classification method based on natural language processing according to claim 1, characterized in that: The preliminary grammatical analysis results include part-of-speech tags, dependency chains and syntactic types; the syntactic dependency structure graph includes node lists, edge connections and structural attribute records; the node feature aggregation results include node weights, feature vectors and aggregation indexes; the dependency feature optimization results include optimization weights, key connections and semantic enhancement items; the text classification results include classification labels, semantic hierarchies and text structure evaluation results.

3. A text classification method based on natural language processing according to claim 1, characterized in that: The specific steps to load the text dataset, perform part-of-speech analysis on each text segment, determine the part-of-speech of each word through natural language processing, extract the basic grammatical features of the text, and obtain preliminary grammatical analysis results are as follows: S101: Load a text dataset, extract each text in the dataset paragraph by paragraph, split each paragraph into sentences, and preliminarily mark the structure in each sentence to obtain text paragraph and sentence division results; S102: using the text paragraph and sentence segmentation results, performing word-level segmentation on the text in each sentence, deconstructing the sentence into words and punctuation marks, marking the position of each word, and obtaining a word and position index; S103: assigning a part-of-speech tag to each word through the word and position index, determining the part-of-speech using context information and typical usage of the word, establishing a part-of-speech file for each word, and obtaining a preliminary grammatical analysis result.

4. The text classification method based on natural language processing according to claim 1, characterized in that: Based on the preliminary grammatical analysis results, a dependency label is constructed for each word, and the dependencies between words are analyzed word by word to reveal the interdependencies of the words in the sentence. The connection between each word and other words is represented as an edge, and the word itself is a node. The specific steps of drawing a syntactic dependency structure diagram are as follows: S201: using the preliminary grammatical analysis result, identifying the function and relative position of each word, assigning dependency labels by comparing the relationship with surrounding words, assigning labels showing dependency to each pair of adjacent words, and obtaining a word dependency list; S202: Based on the word dependency list, use the connection lines between nodes to represent the dependency boundary, each word as a node, reveal the structural relationship between words, and obtain the word connection mapping result; S203: By combining the vocabulary connection mapping result with the data of nodes and edges, the interdependence of the words in the sentence is displayed, the sentence structure is visualized, and a syntactic dependency structure graph is obtained.

5. The text classification method based on natural language processing according to claim 1, characterized in that: The specific steps of performing feature aggregation on the nodes in the syntactic dependency structure graph, identifying the neighbor nodes of each node and integrating the feature information of the neighbor nodes, updating the node status, refining the mutual connection between words, and outputting the node feature aggregation result are as follows: S301: Based on the syntactic dependency structure graph, identify the word represented by each node and its direct neighbors, query the connection information of each node, extract the feature information of the nodes directly connected to each node, and obtain the feature set of the node and its neighbors; S302: Aggregate the feature information of each node and its neighbors through the feature set of the node and its neighbors, merge the data of each node and the node data directly associated with it, update the state of each node, and obtain node state update data; S303: According to the node status update data, refine the mutual connection between each node and its neighbors, organize and display the updated feature data of each node, calculate the aggregated feature values ​​between the nodes, and obtain the node feature aggregation result.

6. A text classification method based on natural language processing according to claim 5, characterized in that: The aggregated feature value is according to the formula: ； Calculate, where represents the aggregate feature value, Representative Node The characteristic vector of Representative Node The characteristic vector of represents the learnable weight matrix, Representative Node The set of adjacent nodes of Representative Node With Node The absolute value of the difference of the eigenvectors.

7. The text classification method based on natural language processing according to claim 1 is characterized in that: Based on the node feature aggregation results, the weight adjustment of key dependencies is performed, the structural information in the text is selectively strengthened, the semantic structure of the text is revealed, the key semantic connections are marked, and the specific steps of generating dependency feature optimization results are as follows: S401: Based on the node feature aggregation result, the dependency weights between each node and its neighbors are evaluated, and the weights that need to be adjusted are determined by comparing the feature aggregation data between the nodes, highlighting the key structural connections, and obtaining the adjusted weight relationship; S402: Using the adjusted weight relationship, screening and marking key structural connections in the text, strengthening the visual and data representation of the structural connections, revealing the semantic structure of the text, and obtaining significant marking results of the key structural connections; S403: Based on the significant marking results of the key structural connections, all key dependency features are sorted, the semantic connections of the text are marked, the deep semantic structure of the text is revealed, and the dependency feature optimization results are obtained.

8. The text classification method based on natural language processing according to claim 1, characterized in that: Using the dependency feature optimization results, text classification is performed in a multi-level structure, the bottom layer analyzes the semantics of the word and phrase level, and the high-level synthesizes the semantic structure of the paragraph and the full text. The specific steps to obtain the text classification result are: S501: Based on the dependency feature optimization result, the dependency structure of words and phrases is analyzed, the relationship strength index between each word and phrase and its dependent item is calculated, key semantic features are extracted, and a semantic feature set of words and phrases is obtained; S502: Using the semantic feature set of the words and phrases, perform semantic structure synthesis on the paragraphs and the entire article in the text, and construct a coherent structure of the entire text by accumulating individual semantic elements to obtain a high-level semantic structure of the text; S503: Based on the high-level semantic structure of the text, a classification operation is performed to classify the text into corresponding categories according to the overall semantic content, accurately reveal the theme and structure of each article, and obtain a text classification result.

9. A text classification method based on natural language processing according to claim 8, characterized in that: The strength index is according to the formula: ； Calculate, where represents the strength index value, Representative The distance between the dependent item and the core word, represents the average distance of all dependencies, Representative From the structural perspective of dependencies, Represents the average value of all structural angles.

10. A text classification system based on natural language processing, characterized in that: According to a text classification method based on natural language processing according to any one of claims 1 to 9, the system comprises: The text preprocessing module loads the text dataset, extracts each text in the dataset segment by segment, splits each segment at the sentence level, and preliminarily marks the structure within each sentence. It also performs word-level segmentation on the text in each sentence, assigns a part-of-speech tag to each word and determines the part of speech, and obtains preliminary grammatical analysis results. The syntactic dependency analysis module uses the preliminary grammatical analysis results to identify the function and relative position of each word, assigns a label showing dependency to each pair of adjacent words, reveals the structural relationship between words, displays the interdependence of words in a sentence, and visualizes the sentence structure to obtain a syntactic dependency structure diagram; The node aggregation analysis module identifies the words represented by each node and its direct neighbors based on the syntactic dependency structure graph, aggregates the features of each node with the feature information of its neighbors, merges the data of each node and its directly related node data, updates the node status, calculates the aggregate feature values ​​between nodes, and obtains the node feature aggregation result; The key structure mining module evaluates the dependency weights between each node and its neighbors based on the node feature aggregation results, highlights key structure connections, screens and marks key structure connections in the text, reveals the semantic structure of the text, marks the semantic connections of the text, reveals the deep semantic structure of the text, and obtains the dependency feature optimization results; The text classification execution module analyzes the dependency structure of words and phrases based on the dependency feature optimization result, calculates the relationship strength index between each word and phrase and its dependent item, synthesizes the semantic structure of paragraphs and the entire article in the text, performs classification operations, divides the text into corresponding categories according to the overall semantic content, and obtains the text classification result.

Images