Text intelligent classification and statistical analysis method based on artificial intelligence

Abstract

The application discloses an artificial intelligence-based text intelligent classification and statistical analysis method, and relates to the field of natural language processing and text analysis; solves the problems of traditional text classification methods in semantic depth analysis, dynamic feature capture and cross-text statistics, and low classification accuracy and poor interpretability; the application divides the text data to be analyzed into continuous semantic units, generates semantic vectors and self-attention distribution; calculates the cognitive tension intensity between the semantic units, constructs a symmetric cognitive tension field matrix; calculates the information entropy of each semantic unit and forms an information entropy change chain; multi-dimensional features are extracted from the matrix and the entropy change chain and input into a pre-trained classification model to obtain a text classification result; based on the classification result, cross-text aggregation statistical analysis is carried out to generate a visual report, fine classification and deep semantic analysis of the text are realized, the accuracy and interpretability are improved, and efficient analysis support is provided for large-scale text processing.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and text analysis, specifically an artificial intelligence-based method for intelligent text classification and statistical analysis. Background Technology

[0002] With the rapid development of information technology, text data is experiencing explosive growth. Traditional text classification methods often rely on keyword matching or shallow semantic models, making it difficult to deeply understand the complex semantic structures and logical connections within text. These methods tend to treat text as a bag-of-words or a static sequence, ignoring its essence as a dynamic information carrier—that is, the deep features within text such as shifts in semantic focus and changes in information density. This results in limited classification accuracy when dealing with content that has complex logic, implicit emotions, or high technical content, and the model's decision-making process is opaque and lacks interpretability. Therefore, existing technologies cannot effectively support the extraction and analysis of higher-order features such as cognitive tension and dynamic evolution of information flow in text, thus hindering the development of high-quality text mining and in-depth statistical analysis.

[0003] The existing technologies have the following problems: traditional text classification methods mainly rely on bag-of-words models or word frequency statistics, which cannot capture the deep semantic connections and logical structures of texts; existing technologies usually treat texts as static word sequences, which cannot effectively model and analyze the dynamic evolution of semantic focus and information density during text reading; the features used for classification are mostly shallow statistics such as word frequency, lacking composite features that can comprehensively reflect the inherent cognitive tension and information flow patterns of texts; and there is a lack of effective tools for cross-text aggregation statistics and comparative analysis of the deep structural patterns of multiple texts within the same category. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a text intelligent classification and statistical analysis method based on artificial intelligence to solve the above-mentioned technical problems.

[0005] The first aspect of this invention provides a method for intelligent text classification and statistical analysis based on artificial intelligence, comprising the following steps:

[0006] S1: Receive the text data to be analyzed, divide it into N consecutive semantic units, and generate a corresponding semantic vector and self-attention distribution for each semantic unit;

[0007] S2: Based on semantic units and their corresponding semantic vectors and self-attention distributions, calculate the cognitive tension intensity between any two semantic units and construct an N×N symmetric cognitive tension field matrix;

[0008] S3: Calculate the information entropy of each semantic unit, and calculate the information entropy change between adjacent semantic units according to the sequence order to form an information entropy change chain;

[0009] S4: Extract features from the symmetric cognitive tension field matrix and information entropy transformation chain, and input the features and the text data to be analyzed into the classification model to obtain the text classification results;

[0010] S5: Based on text classification results, symmetric cognitive tension field matrix, and information entropy transformation chain, perform cross-text aggregation statistical analysis and generate statistical analysis reports.

[0011] Preferably, step S1 includes the following steps:

[0012] The system receives and preprocesses the text data to be analyzed; it uses a pre-trained language coding model to perform dependency parsing on the preprocessed text, identifies clause boundaries in the text, and the clause boundaries are grammatical segmentation points identified by subordinating conjunctions and specific punctuation marks; based on the clause boundaries, the text is divided into N clauses, which are then treated as consecutive semantic units arranged in the original text order.

[0013] Each semantic unit is individually input into a pre-trained language coding model, and a forward propagation computation is performed. The model is then processed sequentially through the embedding layer and the encoder layer to obtain the set of encoded state vectors corresponding to the current input semantic unit, which is output by the last layer of the encoder. The arithmetic mean of all vectors in the encoded state vector set is calculated to generate a semantic vector for the corresponding semantic unit.

[0014] During the forward propagation computation, the attention weight matrix output by the multi-head self-attention mechanism in the last layer of the model encoder is acquired synchronously. The attention weight matrix includes multiple matrices generated by each attention head for the semantic unit of the current input. First, the attention weight matrices generated by each attention head are arithmetically averaged to obtain a joint attention matrix. Then, for the joint attention matrix, a probability vector is generated by calculating the arithmetic mean of the elements in each row and used as the self-attention distribution of the semantic unit.

[0015] Output an ordered list that records each semantic unit and its corresponding semantic vector and self-attention distribution in the original text order.

[0016] Preferably, in step S2, calculating the cognitive tension intensity between any two semantic units includes the following steps:

[0017] Based on the ordered list, we obtain each semantic unit in the text data and its corresponding semantic vector and self-attention distribution;

[0018] Extract the semantic vector of the i-th semantic unit The semantic vector of the j-th semantic unit ;right and Perform L2 norm normalization on each vector, then calculate the dot product of the normalized vectors, and finally calculate... and The semantic similarity is obtained by using the cosine similarity. The value range is [0,1];

[0019] Extract the self-attention distribution of the i-th semantic unit The self-attention distribution of the j-th semantic unit and to and Perform element-wise smoothing based on the constant ε;

[0020] The distribution of self-attention is obtained through a calculation formula. and KL divergence between them serves as a contribution to attention differences. ;

[0021] Calculation formula: The cognitive tension intensity between the i-th and j-th semantic units is obtained. ,in, Contributions to semantic differences; Contributions to attention differences; This is the distance attenuation factor; denoted as the ordinal distance between semantic units i and j in the text sequence; the semantic difference weight coefficient α, the attention distribution difference weight coefficient β, and the distance decay coefficient γ are all adjustable hyperparameters with a value greater than zero.

[0022] Preferably, in step S2, constructing an N×N symmetric cognitive tension field matrix includes the following steps:

[0023] Create an N×N two-dimensional matrix F, where N is the total number of semantic units, and initialize all matrix elements to 0;

[0024] For all unit pairs (i,j) in the semantic unit set, where i,j∈[1,N] and i≠j, a parallel computing thread is started; in each computing thread, the cognitive tension intensity between the i-th semantic unit and the j-th semantic unit is calculated using the cognitive tension intensity calculation method. ;

[0025] All the calculated results The values ​​are filled into the corresponding positions in matrix F, and the diagonal elements of matrix F are also filled. Set to 0;

[0026] Calculate matrix F and its transpose. difference matrix And calculate the Frobenius norm of the difference matrix as the symmetry deviation δ;

[0027] Set a symmetry tolerance threshold τ>0, and compare the symmetry deviation δ with the tolerance threshold τ.

[0028] If δ>τ, then perform symmetry processing on matrix F to obtain the symmetric cognitive tension field matrix. ;

[0029] If δ≤τ, then the original matrix F is directly used as the symmetric cognitive tension field matrix. ;

[0030] Output symmetric cognitive tension field matrix .

[0031] Preferably, in step S3, calculating the information entropy of each semantic unit includes the following steps:

[0032] By using an ordered list, we can obtain each semantic unit in the text data, along with its corresponding semantic vector and self-attention distribution.

[0033] The semantic vectors of N semantic units are combined in order to form an N×D semantic vector matrix V, where D is the dimension of the semantic vector;

[0034] The matrix V is normalized using the L2 norm, and the dot product of the normalized matrices is calculated to obtain an N×N semantic relevance matrix S, where the elements... This represents the semantic association degree between semantic units i and j;

[0035] For each semantic unit i, i∈[1,N], extract the i-th row vector from the semantic relevance matrix S. =[ , ,..., ];

[0036] For the correlation vector The probability distribution is obtained by applying the softmax transform formula for temperature regulation. ;

[0037] Through probability distribution Calculate the information entropy The calculation formula is: ,in, This represents the probability distribution of the semantic influence of the i-th semantic unit on the k-th semantic unit;

[0038] Output information entropy sequence ,in, This represents the information entropy value of the i-th semantic unit; the information entropy sequence satisfies ≥0 and ≤ The mathematical properties of .

[0039] Preferably, in step S3, calculating the information entropy change between adjacent semantic units based on the sequence order to form an information entropy change chain includes the following steps:

[0040] Traversing the information entropy sequence Calculate the information entropy change between each semantic unit and its next adjacent semantic unit;

[0041] All the calculated information entropy changes are combined according to the natural order of their corresponding semantic units in the text to form and output an information entropy change chain sequence.

[0042] Preferably, step S4 includes the following steps:

[0043] From the perspective of symmetric cognitive tension field matrix Multi-dimensional features are extracted, including global statistical features, average cognitive tension intensity features, and graph topological features;

[0044] Global statistical characteristics include the mean of all elements in the matrix, the variance of all elements in the matrix, and the maximum value in the matrix;

[0045] Calculate the arithmetic mean of all elements in the upper triangular part of the matrix except for the main diagonal elements, and use it as the average cognitive tension intensity feature;

[0046] The symmetric cognitive tension field matrix Treating it as the adjacency matrix of a weighted undirected graph, the graph is binarized by setting a predefined tension intensity threshold, and the number of nodes contained in its maximum connected component is calculated and used as the graph topological feature.

[0047] Multi-dimensional features are extracted from the information entropy change chain, including basic statistics, zero-crossing frequency features, persistent pattern length features, and sequence morphology features;

[0048] Basic statistics include the arithmetic mean and variance of the information entropy change chain;

[0049] The frequency of zero crossings is obtained by counting the number of times that adjacent entropy change values ​​in the statistical information entropy change chain have different signs;

[0050] The continuous pattern length feature includes the maximum length of all consecutive positive subsequences and the maximum length of all consecutive negative subsequences in the information entropy change chain;

[0051] The sum of the absolute values ​​of the first-order difference sequences of the information entropy transformation chain is calculated and used as the sequence morphological feature;

[0052] The numerical features extracted from the symmetric cognitive tension field matrix and the information entropy transformation chain are standardized; all standardized features are then input into a pre-trained multilayer perceptron classification model.

[0053] A multilayer perceptron classification model comprises an input layer, at least one hidden layer with a non-linear activation function, and an output layer. The input layer receives features, and the number of its neurons is equal to the total number of standardized features. At least one hidden layer, connected to the input layer, is equipped with a non-linear activation function to perform non-linear transformations and combinations on the input features to generate combined features. The output layer has the same number of neurons as the number of text categories to be classified, and uses the Softmax function to calculate the probability distribution of the text belonging to each category based on the combined features. The category corresponding to the highest probability value in the probability distribution is taken as the final text classification result.

[0054] Preferably, step S5 includes the following steps:

[0055] Based on the text classification results, multiple texts are grouped according to their categories; for each category, the symmetric cognitive tension field matrix and information entropy change chain of all texts in the group are aggregated and then standardized and aligned.

[0056] The symmetric cognitive tension field matrices within the same category, after standardization and alignment, are aggregated to generate the average cognitive tension field matrix of that category and its visualization results; the information entropy change chains within the same category are aggregated to generate the typical information entropy change trajectories of that category and their visualization results.

[0057] For each text, the arithmetic mean of all elements in its symmetric cognitive tension field matrix is ​​calculated as the global intensity index of the cognitive tension field of that text; at the same time, the arithmetic mean of its information entropy change chain is calculated as the information entropy change statistical index of that text; the correlation between the global intensity index of the cognitive tension field and the information entropy change statistical index is calculated.

[0058] By integrating the visualization results of the average cognitive tension field matrix, the visualization results of the typical trajectory of information entropy change, and the correlation calculation results between the global intensity index of cognitive tension field and the statistical index of information entropy change, a structured statistical analysis report is generated.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] This invention quantifies the cognitive association and tension intensity between any two semantic units within a text by constructing a symmetric cognitive tension field matrix, thereby revealing the deep semantic structure of the text.

[0061] This invention accurately depicts the dynamic change process of text information entropy along the sequence order by constructing an information entropy transformation chain, capturing the rhythm and focus shift pattern of text information flow.

[0062] This invention extracts multi-dimensional features such as global statistics, graph topology, and sequence morphology from the cognitive tension field matrix and information entropy transformation chain, providing richer and more interpretable inputs for classification models;

[0063] Based on classification results, this invention performs aggregate analysis on the cognitive tension field and information entropy change patterns of texts in the same category, generates a visualized statistical analysis report, and reveals the common characteristics of text types. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0065] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0066] Please see Figure 1 This invention is a text intelligent classification and statistical analysis method based on artificial intelligence, comprising the following steps:

[0067] S1: Receive the text data to be analyzed, divide it into N consecutive semantic units, and generate a corresponding semantic vector and self-attention distribution for each semantic unit;

[0068] S2: Based on semantic units and their corresponding semantic vectors and self-attention distributions, calculate the cognitive tension intensity between any two semantic units and construct an N×N symmetric cognitive tension field matrix;

[0069] S3: Calculate the information entropy of each semantic unit, and calculate the information entropy change between adjacent semantic units according to the sequence order to form an information entropy change chain;

[0070] S4: Extract features from the symmetric cognitive tension field matrix and information entropy transformation chain, and input the features and the text data to be analyzed into the classification model to obtain the text classification results;

[0071] S5: Based on text classification results, symmetric cognitive tension field matrix, and information entropy transformation chain, perform cross-text aggregation statistical analysis and generate statistical analysis reports.

[0072] Specifically, the system first receives the text data to be analyzed and performs dependency parsing using a pre-trained language model. This identifies clause boundaries marked by subordinating conjunctions or specific punctuation marks, thus segmenting the text into a series of continuous semantic units. Subsequently, a corresponding semantic vector is generated for each semantic unit, and attention weights generated by the model's self-attention mechanism are extracted simultaneously to calculate the self-attention distribution of each unit. The cognitive tension intensity between any two semantic units is calculated, which integrates their semantic similarity, differences in self-attention distribution, and their distance decay effect in the text sequence. The cognitive tension intensities of all unit pairs are filled into a matrix, and through symmetry verification and processing, a symmetric cognitive tension field matrix is ​​constructed to characterize the global semantic tension structure within the text. Simultaneously, the information entropy of each semantic unit based on its semantic association with all units is calculated, and the entropy change between adjacent units is calculated sequentially, forming a dynamic information entropy change chain to reveal the fluctuations in the text's information flow. From the constructed symmetric cognitive tension field matrix and information entropy change chain, multi-dimensional numerical features, including global statistical features, graph topological features, and sequence morphological features, are extracted. These standardized features are then input along with the original text data into a pre-trained multilayer perceptron classification model, which outputs the final text classification result. Finally, multiple texts are grouped according to their classification results. The cognitive tension field matrix and information entropy change chain of all texts within each group are standardized, aligned, and aggregated to generate statistical quantities such as the average cognitive tension field matrix and typical information entropy change trajectories. The correlation between key indicators is calculated, and finally, a structured, visualized statistical analysis report is compiled, thus achieving a complete analytical loop from fine-grained single-text classification to in-depth multi-text statistics.

[0073] In one embodiment of the present invention, step S1 includes the following steps:

[0074] The system receives and preprocesses the text data to be analyzed; it uses a pre-trained language coding model to perform dependency parsing on the preprocessed text, identifies clause boundaries in the text, and the clause boundaries are grammatical segmentation points identified by subordinating conjunctions and specific punctuation marks; based on the clause boundaries, the text is divided into N clauses, which are then treated as consecutive semantic units arranged in the original text order.

[0075] Each semantic unit is individually input into a pre-trained language coding model, and a forward propagation computation is performed. The model is then processed sequentially through the embedding layer and the encoder layer to obtain the set of encoded state vectors corresponding to the current input semantic unit, which is output by the last layer of the encoder. The arithmetic mean of all vectors in the encoded state vector set is calculated to generate a semantic vector for the corresponding semantic unit.

[0076] During the forward propagation computation, the attention weight matrix output by the multi-head self-attention mechanism in the last layer of the model encoder is acquired synchronously. The attention weight matrix includes multiple matrices generated by each attention head for the semantic unit of the current input. First, the attention weight matrices generated by each attention head are arithmetically averaged to obtain a joint attention matrix. Then, for the joint attention matrix, a probability vector is generated by calculating the arithmetic mean of the elements in each row and used as the self-attention distribution of the semantic unit.

[0077] Output an ordered list that records each semantic unit and its corresponding semantic vector and self-attention distribution in the original text order.

[0078] Specifically, the system receives raw text data and preprocesses it; it uses regular expressions to remove noise, removing all tags and special characters that are irrelevant to the semantic content; it performs text standardization processing, converting full-width characters to half-width characters, unifying English characters to lowercase, and merging and standardizing consecutive punctuation marks; at the same time, it performs non-text content processing, converting emojis into semantic descriptive words or directly removing them according to a preset mapping table.

[0079] This method utilizes the dependency parsing capabilities of pre-trained language encoding models (such as BERT) to parse the grammatical structure of preprocessed text, identifying core components such as subject, verb, and object, subordinating conjunctions (e.g., because, but), and specific punctuation marks (e.g., commas, semicolons). Subordinating conjunctions and specific punctuation marks are used as grammatical delimiters, i.e., clause boundaries. Based on these boundaries, the text is divided into N clauses, each maintaining relative grammatical independence and carrying complete sub-semantic meaning; a clause is defined as a semantic unit. Simultaneously, heuristic rules are applied to balance the length of semantic units. Under the premise of grammatical integrity, if the length of a semantic unit exceeds a preset length threshold, a secondary split is forced at an appropriate conjunction. The length of a semantic unit specifically refers to the total number of elements (the smallest text unit generated by the segmenter of the pre-trained language coding model; for example, in BERT models, this might be a WordPiece for English and a character or word for Chinese) after processing by the segmenter. The preset length threshold should be set based on the maximum input capacity of the selected model and the actual text features. The core principle is to prevent excessively long units from being truncated during subsequent vectorization or causing a decrease in semantic representation quality, while ensuring semantic integrity. For a typical model with a maximum input length of 512, this threshold is usually set between 30 and 50 elements. The final result is a list of semantic units arranged in their original order.

[0080] Each semantic unit is individually input into a pre-trained language coding model. During input, model-specific start markers (e.g., [CLS]) and separator markers (e.g., [SEP]) are added before and after the semantic unit, then converted into a corresponding element ID sequence. This element ID sequence is input into the model, and a complete forward propagation computation is performed, sequentially passing through the model's embedding layer and all encoder layers (i.e., a stack of multiple encoder layers). The encoded state vectors of all elements corresponding to the currently input semantic unit, output from the last layer of the model's encoder, are obtained. These vectors form a set of encoded state vectors. The arithmetic mean of all vectors in this set is calculated to generate a dense vector with a fixed dimension of D, which is the semantic vector of that semantic unit. Each semantic unit corresponds to one semantic vector. Here, D is the output dimension of the final output layer of the selected pre-trained language coding model's encoder; for example, D=768 in the BERT-base model. This fixed dimension D is an inherent, predefined hyperparameter of the model architecture and is independent of the specific content and length of the semantic unit.

[0081] During the forward propagation computation, the attention weight matrix output by the last layer of the multi-head self-attention mechanism in the model encoder is directly extracted. This matrix contains multiple [L,L]-dimensional matrices (L being the number of semantic unit elements) generated by all attention heads. The rows and columns in the matrix correspond to each element in the semantic unit, and each value A{m,n} in the matrix represents the degree of attention of the m-th element to the n-th element when constructing the overall semantics of the unit. First, the arithmetic mean of the [L,L]-dimensional attention weight matrices generated by all attention heads is calculated to obtain the joint attention matrix. Then, the arithmetic mean of each row element of the joint matrix is ​​calculated to generate a probability vector of length L that reflects the distribution of lexical importance within the semantic unit. This probability vector is defined as the self-attention distribution, and each semantic unit corresponds to a self-attention distribution.

[0082] The final output is an ordered list that strictly follows the original text order, recording each semantic unit, its corresponding semantic vector, and self-attention distribution.

[0083] In one embodiment of the present invention, step S2, calculating the cognitive tension intensity between any two semantic units, includes the following steps:

[0084] Based on the ordered list, we obtain each semantic unit in the text data and its corresponding semantic vector and self-attention distribution;

[0085] Extract the semantic vector of the i-th semantic unit The semantic vector of the j-th semantic unit ;right and Perform L2 norm normalization on each vector, then calculate the dot product of the normalized vectors, and finally calculate... and The semantic similarity is obtained by using the cosine similarity. The value range is [0,1];

[0086] Extract the self-attention distribution of the i-th semantic unit The self-attention distribution of the j-th semantic unit and to and Perform element-wise smoothing based on the constant ε;

[0087] The distribution of self-attention is obtained through a calculation formula. and KL divergence between them serves as a contribution to attention differences. ;

[0088] Calculation formula: The cognitive tension intensity between the i-th and j-th semantic units is obtained. ,in, Contributions to semantic differences; Contributions to attention differences; This is the distance attenuation factor; denoted as the ordinal distance between semantic units i and j in the text sequence; the semantic difference weight coefficient α, the attention distribution difference weight coefficient β, and the distance decay coefficient γ are all adjustable hyperparameters with a value greater than zero.

[0089] Specifically, the ordered list is parsed to obtain each semantic unit in the text data, along with its corresponding semantic vector and self-attention distribution. The semantic vector is a dense vector with a fixed dimension of D, and the self-attention distribution is a probability vector of length L. The semantic vectors corresponding to the specified i-th and j-th semantic units are then extracted from these vectors. , (Fixed dimension D) and self-attention distribution , (Length is L).

[0090] semantic vectors and Perform L2 norm normalization to ensure that the magnitude is 1; calculate the semantic vector. and The L2 norm is calculated using the following formula: and Where k is the dimension position index, ranging from 1 to D; then, each element of the semantic vector is divided by its corresponding L2 norm to obtain the normalized vector. and Then, the normalized semantic vector is calculated. and The dot product is calculated using the following formula: The result of this dot product is directly used as the cosine similarity between the two. Its value ranges from [0,1]. The closer the similarity value is to 1, the more semantically similar the meaning; the closer it is to 0, the greater the semantic difference. Based on this, the fundamental contribution of semantic difference to cognitive tension is quantified as the difference between 1 and the similarity value, i.e. .

[0091] Distribution of self-attention and Element-wise smoothing based on a constant ε is performed by adding a minimal constant to the original value at each element position k (k ranges from 1 to L). (usually) ), and then renormalize to ensure numerical stability. The calculation formula is as follows: and ; Calculate the self-attention distribution after smoothing. and The KL divergence between them is calculated using the following formula: The KL divergence value As a contribution to attentional differences, it is used to measure the degree of difference in attentional focusing patterns within two semantic units. The greater the difference, the more significant the change in cognitive focus.

[0092] Cognitive tension intensity was calculated using a weighted combination formula. This value measures the degree of difference and conflict between two semantic units, depending on three factors: differences in semantic content, differences in attention distribution, and their distance in the text. The closer the distance in the text and the greater the differences in both semantics and attention, the stronger the cognitive tension. The calculation formula is: ;in, Contributions to semantic differences; Contributions to attention differences; This is the distance attenuation factor; The ordinal distance between semantic units i and j in the text sequence is calculated using the following formula: ;

[0093] The three parameters—semantic difference weighting coefficient α, attention distribution difference weighting coefficient β, and distance decay coefficient γ—are all adjustable hyperparameters with a value greater than zero. α adjusts the contribution of semantic difference to cognitive tension; a larger α value should be set when the text classification task relies more on semantic content differences. β adjusts the contribution of attention distribution differences to cognitive tension; a larger β value should be set when the task focuses more on changes in text structural focus. γ controls the decay rate of the influence of ordinal distance; a larger γ value should be set when it is necessary to emphasize the interaction between local semantic units. The specific values ​​of these parameters need to be determined through hyperparameter tuning techniques such as grid search or Bayesian optimization on the validation set to obtain optimal classification performance. For example, typically, the initial values ​​of these three parameters can be set to α=1.0, β=1.0, and γ=0.1, respectively.

[0094] In one embodiment of the present invention, step S2 involves constructing an N×N symmetric cognitive tension field matrix, including the following steps:

[0095] Create an N×N two-dimensional matrix F, where N is the total number of semantic units, and initialize all matrix elements to 0;

[0096] For all unit pairs (i,j) in the semantic unit set, where i,j∈[1,N] and i≠j, a parallel computing thread is started; in each computing thread, the cognitive tension intensity between the i-th semantic unit and the j-th semantic unit is calculated using the cognitive tension intensity calculation method. ;

[0097] All the calculated results The values ​​are filled into the corresponding positions in matrix F, and the diagonal elements of matrix F are also filled. Set to 0;

[0098] Calculate matrix F and its transpose. difference matrix And calculate the Frobenius norm of the difference matrix as the symmetry deviation δ;

[0099] Set a symmetry tolerance threshold τ>0, and compare the symmetry deviation δ with the tolerance threshold τ.

[0100] If δ>τ, then perform symmetry processing on matrix F to obtain the symmetric cognitive tension field matrix. ;

[0101] If δ≤τ, then the original matrix F is directly used as the symmetric cognitive tension field matrix. ;

[0102] Output symmetric cognitive tension field matrix .

[0103] Specifically, an N×N two-dimensional matrix F is created, where N is the total number of semantic units, and all elements of the matrix are initially set to 0. For all unit pairs (i,j) in the semantic unit set, where i,j∈[1,N] and i≠j, multiple parallel computing threads are started. In each independent computing thread, the cognitive tension intensity between the i-th semantic unit and the j-th semantic unit is calculated using the cognitive tension intensity calculation method. .

[0104] After parallel computation is completed, all the results obtained by each computation thread are... The values ​​are filled into the corresponding positions in matrix F according to their corresponding semantic unit positions. Specifically, they represent the cognitive tension intensity between the semantic unit at position i and the semantic unit at position j. , which is filled into the position of the i-th row and j-th column of matrix F.

[0105] For the diagonal elements of matrix F Explicitly set it to 0; when i=j, semantic similarity =1, semantic difference contribution term is 0; at the same time, self-attention distribution KL divergence with itself =0; therefore, according to the formula for calculating cognitive tension intensity, the theoretical value of the diagonal element should be 0.

[0106] Due to the asymmetric nature of KL divergence, i.e. ≠ This can lead to the calculated matrix F not being perfectly symmetric. To quantify the degree of this asymmetry, the symmetry deviation δ of matrix F is calculated using the following formula: ;in, It is the transpose of matrix F. Let F be a matrix and its transpose. The difference matrix, where N is the total number of semantic units.

[0107] A symmetry tolerance threshold τ>0 is preset, which is usually set to a relatively small positive number (e.g., This is used to determine whether a matrix satisfies symmetry requirements. The relationship between the symmetry deviation δ and the tolerance threshold τ is compared:

[0108] When δ>τ, it indicates that the asymmetry of matrix F exceeds the acceptable range. A symmetry transformation is then performed on matrix F to obtain the symmetric cognitive tension field matrix. The symmetry processing employs an arithmetic average strategy, that is, for each element... ;

[0109] When δ≤τ, it indicates that the asymmetry of matrix F can be ignored, and the original matrix F can be directly used as the symmetric cognitive tension field matrix. Output.

[0110] Finally, the symmetric cognitive tension field matrix is ​​output. The matrix is ​​an N×N real symmetric matrix, where each element... It represents the intensity of symmetric cognitive tension between the i-th and j-th semantic units.

[0111] In one embodiment of the present invention, step S3, calculating the information entropy of each semantic unit, includes the following steps:

[0112] By using an ordered list, we can obtain each semantic unit in the text data, along with its corresponding semantic vector and self-attention distribution.

[0113] The semantic vectors of N semantic units are combined in order to form an N×D semantic vector matrix V, where D is the dimension of the semantic vector;

[0114] The matrix V is normalized using the L2 norm, and the dot product of the normalized matrices is calculated to obtain an N×N semantic relevance matrix S, where the elements... This represents the semantic association degree between semantic units i and j;

[0115] For each semantic unit i, i∈[1,N], extract the i-th row vector from the semantic relevance matrix S. =[ , ,..., ];

[0116] For the correlation vector The probability distribution is obtained by applying the softmax transform formula for temperature regulation. ;

[0117] Through probability distribution Calculate the information entropy The calculation formula is: ,in, This represents the probability distribution of the semantic influence of the i-th semantic unit on the k-th semantic unit;

[0118] Output information entropy sequence ,in, This represents the information entropy value of the i-th semantic unit; the information entropy sequence satisfies ≥0 and ≤ The mathematical properties of .

[0119] Specifically, the ordered list is parsed to extract the semantic vector of each semantic unit. (Dimension D); N semantic vectors Stack them in order to form an N×D semantic vector matrix V. Each row of matrix V corresponds to the semantic vector of a semantic unit, and the columns correspond to the dimensions of the semantic vectors.

[0120] The semantic vector matrix V is normalized using the L2 norm to ensure that the magnitude of each semantic vector is 1; for each semantic vector... Calculate its L2 norm ,in Represents semantic vectors The j-th element; the vector Dividing each element by its L2 norm yields a normalized vector. Calculate the dot product of the normalized matrices to obtain the semantic relevance matrix S: ,in The normalized matrix, This is the transpose of the normalized matrix.

[0121] The semantic relevance matrix S is an N×N real symmetric matrix, whose elements This represents the semantic correlation degree between semantic unit i and semantic unit j. Since the vector has been normalized, The value range is [-1, 1], but the actual value is usually concentrated in the [0, 1] interval; The closer a value is to 1, the more similar the semantics; the closer a value is to 0, the more unrelated the semantics; negative values ​​indicate opposite semantics, but this is less common in practice.

[0122] For each semantic unit i (i∈[1,N]), extract the i-th row vector from the semantic relevance matrix S. =[ , ,..., The vector represents the semantic association strength between the i-th semantic unit and all semantic units in the text (including itself).

[0123] For the correlation vector Perform a softmax transform on the temperature regulation to obtain the probability distribution. , which represents the semantic influence probability of the i-th semantic unit on all semantic units in the text. The calculation formula is: ;in, This represents the probability of the semantic influence of the i-th semantic unit on the k-th semantic unit; It is a vector The maximum value in θ is used for numerical stability processing to prevent overflow in exponential operations; θ is a temperature parameter greater than zero, controlling the concentration of the probability distribution, with a typical value of θ=1.0; when θ>1, the distribution is flatter, considering more weak correlations; when θ<1, the distribution is more concentrated, highlighting the strongest correlations. The value of θ needs to be optimized using a validation set. Probability Distribution Normalization conditions must be met: This ensures its validity as a probability distribution.

[0124] For each semantic unit i (i∈[1,N]), based on the probability distribution The information entropy of the i-th semantic unit is calculated using the Shannon entropy formula in information theory. The calculation formula is as follows: During the calculation process, numerical stability processing is required. When = 0, according to the definition of a limit, =0. In actual calculations, for Apply a minimum truncation. ,in = ,avoid Undefined.

[0125] When information entropy Approaching its theoretical upper limit When, it indicates the probability distribution The information entropy tends to be uniformly distributed, which corresponds to the i-th semantic unit having significant semantic connections with many other semantic units in the text, and its semantic role in the global context is more complex or in a transitional position; conversely, when the information entropy is close to uniformly distributed, it indicates that the i-th semantic unit has significant semantic connections with many other semantic units in the text, and its semantic role in the global context is more complex or in a transitional position; When it approaches its theoretical lower limit of 0, it indicates that the probability distribution... High concentration corresponds to the i-th semantic unit having a clear semantic focus and strong independence, and only being strongly associated with a few specific semantic units. To ensure the correctness of the calculation results, all information entropies are verified. All satisfy the basic mathematical properties of information theory, namely ≥0 (non-negativity), and ≤ (Boundedness).

[0126] Finally, the information entropy calculated for each semantic unit and satisfying basic mathematical properties is... The information is combined strictly according to its order in the original text (i from 1 to N), outputting an ordered sequence of information entropy. .

[0127] In one embodiment of the present invention, step S3, calculating the information entropy change between adjacent semantic units according to the sequence order to form an information entropy change chain, includes the following steps:

[0128] Traversing the information entropy sequence Calculate the information entropy change between each semantic unit and its next adjacent semantic unit;

[0129] All the calculated information entropy changes are combined according to the natural order of their corresponding semantic units in the text to form and output an information entropy change chain sequence.

[0130] Specifically, based on information entropy sequences Each element in the sequence Let represent the information entropy value of the i-th semantic unit, and each All strictly meet ≥0 and ≤ The mathematical properties of the sequence are as follows: Iterate through each position i in the information entropy sequence, where i is an integer index from 1 to N-1 (i.e., i∈[1,N-1]); for each position i, calculate the information entropy change between adjacent semantic units. The calculation formula is: ,in, This represents the information entropy of the i-th semantic unit; This represents the information entropy of the (i+1)th semantic unit; Let be a scalar value representing the change in information entropy from the i-th to the (i+1)-th semantic unit. All calculated information entropy changes... Combining them according to the natural order of their indices i (i.e., i from 1 to N-1), an information entropy change chain sequence is formed: The length of this sequence is N-1, which strictly corresponds to the number of semantic units N, ensuring that each pair of adjacent units is covered. Verify that the length of the information entropy change chain sequence is N-1. If N=1 (i.e., the text has only one semantic unit), then the information entropy change chain is an empty sequence, and an empty result is output directly.

[0131] when A value greater than 0 indicates an increase in information entropy, corresponding to an evolution of information structure from ordered to disordered, commonly seen in the introduction of new concepts, divergent thinking, or descriptive expansions. When When the entropy is less than 0, it indicates a decrease in information entropy, corresponding to the evolution of information structure from disorder to order. This is commonly seen in inductive summaries, convergent arguments, or conclusive statements. (Information entropy change chain sequence) Strictly adhere to the original text order to ensure that the evolutionary trajectory is consistent with the text structure.

[0132] In one embodiment of the present invention, step S4 includes the following steps:

[0133] From the perspective of symmetric cognitive tension field matrix Multi-dimensional features are extracted, including global statistical features, average cognitive tension intensity features, and graph topological features;

[0134] Global statistical characteristics include the mean of all elements in the matrix, the variance of all elements in the matrix, and the maximum value in the matrix;

[0135] Calculate the arithmetic mean of all elements in the upper triangular part of the matrix except for the main diagonal elements, and use it as the average cognitive tension intensity feature;

[0136] The symmetric cognitive tension field matrix Treating it as the adjacency matrix of a weighted undirected graph, the graph is binarized by setting a predefined tension intensity threshold, and the number of nodes contained in its maximum connected component is calculated and used as the graph topological feature.

[0137] Multi-dimensional features are extracted from the information entropy change chain, including basic statistics, zero-crossing frequency features, persistent pattern length features, and sequence morphology features;

[0138] Basic statistics include the arithmetic mean and variance of the information entropy change chain;

[0139] The frequency of zero crossings is obtained by counting the number of times that adjacent entropy change values ​​in the statistical information entropy change chain have different signs;

[0140] The continuous pattern length feature includes the maximum length of all consecutive positive subsequences and the maximum length of all consecutive negative subsequences in the information entropy change chain;

[0141] The sum of the absolute values ​​of the first-order difference sequences of the information entropy transformation chain is calculated and used as the sequence morphological feature;

[0142] The numerical features extracted from the symmetric cognitive tension field matrix and the information entropy transformation chain are standardized; all standardized features are then input into a pre-trained multilayer perceptron classification model.

[0143] A multilayer perceptron classification model comprises an input layer, at least one hidden layer with a non-linear activation function, and an output layer. The input layer receives features, and the number of its neurons is equal to the total number of standardized features. At least one hidden layer, connected to the input layer, is equipped with a non-linear activation function to perform non-linear transformations and combinations on the input features to generate combined features. The output layer has the same number of neurons as the number of text categories to be classified, and uses the Softmax function to calculate the probability distribution of the text belonging to each category based on the combined features. The category corresponding to the highest probability value in the probability distribution is taken as the final text classification result.

[0144] Specifically, the symmetric cognitive tension field matrix It is an N×N real symmetric matrix, where each element It represents the cognitive tension intensity between the i-th and j-th semantic units; features are extracted from the cognitive tension field matrix, including global statistical features, average cognitive tension intensity features, and graph topological features.

[0145] Global statistical characteristics include the mean of all elements in the matrix, the variance of all elements in the matrix, and the maximum value in the matrix; these are calculated using the formula... The symmetric cognitive tension field matrix is ​​obtained. The average value of all elements in This reflects the overall cognitive tension level of the text; it is calculated using a formula. The symmetric cognitive tension field matrix is ​​obtained. variance of all elements This measures the degree of fluctuation in the distribution of cognitive tension throughout the text; it is calculated using the formula... The symmetric cognitive tension field matrix is ​​obtained. The maximum value among all elements This marks the point of strongest cognitive conflict in the text;

[0146] Calculate the symmetric cognitive tension field matrix The arithmetic mean of all elements in the upper triangular portion (excluding the main diagonal elements) yields the average cognitive tension intensity characteristic. The calculation formula is: This feature focuses on the interaction between different semantic unit pairs, avoiding the influence of its own tension being always zero.

[0147] because It is a real symmetric matrix, that is, it satisfies = Its upper triangular part (excluding the main diagonal) already contains all non-repetitive, non-reflexive semantic unit pairs. Therefore, calculating the arithmetic mean of its upper triangular part can effectively characterize the average cognitive tension intensity among all independent unit pairs in the text, while avoiding the influence of diagonal elements on the calculation of the mean.

[0148] The process of extracting graph topological features is as follows: First, the symmetric cognitive tension field matrix is... Viewed as a weighted undirected graph adjacency matrix, where each node represents a semantic unit, the cognitive tension between two semantic units i and j is... If the value is not zero, then there is an edge between them, and the weight of that edge is... The value is then used for image binarization, and a predefined tension strength threshold is set. (This threshold can be determined through hyperparameter tuning techniques such as grid search, for example, by taking...) (The median of all elements); traverse the adjacency matrix to generate a new binary adjacency matrix; the rule is that if ≥ If the value of a node is 1, it indicates a connection between nodes i and j in the binary graph; otherwise, it is 0, indicating no connection. This process is called binarization, and its purpose is to filter out weak connections and retain only significant cognitive tension relationships. Finally, based on this binary adjacency matrix, a depth-first search algorithm is used to identify all connected components in the graph. Specifically, an access marker array is initialized to record whether each node has been visited; each unvisited node is traversed, and all connected nodes are recursively visited from that node, marking it as a connected component; the number of nodes in each connected component is counted; and the maximum number of nodes in all connected components is taken as the graph topological feature. This feature reflects the size of the largest cluster of semantic units tightly connected by strong cognitive tension in the text. The larger the value, the more coherent and unified the cognitive structure of the text tends to be under strong tension relationships.

[0149] Information entropy change chain sequence ,in , representing the change in information entropy between adjacent semantic units; features extracted from the information entropy change chain sequence include basic statistics, zero-crossing frequency features, continuous pattern length features, and sequence morphology features.

[0150] Basic statistics include the arithmetic mean and variance of the information entropy change chain; calculated using the formula... The arithmetic mean of the information entropy change chain is obtained. This value reflects the overall evolution direction of the information structure; a positive value indicates an increase in overall information entropy (information divergence), while a negative value indicates a decrease in information entropy (information convergence). It is calculated using the formula... The variance of the information entropy transformation chain is obtained. This value measures the stability of the information evolution process; the larger the variance, the more volatile the information evolution.

[0151] The frequency feature of zero crossings is obtained by counting the number of times adjacent entropy change values ​​have different signs in the statistical information entropy change chain. Specifically, for i from 1 to N-2, the condition is checked. · If the value is less than 0, the count is incremented by 1. This feature represents the frequency of information structure transitions; a larger value indicates more logical transitions in the text.

[0152] The duration feature includes the maximum length of all consecutive positive subsequences and the maximum length of all consecutive negative subsequences in the information entropy change chain. Traversal Statistical analysis of all consecutive Subsequences with values ​​greater than 0 are identified, and their maximum length is recorded; this is the maximum length of all consecutive positive subsequences. This characteristic reflects the persistence of the information divergence phase. Simultaneously, all consecutive positive subsequences are statistically analyzed. The subsequence with a value <0 is recorded, and its maximum length is recorded, which is the maximum length of all consecutive negative subsequences; this feature reflects the persistence of the information convergence phase.

[0153] Sequence morphological features are characterized by calculating the sum of the absolute values ​​of the first-order difference sequences of the information entropy transformation chain. The first-order difference sequence D is defined as follows: (i ranges from 1 to N-2). Sequence morphological characteristics The calculation formula is: This feature quantifies the local fluctuation amplitude of the information entropy change chain; the larger the value, the more drastic the change in information structure.

[0154] After extracting multi-dimensional features from the symmetric cognitive tension field matrix and information entropy transformation chain, the obtained numerical features need to be standardized. The standardization process uses the Z-score method, which relies on the mean of each feature obtained from a labeled training text set. ) and standard deviation ( The training set contains 1000 texts of various types, covering all text categories to be classified; for any original feature value x, its standardized value formula is: ,in, x represents the original feature value, which is used to standardize the feature value. Then, these standardized feature values ​​are concatenated in a preset fixed order to form a one-dimensional standardized feature vector, the dimension of which is equal to the total number of features extracted.

[0155] The multilayer perceptron classification model is a pre-trained, fixed-parameter feedforward neural network model that can be directly used to receive the aforementioned one-dimensional standardized feature vector and output text classification results. The model's structure includes an input layer, at least one hidden layer, and an output layer.

[0156] The input layer receives a one-dimensional normalized feature vector, and the number of its neurons is equal to the dimension of the feature vector, i.e., the total number of features, ensuring that each feature value is input to a corresponding neuron.

[0157] At least one hidden layer is fully connected to the input layer. Each hidden layer consists of a number of neurons, the number of which is a specific value determined through hyperparameter optimization before model training. The hidden layer performs a non-linear transformation on the input data, and its computation process is expressed as follows: ,in, For the first Layer output vector; superscript Assign the neural network layer number to which this parameter or variable belongs. =1,2,...,L, where L is the total number of layers in the neural network; The output vector of the previous layer (for the first hidden layer, (for the input feature vector x). and The first The weight matrix and bias vector corresponding to each layer. Weight matrix The dimension is × ,in It is the dimension of the input vector. This represents the number of neurons in the hidden layer; the bias vector. The dimension is ×1. These weights and bias parameters are fixed values ​​learned through optimization algorithms during the model training phase. The function is used as a non-linear activation function, and it is defined as follows: The aim is to introduce nonlinear transformation capabilities, enabling the model to learn and combine complex feature patterns.

[0158] The output layer is fully connected to the last hidden layer, and the number of neurons in it is equal to the total number C of the text categories to be classified. The output layer uses the Softmax function to map the high-dimensional combined features output from the hidden layer into a probability distribution vector, where each element of the vector represents the probability that the text belongs to the corresponding category. The calculation process of the output layer is represented as follows: ,in, It is the output vector of the last hidden layer (i.e., the Lth layer), with dimension 1. (i.e., the number of neurons in the hidden layer); It is the weight vector in the output layer corresponding to class c, and its dimension is... same; It is the bias scalar corresponding to category c; This represents the probability calculated by the model that the text belongs to category c. The weights of all output layers... and bias These are also fixed parameters learned during the model training phase.

[0159] The multilayer perceptron model was trained on a labeled text dataset. During training, the labeled text dataset was used, employing backpropagation and the Adam optimizer to minimize the cross-entropy loss function. The learning rate was set to 0.001, the batch size to 32, and the maximum number of iterations to 1000. Training was stopped early when the loss function stopped decreasing after 50 consecutive iterations on the validation set. All parameters in the model, including the weight matrices of each hidden layer, were iteratively adjusted. Bias vector and the weight vector of the output layer and bias scalar The training continues until the model converges; after training is complete, these parameters are fixed to form the pre-trained model.

[0160] The standardized feature vector of the text to be classified is input into this pre-trained multilayer perceptron model. After forward propagation, the model calculates the probability distribution of the text belonging to each category at the output layer. The final text classification result is determined as the category label corresponding to the highest probability value in this probability distribution.

[0161] In one embodiment of the present invention, step S5 includes the following steps:

[0162] Based on the text classification results, multiple texts are grouped according to their categories; for each category, the symmetric cognitive tension field matrix and information entropy change chain of all texts in the group are aggregated and then standardized and aligned.

[0163] The symmetric cognitive tension field matrices within the same category, after standardization and alignment, are aggregated to generate the average cognitive tension field matrix of that category and its visualization results; the information entropy change chains within the same category are aggregated to generate the typical information entropy change trajectories of that category and their visualization results.

[0164] For each text, the arithmetic mean of all elements in its symmetric cognitive tension field matrix is ​​calculated as the global intensity index of the cognitive tension field of that text; at the same time, the arithmetic mean of its information entropy change chain is calculated as the information entropy change statistical index of that text; the correlation between the global intensity index of the cognitive tension field and the information entropy change statistical index is calculated.

[0165] By integrating the visualization results of the average cognitive tension field matrix, the visualization results of the typical trajectory of information entropy change, and the correlation calculation results between the global intensity index of cognitive tension field and the statistical index of information entropy change, a structured statistical analysis report is generated.

[0166] Specifically, based on the classification results of multiple texts, and the symmetric cognitive tension field matrix corresponding to each text. (The superscript (t) represents the t-th text), corresponding to the information entropy change chain for each text. Information entropy sequence corresponding to each text .

[0167] Maintain a database that stores four-tuple data for all analyzed text: {category label, ...} , , When statistical analysis is required, retrieve data from the database by category label according to the analysis objective. Retrieve all text collections belonging to this category. ={t|category(t)= }

[0168] Due to the number of semantic units in different texts (i.e. The total number of semantic units in the t-th text is different for each text, and their cognitive tension field matrix is ​​different for each text. Dimensions × and information entropy change chain Length ( -1) They are also different; before subsequent aggregation operations, data alignment processing is required: for the cognitive tension field matrix, the matrix of different sizes is uniformly scaled to a fixed standard size M×M (M=100) by bilinear interpolation. Specifically, a linear kernel function is used to calculate the new pixel value based on the weighted average of the four nearest neighbor pixel values; for the information entropy change chain, the dynamic time warping (DTW) algorithm is used for sequence alignment. Euclidean distance is used as the cost function. The minimum cost path is found through dynamic programming to align the sequence to a uniform length L (L=50).

[0169] For each text category The cognitive tension field matrix of all texts in this category The matrix is ​​uniformly scaled to a standard M×M size. For all normalized M×M matrices, the arithmetic mean of all matrix elements at each position (p, q) is calculated to obtain the average cognitive tension field matrix for that category. ,Will The matrix is ​​visualized as a heatmap, generating a heatmap of the average cognitive tension field.

[0170] For each text, calculate its cognitive tension field matrix. The global average intensity is calculated using the following formula: Then, for each category Calculate the global average intensity of the cognitive tension field matrix for all texts in this category. Statistic (arithmetic mean) Standard deviation (e.g., median, etc.) constitute an overview of the distribution of cognitive tension field intensity in this type of text.

[0171] For each category The Dynamic Time Warping (DTW) algorithm is used to transform the information entropy chain of all texts in this category. Perform alignment; based on the alignment, calculate the average entropy change value at each alignment position to obtain the typical trajectory of information entropy change. ;Will Plotted as a line graph, the horizontal axis represents the aligned sequence index, and the vertical axis represents the average information entropy change. The ±1σ standard deviation range can be overlaid and displayed.

[0172] For each category For all text below, calculate the statistical distribution of its entropy change chain features, including the mean entropy change. Distribution, entropy variation variance The distribution of zero-crossing frequency, and the distribution of the maximum length of continuous positive or negative entropy change.

[0173] For each text category Two metrics were collected from all texts in this category: the global strength of the cognitive tension field. and the mean of the information entropy change chain Calculate the Pearson correlation coefficient between these two indicators. The calculation formula is: ;in, Indicate category The number of texts in the text Indicate category Global intensity of all cognitive tension fields The arithmetic mean, Indicate category Mean of all information entropy change chains The arithmetic mean; correlation coefficient It is recorded as a field-entropy correlation strength index, used to quantify the intrinsic connection between cognitive structure and information evolution.

[0174] The system automatically integrates all analysis results to generate a structured statistical analysis report. The report comprises three key parts: first, the cognitive tension field analysis results, providing a visual representation of the average cognitive tension field for each category and key statistical indicators, revealing the cognitive conflict patterns of different text categories; second, the information entropy change analysis results, characterizing the evolutionary features of the information structure of each text category through typical trajectory diagrams and statistical measures; and third, the field-entropy correlation analysis, demonstrating the quantitative correlation strength between cognitive tension and information evolution. The report concludes with a summary of the category differences, clearly pointing out the essential differences in cognitive structure and information evolution patterns among different text categories. The final output can be a PDF document containing charts and textual explanations, or a structured data file format.

[0175] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A text intelligent classification and statistical analysis method based on artificial intelligence, characterized in that, Includes the following steps: S1: Receive the text data to be analyzed, divide it into N consecutive semantic units, and generate a corresponding semantic vector and self-attention distribution for each semantic unit; S2: Based on semantic units and their corresponding semantic vectors and self-attention distributions, calculate the cognitive tension intensity between any two semantic units, and construct an N×N symmetric cognitive tension field matrix; including the following steps: Based on the ordered list, we obtain each semantic unit in the text data and its corresponding semantic vector and self-attention distribution; Extract the semantic vector of the i-th semantic unit The semantic vector of the j-th semantic unit ;right and Perform L2 norm normalization on each vector, then calculate the dot product of the normalized vectors, and finally calculate... and The semantic similarity is obtained by using the cosine similarity. The value range is [0,1]; Extract the self-attention distribution of the i-th semantic unit The self-attention distribution of the j-th semantic unit and to and Perform element-wise smoothing based on the constant ε; The distribution of self-attention is obtained through a calculation formula. and KL divergence between them serves as a contribution to attention differences. ; Calculation formula: The cognitive tension intensity between the i-th and j-th semantic units is obtained. ,in, Contributions to semantic differences; Contributions to attention differences; This is the distance attenuation factor; The distance between semantic units i and j in the text sequence is denoted as α; the semantic difference weighting coefficient α, the attention distribution difference weighting coefficient β, and the distance decay coefficient γ are all adjustable hyperparameters with a value greater than zero. S3: Calculate the information entropy of each semantic unit, and calculate the information entropy change between adjacent semantic units according to the sequence order to form an information entropy change chain; S4: Extract features from the symmetric cognitive tension field matrix and information entropy transformation chain, and input the features and the text data to be analyzed into the classification model to obtain the text classification results; S5: Based on text classification results, symmetric cognitive tension field matrix, and information entropy transformation chain, perform cross-text aggregation statistical analysis and generate statistical analysis reports.

2. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 1, characterized in that, S1 includes the following steps: The system receives and preprocesses the text data to be analyzed; it uses a pre-trained language coding model to perform dependency parsing on the preprocessed text, identifies clause boundaries in the text, and the clause boundaries are grammatical segmentation points identified by subordinating conjunctions and specific punctuation marks; based on the clause boundaries, the text is divided into N clauses, which are then treated as consecutive semantic units arranged in the original text order. Each semantic unit is individually input into a pre-trained language coding model, and a forward propagation computation is performed. The model is then processed sequentially through the embedding layer and the encoder layer to obtain the set of encoded state vectors corresponding to the current input semantic unit, which is output by the last layer of the encoder. The arithmetic mean of all vectors in the encoded state vector set is calculated to generate a semantic vector for the corresponding semantic unit. During the forward propagation computation, the attention weight matrix output by the multi-head self-attention mechanism in the last layer of the model encoder is acquired synchronously. The attention weight matrix includes multiple matrices generated by each attention head for the semantic unit of the current input. First, the attention weight matrices generated by each attention head are arithmetically averaged to obtain a joint attention matrix. Then, for the joint attention matrix, a probability vector is generated by calculating the arithmetic mean of the elements in each row and used as the self-attention distribution of the semantic unit. Output an ordered list that records each semantic unit and its corresponding semantic vector and self-attention distribution in the original text order.

3. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 1, characterized in that, In S2, an N×N symmetric cognitive tension field matrix is ​​constructed, including the following steps: Create an N×N two-dimensional matrix F, where N is the total number of semantic units, and initialize all matrix elements to 0; For all unit pairs (i,j) in the semantic unit set, where i,j∈[1,N] and i≠j, a parallel computing thread is started; in each computing thread, the cognitive tension intensity between the i-th semantic unit and the j-th semantic unit is calculated using the cognitive tension intensity calculation method. ; All the calculated results The values ​​are filled into the corresponding positions in matrix F, and the diagonal elements of matrix F are also filled. Set to 0; Calculate matrix F and its transpose. difference matrix And calculate the Frobenius norm of the difference matrix as the symmetry deviation δ; Set a symmetry tolerance threshold τ>0, and compare the symmetry deviation δ with the tolerance threshold τ. If δ>τ, then perform symmetry processing on matrix F to obtain the symmetric cognitive tension field matrix. ; If δ≤τ, then the original matrix F is directly used as the symmetric cognitive tension field matrix. ; Output symmetric cognitive tension field matrix .

4. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 1, characterized in that, In step S3, the information entropy of each semantic unit is calculated, including the following steps: By using an ordered list, we can obtain each semantic unit in the text data, along with its corresponding semantic vector and self-attention distribution. The semantic vectors of N semantic units are combined in order to form an N×D semantic vector matrix V, where D is the dimension of the semantic vector; The matrix V is normalized using the L2 norm, and the dot product of the normalized matrices is calculated to obtain an N×N semantic relevance matrix S, where the elements... This represents the semantic association degree between semantic units i and j; For each semantic unit i, i∈[1,N], extract the i-th row vector from the semantic relevance matrix S. ; For the correlation vector The probability distribution is obtained by applying the softmax transform formula for temperature regulation. ; Through probability distribution Calculate the information entropy The calculation formula is: ,in, This represents the probability distribution of the semantic influence of the i-th semantic unit on the k-th semantic unit; Output information entropy sequence ,in, This represents the information entropy value of the i-th semantic unit; the information entropy sequence satisfies ≥0 and ≤ The mathematical properties of .

5. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 4, characterized in that, In step S3, the information entropy change between adjacent semantic units is calculated according to the sequence order to form an information entropy change chain, including the following steps: Traversing the information entropy sequence Calculate the information entropy change between each semantic unit and its next adjacent semantic unit; All the calculated information entropy changes are combined according to the natural order of their corresponding semantic units in the text to form and output an information entropy change chain sequence.

6. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 1, characterized in that, S4 includes the following steps: Multi-dimensional features are extracted from the symmetric cognitive tension field matrix Fsym, including global statistical features, average cognitive tension intensity features, and graph topological features; Global statistical characteristics include the mean of all elements in the matrix, the variance of all elements in the matrix, and the maximum value in the matrix; Calculate the arithmetic mean of all elements in the upper triangular part of the matrix except for the main diagonal elements, and use it as the average cognitive tension intensity feature; The symmetric cognitive tension field matrix Fsym is regarded as the adjacency matrix of a weighted undirected graph. By setting a predefined tension intensity threshold, the graph is binarized, and the number of nodes contained in its maximum connected component is calculated and used as the graph topological feature. Multi-dimensional features are extracted from the information entropy change chain, including basic statistics, zero-crossing frequency features, persistent pattern length features, and sequence morphology features; Basic statistics include the arithmetic mean and variance of the information entropy change chain; The frequency of zero crossings is obtained by counting the number of times that adjacent entropy change values ​​in the statistical information entropy change chain have different signs; The continuous pattern length feature includes the maximum length of all consecutive positive subsequences and the maximum length of all consecutive negative subsequences in the information entropy change chain; The sum of the absolute values ​​of the first-order difference sequences of the information entropy transformation chain is calculated and used as the sequence morphological feature; The numerical features extracted from the symmetric cognitive tension field matrix and the information entropy transformation chain are standardized; all standardized features are then input into a pre-trained multilayer perceptron classification model. A multilayer perceptron classification model comprises an input layer, at least one hidden layer with a non-linear activation function, and an output layer. The input layer receives features, and the number of its neurons is equal to the total number of standardized features. At least one hidden layer, connected to the input layer, is equipped with a non-linear activation function to perform non-linear transformations and combinations on the input features to generate combined features. The output layer has the same number of neurons as the number of text categories to be classified, and uses the Softmax function to calculate the probability distribution of the text belonging to each category based on the combined features. The category corresponding to the highest probability value in the probability distribution is taken as the final text classification result.

7. The text intelligent classification and statistical analysis method based on artificial intelligence according to claim 1, characterized in that, S5 includes the following steps: Based on the text classification results, multiple texts are grouped according to their categories; for each category, the symmetric cognitive tension field matrix and information entropy change chain of all texts in the group are aggregated and then standardized and aligned. The symmetric cognitive tension field matrices within the same category, after standardization and alignment, are aggregated to generate the average cognitive tension field matrix of that category and its visualization results; the information entropy change chains within the same category are aggregated to generate the typical information entropy change trajectories of that category and their visualization results. For each text, the arithmetic mean of all elements in its symmetric cognitive tension field matrix is ​​calculated as the global intensity index of the cognitive tension field of that text; at the same time, the arithmetic mean of its information entropy change chain is calculated as the information entropy change statistical index of that text; the correlation between the global intensity index of the cognitive tension field and the information entropy change statistical index is calculated. By integrating the visualization results of the average cognitive tension field matrix, the visualization results of the typical trajectory of information entropy change, and the correlation calculation results between the global intensity index of cognitive tension field and the statistical index of information entropy change, a structured statistical analysis report is generated.

Images