Intelligent text retrieval and analysis system driven by natural language processing

Abstract

The disclosed embodiment discloses an intelligent text retrieval and analysis system driven by natural language processing. The system includes: a preprocessing module for acquiring text data from an external system and preprocessing the text data to obtain text data to be processed; a feature analysis module for extracting data features in the text data to be processed, the data features including text features, first semantic features and second semantic features; a retrieval module for responding to search terms input by an external system, and based on the data features, determining retrieval results matching the search terms from the acquired text data; a text analysis module for performing text analysis tasks for the text data to be processed and / or the retrieval results based on the data features, the text analysis tasks including sentiment analysis tasks, topic analysis tasks and summary extraction tasks. The disclosed invention solves the technical problem of low efficiency in processing large-scale, complex semantic text data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text processing, and in particular to an intelligent text retrieval and analysis system driven by natural language processing. Background Art

[0002] With the rapid development of information technology, the amount of data people face is growing exponentially, especially text data, which occupies a large amount of information flow in the fields of Internet, enterprise management, scientific research, etc. At present, traditional text retrieval methods are mainly based on keyword matching, which is inefficient when processing large-scale and complex semantic text data and cannot meet the needs of modern society. Summary of the invention

[0003] The embodiments of the present disclosure provide a natural language processing-driven intelligent text retrieval and analysis system for solving the technical problem of low efficiency in processing large-scale text data with complex semantics.

[0004] According to one aspect of an embodiment of the present disclosure, a natural language processing-driven intelligent text retrieval and analysis system is provided, comprising:

[0005] A preprocessing module, used for acquiring text data from an external system and preprocessing the text data to obtain text data to be processed;

[0006] A feature analysis module, connected to the preprocessing module, for extracting data features from the text data to be processed, wherein the data features include text features, first semantic features for characterizing the dependency relationship between paragraph contents, and second semantic features for characterizing the correlation between contexts;

[0007] A retrieval module, connected to the feature analysis module, for responding to a search term input by the external system and determining a retrieval result matching the search term from the acquired text data based on the data feature;

[0008] A text analysis module is connected to the feature analysis module and the retrieval module, and is used to perform text analysis tasks on the text data to be processed and / or the retrieval results based on the data features, and the text analysis tasks include sentiment analysis tasks, topic analysis tasks, and summary extraction tasks.

[0009] In a possible implementation, the feature analysis module includes:

[0010] A text feature extraction unit, used for converting the text data to be processed into text vector data; inputting the text vector data into a pre-established text feature extraction model to perform feature extraction, and outputting text features containing text keywords;

[0011] The text feature extraction model includes a cascaded convolution layer, an activation function layer, a pooling layer and a fully connected layer, and a connection structure is provided between the convolution layer and the fully connected layer;

[0012] The text feature extraction unit comprises:

[0013] A text feature extraction model processing unit, used for performing feature extraction on the text vector data through the convolution layer to obtain one or more local features;

[0014] Performing nonlinear processing on the local features through the activation function layer;

[0015] The output of the activation function layer is pooled by the pooling layer, and then the output of the pooling layer and the local features output by the convolution layer are integrated by the fully connected layer to obtain text features for the text data to be processed.

[0016] In a possible implementation, the feature analysis module further includes:

[0017] A semantic feature extraction unit is used to convert the text data to be processed into a text sequence, each word or character in the text sequence represents a time step; the text sequence is input into a preset semantic feature extraction model to extract the time series features and long-distance dependencies in the text sequence, and output a first semantic feature for indicating the semantics between paragraph contents.

[0018] In a possible implementation, the feature analysis module further includes a semantic understanding unit;

[0019] The semantic understanding unit comprises:

[0020] A semantic feature extraction unit is used to determine a semantic understanding model related to the text analysis task from a pre-trained BERT model, so that the text data to be processed is input into the semantic understanding model to output a semantic understanding feature associated with the text analysis task as a second semantic feature of the text data to be processed.

[0021] In a possible implementation, the semantic understanding unit further includes a first semantic disambiguation unit connected to the semantic feature extraction unit:

[0022] A first semantic disambiguation unit, configured to determine polysemous words in the text data to be processed, and determine a text semantic scope of the polysemous words according to contextual content of the polysemous words;

[0023] Calculating the matching degree between each word meaning indicated by the polysemous word and the semantic range of the text, so as to screen out the word meaning with the highest matching degree;

[0024] Based on the polysemous word semantic understanding features of each polysemous word and the screened word meanings, the second semantic features of the polysemous word are determined, wherein the polysemous word semantic understanding features are the semantic understanding features output after the semantic understanding model processes the polysemous word.

[0025] In a possible implementation, the semantic understanding unit further includes a second semantic disambiguation unit;

[0026] The second semantic disambiguation unit is used to establish a semantic disambiguation model;

[0027] Determining model parameters to be processed in the semantic disambiguation model;

[0028] Randomizing the parameters of the model to be processed and performing a randomized test so as to adjust the semantic disambiguation model based on the test results;

[0029] The polysemous words in the text data to be processed are input into the adjusted semantic disambiguation model to determine the meaning of the polysemous words for the context in which they are located, and the second semantic feature of the polysemous words is output.

[0030] In a possible implementation, the retrieval module includes:

[0031] A term search unit, used to create an index for the acquired text data, wherein the index records the association between each word and the text data where the word is located;

[0032] Based on the index, at least one of the text data satisfying the Boolean query condition is determined as a text retrieval result, wherein the Boolean query condition is composed of at least one of the search terms and / or preset query conditions;

[0033] The text search results are similarly matched using a preset vector space model to determine the search term matching results.

[0034] In a possible implementation, the retrieval module further includes:

[0035] A semantic analysis unit, connected to the term search unit and the semantic feature extraction unit, for inputting the search terms and the text data into the semantic understanding model to determine the semantic relationship between the search terms and each of the text data, and outputting corresponding semantic understanding features;

[0036] Calculating the similarity between the search term and the semantic understanding features of each of the text data to determine a semantic matching result;

[0037] Based on the semantic matching result and the search term matching result, a retrieval result matching the search term is determined.

[0038] In a possible implementation, the text analysis module includes:

[0039] A sentiment analysis unit, connected to the text feature extraction unit, for inputting the text data to be processed and / or the retrieval results into the text feature extraction model to extract sentiment text features related to the sentiment analysis task; and using a preset classification model to classify the sentiment text features into sentiment tendencies to obtain sentiment trends regarding the text data to be processed and / or the retrieval results;

[0040] A topic analysis unit, used for extracting corresponding text topics from the text data to be processed to form a topic overview;

[0041] The summary extraction unit is used to identify and extract key sentences from the text data to be processed to form an original text summary.

[0042] In a possible implementation, the subject analysis unit includes:

[0043] A topic extraction unit, used to obtain training text data containing multiple topics and mark the topic category to which each training text data belongs;

[0044] Establishing a state set and a conversion strategy in the CMPA model, wherein each state in the state set corresponds to a subject category, and the conversion strategy is used to instruct to convert an associated word related to the subject category into a state related to the subject category;

[0045] Using the training text data to train, test and verify the preset CMPA model to obtain a topic extraction model;

[0046] The text data to be processed is input into the topic extraction model to determine the number of occurrences of the associated words related to each state, so that the state with the largest number of occurrences of the associated words is output as the text topic.

[0047] The technical solution provided by the embodiments of the present disclosure has the following beneficial effects:

[0048] The natural language processing-driven intelligent text retrieval and analysis system provided by the embodiment of the present disclosure includes a preprocessing module for acquiring text data from an external system and preprocessing the text data to obtain text data to be processed; a feature analysis module connected to the preprocessing module for extracting data features from the text data to be processed, wherein the data features include text features, a first semantic feature for characterizing the dependency relationship between paragraph contents, and a second semantic feature for characterizing the correlation between contexts; a retrieval module connected to the feature analysis module for responding to a search term input by the external system and determining, based on the data features, from the acquired text data the text data that is related to the search term. Matching retrieval results; a text analysis module, connected to the feature analysis module and the retrieval module, for performing text analysis tasks for the text data to be processed and / or the retrieval results based on the data features, wherein the text analysis tasks include sentiment analysis tasks, topic analysis tasks, and summary extraction tasks. In this way, by combining technologies such as text feature extraction, semantic understanding, and machine learning, efficient and accurate retrieval and in-depth analysis can be achieved when facing a large amount of text data, providing an intelligent system that can understand text semantics, perform accurate retrieval, and conduct in-depth text analysis, thereby improving the efficiency and accuracy of text data processing and solving the technical problem of low efficiency when processing large-scale, complex semantic text data.

[0049] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A schematic diagram of the structure of a natural language processing-driven intelligent text retrieval and analysis system provided in an embodiment of the present disclosure;

[0052] Figure 2 A schematic diagram of the structure of a text feature extraction model provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0054] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0055] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0056] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0057] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0058] Figure 1 The present invention provides a schematic diagram of a natural language processing-driven intelligent text retrieval and analysis system. The natural language processing-driven intelligent text retrieval and analysis system 100 includes:

[0059] A preprocessing module 101 is used to obtain text data from an external system and preprocess the text data to obtain text data to be processed;

[0060] A feature analysis module 102, connected to the preprocessing module 101, is used to extract data features from the text data to be processed, wherein the data features include text features, first semantic features for characterizing the dependency relationship between paragraph contents, and second semantic features for characterizing the correlation between contexts;

[0061] A retrieval module 103, connected to the feature analysis module 102, for responding to a search term input by the external system and determining a search result matching the search term from the acquired text data based on the data feature;

[0062] The text analysis module 104 is connected to the feature analysis module 102 and the retrieval module 103, and is used to perform text analysis tasks on the text data to be processed and / or the retrieval results based on the data features, and the text analysis tasks include sentiment analysis tasks, topic analysis tasks and summary extraction tasks.

[0063] In the present disclosure, the natural language processing driven intelligent text retrieval and analysis system is a device system with functions such as text preprocessing, feature extraction, semantic analysis, retrieval and text analysis, and the system is connected to an external system to receive data input from the external system. Specifically, the preprocessing module is a system device with preprocessing functions such as data cleaning, standardization, word segmentation, and annotation. More specifically, the preprocessing module is connected to the external system so that the text data from the external system is preprocessed, for example, word segmentation, stop word removal, punctuation removal, part-of-speech tagging, unified word form, etc. Therefore, the present disclosure realizes the removal of irrelevant information, standardization of text, recognition and resolution of certain language characteristics by configuring the preprocessing module, thereby improving the subsequent processing of the text. The feature analysis module is a system device with functions such as text feature extraction, semantic feature extraction, and context-based semantic understanding, the retrieval module is a system device with a post-retrieval function in response to a search term input by a user, and the text analysis module is a system device with text analysis functions such as sentiment analysis, topic model, and text summary.

[0064] The natural language processing-driven intelligent text retrieval and analysis system provided in this embodiment is configured to obtain text data from an external system through a preprocessing module, and preprocess the text data to obtain text data to be processed; a feature analysis module connected to the preprocessing module, and configured to extract data features from the text data to be processed, wherein the data features include text features, a first semantic feature for characterizing the dependency relationship between paragraph contents, and a second semantic feature for characterizing the correlation between contexts; and a retrieval module connected to the feature analysis module, and configured to respond to a search term input by the external system, and to determine, based on the data features, a text data that matches the search term from the acquired text data. The text analysis module is connected with the feature analysis module and the retrieval module, and is used to perform text analysis tasks for the text data to be processed and / or the retrieval results based on the data features, and the text analysis tasks include sentiment analysis tasks, topic analysis tasks and summary extraction tasks. In this way, by combining the technologies of text feature extraction, semantic understanding and machine learning, efficient and accurate retrieval and in-depth analysis can be achieved when facing a large amount of text data, an intelligent system that can understand text semantics, accurate retrieval and in-depth analysis of text is provided, the efficiency and accuracy of text data processing are improved, and the technical problem of low efficiency in processing large-scale text data with complex semantics is solved.

[0065] In some embodiments, the feature analysis module includes:

[0066] A text feature extraction unit, used for converting the text data to be processed into text vector data; inputting the text vector data into a pre-established text feature extraction model to perform feature extraction, and outputting text features containing text keywords;

[0067] The text feature extraction model includes a cascaded convolution layer, an activation function layer, a pooling layer and a fully connected layer, and a connection structure is provided between the convolution layer and the fully connected layer;

[0068] The text feature extraction unit comprises:

[0069] A text feature extraction model processing unit, used for performing feature extraction on the text vector data through the convolution layer to obtain one or more local features;

[0070] Performing nonlinear processing on the local features through the activation function layer;

[0071] The output of the activation function layer is pooled by the pooling layer, and then the output of the pooling layer and the local features output by the convolution layer are integrated by the fully connected layer to obtain text features for the text data to be processed.

[0072] In this embodiment, a convolutional neural network (CNN) is used to extract text features. When processing text data, the convolutional neural network extracts local features in the text, such as keywords or phrases, by setting a convolutional layer, and then through a combination of multiple convolutional layers and pooling layers, higher-level features (i.e., text features in the present disclosure) can be extracted from these local features to facilitate the identification of keywords, phrases, and sentence structures. At the same time, this embodiment can be applied to a large amount of data and a variety of text analysis operations, such as text classification, sentiment analysis, translation, etc., with the help of a convolutional neural network, and has a relatively wide range of applications.

[0073] For example, the text data to be processed is converted into a numerical vector, i.e., text vector data, by word embedding (such as Word2Vec or GloVe), and each word is converted into a dense vector of fixed length. Figure 2 The structural diagram of the text feature extraction model provided by the embodiment of the present disclosure includes a cascaded convolution layer 201, an activation function layer 202, a pooling layer 203 and a fully connected layer 204, and a connection structure is provided between the convolution layer 201 and the fully connected layer 204. Specifically, at least one convolution layer is set in the convolutional neural network, and convolution kernels of different sizes (such as 2-5 word lengths) are used to extract different local features to capture key words and phrases in the text. In the activation function layer, an activation function (such as ReLU) is used to process the output of the convolution layer, increase the nonlinear characteristics of the model, and facilitate the model to learn complex text features. Subsequently, a pooling layer (such as maximum pooling) is used to reduce the feature dimension, extract the most significant features, reduce the computational complexity, and highlight important features. Further, the output of the convolution layer and the output of the pooling layer are integrated through the fully connected layer, so that the output of the fully connected layer is represented as a text feature, providing accurate feature representation for subsequent text processing (such as text analysis tasks, retrieval).

[0074] Therefore, this embodiment can use convolutional neural networks to efficiently extract key local features in text data, further strengthen the recognition and analysis of key words and phrases in the text, obtain local structural information, and improve the accuracy of text feature extraction. At the same time, by adjusting the size and number of convolution kernels in the convolution layer, the convolutional neural network can adapt to text data of different lengths and complexities, can be used in a variety of application scenarios, and also improve the flexibility of text processing.

[0075] In some embodiments, the feature analysis module further includes:

[0076] A semantic feature extraction unit is used to convert the text data to be processed into a text sequence, each word or character in the text sequence represents a time step; the text sequence is input into a preset semantic feature extraction model to extract the time series features and long-distance dependencies in the text sequence, and output a first semantic feature for indicating the semantics between paragraph contents.

[0077] In this embodiment, the semantic feature extraction model can be a recurrent neural network (RNN) or a long short-term memory network (LSTM). A recurrent neural network (RNN) is a neural network specifically used to process sequence data, which can consider previous inputs when processing current inputs, which makes it particularly suitable for processing sequence data such as text. Long short-term memory network (LSTM) is an improved version of RNN, which can more effectively learn long-distance dependencies and avoid the gradient vanishing problem in traditional RNN. Specifically, the text data to be processed is converted into a text sequence that adapts to the input format of the recurrent neural network and the long short-term memory network. Then, a recurrent neural network or a long short-term memory network is added to process the serialized text data to capture the time series features and long-distance dependencies in the text. In this way, when the model is used to process the text sequence, the hidden state is maintained and transmitted, which contains the information of the previous step, ensuring that the model can remember and utilize the long-distance information in the text. Further, the output of the recurrent neural network or the long short-term memory network is connected to one or more fully connected layers, and the required features or classification results are finally output, and the results of the sequence processing are converted into a usable output format.

[0078] Therefore, this embodiment uses a recurrent neural network or a long short-term memory network to build a semantic feature extraction model, which can capture the time series dependency and context information in the text, and is conducive to understanding the relationship between sentences or phrases in the text, such as the semantic relationship at the paragraph or article level, so as to extract features more accurately. At the same time, this embodiment can process complex and long text data and adapt to text inputs of different lengths without performing additional segmentation or padding operations, providing flexibility for processing texts of various lengths.

[0079] In some embodiments, the feature analysis module further includes a semantic understanding unit;

[0080] The semantic understanding unit comprises:

[0081] A semantic feature extraction unit is used to determine a semantic understanding model related to the text analysis task from a pre-trained BERT model, so that the text data to be processed is input into the semantic understanding model to output a semantic understanding feature associated with the text analysis task as a second semantic feature of the text data to be processed.

[0082] In this embodiment, a semantic understanding model is constructed using the BERT model, and the context of the text is understood through a two-way training method, so that richer and deeper semantic information can be captured. Specifically, the text data is standardized, including word segmentation, adding special tags (such as CLS and SEP tags), etc., to obtain an input format suitable for BERT model processing. Then, a suitable model version (such as BERT-Base or BERT-Large) is selected from the pre-trained BERT model library, so that the model trained with a large-scale corpus is used to obtain in-depth semantic understanding capabilities. More specifically, a BERT model is selected based on subsequent specific text operations (such as text analysis tasks and retrieval) to determine a semantic understanding model so that the semantic understanding model can adapt to the characteristics and operational requirements of specific text data. Furthermore, the text is processed using a semantic understanding model to extract deep semantic features and obtain high-quality text representations for subsequent retrieval and text analysis tasks. It should be noted that the semantic feature extraction unit is connected to the retrieval module and the text analysis module to call the semantic understanding model in real time during the retrieval and text analysis process, so as to implement operations such as text retrieval and text classification, thereby improving the accuracy and efficiency of operation processing. This embodiment obtains the second semantic feature that represents the contextual connection through the semantic understanding model, which can more deeply understand the meaning of the text, especially when processing polysemous words and complex sentences. It also demonstrates strong performance in multiple languages ​​and tasks, ensuring its wide applicability.

[0083] Therefore, the present invention extracts a first semantic feature and a second semantic feature, wherein the first semantic feature is to capture the time series dependency and contextual information in the text, and the second semantic feature is to have a stronger contextual understanding capability, so that the first semantic feature and the second semantic feature are used for retrieval and text analysis, thereby improving the comprehensiveness and accuracy of semantic understanding, thereby more accurately understanding the user's query intent, improving the accuracy of retrieval, and improving the accuracy of text analysis.

[0084] In some embodiments, the semantic understanding unit further includes a first semantic disambiguation unit connected to the semantic feature extraction unit:

[0085] A first semantic disambiguation unit, configured to determine polysemous words in the text data to be processed, and determine a text semantic scope of the polysemous words according to contextual content of the polysemous words;

[0086] Calculating the matching degree between each word meaning indicated by the polysemous word and the semantic range of the text, so as to screen out the word meaning with the highest matching degree;

[0087] Based on the polysemous word semantic understanding features of each polysemous word and the screened word meanings, the second semantic features of the polysemous word are determined, wherein the polysemous word semantic understanding features are the semantic understanding features output after the semantic understanding model processes the polysemous word.

[0088] In this embodiment, a polysemous word may have one or more meanings. For this, the semantic disambiguation operation in the first semantic disambiguation unit is used to identify and interpret the correct meaning of the polysemous word in the text, so as to facilitate the interpretation of complex texts. Specifically, the text content of the context around the polysemous word is analyzed to determine its possible text semantic scope, and to provide sufficient context information for the polysemous word for accurate understanding. Then, a word meaning similarity calculation method (such as similarity calculation based on WordNet) is used to compare the degree of matching between different word meanings and the text semantic scope, so as to determine the word meaning with the highest matching degree as the word meaning of the polysemous word at the current position, so as to achieve the contextual meaning and improve the accuracy of polysemous word recognition. Further, a deep learning model (such as calling a semantic understanding model) is used to conduct an in-depth analysis of the polysemous word and its context to determine the semantic understanding features of the polysemous word and improve the effect of disambiguation. Combining the semantic understanding features of the polysemous word with the filtered word meaning, the second semantic features of the polysemous word are determined to ensure that the polysemous word is accurately understood and processed in various texts, which greatly improves the accuracy of polysemous word recognition and improves the accuracy of disambiguation.

[0089] Therefore, through effective semantic disambiguation, the system of this embodiment can more accurately understand and process complex texts, especially those containing rich polysemous words. With the help of the semantic understanding model, it not only relies on the vocabulary matching of the context, but also can deeply understand the real semantics of the vocabulary, thereby improving the overall text understanding ability, and can more effectively process and understand various complex text data, such as scenes involving detailed semantic distinctions.

[0090] In some embodiments, the semantic understanding unit further includes a second semantic disambiguation unit;

[0091] The second semantic disambiguation unit is used to establish a semantic disambiguation model;

[0092] Determining model parameters to be processed in the semantic disambiguation model;

[0093] Randomizing the parameters of the model to be processed and performing a randomized test so as to adjust the semantic disambiguation model based on the test results;

[0094] The polysemous words in the text data to be processed are input into the adjusted semantic disambiguation model to determine the meaning of the polysemous words for the context in which they are located, and the second semantic feature of the polysemous words is output.

[0095] In this embodiment, the correct meaning of polysemous words is identified by constructing a semantic disambiguation model. In the process of constructing the semantic disambiguation model, a model parameter randomization test (MPRT) is performed to evaluate and improve the robustness of the machine learning model. Specifically, by randomly changing the parameters during the model training process, MPRT aims to identify the performance and stability of the model in the face of non-standard input or condition changes. Specifically, a benchmark NLP model for semantic disambiguation, such as a semantic disambiguation model, is established to ensure that there is a stable and effective model as the starting point of MPRT. Then, the model parameters to be randomized in the semantic disambiguation model, such as weights, biases, or layer outputs, are determined. Therefore, by changing the internal characteristics of the model, the sensitivity and stability of the model to parameter changes are detected. Further, the model parameters are randomized during the model training process, and the performance of the model under different model parameter settings is recorded to evaluate the stability and performance retention of the model in the face of parameter fluctuations. Further, the performance of the model on the semantic disambiguation task under different model parameter randomization conditions is analyzed to identify the weaknesses and instability factors of the model and evaluate the reliability of the model in practical applications. Therefore, the model is adjusted and optimized according to the analysis results to enhance its performance and stability under parameter changes and improve the robustness of the model. Therefore, the adjusted semantic disambiguation model is used to process polysemous words, and the meaning of the polysemous word at the current position is output, thereby improving the accuracy of disambiguation.

[0096] Exemplarily, select the model parameters to be randomized, and select the weights and biases of specific layers (such as the first few Transformer layers) in the semantic disambiguation model for randomization. Determine the degree of randomization, for example, multiply a small portion of the weights (such as 5%) by a random factor that varies between 0.9-1.1. Define the frequency of randomization, for example, apply randomization in each training iteration or at the beginning of each epoch, and determine the randomization strategy, for example, whether to randomize the same parameters each time or select different parameters each time. Then, apply parameter randomization during training, for example, if randomization at the beginning of each epoch is selected, then at each new epoch, the selected parameters will be randomly adjusted according to the set rules. Further, record the performance of the model on the validation set after randomizing the parameters, pay special attention to the accuracy of semantic disambiguation, and record the changes in model performance under different randomization strategies. Therefore, the following analysis results can be obtained: compare the performance of the model on the semantic disambiguation task before and after randomization. For example, if the model's performance decreases when identifying the correct meaning of a polysemous word, this may indicate that the model is particularly sensitive to the adjustment of this parameter; analyze the impact of different degrees of randomization on model performance. For example, it is found that the impact of weight randomization on model performance is more significant than bias randomization. Through this example, it can be seen that through a carefully designed parameter randomization test scheme, the robustness of the model on the semantic disambiguation task can be evaluated and improved. In this way, according to the analysis results, it is helpful to identify the sensitive points of the model and indicate the optimization and adjustment operations of the model.

[0097] Therefore, this embodiment helps to improve the performance of the model and can also deeply reveal the internal working mechanism of the model, especially in complex tasks such as semantic disambiguation. It is applicable to a variety of complex semantic environments and improves the disambiguation effect.

[0098] In some embodiments, the retrieval module includes:

[0099] A term search unit, used to create an index for the acquired text data, wherein the index records the association between each word and the text data where the word is located;

[0100] Based on the index, at least one of the text data satisfying the Boolean query condition is determined as a text retrieval result, wherein the Boolean query condition is composed of at least one of the search terms and / or preset query conditions;

[0101] The text search results are similarly matched using a preset vector space model to determine the search term matching results.

[0102] In this embodiment, the user inputs the search terms through the external system, and combines Boolean retrieval and vector space model to achieve efficient keyword search. Specifically, a reverse index is created for the document set (i.e., the text data of this system or the system) to record the association relationship between each word and the document where the word is located, such as which documents each word appears in, so as to accelerate the retrieval process and improve the search efficiency through the index. Then, Boolean query processing is implemented, that is, judging whether the text data meets the Boolean query condition, and the Boolean query condition can be a combination of multiple search terms and query conditions to extract text data that meets the Boolean query condition. Further, the search terms and text data are converted into vector representations, and the vector space model is applied. The model quantifies the similarity between texts by calculating the distance between text vectors (i.e., the distance between the search term vector and each text data vector), so that in the document set that meets the Boolean query condition, the vector space model is used to perform similarity matching on the search terms and text data, further improving the relevance and accuracy of the search term matching results.

[0103] Based on the above embodiment, in some embodiments, the retrieval module further includes:

[0104] A semantic analysis unit, connected to the term search unit and the semantic feature extraction unit, for inputting the search terms and the text data into the semantic understanding model to determine the semantic relationship between the search terms and each of the text data, and outputting corresponding semantic understanding features;

[0105] Calculating the similarity between the search term and the semantic understanding features of each of the text data to determine a semantic matching result;

[0106] Based on the semantic matching result and the search term matching result, a retrieval result matching the search term is determined.

[0107] In this embodiment, the semantic understanding model is called to perform semantic understanding on the search terms and text data to capture the deep semantic relationship between the search terms and the text data. Then, the similarity between the semantic understanding features of the search terms and the semantic understanding features of each text data is calculated to obtain a semantic matching result, so as to accurately match the user's search intent and text content by comparing the deep semantic features. Further, the semantic matching results are combined with the search term matching results to obtain retrieval results that match the search terms, while ensuring the retrieval efficiency, improving the relevance and accuracy of the results. Optionally, based on user feedback on the retrieval results, the implementation steps of the retrieval module, such as search term matching and semantic matching, are optimized, which greatly meets user needs and improves the accuracy of retrieval.

[0108] Therefore, this audit combines Boolean retrieval and vector space model for keyword matching, which can handle complex queries and reflect text similarity. Secondly, through semantic matching, it is not limited to the surface matching of keywords, and deeply understands the real intention of user queries and the actual content of documents, which improves the accuracy of retrieval and meets user needs, ensuring that users can find the most relevant and accurate information in the face of large amounts of data.

[0109] In some embodiments, the text analysis module includes:

[0110] A sentiment analysis unit, connected to the text feature extraction unit, for inputting the text data to be processed and / or the retrieval results into the text feature extraction model to extract sentiment text features related to the sentiment analysis task; and using a preset classification model to classify the sentiment text features into sentiment tendencies to obtain sentiment trends regarding the text data to be processed and / or the retrieval results;

[0111] A topic analysis unit, used for extracting corresponding text topics from the text data to be processed to form a topic overview;

[0112] The summary extraction unit is used to identify and extract key sentences from the text data to be processed to form an original text summary.

[0113] In this embodiment, regarding the sentiment analysis unit, the text data to be processed and / or the search results are preprocessed, such as removing irrelevant characters, part-of-speech tagging, using stem extraction, etc. Then, the text feature extraction unit in the feature analysis module is called to extract text features, and the semantic feature extraction unit is called to extract semantic features between paragraph contents, so as to capture key information related to sentiment analysis. Further, a classification model (such as a support vector machine, a neural network) is used to classify the sentiment tendency (such as positive, negative, neutral) of the text data to be processed and / or the search results to output the sentiment trend of the text. Therefore, this embodiment understands and processes user-generated content, comments, etc. by performing sentiment analysis tasks, and is suitable for fields such as market analysis and public relations strategies.

[0114] Optionally, regarding the topic analysis unit, a topic model (such as an LDA statistical model) can be used to extract topics in the text data to be processed. Specifically, the text data to be processed and / or the search results are preprocessed, such as word segmentation, stop word removal, etc., to provide a suitable input format for the topic model. Then, appropriate parameters (such as the number of topics) are selected, and an LDA model is established and applied to extract the main topics from the document set, so that the topic distribution of each document is analyzed, and the topics are interpreted, thereby forming a topic overview of the document to help understand the content of the document set. Therefore, this embodiment reveals the hidden topics of the text by performing the topic analysis task, which helps to organize and summarize a large number of documents, such as news articles, academic papers, etc.

[0115] Optionally, regarding the summary extraction unit, a suitable summary method is selected according to the application requirements, such as extractive summary (extracting key sentences from the text) or generative summary (generating summary content using a model). For extractive summary, an algorithm (such as a graph-based ranking algorithm) is used to identify and extract key sentences to extract the most important information from the original text. For generative summary, a deep learning model (such as a sequence-to-sequence model) is used to generate a summary based on the original text to generate a more fluent and coherent summary content. Furthermore, the implementation steps of the summary extraction unit can be continuously optimized through user feedback on the original text summary and automatic evaluation (such as BLEU score) to improve the quality and accuracy of the summary. Therefore, this embodiment improves the efficiency of information retrieval and quickly understands a large amount of text content by performing summary extraction tasks.

[0116] For example, taking the rapid retrieval and content analysis of a large number of documents within an enterprise as an example, the enterprise documents are formatted uniformly through the preprocessing module, and then efficiently retrieved through the retrieval module. The text analysis module performs sentiment analysis and abstract extraction on the retrieval results. Taking the precise positioning and recommendation of a large number of teaching resources on the education platform as an example, the educational content is deeply semantically analyzed to achieve personalized recommendations based on students' learning needs. Taking the analysis of hot topics and public sentiment on social media as an example, social media texts are monitored and analyzed in real time to provide insights into emotional tendencies and hot topics.

[0117] In one embodiment, the system also includes an interactive module for obtaining search terms input by users, displaying search results and execution results of text analysis tasks to users, establishing a user feedback mechanism, and continuously optimizing system performance.

[0118] In some embodiments, the subject analysis unit comprises:

[0119] A topic extraction unit, used to obtain training text data containing multiple topics and mark the topic category to which each training text data belongs;

[0120] Establishing a state set and a conversion strategy in the CMPA model, wherein each state in the state set corresponds to a subject category, and the conversion strategy is used to instruct to convert an associated word related to the subject category into a state related to the subject category;

[0121] Using the training text data to train, test and verify the preset CMPA model to obtain a topic extraction model;

[0122] The text data to be processed is input into the topic extraction model to determine the number of occurrences of the associated words related to each state, so that the state with the largest number of occurrences of the associated words is output as the text topic.

[0123] In this embodiment, the CMPA model is a counting message passing automaton, which can perform complex counting and decision-making when processing data. In text analysis, the CMPA model can efficiently process and analyze large-scale text data, and is particularly suitable for performing complex tasks such as topic identification. First, the training text data and the text data to be processed are preprocessed, such as word segmentation, stop word removal, and converted into a format suitable for CMPA model processing to ensure that the text data can be correctly understood and processed by the CMPA model. Then, the CMPA model is defined: first, the state set is determined. Each state in the CMPA model represents a specific stage or feature in the text analysis process. For example, different topics or topic-related features can be defined as different states; second, the conversion strategy between states is defined, and the associated words related to the topic category are converted into the state related to the topic category, that is, the state conversion is triggered based on specific keywords or phrases; third, the counting logic is implemented to track the number of occurrences of specific elements in the text, that is, the number of occurrences of the related words of the counting topic; fourth, the output is defined, and the frequency or counting result of reaching a certain state is regarded as the output of the CMPA model. For example, taking the topic analysis of news articles as an example, the CMPA model is used to perform topic analysis on news articles. The goal is to distinguish whether the article is about politics, economy or sports. The states are set to "politics", "economy" and "sports". For the conversion strategy, if words such as "government" and "election" appear in the article, it is converted to the "politics" state. If words such as "stock market" and "trade" appear, it is converted to the "economy" state. If words such as "Olympic Games" and "football" appear, it is converted to the "sports" state. Therefore, for each state, the number of occurrences of related words is calculated, and the state with the largest number of occurrences of related words is used as the theme of the news article.

[0124] Further, for model training, the CMPA model is trained using training text data, so that the model optimizes its state transition strategy by learning data features. During the training process, the performance of the model, such as accuracy, recall, and other indicators, needs to be monitored to ensure that the model is learning correctly. Then, according to the performance of the model on the training text data, the hyperparameters (such as learning rate, batch size, etc.) are adjusted to achieve better training results and achieve performance evaluation and adjustment. Then, the model performance is tested on an independent validation set to ensure that the model has good generalization ability. Thus, the performance of the model on different topics is analyzed to identify the weaknesses of the model, such as inaccurate or confusing identification of certain topics. And, the model is adjusted according to the test results, which may include modifying the state and transition rules, or further tuning the hyperparameters. Optionally, cross-validation is performed to ensure the robustness of the model, which means repeating the training and evaluation process on different data partitions to verify the consistency of the model. Therefore, the CMPA model can effectively learn and identify topics in text data, providing strong support for subsequent text analysis. This process ensures that the model has good performance and reliability in real scenarios.

[0125] Therefore, the text data to be processed is input into the topic extraction model to efficiently and accurately extract key topics from a large amount of text. Optionally, according to the performance of the CMPA model in practical applications, the model is adjusted and optimized to improve its accuracy and efficiency, and continuously improve the performance and reliability of the model in text topic analysis. At the same time, the CMPA model is suitable for processing large-scale text data sets, and can effectively extract and analyze information from them. Through its complex state and conversion rules, it can identify subtle and complex topics in text data and provide in-depth analysis.

[0126] Exemplary, taking social media content analysis as an example, the main topics in social media content are identified according to the CMPA model. The model can identify different topics such as politics, economy, technology, and entertainment. First, collect social media posts within a certain period of time, such as obtaining public posts from Twitter or Facebook in the past week. The collected social media posts are preprocessed, including cleaning (removing invalid characters, emoticons, etc.), word segmentation, and removal of stop words. The preprocessed post data is input into the CMPA model, and the model analyzes each post according to its defined state and conversion strategy to identify the corresponding topic. Thus, the model outputs the topic classification of each post. For example, one post may be classified as "technology" and another may be classified as "entertainment". In this regard, the topics identified by the model are summarized and counted to understand the distribution and popularity of different topics on social media. In addition, the audience trend and application value are deeply analyzed, such as which topics are more popular in specific groups or regions, or the trend of topics changing over time, and by identifying popular topics on social media, market trends and public interests can be gained. Thus, more effective content strategies and marketing plans can be formulated. This example shows how to apply the CMPA model to actual social media content to achieve effective topic identification and analysis. In this way, valuable information can be quickly extracted from a large number of social media posts to help companies or researchers understand public opinion and trends.

[0127] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0128] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0129] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0130] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0131] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0132] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A natural language processing driven intelligent text retrieval and analysis system, characterized in that: include: A preprocessing module, used for acquiring text data from an external system and preprocessing the text data to obtain text data to be processed; A feature analysis module, connected to the preprocessing module, for extracting data features from the text data to be processed, wherein the data features include text features, first semantic features for characterizing the dependency relationship between paragraph contents, and second semantic features for characterizing the correlation between contexts; A retrieval module, connected to the feature analysis module, for responding to a search term input by the external system and determining a retrieval result matching the search term from the acquired text data based on the data feature; A text analysis module, connected to the feature analysis module and the retrieval module, for performing a text analysis task for the text data to be processed and / or the retrieval results based on the data features, wherein the text analysis task includes a sentiment analysis task, a topic analysis task, and a summary extraction task; Wherein, the text analysis module includes: A topic analysis unit, used for extracting corresponding text topics from the text data to be processed to form a topic overview; The subject analysis unit includes: A topic extraction unit, used to obtain training text data containing multiple topics and mark the topic category to which each training text data belongs; A state set and a conversion strategy in the CMPA model are established, each state in the state set corresponds to a subject category, and the conversion strategy is used to indicate that the associated words related to the subject category are converted into states related to the subject category; the CMPA model tracks the number of occurrences of specific elements in the text to implement counting logic, and the frequency or counting result of reaching a certain state is regarded as the output of the CMPA model; Using the training text data to train, test and verify the preset CMPA model to obtain a topic extraction model; The text data to be processed is input into the topic extraction model to determine the number of occurrences of the associated words related to each state, so that the state with the largest number of occurrences of the associated words is output as the text topic.

2. The natural language processing driven intelligent text retrieval and analysis system according to claim 1, characterized in that: The feature analysis module comprises: A text feature extraction unit, used for converting the text data to be processed into text vector data; inputting the text vector data into a pre-established text feature extraction model to perform feature extraction, and outputting text features containing text keywords; The text feature extraction model includes a cascaded convolution layer, an activation function layer, a pooling layer and a fully connected layer, and a connection structure is provided between the convolution layer and the fully connected layer; The text feature extraction unit comprises: A text feature extraction model processing unit, used for performing feature extraction on the text vector data through the convolution layer to obtain one or more local features; Performing nonlinear processing on the local features through the activation function layer; The output of the activation function layer is pooled by the pooling layer, and then the output of the pooling layer and the local features output by the convolution layer are integrated by the fully connected layer to obtain text features for the text data to be processed.

3. The natural language processing driven intelligent text retrieval and analysis system according to claim 2, characterized in that: The feature analysis module further includes: A semantic feature extraction unit is used to convert the text data to be processed into a text sequence, each word or character in the text sequence represents a time step; the text sequence is input into a preset semantic feature extraction model to extract the time series features and long-distance dependencies in the text sequence, and output a first semantic feature for indicating the semantics between paragraph contents.

4. The natural language processing driven intelligent text retrieval and analysis system according to claim 3, characterized in that: The feature analysis module also includes a semantic understanding unit; The semantic understanding unit comprises: A semantic feature extraction unit is used to determine a semantic understanding model related to the text analysis task from a pre-trained BERT model, so that the text data to be processed is input into the semantic understanding model to output a semantic understanding feature associated with the text analysis task as a second semantic feature of the text data to be processed.

5. The natural language processing driven intelligent text retrieval and analysis system according to claim 4, characterized in that: The semantic understanding unit further includes a first semantic disambiguation unit connected to the semantic feature extraction unit: A first semantic disambiguation unit, configured to determine polysemous words in the text data to be processed, and determine a text semantic scope of the polysemous words according to contextual content of the polysemous words; Calculating the matching degree between each word meaning indicated by the polysemous word and the semantic range of the text, so as to screen out the word meaning with the highest matching degree; Based on the polysemous word semantic understanding features of each polysemous word and the screened word meanings, the second semantic features of the polysemous word are determined, wherein the polysemous word semantic understanding features are the semantic understanding features output after the semantic understanding model processes the polysemous word.

6. The natural language processing driven intelligent text retrieval and analysis system according to claim 4, characterized in that: The semantic understanding unit also includes a second semantic disambiguation unit; The second semantic disambiguation unit is used to establish a semantic disambiguation model; Determining model parameters to be processed in the semantic disambiguation model; Randomizing the parameters of the model to be processed and performing a randomized test so as to adjust the semantic disambiguation model based on the test results; The polysemous words in the text data to be processed are input into the adjusted semantic disambiguation model to determine the meaning of the polysemous words for the context in which they are located, and the second semantic feature of the polysemous words is output.

7. The natural language processing driven intelligent text retrieval and analysis system according to claim 6, characterized in that: The retrieval module comprises: A term search unit, used to create an index for the acquired text data, wherein the index records the association between each word and the text data where the word is located; Based on the index, at least one of the text data satisfying the Boolean query condition is determined as a text retrieval result, wherein the Boolean query condition is composed of at least one of the search terms and / or preset query conditions; The text search results are similarly matched using a preset vector space model to determine the search term matching results.

8. The natural language processing driven intelligent text retrieval and analysis system according to claim 7, characterized in that: The retrieval module further includes: A semantic analysis unit, connected to the term search unit and the semantic feature extraction unit, for inputting the search terms and the text data into the semantic understanding model to determine the semantic relationship between the search terms and each of the text data, and outputting corresponding semantic understanding features; Calculating the similarity between the search term and the semantic understanding features of each of the text data to determine a semantic matching result; Based on the semantic matching result and the search term matching result, a retrieval result matching the search term is determined.

9. The natural language processing driven intelligent text retrieval and analysis system according to claim 1 or 8, characterized in that: The text analysis module further includes: A sentiment analysis unit, connected to the text feature extraction unit, for inputting the text data to be processed and / or the retrieval results into the text feature extraction model to extract sentiment text features related to the sentiment analysis task; and using a preset classification model to classify the sentiment text features into sentiment tendencies to obtain sentiment trends regarding the text data to be processed and / or the retrieval results; The summary extraction unit is used to identify and extract key sentences from the text data to be processed to form an original text summary.

Images