A natural language analysis processing platform

Abstract

This invention provides a natural language processing (NLP) platform, comprising a data pool module, a topic type selection module, and a functional component module. The data pool module interfaces with third-party systems to acquire natural language data. The topic type selection module selects topic types. The functional component module selects natural language analysis models. The interactive platform of the NLP platform allows for free combination of components, enabling the flexible arrangement of the data pool module, topic type selection module, and functional component module, and providing a clear view of data flow and analysis results. This platform integrates multiple natural language analysis models, improving the efficiency of NLP analysis and enhancing hardware computing power.

Description

Technical Field

[0001] This invention relates to the fields of data mining, machine learning, artificial intelligence, and multilingual text analysis technology applications, specifically to a natural language analysis and processing platform. Background Technology

[0002] With the rapid development of internet information, there is a need to process the various languages ​​contained within it. Natural Language Processing (NLP) is a field of study focusing on the interaction between computers and human (natural) language, encompassing computer science, artificial intelligence, and linguistics. It is an important direction within computer science and artificial intelligence. NLP research covers various theories and methods that enable effective communication between humans and computers using natural language, including areas such as natural semantic understanding, retrieval, information extraction, machine translation, and automatic question-answering systems.

[0003] Traditional machine learning algorithms, such as SVM and LR, analyze and process text features mapped to high-dimensional space. When the training sample size is large, their performance is poor and underfitting may occur.

[0004] Therefore, it is necessary to provide a new natural language processing platform that improves hardware computing power. Summary of the Invention

[0005] Technical problems to be solved

[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a natural language analysis and processing platform that integrates multiple natural language analysis models into the entire platform, thereby improving the efficiency of natural language analysis and enhancing hardware computing power.

[0007] Technical solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] This invention provides a natural language processing (NLP) platform, comprising a data pool module, a topic type selection module, and a functional component module. The data pool module is used to interface with third-party systems to acquire natural language data; the topic type selection module is used to select topic types; and the functional component module is used to select natural language analysis models. The interactive platform of the NLP platform allows for free combination of components, enabling the free combination of the data pool module, the topic type selection module, and the functional component module, and intuitively displaying data flow and analysis results.

[0010] Furthermore, the functional component module integrates multiple natural language analysis models, including keyword extraction model, long text summarization model, sentiment classification model, text classification model, named entity recognition model, and hot word discovery model.

[0011] Furthermore, all of the aforementioned natural language analysis models use BERT as a pre-trained model and are fine-tuned on a proprietary dataset.

[0012] Furthermore, all of the aforementioned natural language analysis models are deployed using Docker container technology and provide REST interfaces.

[0013] Furthermore, the data pool module generates a language detection training model by collecting various types of training data from different languages ​​using a multinomial-based Naive Bayes classifier. Then, it calculates the posterior probability value of the input natural sentence text information in each language category using the generated language detection training model, and selects the language category with the largest posterior probability value as the language type of the input natural sentence text information.

[0014] Furthermore, using a multinomial-based Naive Bayes classifier, various types of data in different languages ​​are collected as training data. The training data is labeled with different languages ​​to prepare for model training. For a document D containing n features, the probability that document D belongs to a certain category Ci in the closed set C is calculated, where i refers to the number of language categories and i>0. Finally, the posterior probability of document D for each category is obtained, and the one with the highest probability is selected as the language category of the target document D.

[0015] Furthermore, the natural language analysis and processing platform adopts front-end and back-end separation technology and B / S architecture, uses Redis as a high-speed data caching unit, uses Elastic Search non-relational database to store raw natural language data, and uses NVIDIA graphics cards to train the multiple natural language analysis models and accelerate inference.

[0016] Beneficial effects

[0017] This invention designs a natural language analysis and processing platform. The platform adopts artificial intelligence technology and deep learning methods, using BERT as a pre-trained model to train models such as keyword extraction model, long text summarization model, sentiment classification model, text classification model, named entity recognition model, and hot word discovery model, and integrates them into the entire platform to improve the efficiency of natural language analysis and enhance hardware computing power. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the interaction of a natural language analysis and processing platform provided in an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of the natural language analysis and processing platform provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] See Figure 1 This invention provides a natural language processing (NLP) platform, including a data pool module, a topic type selection module, and a functional component module. The data pool module interfaces with third-party systems to acquire natural language data. The topic type selection module selects topic types. The functional component module selects natural language analysis models. The interactive platform of the NLP platform allows for free combination of components, enabling the free combination of the data pool module, topic type selection module, and functional component module, and intuitively displaying data flow and analysis results. This platform integrates multiple natural language analysis models into the entire platform, improving the efficiency of natural language analysis and enhancing hardware computing power.

[0023] For specific implementation, please refer to Figure 1 After a user logs into the platform, all functional component information of the aforementioned functional component module is automatically retrieved and automatically categorized into a navigation list. This navigation list includes the data pool module, the topic type selection module, and all functional components of the aforementioned functional component module. After importing natural language data, users can perform operations such as dragging, connecting, moving, and deleting components. The data pool, topics, and functional components can be freely combined, providing a clear visual representation of data flow and analysis results.

[0024] In this embodiment, the functional component module integrates multiple natural language processing (NLP) models, including a keyword extraction model, a long text summarization model, a sentiment classification model, a text classification model, a named entity recognition model, and a hot word discovery model. All of these NLP models use BERT as a pre-trained model and are fine-tuned on a proprietary dataset. All of these NLP models are deployed using Docker container technology and provide REST interfaces. The BERT model is a language representation model proposed by Google in the second half of 2018 in the paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." BERT stands for Bidirectional Encoder Representation from Transformers, and structurally, it is a multi-layered bidirectional transformer encoder.

[0025] In this embodiment, the keyword extraction model, long text summarization model, sentiment classification model, text classification model, named entity recognition model, and hot word discovery model are trained using BERT, the best performing model in the industry, as the pre-trained model. Fine-tuning training is performed on a proprietary dataset. Furthermore, for the sentiment classification problem, the text feature vectors extracted by the original BERT model are applied, followed by a suitable classifier model, and finally the natural language is divided into three categories: positive, negative, and neutral.

[0026] In this embodiment, the model deployment application uses Docker container technology to deploy the trained model, which effectively solves the cross-platform compatibility problem, allows for the simultaneous deployment of multiple models, integrates servers, reduces costs, and enables rapid automated deployment. TensorFlow / serving is used as the base image, and the container provides an API via the HTTP protocol after startup. The inference process is accelerated using GPU.

[0027] In this embodiment, refer to Figure 2Step 1: Data was crawled using web scraping technology, cleaned, and persisted locally. Data was manually labeled for different model tasks; for example, in sentiment classification, each entry was labeled as positive, negative, or neutral using 0, 1, and 2. The total data volume was 100,000 entries. Step 2: PyCharm was used as the integrated development environment, Python as the programming language, and TensorFlow as the AI ​​framework to train the data, iteratively generating the optimal power-bench (PB) model. Step 3: The model files were deployed to Docker containers and then to inference systems with NVIDIA GPUs, providing REST interfaces. Step 4: The front-end interface was developed using Vue and Element UI technologies, the back-end service interface was developed using the Spring Boot framework, and Nginx was used as the web container to provide reverse proxying.

[0028] In this embodiment, a language detection training model is generated by collecting training data of various types in different languages ​​using a multinomial-based Naive Bayes classifier. The generated language detection training model is then used to calculate the posterior probability value of an input natural sentence text in each language category, and the language category with the highest posterior probability value is selected as the language type of the input natural sentence text. Language detection means that the input is a natural sentence text, and the output result is the determination of the language type to which the natural sentence belongs.

[0029] Specifically, the process is as follows: Using a multinomial-based Naive Bayes classifier, various types of data in different languages ​​are collected as training data. The training data is then labeled with language classifications to prepare for model training. For a document D containing n features (i.e., input natural language text information), the probability that document D belongs to a certain category Ci in a closed set C is calculated (i refers to the number of language categories and i>0, for example: C1 is Chinese, C2 is English, C3 is French, etc., where i=1, 2, or 3), and this probability is assigned to the most probable category. Through this calculation, a language detection model is trained. Given the parameter values ​​of the above model, the posterior probability of the target document D for each category can be calculated, and the one with the highest probability is selected as the language category of the target document D.

[0030] In this embodiment, the present invention can employ a TensorFlow-based convolutional neural network (CNN) and bidirectional GRU parallel architecture for model construction, including:

[0031] Embedding layer: This layer uses embedding technology to transform the input sentence into a dense spatial vector. The related technologies are similar to those described in the word2vec section and will not be repeated here.

[0032] Convolutional Layer: The first step in a convolutional neural network (CNN), used for feature extraction. In a convolutional layer, a neuron is connected to only a subset of neurons in neighboring layers. A convolutional layer in a CNN typically contains several feature maps, each composed of neurons arranged in a rectangular pattern. Neurons within the same feature map share weights, which are called the convolutional kernel. The convolutional kernel is usually initialized as a random fractional matrix, and during network training, it learns appropriate weights. The direct benefit of shared weights (convolutional kernel) is reduced connections between network layers, while also lowering the risk of overfitting.

[0033] Pooling layer: The second step in a convolutional neural network, used for feature extraction. Pooling, also known as subsampling, typically takes two forms: mean pooling and max pooling. Subsampling can be viewed as a special type of convolution. Convolution and subsampling greatly simplify model complexity and reduce the number of model parameters. Bidirectional GRU: GRU (Gated Recurrent Units) is a recurrent neural network based on a "gating" mechanism.

[0034] Bidirectional refers to the use of forward and backward neural networks, generally referring to two different sequence directions of the input sequence. (Human thinking doesn't always start from a blank slate. For example, when reading, we infer the meaning of the current word based on our existing understanding of previously seen words, not discarding everything and starting with a blank mind. Traditional neural networks cannot achieve this; this is a limitation of traditional neural networks. For example, suppose you want to classify the time type of each point in a movie. Traditional neural networks would struggle to handle this problem—inferring subsequent events from previous events in the movie. RNNs (Recurrent Neural Networks) solve this problem. RNNs are networks that contain recurrences, allowing information persistence. One of the key points of RNNs is that they can be used to connect previous information to the current task, such as using past video segments to infer the understanding of the current segment. For example, a language model uses previous words to predict the next word. If you try to predict "the clouds are in the...",...) The last word "sky" doesn't require any further context—therefore, the next word should obviously be "sky." In such scenarios, the gap between relevant information and the predicted word position is very small, and the RNN can learn to use previous information. However, as the gap between relevant information and the predicted word position increases, the RNN loses its ability to learn to connect information that is so far apart, which is a drawback of RNNs. This application uses a bidirectional GRU to extract sentence features, which enables the RNN to learn to connect information that is so far apart even as the gap between relevant information and the predicted word position increases.

[0035] The advantage of this invention lies in the design of a natural language analysis and processing platform. This platform uses artificial intelligence technology and deep learning methods, with BERT as a pre-trained model, to train models such as keyword extraction, long text summarization, sentiment classification, text classification, named entity recognition, and hot word discovery. These models are then integrated into the entire platform, improving the efficiency of natural language analysis and enhancing hardware computing power.

[0036] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A natural language processing platform, characterized in that, The system comprises a data pool module, a topic type selection module, and a functional component module. The data pool module interfaces with third-party systems to acquire natural language data. The topic type selection module selects topic types. The functional component module selects a natural language analysis model. The interactive platform of the natural language analysis and processing platform allows for free combination of components, enabling the data pool module, topic type selection module, and functional component module to be freely combined, and intuitively displaying data flow and analysis results. The natural language analysis model includes a model constructed in parallel using TensorFlow's Convolutional Neural Network (CNN) and Bidirectional GRU. The parallel-constructed model includes an embedding layer, convolutional layers, pooling layers, and a bidirectional GRU. The embedding layer uses embedding technology to convert the input sentence into a dense spatial vector. In the convolutional layers, each neuron connects only to a subset of neighboring neurons for feature extraction. The pooling layer uses mean subsampling or maximum subsampling for feature extraction and reducing model parameters. The bidirectional GRU uses both forward and backward neural networks to extract sentence features.

2. The natural language analysis and processing platform according to claim 1, characterized in that, The functional component module integrates multiple natural language analysis models, including keyword extraction model, long text summarization model, sentiment classification model, text classification model, named entity recognition model, and hot word discovery model.

3. The natural language analysis and processing platform according to claim 2, characterized in that, All of the aforementioned natural language analysis models use BERT as a pre-trained model and are fine-tuned on a proprietary dataset.

4. The natural language analysis and processing platform according to claim 3, characterized in that, All of the natural language analysis models are deployed using Docker container technology and provide REST interfaces.

5. The natural language analysis and processing platform according to claim 1, characterized in that, The data pool module generates a language detection training model by collecting various types of training data from different languages ​​using a multinomial-based Naive Bayes classifier. The generated language detection training model then calculates the posterior probability value of the input natural sentence text information for each language category and selects the language category with the highest posterior probability value as the language type of the input natural sentence text information.

6. The natural language analysis and processing platform according to claim 5, characterized in that, Using a Naive Bayes classifier based on multinomials, we collect various types of data in different languages ​​as training data, label the training data by language, and begin preparing to train the model. For a document D containing n features, we calculate the probability that the document D belongs to a certain category Ci in the closed set C, where i refers to the number of language categories and i>0. Finally, we obtain the posterior probability of document D for each category, and select the one with the highest probability as the language category of the target document D.

7. The natural language analysis and processing platform according to claim 1, characterized in that, The natural language analysis and processing platform adopts front-end and back-end separation technology and B / S architecture, uses Redis as a high-speed data caching unit, uses Elastic Search non-relational database to store raw natural language data, and uses NVIDIA graphics cards to train the multiple natural language analysis models and accelerate inference.

Images