An intelligent traffic text analysis method based on natural language processing

Abstract

The present disclosure relates to a natural language processing-based intelligent traffic text analysis method, belonging to the technical field of intelligent traffic, comprising the following steps: text preprocessing, self-defined named entity recognition, topic analysis and clustering, and word embedding and citation analysis; wherein, the target text is segmented, stop words are removed, and text sentence labels are marked; a named entity recognition model is trained, and the named entity recognition model is used to specify information in advance for the summary and title of the target text to filter out the required articles; the topics of the target text are analyzed and clustered, new data sets are created according to the topics, and model evaluation is carried out by using coherence; wherein, word embedding is used to find at least one word as a context from each topic cluster, and the at least one word is used as a keyword to be checked, and when the keyword is set, the keyword is used to find words in each topic cluster that have the same context as the keyword.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent transportation technology, and more specifically, to an intelligent transportation text analysis method based on natural language processing. Background Technology

[0002] As an application branch of digital twins, intelligent transportation (ITS) evaluation systems play a crucial role in improving ITS systems. ITS text evaluation employs various methods. One is traditional manual evaluation, which involves manually analyzing numerous ITS evaluation articles covering concepts, technologies (applications and core sciences), policies, management and strategies, methodological development, and other topics. System developers manually read these articles and extract evaluation viewpoints. Another method is traditional machine learning evaluation, which uses traditional text processing methods and classification models to analyze the sentiment of the articles, or employs unsupervised methods such as clustering to analyze unlabeled articles and identify research articles with similar viewpoints. Traditional machine learning evaluation methods mainly include word frequency-based machine learning classifier models, clustering-based evaluation text analysis methods, and DNN-based text analysis models.

[0003] For evaluation methods of "machine learning classifier models based on word frequency models", there are several specific types: First, the evaluation method can be a classification model based on simple word frequency, that is, after word segmentation, for each word w in document D... i Calculate its frequency c in all documents. i As the word frequency value, this evaluation method allows any text to be represented as a vector D = (c1, c2, c3, ..., c) in the dictionary space. n Here, n represents the dimension of the dictionary space. This vector and the final text classification label are used as training samples to train the classification model, which is typically a Bayesian classifier. Secondly, this evaluation can be based on a TF-IDF-based classification model. Simply using word frequency to represent text can lead to feature bias because certain high-frequency words may appear extensively in various texts, causing weight bias in certain dimensions of the text vector and severely affecting the final classification result. Therefore, TF-IDF is proposed to represent text. The TF-IDF calculation method is as follows:

[0004]

[0005]

[0006] After calculating TF and IDF separately, the TF-IDF can be obtained as follows:

[0007] TF-IDF = TF*IDF

[0008] Thus, this evaluation method can largely solve the problem of weight bias in certain dimensions of text vectors caused by high-frequency words. High word frequency within a specific document, and low document frequency of that word across the entire document set, can produce a high-weighted TF-IDF. Therefore, TF-IDF can filter out common high-frequency words and retain important words.

[0009] Furthermore, this evaluation method obtains the TF-IDF value w for each word. i Then, for document D = (w1, w2, ..., w n ), where n is the dimension of the dictionary space, and then the document D is used as a training sample to input into a machine learning classifier, such as Bayesian, SVM, XGBOOST, LightGBM, etc., to train and obtain a classification model.

[0010] For evaluation methods based on clustering in text analysis, "clustering" means comparing the similarity of several texts and grouping those with high similarity into one category. After modeling a large amount of text, it's necessary to cluster the topic distribution to obtain more accurate and concise topics; therefore, clustering is crucial in topic detection technology. Clustering is an unsupervised learning method that aims to divide a dataset into multiple sub-data sets according to certain rules; each sub-data set is called a cluster. Cluster analysis has wide applications in text analysis, business applications, web search, recommendation systems, biomedicine, and many other fields. Due to the different data application scenarios, different clustering methods have different focuses, advantages, and disadvantages; therefore, there is currently no universal clustering algorithm. Currently, clustering algorithms are mainly classified into the following categories: partition-based clustering algorithms (K-means, Single-Pass, PAM, etc.), hierarchical clustering algorithms (Chameleon algorithm, AGNES, CURE, etc.), density-based clustering algorithms (DBSCAN, OPTICS, and DENCLUE, etc.), grid-based clustering algorithms (STING algorithm, CLIQUE algorithm, etc.), model-based clustering algorithms (GMM, SOM), and fuzzy clustering algorithms.

[0011] For evaluation methods of "DNN-based text analysis models", typical types include RNN, TextCNN and LSTM. Among them, LSTM can also be considered a type of RNN, which can learn the sequence relationship between text words in a multi-layer network structure.

[0012] It should be noted that the network model structure of text-CNN can be shown in the attached figure. Figure 2As shown; the RNN network model structure can be as follows: Figure 3 As shown, RNNs have proven to be very successful in NLP in practice, for example, in word vector representation, sentence validity checking, and part-of-speech tagging. Among the many RNN models, the most widely used and successful model is currently LSTM (Long Short-Term Memory) model. This model can generally represent long and short-term dependencies better than vanilla RNNs. Compared with general RNNs, this model only makes some optimizations in the hidden layers.

[0013] However, at present, the above methods have at least the following problems:

[0014] First, word frequency or TF-IDF methods cannot characterize the contextual relationships and word order features between words in a sentence. Therefore, the evaluation method of "machine learning classifier model based on word frequency model" essentially treats each word as an independent feature for classification, resulting in poor classification performance when the number of samples is insufficient.

[0015] Secondly, in the classification stage, if traditional machine learning classification models (SVM, Bayes, XGBoost, and LightGBM, etc.) are used, relying on statistical methods of word frequency or decision tree classification methods, it will lead to a significant deficiency in semantic understanding.

[0016] Third, in terms of clustering methods, traditional clustering methods such as k-means still use traditional machine learning methods. If mathematical models are used to simply train the model, it will result in poor performance in terms of generalization and semantics.

[0017] Fourth, the evaluation methods for "DNN-based text analysis models" have at least the following shortcomings:

[0018] 1. This evaluation method requires high-performance hardware computing resources and has a long training cycle, which is not conducive to fast processing.

[0019] 2. This evaluation method does not provide good support for long texts, and usually only supports texts of up to 1000 characters, making it not very practical.

[0020] 3. The model complexity of this evaluation method is high, which may lead to overfitting.

[0021] 4. If only classification or clustering models are used, it is impossible to analyze the evaluation articles in detail. It can only give a confidence value between 0 and 1 or one of the classification labels in terms of sentiment. It cannot obtain information such as the theme, abstract, core entities and the relationships between entities in the article. Summary of the Invention

[0022] The purpose of this disclosure is to provide an intelligent transportation text analysis method based on natural language processing. In order to efficiently analyze professional articles on intelligent transportation evaluation in the context of current cloud storage and big data, this intelligent transportation text analysis method based on natural language processing can solve at least one of the above-mentioned technical problems.

[0023] To achieve the above objectives, according to a first aspect of this disclosure, this disclosure provides an intelligent traffic text analysis method based on natural language processing, comprising the following steps:

[0024] Text preprocessing: Tokenization, stop word removal, and sentence tagging of the target text; if the target text is in English, part-of-speech tagging is performed.

[0025] The word segmentation is the process of dividing a text sentence into words; the stop word removal is the process of removing frequently used words that are insufficient to represent the text sentence from the vocabulary set after word segmentation; the text sentence tagging is the process of tagging each text sentence to give it a unique identifier.

[0026] Custom Named Entity Recognition: Train a named entity recognition model and use this model to pre-specify information for the summary and title of the target text in order to filter out the required articles;

[0027] Topic analysis and clustering: By analyzing and clustering the topics of the target text, new datasets are created according to the topics, and the model is evaluated using coherence.

[0028] Word embedding and citation analysis: Word embedding is used to find at least one word as context in each topic cluster, and at least one word is used as the keyword to be checked. After the keyword is set, the keyword is used to find words in each topic cluster that have the same context as the keyword. The number of citations is used to determine the text most frequently cited by other researchers.

[0029] As an optional technical solution, the text preprocessing further includes:

[0030] Remove special characters and numbers from the text.

[0031] Tokenize the target text, which means converting each character or word in the target text into a corresponding dictionary ID.

[0032] As an optional technical solution, the text preprocessing further includes:

[0033] Error correction steps: For spelling errors in the target text, error correction is performed using methods based on language perplexity and / or corpus dictionaries.

[0034] As an optional technical solution, the error correction step includes: training a language error correction model by combining a language perplexity model and a corpus dictionary to detect erroneous words;

[0035] The target text is replaced with similar-sounding or similar-looking characters, words, or phrases from the corpus dictionary. The replaced text sentences are then used as a candidate set for error correction. The language perplexity of the replaced text sentences is calculated using a language perplexity model. If the language perplexity of the replaced text sentences is less than that of the original text sentences, then the replaced text sentences from the error correction candidate set are used to replace the corresponding original text sentences.

[0036] As an optional technical solution, the named entity recognition model uses a pre-trained model + conditional random field method to fine-tune the training of downstream tasks. The training data is labeled using the BIO labeling method, which requires setting "ITS-POS" to label the text subsequences related to the intelligent transportation system, and simultaneously labeling organizational structure elements and date elements.

[0037] As an optional technical solution, the topic analysis and clustering includes the step of automatically finding the optimal hyperparameter values.

[0038] As an optional technical solution, the step of automatically finding the optimal hyperparameter values ​​adopts a Bayesian hyperparameter optimization method based on Gaussian process regression to find the optimal hyperparameters.

[0039] As an optional technical solution, the topic analysis and clustering includes the step of automatically finding the optimal hyperparameter values:

[0040] The initial number of topics is set to k = 10, thus establishing a consistent baseline with a consistency score of 0.41. Hyperparameters are then tuned to determine the hyperparameters k, α, and β. The default values ​​of α and β are 1, and the range of k is [2, 11]. The hyperparameter values ​​are determined by grid search or heuristic optimization algorithms to achieve the highest consistency score for the potential Dirichlet assignment model. The optimal hyperparameter values ​​are k = 10, α = 0.3, and β = 0.9, with a consistency score of 0.45.

[0041] Create another baseline with initial values ​​of k=10, α=0.3, and β=0.9 to create a visual graph showing the specific value of hyperparameter k and the number of iterations when the model achieves the highest consistency score.

[0042] The latent Dirichlet assignment model produces phi and θ values;

[0043] Where phi represents the distribution of words within the topic, and θ represents the distribution of topics within a text.

[0044] As an optional technical solution, when the potential Dirichlet allocation model achieves the highest consistency score, the specific values ​​of the hyperparameter k are k=6, the specific values ​​of the hyperparameter α are α=0.3, the specific values ​​of the hyperparameter β are β=0.9, and the number of iterations is 2000.

[0045] As an optional technical solution, the word embedding adopts a skip-word model and internal evaluation.

[0046] Through the above technical solution, on the one hand, compared with existing related technologies, the technical solution of this disclosure uses self-annotated intelligent transportation system (ITS) related text data to establish a named entity recognition model, and uses the established named entity recognition model to filter related target texts. Compared with the whole text binary classification or keyword filtering methods in related technologies, the named entity recognition model used in this disclosure can perform filtering at a smaller information granularity. Therefore, the intelligent transportation text analysis method of this disclosure can more accurately and meticulously filter out articles related to intelligent transportation systems from massive target texts, and also has better generalization, that is, it can filter target texts at the semantic level. On the other hand, the intelligent transportation text analysis method of this disclosure can also achieve a certain degree of matching for some unknown text series, thereby filtering out more valuable evaluation texts. Furthermore, compared with traditional text deep models, in the method of this disclosure, the combination of various algorithm processes can not only overcome the problem of not being able to review long texts, but also support data with a length of millions or more characters, and can also shorten the processing time. The processing time of the analysis method of this disclosure can be within seconds.

[0047] Furthermore, this disclosure employs topic analysis and clustering methods, which offer superior performance for analyzing massive amounts of related articles. It can quickly establish "groups" of related articles and extract core vocabulary, providing excellent support for keyword retrieval based on word embedding.

[0048] According to a second aspect of this disclosure, this disclosure provides an intelligent traffic text analysis device based on natural language processing, the device comprising: a processor and a communication interface; the communication interface and the processor are coupled, the processor being configured to run computer programs or instructions to implement the intelligent traffic text analysis method based on natural language processing as described in the first aspect and any possible implementation thereof.

[0049] According to a third aspect of this disclosure, a computer-readable storage medium is provided that stores instructions which, when executed on a terminal, cause the terminal to perform the intelligent traffic text analysis method based on natural language processing as described in the first aspect and any possible implementation thereof.

[0050] According to a fourth aspect of this disclosure, embodiments of this disclosure provide a computer program product containing instructions that, when run on a natural language processing-based intelligent traffic text analysis device, cause the natural language processing-based intelligent traffic text analysis device to perform the natural language processing-based intelligent traffic text analysis method as described in the first aspect and any possible implementation thereof.

[0051] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a schematic diagram illustrating the steps of an intelligent traffic text analysis method based on natural language processing provided in an exemplary embodiment of this disclosure;

[0054] Figure 2 This is a schematic diagram of the text-CNN network model structure;

[0055] Figure 3 This is a schematic diagram of the RNN network model structure;

[0056] Figure 4 This is a schematic diagram illustrating the steps of an intelligent traffic text analysis method based on natural language processing provided in an exemplary embodiment of this disclosure;

[0057] Figure 5 In the topic analysis and clustering process, which includes the step of automatically finding the optimal hyperparameter values, when creating another baseline, the initial values ​​are set to k=10, α=0.3, and β=0.9, creating a visual graphical representation.

[0058] Figure 6 This is a schematic diagram of a hyperparameter optimization process provided by an exemplary embodiment of this disclosure. Detailed Implementation

[0059] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0060] like Figures 1 to 6 As shown, according to a first aspect of this disclosure, this disclosure provides an intelligent traffic text analysis method based on natural language processing, comprising the following steps:

[0061] Text preprocessing: Tokenization, stop word removal, and sentence tagging of the target text; if the target text is in English, part-of-speech tagging is performed.

[0062] Word segmentation is the process of dividing a text sentence into words; stop word removal is the process of removing frequently used words that are insufficient to represent the text sentence from the vocabulary set after word segmentation; text sentence tagging is the process of tagging each text sentence to give it a unique identifier.

[0063] Custom Named Entity Recognition: Train a named entity recognition model and use this model to pre-specify information for the summary and title of the target text in order to filter out the required articles;

[0064] Topic analysis and clustering: By analyzing and clustering the topics of the target text, new datasets are created according to the topics, and the model is evaluated using coherence.

[0065] Word embedding and citation analysis: Word embedding is used to find at least one word as context in each topic cluster and to use at least one word as the keyword to be examined. After the keyword is set, the keyword is used to find words in each topic cluster that have the same context as the keyword. The number of citations is used to determine the text most frequently cited by other researchers.

[0066] In this disclosure, word segmentation and stop word removal of the target text are performed to remove noise data, thereby improving the effectiveness and efficiency of text classification. For example, noise words that have no effect on text classification, such as "today," "tomorrow," "morning," "of," "this," and "that," can be removed by the text preprocessing step. If the target text is in English, part-of-speech tagging is performed, which means removing affixes from words and extracting the core parts of the words. Generally, the tagged words will correspond to a word in a corpus dictionary or thesaurus.

[0067] In the topic analysis and clustering steps of this disclosure, creating new datasets according to topics allows for the evaluation of changes in topic clusters over time, i.e., using consistency scores for model evaluation. Topic consistency reflects the degree of semantic matching of high-scoring tokens within each topic cluster.

[0068] In the word embedding and citation analysis steps, word embedding and citation analysis can minimize the workload of manual reading. Word embedding can be used to find specific words as context within each topic cluster, using certain words as keywords to be examined. Once keywords are set, these words are used to find words in each topic cluster that have the same context as the keywords.

[0069] Word embedding can employ a Skip-Gram approach because this disclosure uses multiple keywords to extract words or context from similar topic clusters. For English articles, a problem arises because the context of words may be phrases, often consisting of two or more words. Before preprocessing the text into the model, a bigram dictionary needs to be built for each topic cluster. For word embedding, this patent can also employ Intrinsic Evaluation, which evaluates the quality of word vectors on an intermediate task. This method is computationally fast and helps us better understand the system.

[0070] Thus, through the above technical solution, on the one hand, compared with existing related technologies, the technical solution of this disclosure uses self-annotated intelligent transportation system (ITS) related text data to establish a named entity recognition model, and uses the established named entity recognition model to filter related target texts. Compared with the whole text binary classification or keyword filtering methods in related technologies, the named entity recognition model used in this disclosure can perform filtering at a smaller information granularity. Therefore, the intelligent transportation text analysis method of this disclosure can more accurately and meticulously filter out articles related to intelligent transportation systems from massive target texts, and also has better generalization, that is, it can filter target texts at the semantic level. On the other hand, the intelligent transportation text analysis method of this disclosure can also achieve a certain degree of matching for some unknown text series, thereby filtering out more valuable evaluation texts. Furthermore, compared with traditional text deep models, in the method of this disclosure, the combination of various algorithm processes can not only overcome the problem of not being able to review long texts, but also support data with a length of millions or more characters, and can also shorten the processing time. The processing time of the analysis method of this disclosure can be within seconds.

[0071] Furthermore, this disclosure employs topic analysis and clustering methods, which offer superior performance for analyzing massive amounts of related articles. It can quickly establish "groups" of related articles and extract core vocabulary, providing excellent support for keyword retrieval based on word embedding.

[0072] This patent uses citation counts to determine the documents most frequently cited by other researchers. Broadly speaking, after completing modeling and pre-training the Skip-Gram, we search for context using keywords as a reference (guided and within the scope of the research question). Subsequently, we search for articles containing that context. Articles considered influential or with the greatest impact are those with a considerable number of citations.

[0073] As an optional technical solution, text preprocessing also includes:

[0074] Remove special characters and numbers from the text.

[0075] Tokenize the target text, which means converting each character or word in the target text into a corresponding dictionary ID.

[0076] As an optional technical solution, text preprocessing also includes:

[0077] Error correction steps: For spelling errors in the target text, error correction is performed using methods based on language perplexity and / or corpus dictionaries.

[0078] Error correction steps may include: training a language error correction model by combining a language perplexity model and a corpus dictionary to detect erroneous words;

[0079] The target text is replaced with similar-sounding or similar-looking characters, words, or phrases from the corpus dictionary. The replaced text sentences are then used as a candidate set for error correction. The language perplexity of the replaced text sentences is calculated using a language perplexity model. If the language perplexity of the replaced text sentences is less than that of the original text sentences, then the replaced text sentences from the error correction candidate set are used to replace the corresponding original text sentences.

[0080] Thus, compared to existing steady preprocessing steps, the technical solution of this disclosure introduces a spelling correction mechanism based on a language perplexity model and a dictionary / lexicon, which can greatly improve the text quality of the target text, thereby creating a strong optimization effect on the subsequent processing results and improving the quality and efficiency of the intelligent traffic text analysis of this disclosure.

[0081] As an optional technical solution, the Named Entity Recognition model uses a pre-trained model + Conditional Random Field method to fine-tune the training of downstream tasks. The BIO labeling method is used to label the training data for sequence labeling. In this method, "ITS-POS" needs to be set to label the text subsequences related to the intelligent transportation system. At the same time, organizational structure elements and date elements are also labeled.

[0082] Using the named entity recognition model disclosed herein, if the text contains "ITS-POS", it can be considered as text related to Intelligent Transportation System (ITS) and can be used as the object for further processing.

[0083] It should be noted that, after training and evaluation of the named entity recognition model of this disclosure, the output results show that its F1 score (F1-score, the harmonic mean of precision and recall) reaches 88%, precision exceeds 90%, and recall exceeds 85%. Thus, the accuracy and precision of the named entity recognition model of this disclosure are sufficient to meet the requirements of text analysis in the field of intelligent transportation.

[0084] In this disclosure, the pre-trained model can be a BERT model (Bidirectional Encoder Representation from Transformers, hereinafter referred to as BERT) or preferably an autoregressive pre-trained model (XLNET model). The Conditional Random Field (CRF) disclosed herein is a discriminative probability model, a type of random field, commonly used for labeling or analyzing sequence data, such as natural language text or biological sequences. In principle, the graphical model layout of a Conditional Random Field can be arbitrarily given. A commonly used layout is a linked architecture. Efficient algorithms exist for calculating linked architectures in training, inference, and decoding.

[0085] In this disclosure, the BIO (B-begin, I-inside, O-outside) notation method refers to the three-dimensional BIO notation method (that is, B represents the beginning of an entity, I represents the middle or end of an entity, and O represents not belonging to that entity).

[0086] As an optional technical solution, topic analysis and clustering includes the step of automating the search for optimal hyperparameter values.

[0087] As an optional technical solution, the step of automating the search for optimal hyperparameter values ​​adopts a Bayesian hyperparameter optimization method based on Gaussian process regression to find the optimal hyperparameters.

[0088] Thus, for hyperparameter optimization in topic model training, this disclosure can provide an automated hyperparameter optimization method based on Gaussian process regression that can obtain the global optimal solution. During model iterative training, the hyperparameters converge at an extremely fast speed, thereby selecting the best combination of hyperparameters.

[0089] As an optional technical solution, topic analysis and clustering includes the step of automating the search for optimal hyperparameter values:

[0090] The initial number of topics is set to k = 10, thus establishing a consistent baseline with a consistency score of 0.41. Hyperparameters are then tuned to determine the hyperparameters k, α, and β. The default values ​​of α and β are 1, and the range of k is [2, 11]. The hyperparameter values ​​are determined by grid search or heuristic optimization algorithms to achieve the highest consistency score for the potential Dirichlet assignment model. The optimal hyperparameter values ​​are k = 10, α = 0.3, and β = 0.9, with a consistency score of 0.45.

[0091] Create another baseline with initial values ​​of k=10, α=0.3, and β=0.9 to create a visual graph showing the specific value of hyperparameter k and the number of iterations when the model achieves the highest consistency score.

[0092] The latent Dirichlet assignment model produces phi and θ values;

[0093] Where phi represents the distribution of words within the topic, and θ represents the distribution of topics within a text.

[0094] Thus, using θ values ​​can help identify the dominant topic in the research texts within each topic cluster. This method simplifies the systematic evaluation process, leading to better improvement of ITS knowledge. Furthermore, since the Latent Dirichlet Allocation Model is unsupervised learning, it can be directly trained using massive amounts of target text data without requiring extensive manual data labeling. This effectively improves the efficiency of model building and enhances the efficiency of the intelligent transportation text analysis method disclosed in this paper.

[0095] In one specific example of this disclosure, as shown in Table 1 below,

[0096]

[0097]

[0098] Table 1. Experimental results of the topic model

[0099] As can be seen from Table 1, some thematic groups have completely different research subjects, such as Theme 1 and Theme 3. Although these documents share the same research theme, they differ in their research subjects, thus providing different perspectives for ITS evaluation.

[0100] The heuristic optimization algorithms disclosed herein include various implementations, such as particle swarm optimization (PSO), simulated annealing, genetic algorithms, and ant colony optimization (ACO). Ant colony optimization (ACO) is a probabilistic algorithm used to find optimal paths. It features distributed computation, positive feedback, and heuristic search, and is essentially a heuristic global optimization algorithm within evolutionary algorithms. Particle swarm optimization (PSO), also known as particle swarm optimization algorithm, has the following advantages: it does not depend on problem information, uses real numbers for solving, and has strong versatility; its principle is simple, it is easy to implement, and requires few parameters to be adjusted; it converges quickly and has low memory requirements; and its leapfrog nature makes it easier to find the global optimum rather than getting trapped in local optima. Simulated annealing (SAA) is an optimization algorithm that avoids getting trapped in local minima and eventually converges to the global optimum by giving the search process a time-varying probability of sudden jumps that eventually reach zero. Genetic algorithms, on the other hand, are methods that search for optimal solutions by simulating natural evolution. These algorithms use mathematical methods and computer simulations to transform the problem-solving process into processes similar to the crossover and mutation of chromosomes and genes in biological evolution. When solving complex combinatorial optimization problems, they often achieve better optimization results faster than some conventional optimization algorithms.

[0101] Furthermore, the Latent Dirichlet Allocation (LDA) model disclosed herein is also an abbreviation for Linear Discriminant Analysis. The Latent Dirichlet Allocation model is a topic extraction model, an unsupervised algorithm, whose function is to extract the topics of a document and the words within those topics.

[0102] In one specific embodiment of this disclosure, such as Figure 5 As shown, Figure 5There are multiple k values ​​that maximize the consistency score. Choosing a small k value leads to overgeneralization of the dataset. Conversely, choosing a large k value limits the number of topic words in each topic cluster, potentially affecting subsequent word embeddings. Furthermore, an inappropriate k value can cause topic clusters to become redundant. Model decisions are based on coherence and whether topics contain redundant information or include research articles with some similarity. Therefore, in this disclosure, the optimal hyperparameter values ​​are k = 6, iterations = 2000, α = 0.3, and β = 0.9.

[0103] As an optional technical solution, word embedding adopts a skip-word model and internal evaluation.

[0104] In this disclosure, Named Entity Recognition (NER), also known as proper name recognition, is a fundamental task in natural language processing with a wide range of applications. Named entities generally refer to entities in text that have specific meaning or strong referentiality, typically including names of people, places, organizations, dates and times, proper nouns, etc. The NER model extracts these entities from unstructured input text (e.g., the target text of this disclosure) and can identify more categories of entities according to business needs, such as product names, models, and prices.

[0105] Furthermore, it is understood that the term "entity" in this disclosure should be interpreted broadly, meaning that any specific text fragment required for business purposes can be called an entity.

[0106] The following is a specific embodiment to explain the intelligent traffic text analysis method based on natural language processing disclosed herein. For example, Python code is used to write the code, and the Django Rest Framework is used to encapsulate the text classification service as a RESTful API. This allows users to remotely submit the article object to be analyzed using the HTTP protocol's POST or GET methods, and the execution results are returned. This method can minimize the coupling between users and services, which is more in line with the "high cohesion, low coupling" principle proposed in software engineering. Furthermore, the service can be deployed in Docker to form an image, and multiple services can be started in a cluster environment. For these multiple services, Nginx is used to load balance user requests, thereby significantly improving the performance of user API calls.

[0107] Write Python code to train the model on the training data, obtaining the language model and the XLNET+CRF NER model, and save the model files. Then, use the Django Rest Framework to write code to encapsulate the NLP service, load the model, and start an API service. Package the NLP service and its environment into a Docker Image. During deployment, start the Docker Image on multiple machines or virtual machines to obtain containers, which are the NLP services. This starts multiple NLP services. In the forwarding layer, use the same technique to start several forwarding services to handle the URL request queue and perform asynchronous processing. The scheduling layer schedules the user's URL requests so that the requests can be evenly distributed to various services to improve execution efficiency. After the service layer finishes execution, it encapsulates the result in JSON format and returns it to the forwarding layer. The forwarding layer then encapsulates the execution result again and returns it to the user.

[0108] To improve service speed, model acceleration technology is used for the XLNET+CRF deep model. The PyTorch model is converted to ONNX, then to TensorRT Engine files, and finally deployed using Triton Server.

[0109] In addition, to automate the collection and processing of article data, an automated incremental crawler system can be developed to periodically crawl documents from the target website and store them in Elasticsearch (ES). The NLP service can then automatically read and process the articles from ES.

[0110] According to a second aspect of this disclosure, this disclosure provides an intelligent traffic text analysis device based on natural language processing, the device comprising: a processor and a communication interface; the communication interface and the processor are coupled, the processor being configured to run computer programs or instructions to implement the intelligent traffic text analysis method based on natural language processing as described in the first aspect and any possible implementation thereof.

[0111] According to a third aspect of this disclosure, a computer-readable storage medium is provided that stores instructions which, when executed on a terminal, cause the terminal to perform the intelligent traffic text analysis method based on natural language processing as described in the first aspect and any possible implementation thereof.

[0112] According to a fourth aspect of this disclosure, embodiments of this disclosure provide a computer program product containing instructions that, when run on a natural language processing-based intelligent traffic text analysis device, cause the natural language processing-based intelligent traffic text analysis device to perform the natural language processing-based intelligent traffic text analysis method as described in the first aspect and any possible implementation thereof.

[0113] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0114] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0115] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for intelligent traffic text analysis based on natural language processing, characterized in that, Includes the following steps: Text preprocessing: Tokenization, stop word removal, and sentence tagging of the target text; if the target text is in English, part-of-speech tagging is performed. The word segmentation is the process of dividing a text sentence into words; the stop word removal is the process of removing frequently used words that are insufficient to represent the text sentence from the vocabulary set after word segmentation; the text sentence tagging is the process of tagging each text sentence to give it a unique identifier. Custom Named Entity Recognition: Train a named entity recognition model and use this model to pre-specify information for the summary and title of the target text in order to filter out the required articles; Topic Analysis and Clustering: By analyzing and clustering the topics of the target text, a new dataset is created according to the topics, and coherence is used for model evaluation. The topic analysis and clustering steps include an automated process for finding the optimal hyperparameter values. This automated process employs a Bayesian hyperparameter optimization method based on Gaussian process regression to find the optimal hyperparameters. Specifically: The initial number of topics is set to k=10 to establish a consistent baseline with a consistency score of 0.

41. Hyperparameters are then tuned to determine the hyperparameters k, α, and β. The default values ​​of α and β are 1, and the range of k is [2, 11]. The hyperparameter values ​​are determined by grid search or heuristic optimization algorithms to achieve the highest consistency score for the potential Dirichlet assignment model. The optimal hyperparameter values ​​are k=10, α=0.3, and β=0.9, with a consistency score of 0.

45. Create another baseline with initial values ​​of k=10, α=0.3, and β=0.9 to create a visualization graph showing the specific value of hyperparameter k and the number of iterations when the model achieves the highest consistency score. The latent Dirichlet assignment model produces phi and θ values; Where phi represents the distribution of words within the topic, and θ represents the distribution of topics within a text; Word embedding and citation analysis: Word embedding is used to find at least one word as context in each topic cluster, and at least one word is used as the keyword to be checked. After the keyword is set, the keyword is used to find words in each topic cluster that have the same context as the keyword. The number of citations is used to determine the text most frequently cited by other researchers.

2. The intelligent traffic text analysis method based on natural language processing according to claim 1, characterized in that, The text preprocessing also includes: Remove special characters and numbers from the text. Tokenize the target text, which means converting each character or word in the target text into a corresponding dictionary ID.

3. The intelligent traffic text analysis method based on natural language processing according to claim 1, characterized in that, The text preprocessing also includes: Error correction steps: For spelling errors in the target text, error correction is performed using methods based on language perplexity and / or corpus dictionaries.

4. The intelligent traffic text analysis method based on natural language processing according to claim 3, characterized in that, The error correction steps include: training a language error correction model by combining a language perplexity model and a corpus dictionary to detect erroneous words; The target text is replaced with similar-sounding or similar-looking characters, words, or phrases from the corpus dictionary. The replaced text sentences are then used as a candidate set for error correction. The language perplexity of the replaced text sentences is calculated using a language perplexity model. If the language perplexity of the replaced text sentences is less than that of the original text sentences, then the replaced text sentences from the error correction candidate set are used to replace the corresponding original text sentences.

5. The intelligent traffic text analysis method based on natural language processing according to claim 1, characterized in that, The named entity recognition model uses a pre-trained model + conditional random field method to fine-tune the downstream task. The training data is labeled using the BIO labeling method. Specifically, "ITS-POS" needs to be set to label the text subsequences related to intelligent transportation systems. At the same time, organizational structure elements and date elements are also labeled.

6. The intelligent traffic text analysis method based on natural language processing according to claim 1, characterized in that, When the Latent Dirichlet Allocation Model achieves the highest consistency score, the corresponding hyperparameter k is k=6, the hyperparameter α is α=0.3, the hyperparameter β is β=0.9, and the number of iterations is 2000.

7. The intelligent traffic text analysis method based on natural language processing according to any one of claims 1-5, characterized in that, The word embedding adopts a skip-word model and internal evaluation.

Images