127 results about "Topic model" patented technology

The invention provides an AI auxiliary reading method and system based on dynamic partitioning knowledge classification self-verification, and relates to the technical field of artificial intelligence, the method comprises the following steps: carrying out dynamic semantic partitioning on a super-long text to obtain a plurality of initial text units with consistent internal semantics; performing topic model construction according to the initial text unit and a layered Dirichlet process to obtain a topic distribution tag; receiving a user query request, and extracting a retrieval text set according to the hierarchical retrieval strategy and a retrieval text corresponding to the user query request; inputting the constraint cue word extracted by the topic distribution tag corresponding to the retrieval text set, the retrieval text set and the retrieval text into a text generation model to output a to-be-calibrated text; if the error scoring function of the to-be-calibrated text and any initial text unit in the retrieval text set is smaller than a preset error threshold value, the to-be-calibrated text serves as a credible text to be output to a user. According to the invention, the credibility and accuracy of the answer can be enhanced.

The invention relates to an LDA topic model recognition method which comprises the following steps: S1, collecting high-frequency search keywords of archives by utilizing a Python library, preprocessing the collected data, including removing useless punctuations and special symbols, only extracting Chinese and performing text word segmentation, and meanwhile, deepening to the Internet or a specific database by adopting a crawler technology, so as to obtain an LDA topic model; according to the method, richer and more comprehensive archive data are collected, a natural language processing (NLP) technology is introduced, deeper semantic understanding and information extraction are performed, non-key vocabularies are filtered by adopting part-of-speech tagging, and the data quality and the analysis precision are improved. Based on an LDA topic model, various sampling algorithms are adopted to allocate categories for training data, after convergence is achieved, the score of each category of to-be-classified books is counted, the category of to-be-classified archives is obtained accordingly, and aiming at the defects that a traditional archive format is not standard, naming is not standard, and archive sharing access is difficult in organizations at all levels, the classification efficiency is improved. The improvement is obvious.

Computer-implemented methods for a large language model privacy preservation system. Aspects include receiving prompt data from a user device. Aspects further include generating pre-processed prompt data using the prompt data from the user device. Aspects also include identifying a category for the pre-processed prompt data using topic modeling. Aspects include generating normalized prompt data using the pre-processed prompt data. Aspects further include storing the category and the normalized prompt data.

The invention discloses an earth and rockfill dam illness feature mining method and system based on historical text data, and the method comprises the steps: collecting earth and rockfill dam historical illness data, constructing an earth and rockfill dam illness diagnosis text corpus set, carrying out the structured preprocessing of the corpus set, and obtaining an illness feature and danger removal measure text corpus set; generating a structured word sequence subset through a word segmentation tool; processing the structured word sequence subset by adopting an LDA topic model, determining an optimal topic number through a confusion degree curve, and outputting a final topic and a corresponding topic word; constructing a visual network graph based on the subject term co-occurrence frequency; and carrying out centrality analysis on nodes in the visual network diagram, identifying key nodes in the visual network diagram, quantitatively analyzing association rules of the danger characteristics and danger removing measures, and completing feature mining of the earth and rockfill dam danger. The problems that traditional manual diagnosis is high in subjectivity and low in utilization rate of historical engineering data are solved, and intelligent auxiliary decision making of the earth and rockfill dam danger is achieved.

Systems, apparatuses, methods, and computer program products are disclosed for automated prototyping of a topic model. An example method includes a data manipulation engine ingesting and pre-processing source data from a set of data sources, a feature extraction engine that thereafter transforms the pre-processed data into a set of numeric representations of the pre-processed data, and an autonomous model generator that automatically generates a trained topic model using the set of numeric representations. Embodiments further enable visualization of topic model output, which permits a user to easily consume and utilize information from a topic model for any number of purposes.

The invention provides a cross-platform theme decoupling Chinese aggressive language detection method based on a causal relationship, and belongs to the field of natural language processing in an artificial intelligence technology.The method comprises the steps that firstly, a causal graph is constructed, and themes related to a platform and aggressive languages unrelated to the platform are decoupled; constructing a Chinese aggressive language detection model based on the causal graph; training the Chinese aggressive language detection model by using the Chinese aggressive language data set, performing topic modeling on the Chinese aggressive language data set by using a probability topic model, and constructing a loss function for optimizing the Chinese aggressive language detection model by combining results of the two; and detecting a to-be-detected text by using the Chinese aggressive language detection model combined with the loss function. According to the method, the causal relationship is introduced to effectively decouple the Chinese aggressive language data, so that more accurate Chinese aggressive language detection is realized in a cross-platform environment.

The invention provides an archive association method based on theme association relationship analysis, and relates to the technical field of archive management businesses, and the method comprises the following steps: S1, data preparation and preprocessing: collecting archive data needing to be analyzed, preprocessing the archive data through a natural language processing technology, and then carrying out deep analysis to obtain a processed archive; s2, constructing a model: modeling potential topics in the processed archive set by applying an LDA algorithm, and constructing a layered LDA model; and S3, archive association: calculating the similarity between the processed archives based on the topic distribution output by the hierarchical LDA model. Through comprehensive application of advanced technologies such as natural language processing, topic model and graph theory analysis, a brand-new and efficient archive management and retrieval mode is realized, so that the efficiency and accuracy of archive management are remarkably improved, and richer and deeper possibilities are provided for archive utilization and information mining.

Certain aspects of the disclosure provide a method for providing user support by generating recommended response for a customer verbatim with an ensemble of machine learning models. The method includes processing a customer verbatim with a topic model trained to identify a topic associated with the customer verbatim. The method further includes processing the customer verbatim with a sentiment model trained to determine a sentiment of the customer verbatim. The method further includes processing the customer verbatim with an actionability model trained to assign an actionability score to the customer verbatim. The method includes processing the topic, the sentiment, and the actionability score with a recommendation model to generate the recommended response to the customer verbatim.

A computer-implemented method executed using a computing device comprises digitally generating and storing a machine learning statistical topic model in computer memory, the topic model being programmed to model call transcript data representing words spoken on a call as a function of one or more topics of a set of topics, the set of topics being modeled to comprise a set of pre-seeded topics and a set of non-pre-seeded topics, and the one or more topics being modeled as a function of a probability distribution of topics; programmatically pre-seeding the topic model with a set of keyword groups, each keyword group associating a respective set of keywords with a topic of the set of pre-seeded topics; programmatically training the topic model using unlabeled training data; conjoining a classifier to the topic model to create a classifier model, the classifier defining a joint probability distribution over topic vectors and one or more observed labels; programmatically training the classifier model using labeled training data; receiving target call transcript data comprising an electronic digital representation of a verbal transcription of a target call; programmatically determining, using the classifier model, at least one of one or more topics of the target call or one or more classifications of the target call; digitally storing the target call transcript data with additional data indicating the determined one or more topics of the target call and / or the determined one or more classifications of the target call; accessing, in computer storage, a first digitally stored electronic document comprising a first text; receiving computer input specifying a search query comprising one or more search terms; processing the search query using the classifier model to output a query topic vector representing a thematic content of the search query; processing the first text using the classifier model to output and store in the computer memory a first plurality of topic vectors each representing a topic in the text; using the query topic vector and the first plurality of topic vectors, calculating a plurality of similarity values, each of the similarity values representing a similarity of the query topic vector to a particular topic vector among the first plurality of topic vectors; outputting a visual display that specifies one or more topic vectors among the first plurality of topic vectors having one or more corresponding similarity values that are greater than a specified threshold similarity value.

The invention provides a low-altitude economy teaching text classification method and system based on a topic model, and the method comprises the steps: firstly obtaining a low-altitude economy teaching text set, constructing a low-altitude economy teaching term association network, enabling nodes to be teaching terms, and enabling edges to represent the term co-occurrence association strength, then, based on the low-altitude economy teaching term association network, mining an initial low-altitude economy teaching subject set which comprises a plurality of subject units formed by close co-occurrence terms, then, performing semantic vector representation on text units, optimizing the initial subject set, and based on the optimized subject set, constructing a low-altitude economy teaching text subject classification model; the method comprises the following steps of: inputting a to-be-classified text into a topic feature library and a topic classification rule set, finally inputting the to-be-classified text into a low-altitude economy teaching text topic classification model, and generating a topic classification result through topic feature matching and classification decision, so that the accuracy and efficiency of low-altitude economy teaching text classification are improved.

The invention relates to the technical field of information retrieval, in particular to a recall method and device of a retrieval enhancement system. The method comprises the following steps: S1, storing knowledge points; s2, carrying out recall enhancement; s3, checking the consistency of recall results; and S4, summarizing the model. The device comprises a knowledge point storage module, a recall enhancement module, a recall result consistency checking module and a model summarization module. According to the method, the features of the knowledge points are extracted and analyzed through the topic model and the clustering model, and the accuracy and availability of the knowledge points are improved. And the ES recall result is converted into the vector, so that the text information is quantized and represented, and the recall accuracy is improved. And the accuracy and the reliability of the recall result are ensured by performing consistency check on the recall result.

The present disclosure relates to a system, a method, and a product for topic discovery. The system includes a memory storing instructions; and a processor in communication with the memory. When the processor executes the instructions, the instructions are configured to cause the processor to: obtain text data, conduct pre-processing on the text data to obtain pre-processed text data, extract an entity list and a keyword list based on the pre-processed text data, generate an entity embedding list based on the entity list, clusterize the entity list based on the entity embedding list to obtain a plurality of entity clusters, each entity cluster comprising at least one entity, retrieve a co-occurring keyword list based on the plurality of entity clusters, the entity list, and the keyword list, and obtain a topic for each entity cluster of the plurality of entity clusters based on the co-occurring keyword list.

The invention provides a public text analysis method and system based on an LDA model and an LSTM algorithm. The method comprises the steps of obtaining target text data of a science and technology project; extracting feature words in the target text data, and constructing a feature word vector matrix according to the feature words; constructing a confusion degree curve based on the feature word vector matrix, and constructing a topic model based on the confusion degree curve; based on the topic model, determining the topic attention of the target topic, and based on the topic attention, determining an attention trend feature which is used for indicating the change trend of the target topic in the time dimension, the space dimension and the topic dimension; analyzing the target text data by using a pre-established sentiment analysis model to obtain a sentiment analysis result; and generating a text analysis result of the science and technology project based on the target topic, the attention trend feature and the sentiment analysis result. According to the method and the device, intelligent analysis of the public text is realized, and the text interpretation efficiency and accuracy are improved.

The invention relates to the field of technical opportunity recognition, in particular to a technical opportunity recognition method of a double-layer GTM map based on papers and patents. According to the recognition method, a BERTopic topic model, a double-layer generation type topological mapping map and link prediction are mainly combined to carry out recognition on technical opportunities, and calculation and analysis are carried out based on the BERTopic topic model to obtain the trend of a technical topic; processing and calculating the intersection of the important keywords of the thesis and the patent based on GTM to obtain a thesis map and a patent map, and analyzing vacuum points and non-vacuum points in the two maps to identify technical vacuum and technical opportunities; and a prediction similarity index is constructed by combining cosine similarity calculation of keyword vectors and a link prediction index, and selection and interpretation of a target technical opportunity are realized. Compared with an existing identification method, the identification method has the advantages that intervention of field experts is removed, information interpretation of the patent vacuum point is more objective and reasonable, and meanwhile a more comprehensive result can be obtained.

The invention discloses a traffic transportation government affair hotline mining method based on natural language processing, and belongs to the technical field of artificial intelligence and machine learning. According to the method, for structured and unstructured data in a traffic transportation government affair hotline, firstly, a five-class word segmentation dictionary containing cleaning words, noise words, synonyms, additive words and stop words is constructed; converting the unstructured text into structured data through improved data cleaning, text word segmentation and feature representation; carrying out clustering analysis by utilizing an LDA topic model, and extracting public demand topics and high-frequency keywords; hot appeals and trend changes are mined in combination with space-time analysis and association analysis, and finally a visual analysis report is generated. According to the method, the problems of poor model interpretability and insufficient field adaptability in the prior art are solved, the accuracy of hotline data processing and the effectiveness of theme recognition are remarkably improved through multi-dictionary collaborative optimization and field knowledge fusion, and accurate decision support is provided for a traffic transportation management department. A real taxi field case in a certain city is used as an example for research, an experiment proves that the method has an accurate theme identification function, and the complaint and report work order amount in the traffic transportation government affair hotline taxi field in the city is reduced by 20% on year-on-year basis in 2024.

A method includes obtaining at least one feedback response from a developer regarding an answer and corresponding source documents of the answer in a discussion thread initiated by the developer. The method further includes converting the discussion thread into a topic model clustering input, responsive to the feedback response specifying an unsatisfactory category of feedback responses, to obtain a multitude of topic clustering model inputs. The method further includes periodically processing the multitude of topic clustering model inputs by a thread-topic clustering model to obtain a multitude of candidate topics. The method further includes processing, by an answer generation model, a first candidate topic of the multitude of candidate topics to obtain a multitude of corresponding documentation recommendations for the candidate topic. The method further includes presenting the first candidate topic and the multitude of corresponding documentation recommendations.

Learning frameworks for processing text transcripts may include receiving, by an application, a query comprising a topic. A topic model may determine a plurality of subtopics based on the topic. The application may receive, from a database based on the topic and plurality of subtopics, a plurality of text transcripts. A sentiment model may compute, for each text transcript, a respective sentiment score based on a text of the respective text transcript. The application may determine, for each text transcript, a duration of a communication session associated with the respective text transcript. The application may compute, for each text transcript, a total score based on the sentiment score and the duration of the respective text transcript. The application may return, as responsive to the query, a subset of the plurality of text transcripts having a total score that exceeds a threshold.

The invention discloses an equipment software test case recommendation method and device based on semantic similarity. The method comprises the steps of special noun dictionary construction and keyword extraction, demand and case semantic similarity calculation and software test case reuse. According to the method, the historical equipment software quality data serves as knowledge, the intelligent and automatic reuse capability of the historical equipment software test case is formed by means of artificial intelligence algorithms such as a topic model and a word vector model, the reuse efficiency of the equipment software test case is enhanced, and the sufficiency and effectiveness of new project software test case design are improved.

The embodiment of the application discloses a kind of theme model updating method, system and storage medium and server, apply to the information processing technical field based on artificial intelligence.Theme model system will obtain the first label semantic feature and the second label semantic feature corresponding respectively to multiple old theme labels in the first theme model and multiple new theme models in the second theme model, and based on the first label semantic feature and the second label semantic label, mapping relationship is established between old theme label and new theme label, and then the old theme label in the first theme model is updated based on the mapping relationship.The updating of the first theme model existing in the system is realized automatically, the efficiency of the theme model is improved, and the theme model with larger dimensionality can also be updated, and the updating of the first theme model is not limited by the theme model acquisition method.

The invention relates to the technical field of style recognition, and discloses a text style recognition method. The method comprises the steps of obtaining a to-be-recognized text, and determining a text publishing platform corresponding to the to-be-recognized text; performing topic recognition on the to-be-recognized text through an LDA topic model corresponding to the text publishing platform to obtain at least one text topic and topic keywords corresponding to the text topics; performing topic clustering on all the text topics and the topic keywords corresponding to the text topics through a visual clustering tool to obtain a text topic cluster; sending the text theme cluster to the client, and receiving an optimal theme quantity fed back by the client; and performing style mapping on the target theme in the optimal theme quantity to obtain a text style. According to the method, through the LDA topic model and the style mapping rule, the topic and the style in the text are recognized, the accuracy of style recognition is improved, and the efficiency of style recognition is improved.

This invention discloses a method for spatial gene identification and extraction based on social media text data, comprising the following steps: collecting online text data about a city, then preprocessing the data to obtain dataset D1; constructing a dictionary and vector space in analysis software, introducing an LDA topic model, and classifying the obtained dataset D1 into topics; merging synonyms in each topic, and performing synonym replacement in dataset D1 to obtain dataset D2; counting the co-occurrence frequency of keywords in dataset D2 and constructing a co-occurrence matrix M; and using a hierarchical clustering model to cluster the semantic network analysis results to obtain spatial combination patterns, i.e., spatial genes. This invention collects online text data about a specific city from multiple social media platforms, providing a practical technical means for urban researchers to identify urban spatial genes by obtaining rich, non-invasive data.

The present invention relates to the field of financial technology, and discloses a method, device, equipment and computer-readable storage medium for extracting a topic. The method for extracting a topic includes: obtaining search log information and each search statement; determining a session in the search log information according to each search statement, wherein each search statement in the session is semantically associated; determining the topic distribution of the session through a topic model, and determining the uniform resource locator corresponding to the session according to the topic distribution; and extracting a target topic from each topic corresponding to the session according to the uniform resource locator corresponding to the session. The present invention improves the accuracy of topic extraction.

The invention relates to a text sentiment analysis method and a text sentiment analysis system based on natural language processing, and further relates to the technical field of natural language processing and classification algorithms, and the specific implementation scheme is as follows: firstly obtaining data extraction, then distinguishing segmented words described by a specific character in water upload through data preprocessing, removing stop words, tagging parts of speech and the like. And finally, based on sentiment analysis and positive, negative and neutral character sentiment classification, obtaining a sentiment type corresponding to the sentiment analysis, so that subjective and objective sentiment types of a sentiment analysis text can be recognized, and each kind of water-uploaded character can analyze a detailed sentiment type by using a corresponding topic model. The accuracy and pertinence of text sentiment analysis are effectively improved, so that the classification accuracy of texts in public opinion products is improved.

The embodiment of the invention provides a topic model generation method and device, a storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: preprocessing an obtained user comment text and comment metadata, and correspondingly obtaining a word frequency vector and a text label; inputting the word frequency vector into a preset topic model, converting the word frequency vector into potential space representation, and performing normalization processing to obtain topic distribution corresponding to a user comment text output by the topic model; inputting the user comment text into a preset attention mechanism model to obtain a text embedding vector corresponding to the user comment text; and based on the topic distribution, the text embedding vector and the text label, carrying out joint training, and determining a trained target model. The attention mechanism and the comment metadata are combined in the modeling process, the trained target model is determined, and the problems that in the prior art, a topic model is insufficient in real scene data processing capacity and low in efficiency are solved.

The invention discloses a long text abstract generation method based on hierarchical graph comparison themes, which comprises the following steps of: 1, preprocessing an original document, dividing sentence sequences, and obtaining global context-aware sentence and document representation through a hierarchical encoder network; 2, deducing document-level and sentence-level topic distribution by using a neural topic model; and 3, constructing a supervision graph based on a standard abstract to perform graph comparison learning so as to close the topic representation of a document and a key sentence and push redundant information. According to the method, the deep semantic structure of the long document can be effectively captured, so that the theme consistency and the information coverage degree of the abstract can be improved, and the redundancy is reduced.

A method for technical issue resolution. The method includes: receiving, from a user, a text query concerning a technical issue; obtaining query-related context relevant to the text query; and processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. More specifically, embodiments described herein utilize text topic and zero shot classification models to translate multimodal technical documentation (e.g., including text and images) into topic relevant metadata; and process queries, pertaining to technical issues, using a multimodal LLM provided with query-related text and image context derived from said topic relevant metadata.

A method for identifying and ranking potentially privileged documents using a machine learning topic model may include receiving a set of documents. The method may also include, for each of two or more documents in the set of documents, extracting a set of spans from the document, generating, using a machine learning topic model, a set of topics and a subset of legal topics for the set of spans, generating a vector of probabilities for each span with a probability being assigned to each topic in the set of topics for the span, assigning a score to one or more spans in the set of spans by summing the probabilities in the vector that are assigned to a topic in the subset of legal topics, and assigning a score to the document. The method may further include ranking the two or more documents by their assigned scores.

The invention discloses a text sentiment analysis method of an LDA topic model based on a BERT model and a K-means thought. The text sentiment analysis method comprises the following steps: (1) preprocessing a text data set; (2) obtaining topic distribution and word distribution of each text by using an LDA topic model, extracting a topic word with the highest weight, and splicing the topic word with the original text to generate a new data set; (3) in combination with a K-means clustering thought, iteratively extracting subject terms and repeatedly splicing the subject terms with the original text by applying the LDA subject model to the new data set again until subject distribution converges; (4) inputting the text of the final spliced subject term into a BERT model to generate a context-aware word vector; (5) outputting a text emotion category through a SoftMax classifier; according to the method, the conformity between the text and the subject term is improved, and the sentiment polarity of the text is enhanced.

Certain aspects of the disclosure provide a method for providing user support by generating recommended response for a customer verbatim with an ensemble of machine learning models. The method includes processing a customer verbatim with a topic model trained to identify a topic associated with the customer verbatim. The method further includes processing the customer verbatim with a sentiment model trained to determine a sentiment of the customer verbatim. The method further includes processing the customer verbatim with an actionability model trained to assign an actionability score to the customer verbatim. The method includes processing the topic, the sentiment, and the actionability score with a recommendation model to generate the recommended response to the customer verbatim.

This invention discloses a short text topic modeling method based on dynamic clustering and word embedding enhancement. First, short text data is collected to obtain a short text stream. Then, the short text stream is clustered using the FastStream clustering method, and pseudo-documents are constructed based on the clustering results. Next, a word embedding matrix and a word co-occurrence matrix are formed using a word embedding model pre-trained on a large corpus. Subsequently, the topic distribution of the pseudo-documents and its word distribution are modeled using Dirichlet distribution to obtain a topic model. Then, the topic model is trained on the pseudo-documents using the Gibbs sampling method, updating the topic assignment of each word in the pseudo-documents and the topic distribution parameters of the pseudo-documents until the topic model converges. Finally, new short texts are acquired for topic inference to obtain the topic distribution. This invention utilizes the combination of FastStream clustering and word embedding techniques to effectively improve the topic modeling performance of short text data by creating pseudo-document views and enhancing word embeddings.