70 results about "Topic analysis" patented technology

The invention discloses an LLM capability enhancement method of a hybrid enhancement strategy, and particularly relates to the technical field of natural language processing. The method comprises the following steps: obtaining corpus data and external literature data of a target application field, carrying out denoising, word segmentation and semantic indexing processing, and establishing a searchable heterogeneous knowledge base; secondly, classifying the corpus data by adopting a topic analysis and word vector clustering method to form a basic corpus set; performing multi-round parameter updating on the large language model by using a domain fine tuning algorithm based on the basic corpus set, and setting a fusion threshold value between a new parameter and an original parameter; and retrieving related literature fragments from the heterogeneous knowledge base through a matching algorithm, screening content conforming to field correlation by utilizing semantic similarity evaluation, fusing the content with the fine-tuned model, processing a fusion result by adopting a context fusion algorithm and an entity linking method, completing syntactic analysis, and generating a precise solution oriented to the target application field.

The invention belongs to the field of data processing, and discloses a social media comprehensive public opinion analysis method and related device.The method comprises the steps that feature vectors of all cross-modal public opinion data are obtained through a pre-trained cross-modal large model, and public opinion events are clustered through an unsupervised clustering method according to the feature vectors; utilizing a pre-training sentiment analysis model and a pre-training topic analysis model to respectively obtain sentiment scores and topic sensitivity scores of the public opinion events in the public opinion event clusters; according to the emotion score and the topic sensitivity score of each public opinion event in each public opinion event cluster, respectively obtaining an emotion score index score and a topic sensitivity index score of each public opinion event cluster, and obtaining a comprehensive public opinion analysis score of each public opinion event cluster in combination with the exposure index score, the interactivity index score and the propagation intensity index score; and obtaining a public opinion evaluation result of each public opinion event cluster according to the comprehensive public opinion analysis score of each public opinion event cluster, thereby realizing comprehensive perception and accurate study and judgment of the public opinion situation.

The invention belongs to the technical field of natural language processing and intelligent knowledge retrieval, and provides an open type question implicit intention recognition method and system based on multi-agent reasoning. According to the technical scheme, multi-language background knowledge is constructed through a multi-language translation agent and an information retrieval agent; based on the constructed multi-language background knowledge, subject analysis is carried out, multi-field expert agent information is generated, and a multi-field expert agent is dynamically constructed; according to the guide problem proposed by the host agent, generating respective implicit intentions and reasoning argument text contents by using the multi-field expert agent according to the guide problem proposed by the host agent; and performing quality evaluation on the implicit intention and the reasoning argumentation text content by utilizing the host agent, and constructing an implicit intention library by taking the speech of each screened expert agent as a node. The defects in the aspects of processing complex semantic understanding and the like are effectively overcome, and the implicit intention in the open type problem is recognized more accurately and comprehensively.

The invention discloses a topic analysis method based on a large language model. The method comprises the following steps: performing data preprocessing on dialogue content; performing topic classification on the preprocessed dialogue contents, and collecting contents belonging to the same topic together; discussion objects, abbreviations and key phrases appearing in the content are extracted, and additional background knowledge is collected; summarizing contents under the topic by using a large language model to form a topic name and a topic abstract; and performing the same processing on a plurality of topics in the dialogue process to form a topic analysis report of the discussion content. Important information can be efficiently and accurately extracted, and a topic report is formed.

The invention provides a multi-dimensional flood disaster analysis method. The method comprises disaster space-time analysis, disaster semantic analysis, response adequacy analysis and population exposure degree analysis. According to the disaster situation space-time analysis, disaster situation related information extracted from a text is associated with a time attribute and a space coordinate attribute; analyzing the disaster information from the angles of time sequence and space distribution; the time sequence analysis adopts a quantitative statistics mode, and key nodes of the flood event are analyzed from the precipitation trend, the flood discharge trend, the microblog number trend, the ponding point number trend, the help seeking information number trend and the taken measure number trend; according to disaster semantic analysis, subject analysis is carried out on the text of each help-seeking point, and semantic information of disaster manifestation in space is mined; according to the response measure sufficiency analysis, the disaster response sufficiency is obtained by analyzing the correlation between the number of response measures and the number of help seeking information.

The present invention belongs to the field of natural language processing technology and intelligent knowledge retrieval technology, and provides an open-ended question implicit intention recognition method and system based on multi-agent reasoning. The technical solution is to use multi-language translation agents and information retrieval agents to construct multi-language background knowledge; based on the constructed multi-language background knowledge, conduct topic analysis, generate multi-domain expert agent information, and dynamically construct multi-domain expert agents; according to the guiding questions raised by the host agent, use multi-domain expert agents to generate their own implicit intentions and reasoning text content according to the guiding questions raised by the host agent; use the host agent to evaluate the quality of the implicit intentions and reasoning text content, and use the speech of each screened expert agent as a node to construct an implicit intention library. Effectively solve the deficiencies in processing complex semantic understanding and other aspects, and more accurately and comprehensively identify the implicit intentions in open-ended questions.

The invention discloses a digital financial management system based on big data, and relates to the technical field of financial management. Comprising a data acquisition module for acquiring financial related data through a web crawler unit and a data interface unit and storing the financial related data in an original database; the data preprocessing module is used for performing cleaning, word segmentation and labeling operation on the original data to obtain text data; and the topic analysis module is used for analyzing the preprocessed text data, identifying hot topics, judging sources of the hot topics and adding feature tags. The system covers a plurality of modules of data acquisition, preprocessing, topic analysis, trend analysis, matching comparison, scoring mechanism, investment suggestion generation and the like, and a complete digital financial management ecology is formed. The system can identify hot topics, judge sources of the hot topics and provide personalized investment suggestions according to risk preferences and market conditions of investors; and the satisfaction degree and the investment success rate of investors can be improved.

The invention discloses a public opinion analysis system based on text hotspots and emotions, and the system comprises a data obtaining module which is used for obtaining text data on a social platform and a corresponding user ID; the data cleaning module is used for cleaning the acquired text data; the topic analysis module is used for performing feature extraction and clustering on the text data; the sentiment analysis module is used for extracting sentiment information and reflecting the sentiment tendency and change of the social platform user; and the comprehensive analysis module is used for comprehensively analyzing the topic display result and the negative emotion new classification result. According to the method, topic analysis and sentiment analysis are included at the same time, and the results of topic analysis and sentiment analysis are further fused; and according to a fusion result of topic analysis and sentiment analysis, performing comprehensive analysis to obtain whether a person performs public opinion guidance or not so as to fully and comprehensively use text data to perform public opinion analysis.

The invention provides a social listening data noise reduction method and device based on a large language model, and belongs to the technical field of social media data processing. The method comprises the steps that social media data are collected and preliminarily filtered to obtain preliminarily filtered data; generating a semantic embedding vector by using a pre-trained large language model, and screening semantic related data in combination with cosine similarity and context; identifying and filtering duplicate and variant contents through an embedded model and a generative adversarial network to obtain duplicate-removed data; performing multi-modal feature extraction on the de-duplicated data, and screening credible data in combination with generative AI detection and sentiment analysis; and filtering semantic inconsistency and non-topic continuation texts by adopting a clustering algorithm and semantic relationship analysis to finish noise reduction. According to the method, through a multi-stage and multi-dimensional noise reduction strategy, the purity and credibility of social listening data are effectively improved, and high-quality data support is provided for subsequent topic analysis and decision making.

The invention belongs to the technical field of named entity recognition, and particularly relates to an entity labeling and recognition method and device based on deep learning, and the method comprises the steps: obtaining a training text, and labeling the training text through a BIO labeling method; the marked training text is processed; the processed text is input into a pre-trained BERT-CRF model, and fine tuning of the model on named entity recognition is completed; inputting a to-be-recognized text A into the fine-tuned BERT-CRF model to obtain an entity set of each category; performing topic analysis on a to-be-recognized text A by utilizing the trained LDA model to obtain a topic of the text A and a keyword of which the probability is greater than a set value under the topic; and verifying the entity set of each category, and correcting the entities which are not matched with the main body and the keyword in the set. The entity can be manually corrected, and the accuracy of named entity recognition is improved.

The present invention discloses a knowledge point extraction method and system based on a large model, which comprises the following steps: obtaining a knowledge text to be extracted, and obtaining important words of the text through large model preprocessing; vectorizing the important words of the text obtained through preprocessing to obtain text word vectors, and obtaining a text vector data set through aggregation; clustering analysis of the text vector data set to divide the text vector data set into different clusters; analyzing the text word vectors, selecting keywords representing the theme of the cluster from each cluster as the knowledge theme of the cluster; analyzing the degree of association between each important word of the text and the knowledge theme in each cluster, and extracting the important words of the text that meet preset conditions as the knowledge points of each cluster; and continuously aggregating and updating the knowledge points extracted from each cluster to obtain a final knowledge point set. The present invention can extract useful knowledge information from text data, and help better understand the content and theme of the text data by continuously updating and improving the knowledge point set.

This invention relates to a big data topic analysis method based on an embedding model. First, the Sentence-BERT model is used to perform sentence embedding representation on preprocessed Chinese text data. Then, the UMAP projection dimensionality reduction algorithm is used to reduce the dimensionality of the embedded vectors. Next, the HDBSCAN clustering algorithm is used to cluster the dimensionality-reduced vectors. Based on the assignment of each Chinese text in the target Chinese dataset to a corresponding topic class, the Chinese words with the highest c-TF-IDF scores are selected to represent each topic class. Finally, the DSG model is used to perform word embedding representation on the topic words, calculating the similarity between different topic words and between different topic classes, thereby detecting the volatility of newly emerging topic classes. The entire scheme design has higher topic consistency and topic diversity, and can detect new hot topics in a timely and accurate manner, providing early warnings.

This application relates to a method, apparatus, computer device, and storage medium for detecting hidden encryption attacks. The method includes: performing topic analysis on input text content to determine the topic semantics of the text content; performing intent parsing on the text content to extract multiple intent information contained in the text content and obtain the intent semantics corresponding to each intent information; performing anomaly analysis based on multiple intent semantics to determine abnormal intent information; determining the semantic similarity between the intent semantics of the abnormal intent information and the topic semantics, and identifying abnormal intent information with a semantic similarity lower than a preset similarity threshold as irrelevant intent information; and performing instruction nature identification on the irrelevant intent information to determine the hidden encryption attack detection result of the text content. Using this application can improve the ability to identify malicious content with high concealment and semantic complexity.

The present invention discloses a keyword-related topic analysis method based on local word embedding technology. When a user enters a keyword, the method uses global word embedding technology to analyze the keyword in text big data, sorts the cosine similarity between the vocabulary and the keyword in the data, and selects the subject words. The subject dictionary is constructed based on the subject words, and the text big database is segmented by time, author, article type, etc. The local word embedding analysis is performed on the keywords and the subject dictionary within the segment, and the local cosine similarity in each segment is used to obtain the theme of each segment. Then, the theme change of the keyword in the text big database can be analyzed. The present invention can be widely used in the fields of social science, public opinion monitoring, historical document analysis, etc., and provides an efficient and accurate theme dynamic analysis tool for text mining.

The present invention provides a text topic analysis method, device, electronic device and storage medium, wherein the method includes: determining a number of analysis topics to be implemented and the corresponding analysis result label range; obtaining a training sample set containing multiple sample texts; each sample text may correspond to at least one analysis topic; determining the analysis result label of each sample text on the corresponding analysis topic; the analysis result label is located within the analysis result label range of the corresponding analysis topic; taking each sample text and the corresponding analysis topic as input, and the analysis result label as output, training the XLM‑RoBERTa network to obtain an XLM‑RoBERTa model; inputting the text to be evaluated and the target analysis topic into the XLM‑RoBERTa model, and obtaining the target analysis result label output by the XLM‑RoBERTa model. This solution can perform topic analysis on text according to different requirements, thereby improving the flexibility of topic analysis.

The application provides a self-adaptive LDA topic model training system based on public opinion real-time data flow, comprising a data gathering module, a data preprocessing module, a self-adaptive LDA model training module and an incremental LDA model fusion module; the data gathering module is used for extracting and converting and loading the structured and semi-structured data and inputting into a distributed message bus kafka; the data preprocessing module is used for preprocessing the data in the message bus kafka and finally forming a weighted word vector; the self-adaptive LDA model training module is used for training to obtain an LDA model result and merging the training result; the incremental LDA model fusion module is used for fusion training to generate a new round of LDA model. The application is superior to the traditional LDA topic analysis method in accuracy and performance, and is applied to network public opinion field event detection, recommendation, word cloud and retrieval and other practical engineering projects, and creates commercial value.

The invention relates to the technical field of data analysis, and discloses an intelligent medical literature data analysis method and system based on natural language processing. The method comprises the following steps: collecting historical medical literature data, processing the collected historical medical literature data, constructing a sample set based on the obtained processed historical medical literature data, and constructing a literature topic analysis model based on the processed historical medical literature data in the sample set. The subject of the processed historical medical literature data is determined, and meanwhile, based on the determined subject of the processed historical medical literature data, the subject of the processed historical medical literature data is analyzed and trained through an intelligent analysis method; after the training is completed, the medical literature data is collected in real time, analysis and matching are carried out through a natural language processing mode based on the obtained trained historical medical literature data theme, and the accuracy of medical literature data analysis is improved.

The present technology assists a user in having an effective conversation in remote multiplex conversation. A display processing unit performs processing of displaying, on the basis of a result of topic analysis processing of identifying an ongoing topic on the basis of multiplex conversation data and giving a title indicating a summary of each topic, the ongoing topic using the title. The user in remote multiplex conversation can easily recognize not only a topic in which the user is participating but also other topics that are ongoing at the same time, and can easily find an ongoing topic of interest and newly participate in the topic. The present technology is therefore capable of assisting the user in having an effective conversation in the remote multiplex conversation.

The application relates to a customer complaint text classification method and device, computer equipment and a storage medium. The method comprises the following steps: preprocessing text data to be processed to obtain a plurality of word segmentation results; performing vectorization processing on the plurality of word segmentation results to obtain first vector values of the word segmentation results; inputting the plurality of word segmentation results into a pre-trained topic analysis model to obtain second vector values of each customer complaint topic to which the text data to be processed belongs and third vector values of keywords of the text data to be processed; performing splicing processing on the first vector values, the second vector values and the third vector values to obtain splicing features, and taking the splicing features as features of the text data to be processed; and inputting the features of the text data to be processed into a topic classification model and a perception classification model of a pre-trained classification model respectively to obtain a customer complaint topic category and a customer complaint perception category to which the text data to be processed belongs. The method can deeply analyze specific customer complaint contents under a field category.

The invention provides a social governance-oriented agent non-inductive management method, which comprises the following steps of: acquiring data related to social events from a plurality of information sources, analyzing the acquired data by utilizing a natural language processing technology and a large language model, and extracting key information related to the social events; subject analysis is carried out on the extracted key information, the key information is converted into a specific governance problem, and the governance problem is split from multiple angles; based on the split governance problem, data related to the current event is retrieved from a plurality of data sources, deep analysis is carried out through expert reasoning, an expert reasoning analysis result is generated, the data sources comprise a pre-constructed knowledge graph, social media and an instant webpage, and expert reasoning adopts a multi-round dialogue reasoning mode; and on the basis of the expert reasoning analysis result, updating a figure portrait, automatically generating a social governance report meeting requirements, and displaying related geographical location information through a geographical information system.

The present invention relates to a method and a system utilizing a large language model for topic allocation based on quotations. According to the present invention, a computer-implemented method utilizing a large language model for topic allocation based on quotations may comprise the steps of: specifying content to be analyzed, which includes at least one sentence; specifying at least one topic related to the at least one sentence included in the content to be analyzed; extracting, by using a large language model, at least one quotation corresponding to each of the at least one topic from the content to be analyzed; and providing, by using the topic and the quotation, a topic analysis result for the content to be analyzed.

The disclosed embodiment discloses an intelligent text retrieval and analysis system driven by natural language processing. The system includes: a preprocessing module for acquiring text data from an external system and preprocessing the text data to obtain text data to be processed; a feature analysis module for extracting data features in the text data to be processed, the data features including text features, first semantic features and second semantic features; a retrieval module for responding to search terms input by an external system, and based on the data features, determining retrieval results matching the search terms from the acquired text data; a text analysis module for performing text analysis tasks for the text data to be processed and / or the retrieval results based on the data features, the text analysis tasks including sentiment analysis tasks, topic analysis tasks and summary extraction tasks. The disclosed invention solves the technical problem of low efficiency in processing large-scale, complex semantic text data.

The present technology assists a user in having an effective conversation in remote multiplex conversation.A display processing unit performs processing of displaying, on the basis of a result of topic analysis processing of identifying an ongoing topic on the basis of multiplex conversation data and giving a title indicating a summary of each topic, the ongoing topic using the title. The user in remote multiplex conversation can easily recognize not only a topic in which the user is participating but also other topics that are ongoing at the same time, and can easily find an ongoing topic of interest and newly participate in the topic. The present technology is therefore capable of assisting the user in having an effective conversation in the remote multiplex conversation.

The invention relates to the field of natural language processing, and discloses a Chinese language and literature text topic analysis system based on big data, and the system comprises a core semantic primitive library construction module which is used for extracting and constructing a knowledge base containing semantics, culture and timestamps from a meta-poeia corpus offline; the text multi-dimensional analysis module is used for extracting four-dimensional features of grammar, core semantics, intertext and emotion time sequence of the to-be-analyzed text in parallel; the feature collaborative analysis module is used for fusing multi-dimensional features, performing context dynamic activation on primitives, and innovatively quantifying and detecting deep semantic conflicts by calculating the dispersion of feature vectors in a semantic space; and the topic report generation module is used for synthesizing a macroscopic topic, integrating all analysis results and generating a structured multi-level report. The system can objectively and deeply reveal complex and even contradictory theme connotation in literature works through modular cooperation, and the accuracy and depth of text analysis are remarkably improved.

The invention belongs to the field of social network derived topic analysis, and particularly relates to a time-varying network-oriented key influence user discovery method, which comprises the following steps of: preprocessing original topic network data to obtain topic network data; slicing the topic network data along the time dimension to obtain topic network data at different time points; calculating dynamic importance of the user nodes at different time points, and obtaining a transfer sequence of each user node according to the dynamic importance; constructing a feature vector of each user node by adopting an attention mechanism; obtaining an accumulated interaction sequence and a recent interaction sequence of each user node, and obtaining an interest vector of each user node through a dual Transform structure; splicing the feature vector and the interest vector of each user node to obtain a comprehensive feature; inputting all the comprehensive features into a full connection layer and a softmax function layer to obtain a key influence user identification result; according to the method, the key influence elements in the topic space can be effectively found.

The invention belongs to the technical field of data processing, and discloses a social media data processing method and device, computer equipment and a medium. The method comprises the following steps: acquiring various types of media data from a plurality of social media platforms through an interface adapter; converting the media data into structured data according to a preset data conversion rule, and storing the structured data in a database; receiving a data analysis instruction, and extracting to-be-processed target data from the database based on a screening condition in the data analysis instruction; inputting the target data into a pre-trained topic analysis model for topic extraction to obtain a topic recognition result; inputting the target data into a pre-trained sentiment analysis model for sentiment analysis to obtain a sentiment classification result; and combining the subject recognition result and the sentiment classification result to obtain a final data analysis result. The media data can be comprehensively collected and deeply analyzed, so that the value of the media data is fully utilized.

The invention discloses an NLP-based method for realizing automatic transcription of hot word extraction specifications, which comprises the following steps of: presetting different scenes, defining hot word types in the scenes and constructing a common hot word library, and preprocessing and cleaning a document; automatically analyzing the cleaned context data, and automatically extracting hot words in the corresponding file content; the extracted hot words are intelligently classified, and a dynamic hot word library is constructed; meanwhile, a voice transfer result is transferred into a hot word verification module in real time; carrying out semantic analysis according to the context of the voice recognition result, judging the most coincident hot word, and carrying out replacement display on the extracted hot word result; according to the method, a normalized result of hot word replacement is displayed, a text result of speech recognition is automatically adjusted, weights of hot words in a dynamic hot word library are updated in real time, representative hot words are extracted from preprocessed texts based on a trained NLP semantic model by applying technologies such as a clustering algorithm and topic analysis, and the accuracy of transcription is improved.

The invention provides a method for automatically generating and distributing network guidance content based on a large model, which comprises the following steps of: 1, data monitoring: collecting related data of a monitoring target in the Internet; 2, analyzing a hot topic, processing the related data collected in the step 1, and obtaining the topic of the hot event and the corresponding data content; step 3, guide content generation: setting a caliber requirement of the guide content according to the theme of the hot event obtained in the step 2, and generating the guide content through a large language model; and 4, task distribution management: selecting a distribution method and a distribution strategy according to the guide content obtained in the step 3, and carrying out guide content distribution.

The application relates to an open domain long text classification method and device based on theme analysis. The method comprises the following steps: constructing a field self-adaptive word table; preprocessing original long text to obtain purified text; using an LDA model to construct a latent theme space; extracting a core sentence group from the long text through semantic clustering to generate an abstract; mapping the abstract to the theme space to calculate a correlation degree score and output a theme identification; and converting the theme identification into a business classification label by querying a theme-label mapping table. The application effectively solves the problems of complex long text semantics and dynamic label system changes, and improves classification accuracy and expansibility.

The invention discloses a keyword related topic analysis method based on a local word embedding technology, which comprises the following steps of: after a user inputs a keyword, analyzing the keyword in text big data by utilizing a global word embedding technology, and sorting and screening out a topic word according to cosine similarity of vocabularies and the keyword in the data; the method comprises the following steps: constructing a subject dictionary through subject terms, segmenting a text big database according to modes of time, authors, article types and the like, performing local word embedding analysis on keywords and the subject dictionary in segments, and obtaining a subject of each segment through local cosine similarity in each segment; and furthermore, the theme change condition of the keyword in the text big database can be analyzed. The method can be widely applied to the fields of social science, public opinion monitoring, historical literature analysis and the like, and an efficient and accurate theme dynamic analysis tool is provided for text mining.