160 results about "Text mining" patented technology

The invention relates to the field of financial science and technology, and discloses a financial risk dynamic assessment system and method based on artificial intelligence. The system comprises a multi-source heterogeneous data acquisition module which acquires transaction data, public opinion texts and association maps in real time; the adaptive feature engineering module dynamically screens key risk factors; the dynamic risk map construction module calculates a risk conduction coefficient through a map neural network; the multi-modal AI analysis engine cooperatively runs a time sequence prediction model, a text mining model and a graph calculation model; a risk conduction simulator quantifies a systematic risk path. The problems of data splitting processing, model static solidification and correlation risk quantification deficiency in the prior art are solved, the false alarm rate is reduced to 12%, the response speed reaches 90 seconds, the prediction deviation is reduced to 22%, and an interpretable supervision report is generated.

The invention discloses a user demand comprehensive analysis method based on online user comment data. The method comprises: calculating a user demand intensity value; calculating a user demand emotion score; calculating a user demand weight value by adopting an IGR-AHP model based on the user demand emotion score; and respectively endowing the user demand intensity value and the user demand weight value with weight coefficients by using a weighted average method, thereby calculating a user comprehensive evaluation index value, and finally carrying out priority ranking on the user demands according to the user comprehensive evaluation index value. The method has the advantages that the LDA topic model is used for text mining, and the problem of how to effectively extract user demand information from a large number of user comments is solved; a user demand emotion score is calculated by fine tuning the BERT model, and the problem that a traditional emotion dictionary analysis method is tedious in process is solved; and finally, combining and considering the strength value and the weight value of the user demand through a weighted average method, and carrying out priority ranking on the user demand.

The present invention relates to the field of artificial intelligence natural language processing, and in particular to a topic clustering method driven by a large language model; the method comprises the following steps: S1) arranging document set data; S2) using the HDBSCAN clustering algorithm to perform preliminary clustering on the document set D to obtain a plurality of clusters {C k}; S3) Model the document topics using the LDA topic model to obtain the topic distribution of each document; S4) Adjust the clustering results based on the results obtained from the LDA model; S5) Further refine the topics of each cluster using LLM; S6) Establish effective and explainable associations between the generated topic list and the documents in the data set. The topic clustering system driven by the large language model of the present invention achieves a deep understanding and fine clustering of document topics by integrating HDBSCAN clustering, LDA and the large language model, providing a solid foundation for subsequent information retrieval, text mining and knowledge graph construction.

The invention relates to the technical field of performance appraisal management, in particular to a mine workshop performance appraisal management method, which comprises data acquisition and preprocessing, dynamic performance evaluation, task difficulty adjustment, an incentive mechanism and performance feedback and optimization. Optimizing factor weights and predicting employee performance based on a multi-factor weighting model in combination with a neural network and reinforcement learning, and calculating performance scores in real time; the task difficulty is quantified by applying an analytic hierarchy process and fuzzy comprehensive evaluation, and real-time adaptive optimization is analyzed in combination with employee ability; a market supply and demand and collaborative filtering algorithm is adopted to adjust an integral value and a customization scheme, and an award standard is analyzed and optimized according to cost effectiveness and performance sensitivity; a report is generated by means of a visual library, suggestions are generated by means of text mining, and through prediction and causal analysis early warning, a management strategy adjustment scheme is recommended according to reinforcement learning, and feedback optimization is performed; according to the invention, the performance of the mine workshop can be comprehensively, accurately and dynamically assessed and managed, the operation efficiency and the enthusiasm of employees are improved, and sustainable development of enterprises is promoted.

The present application provides a method and system for determining regulation-violating behaviors underground. The method comprises: acquiring historical personnel behavior hidden danger data within a target area underground, and performing text mining and data processing on the historical personnel behavior hidden danger data to obtain spatial-temporal patterns of hidden dangers and prediction and early-warning results; performing association analysis on the basis of personnel regulation-violating data, the spatial-temporal patterns of the hidden dangers and the prediction and early-warning results, and carrying out knowledge modeling to determine an ontology model; using a knowledge extraction method to extract knowledge elements of a knowledge graph from the ontology model; performing knowledge fusion processing on the knowledge elements to determine knowledge fusion data; on the basis of the knowledge fusion data, using a top-down and bottom-up hybrid construction method to construct the knowledge graph; and determining regulation-violating behaviors of personnel underground on the basis of the knowledge graph. The present application can dynamically diagnose safety production risks and regulation-violating behaviors in mines, and can discover and predict related accident signs and provide early warnings to ensure safety production.

A text mining engine running on an artificial platform is trained to perform conversation role identification, semantic analysis, summarization, language detection, etc. The text mining engine analyzes words in a transcript that represent unique characteristics of a conversation and, based on the unique characteristics and utilizing classification predictive modeling, determines a conversation role for each participant of the conversation and metadata describing the conversation such as tonality of words spoken by a participant in a particular conversation role. Outputs from the text mining engine are indexed and useful for various purposes. For instance, because the system can identify which speaker in a customer service call is likely an agent and which speaker is likely a customer, words spoken by the agent can be analyzed for compliance reasons, training agents, providing quality assurance for improving customer service, providing feedback to improve the performance of the text mining engine, etc.

The invention discloses a data analysis decision-making method, device and equipment and a medium, and is suitable for data analysis decision-making in the financial and medical fields. The data analysis decision-making method comprises the following steps: firstly, performing distributed data acquisition from a target data source by combining a passive collection mode and an active collection mode to obtain original data; and then performing efficient cleaning and processing on the original data through a preset segmentation rule and a pre-trained detection model. And through text mining analysis and association rule mining technologies, key modes and potential associations in a large amount of data are extracted, so that an accurate target portrait is constructed, and dynamic adjustment is performed according to real-time updated original data. And finally, through a preset decision-making algorithm, based on the target portrait, carrying out deep analysis and generating a decision-making report. According to the method, the efficiency and accuracy of data processing and decision support are remarkably improved.

The invention provides a multi-document key phrase extraction method based on graph structure node influence, and relates to the technical field of natural language processing and text mining. Firstly, a candidate phrase set is generated through noun phrase extraction and standardization; secondly, a semantic relation between phrases is captured through local subgraph construction and a sliding window mechanism, and the semantic relation is integrated into a global phrase co-occurrence graph; thirdly, dynamically dividing theme communities based on two-dimensional structure entropy minimization and a potential game model, and identifying phrase groups with high semantic aggregation; then, cross-topic nodes are processed through a structure entropy heuristic function, and flexibility of topic division is enhanced; and finally, in combination with node influence sorting, extracting key phrases with theme representativeness and propagation capability. The method does not need to label data, is suitable for multiple fields of academic literatures, news texts and the like, has high efficiency, accuracy and universality, and provides an innovative solution for multi-document key phrase extraction.

The invention relates to the technical field of artificial intelligence, and discloses a dialogue text mining method and system based on code scanning consumption cash back of big data, and the system comprises a consumption data collection module, a multi-modal analysis module, a strategy decision engine and a dynamic execution module. When code scanning consumption cash-back strategy decision making is carried out, real-time dialogue text semantics and consumption time-space behavior data are fused, a deep association mechanism of consumption intentions and payment scenes is established, the matching deviation of user language expression and actual consumption behaviors can be eliminated in real time, the consistency of a cash-back strategy and real consumption appeals is ensured, and the payment efficiency is improved. The problem of strategy mismatching caused by semantic understanding isolation in a traditional system is effectively avoided, the precision of personalized marketing is improved, double feedback of dialogue emotion tendency and historical strategy effects is dynamically analyzed, the deviation value of strategy feature matching is corrected in real time, and the system can actively adjust weight distribution of feature combinations in a strategy library.

The invention relates to the technical field of safety production management and intelligent analysis, in particular to a production safety accident key cause link identification method based on a complex network, which comprises the following steps of: acquiring structured production safety accident report data from a safety production supervision platform by utilizing a web crawler technology; a TF-IDF algorithm is combined with an industrial standard term library to optimize Jieba word segmentation, and a chi-square statistical method is adopted to calculate the correlation between each risk keyword and an accident category. By fusing text mining, statistical analysis and complex network theories and combining field dictionary optimization word segmentation, multi-accident model adaptation and triple centrality index analysis, the problems that a traditional method is single in analysis dimension, depends on subjective experience and is insufficient in technology integration are solved; efficient utilization of unstructured data, scientific layering of risk factors and accurate identification of key cause links are realized, and objectivity and effectiveness of safety management of a complex production system are effectively improved.

The invention discloses a quality evaluation method and system for an AI large model text training data set. The method comprises the steps that the large model text training data set is preprocessed; setting an evaluation index system, and defining a threshold value and a weight of each index; setting a data set sampling rule, and extracting a data set; counting preset data indexes; performing prejudice and compliance evaluation on the data set by using a large model through cue words and evaluation specifications; identifying illegal toxic contents in the data set according to evaluation specifications by utilizing cooperation of a text mining technology and a large model; and calculating a text confusion degree of the data set based on the BERT model, and calculating a confusion degree evaluation result according to the evaluation specification. Through the technical scheme of the invention, invalid training and influence on large model capability output caused by a data set problem are avoided, the AI large model training efficiency and the model effect are effectively improved, meanwhile, the large model training cost is greatly reduced, and the evaluation effect is remarkably improved.

A method for monitoring bills and legal regulations according to one embodiment of the present invention is a method for monitoring bills and legal regulations performed by a computing system including at least one memory and at least one processor, and includes the steps of storing template information including predetermined items, receiving basic information including reasons for proposing the bill and main contents from a user terminal, storing the basic information in an analysis server, forming extracted information from the basic information through text mining based on the predetermined items, generating regulatory impact analysis data based on the extracted information and the template information, and outputting a regulatory impact analysis interface for the bill or legal regulation to the user terminal based on the generated regulatory impact analysis data.

The invention relates to a record text mining method for telecommunication fraud prevention, which belongs to the field of clue mining, and comprises the following steps: inputting telecommunication fraud records formed in a certain region within a period of time into a KnowLM-13b-i e model, extracting 9 types of entities and relationships, and generating structured data; importing the generated structured data into a Neo4j database, defining three layers of entities of "victim-attribute-structuring" and a corresponding relationship, performing structured analysis and centrality calculation, and further constructing a knowledge graph; initializing the constructed knowledge graph by using a reasoning technology, generating a data set and optimizing a loss function, performing model training on a reasoning model by using the data set, performing multi-task fine tuning in combination with the reasoning model, and enhancing the fraud recognition capability; and inputting a case description, and outputting a fraud type and a subsequent fraud means by the reasoning model. According to the method, the electric fraud record text can be combined with the pre-trained large model to realize knowledge reasoning of the electric fraud record.

In one aspect, a computerized method for document extraction workflow for unstructured documents includes the steps of implementing a text mining operation on a set of digital documents the incoming documents. This is done by defining a document type of each digital document. Based on the document type, the method defines a set of data dictionaries to extract any data from each digital document. The method uses the defined set of data dictionaries to extract any data from each digital document.

The present application discloses a sensitive data identification method and apparatus, a device, and a computer storage medium. A text mining technology is used to mine a plurality of sensitive data rules from a data security specification file of a target industry to form a sensitive data rule base, the rule base is continuously augmented by using technologies such as NLP and NER, and after data to be identified of the target industry is obtained, a sensitivity class and a sensitivity level of the data to be identified can be identified by matching the sensitive data rules in the sensitive data rule base corresponding to the target industry with the data to be identified.

The invention relates to a multi-modal medical text extraction method based on adaptive domain knowledge fusion, and belongs to the technical field of medical text mining, and the method comprises the following steps: text entity-relationship mining: extracting related entities and semantic relationships from clinical structured and unstructured texts, and generating a preliminary text disease network; professional knowledge linking: performing knowledge alignment by using an external medical knowledge graph; and extracting a medical spatio-temporal event chain: extracting an event sequence with a time sequence and a spatial position from a clinical text, constructing a dynamic text disease knowledge network, and extracting a multi-modal medical text.

The invention discloses a civil aviation control operation safety risk assessment method based on text mining and network analysis, and relates to the field of civil aviation control operation safety risk detection, and the method comprises the following steps: S1, collecting domestic and overseas air traffic control operation unsafe event data, and carrying out the preprocessing, including the processing of irrelevant and duplicated data, and the processing of the unsafe event data; constructing an air traffic control operation safety risk management stop word and segmented word library; s2, based on the processed data, a BERTopic topic model is used for identifying civil aviation control operation safety risk topics, model parameters of the BERTopic are modified through topic coherence (TC) and topic diversity (TD) among the topics, and the risk topics are optimized; s3, setting causal relationship network nodes based on the civil aviation control operation safety risk theme, and constructing a causal relationship network to determine the mutual relationship between civil aviation control operation safety risk factors; and S4, identifying high-risk nodes in the civil aviation control operation safety causal relationship network based on topology analysis, determining key cause factors in civil aviation control operation safety events, and proposing a risk control strategy. By adopting the steps, the civil aviation control operation safety risk theme is identified, the causal relationship network is constructed, the risk cause factors are determined, and finally the targeted risk control strategy is put forward, so that stable and safe operation of the air transportation system is facilitated.

The invention relates to the technical field of clinical test information extraction, and discloses an information extraction method of a clinical test scheme. According to the method, clinical test scheme files, registration information, research papers and related literatures are collected, data are sorted, an electronic database is formed, a text mining technology and a natural language processing tool are used, the collected literatures are analyzed, key information is labeled, and the literatures are analyzed. Calculating a random control test design type proportion and an open label design type proportion, extracting the marked data into a spreadsheet, checking the extracted data, implementing double check, calculating a consistency proportion and an error rate in the checking process, and performing statistical analysis according to the extracted data, so as to obtain a statistical analysis result; calculating a mean value, a standard deviation and a confidence interval, extracting a minimum value and a maximum value of data and a combination effect quantity of a combination end point, arranging analysis results into a report, including detailed description of test design, key results and conclusions, and displaying by using charts and graphs.

The invention belongs to the technical field of natural language processing and text mining, and discloses an online forum-oriented low-resource topic key topic extraction method, which comprises the following steps of: performing semantic-preserving data enhancement on an original text through a large language model to generate an enhanced document set; utilizing a pre-training language model to extract context-aware semantic representation of the document; constructing a learnable topic embedding matrix, and calculating and generating topic distribution; designing a semantic perception contrast learning framework, and optimizing theme diversity by adopting a dynamic negative sample screening strategy; and meanwhile, priori alignment loss is used for ensuring theme consistency. According to the invention, an LLM enhanced data expansion mechanism and a lightweight theme coding architecture are creatively fused, and through dual optimization of contrast learning regularization and prior distribution matching, three technical problems of data sparsity, model over-fitting and noise sensitivity in a low-resource scene are effectively solved; and an efficient and reliable theme modeling solution is provided for social media public opinion analysis.

The invention discloses an enterprise hidden danger-accident association early warning method based on text mining, and relates to the technical field of association early warning and oil gas safety. According to the method, the high relevance between hidden dangers and accidents can be better revealed, the incidence relation between'accident cause segmented words' and'hidden dangers' is used as a data source of the model, and an association rule mining algorithm (Apriori algorithm) is used for finding out the hidden dangers; a potential association mode between the hidden danger and the accident is mined through a chain of hidden danger-keyword-accident. The method not only comprehensively reflects the relationship between the hidden danger and the accident from the perspective of data, but also reveals that the hidden danger is the key cause of the occurrence of some accidents, and the analysis method is obviously superior to the traditional hidden danger analysis means depending on subjective judgment or single dimension.

The invention discloses a brain connection event causal relationship extraction method based on weighted joint constraint learning. The method aims at a text mining task-brain connection event causal relationship extraction in an event-level neural image. According to the method, a novel model named as a brain connection event relation miner is developed by utilizing weighted joint constraint learning. A lightweight model is developed by integrating a BiLSTM (Bidirectional Long Short Term Memory) network and a multilayer perceptron (MLP), and is used for jointly extracting a plurality of event causal relationships from brain connection literatures. In view of scarcity of an annotation brain connection corpus, a weighted joint constraint learning framework is provided. The framework integrates dual consistency constraints, including common sense constraints and domain constraints, and combines the dual consistency constraints with adaptive weight learning to enhance the small sample learning ability of the model.

The invention belongs to the technical field of computer software data processing and text mining, and particularly relates to a similarity detection method and system for science and technology project texts. The method comprises the following steps: step 1, obtaining a text, and preprocessing the text; 2, judging an application scene of the text, performing feature extraction according to selection of the application scene, then converting the extracted features into character strings with fixed lengths, and finally obtaining fingerprint features of the text through Hash calculation; 3, creating an index instance, adding the data into an index, constructing a Faiiss vector retrieval library, then performing similarity retrieval through an AAN algorithm, and obtaining the similarity between the fingerprint features of the text and the text in the index library; and 4, sorting and filtering the retrieval results, and judging whether text repetition exists or not according to a set threshold value and outputting the text repetition. According to the invention, the accuracy and reliability of the detection result are improved, and the detection efficiency is improved.

A source content processor receives content from a crawler and calls a text mining engine. The text mining engine mines the content and provides metadata about the content. The source content processor applies a source content filtering rule to the content utilizing the metadata from the text mining engine. The source content filtering rule is previously built based on at least one of a named entity, a category, or a sentiment. The source content processor determines whether to persist the content according to a result from applying the source content filtering rule to the content and either stores the content in a data store or deletes the contents from the data ingestion pipeline such that the content is not persisted anywhere. Embodiments disclosed herein can significantly reduce the amount of irrelevant content through the data ingestion pipeline, prior to data persistence.

This invention relates to systems and methods for adaptive discovery and mixed-variable optimization of synthesizable microelectronic materials, and applications of the same. Specifically, an exemplary system includes a virtual screening (VS) module to extract information from literatures of a knowledge base by text mining, a ML-assisted conceptual exploration (CE) module to identify candidate material families for the specific class of compound materials based on the extracted information via a combination of ML models and to generate exogenous models of objective functions f(x, y) and constraint functions g(x, y), and an adaptive discovery (AD) engine to generate and optimize design of the newly discovered compound materials. The AD engine includes a mixed-variable ML module, a mixed-integer optimization (MIO) module, and a high-fidelity evaluation (HFE) module, which are iteratively and sequentially executed.

The invention discloses an e-commerce user portrait construction method based on reinforcement and increase learning, and the system comprises a data collection module which collects the information of a user, and provides a data source for the comprehensive analysis of user behaviors; the data processing and analyzing module is used for cleaning and preprocessing the data and analyzing the behavior pattern and preference of the user according to the processed data; and the label generating and grading module is used for automatically generating labels for the user according to the result output by the data processing and analyzing module in combination with a preset label rule base, and calculating the comprehensive score of the user portrait. According to the method, browsing behavior characteristics are calculated through a specific formula, interest preferences are analyzed by applying a text mining technology and a deep learning model, and user behavior modes and interest points can be accurately grasped; a reward mechanism is constructed based on feedback, a push strategy is evaluated by using a reinforcement learning algorithm, an optimal strategy is continuously selected, and the push effect and the accuracy of a user portrait are improved.

The invention discloses a method for determining the importance degree of seepage monitoring points of an earth and rockfill dam and application of the method. The method comprises the following steps that six seepage safety parts of the earth and rockfill dam are recognized, wherein the six seepage safety parts comprise a dam body, an anti-seepage body area, an inverted filtration drainage area, a dam penetrating building area, a dam foundation and a bank slope area; constructing an earth and rockfill dam seepage Bayesian network according to a historical monitoring text, and calculating importance degrees of six seepage safety parts; acquiring three-dimensional space coordinates of the seepage monitoring points: stake numbers, wheelbases and elevations; calculating the comprehensive importance degree of each part to each section of the monitoring point; and superposing the comprehensive importance degrees of the cross sections in the transverse, longitudinal and horizontal directions to obtain the importance degrees of the monitoring points. According to the method, historical monitoring text mining, a Bayesian network probability model and a three-dimensional space attenuation algorithm are coupled, and the importance degree of a quantitative measuring point is calculated, so that the problem that a traditional method depends on experience and neglects local structure defects is solved; and the high-weight section automatically triggers special inspection of the corresponding seepage safety part, so that timely further disposal is facilitated, and the safe operation life of the dam is effectively prolonged.

The invention discloses a knowledge semantic tree construction method based on text data, and belongs to the technical field of natural language processing. Based on related text data in the professional field and on the basis of preprocessing, feature information is extracted through text mining, word bag modeling and clustering analysis are conducted on related vocabularies, related attributes are continuously filled, the relation between entities is expanded, a semantic tree is gradually constructed, and priori knowledge capable of being recognized by a computer is formed. By constructing the knowledge semantic tree, an intuitive knowledge system representation mode is provided, different types of knowledge can be conveniently integrated and fused, knowledge service application tasks such as information retrieval, intelligent question and answer and semantic analysis are supported, efficient reasoning is supported, and the knowledge system representation method is suitable for popularization and application. The method is easier to popularize in the aspect of data standardization, and corresponding tools are more convenient to unify.

The invention discloses a customer loss risk assessment method and device based on a neural network model, and relates to the field of artificial intelligence or other related technical fields, and the method comprises the steps: obtaining the assessment data of a target customer, the assessment data at least comprises a user portrait, a journey map and a behavior data set, the journey map is used for describing the experience feeling of the target customer in the processes of purchasing financial products and using financial services; performing feature extraction on the user portrait and the journey map of the target customer to obtain a user portrait feature vector and a journey map feature vector; drawing a behavior graph based on the behavior data set of the target customer; and inputting the user portrait feature vector, the journey map feature vector and the behavior graph into a loss risk assessment model, and outputting a loss risk value of the target customer. According to the method and the device, the technical problem of relatively low accuracy of an assessment result caused by relatively single assessment data in a mode of performing loss risk assessment based on text mining or user portraits in related technologies is solved.

The invention relates to the technical field of text mining and data clustering, and discloses a state grid document collaborative clustering analysis method, system and device based on a graph model and a medium, comprising: constructing a joint graph model, the joint graph model comprising a document node subset and a word node subset, the edge weight between document nodes being determined based on the diversity between documents, and the word node subset being determined based on the difference between the document nodes; the edge weight between the word nodes is determined based on the diversity between the words, and the document nodes and the word nodes are not directly connected; on the basis of the joint graph model, constructing and solving a collaborative clustering target function to synchronously perform graph division on the document node subset and the word node subset so as to obtain a clustering result of the document and a clustering result of the word at the same time; wherein the collaborative clustering objective function is configured to maximize the sum of the cut value of the document sub-graph division and the cut value of the word sub-graph division. According to the method, the complex dissimilar relationship between the documents and between the words can be visually and effectively captured by utilizing the natural relationship representation capability of the graph model.

Systems and methods for power generation forecasting in a microgrid are disclosed. In some embodiments, a system comprises at least one processor; and memory storing instructions executable by the at least one processor, the instructions when executed cause the system to: obtain weather information; extract, from the weather information, one or more weather events, the one or more weather events including a first event, wherein the first event causes reduction of power generation of a power generation system; determine a condition of one or more components of the power generation system based on the first weather event; and determine future power generation by the power generation system based on the determined condition of the power generation system.