84 results about "Text cluster" patented technology

The invention discloses an event determination method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: clustering a plurality of obtained texts to obtain a plurality of candidate text clusters; in the multiple candidate text clusters, the candidate text clusters with the number of texts meeting a first number condition are determined as target text clusters, and events associated with the multiple texts in the target text clusters belong to target type events; determining weights of the plurality of texts in the target text cluster based on the interaction information of the plurality of texts in the target text cluster; based on the multiple texts in the target text cluster and the weights of the multiple texts in the target text cluster, an event description text is generated, the semantics of the event description text is the same as the semantics of the multiple texts in the target text cluster, and the event description text is used for describing target type events associated with the multiple texts in the target text cluster. According to the method, computing resources can be saved, and the interpretability of the event determination method can be improved.

The present application provides a text clustering method, apparatus, electronic device and computer-readable storage medium. Based on anchor words corresponding to the full-text semantics of each training text, anchor word model features corresponding to the anchor words are obtained to avoid the introduction of additional noisy features. Then, a first clustering result and a second clustering result of each training text are obtained based on the anchor word model features, and a self-training objective function of multiple training texts is determined based on each first clustering result and each second clustering result, as well as a self-training target value of the self-training objective function. Finally, the text clustering model is updated based on the self-training target value until the text clustering model converges, and the converged text clustering model is applied to text clustering. After continuous training until the convergence of the text clustering model, the accuracy and stability of the text clustering model are continuously improved, and the accuracy of text clustering is avoided from being affected by additional noisy features.

The present invention discloses a method and device for text data clustering, and relates to the field of computer technology. A specific implementation of the method includes: obtaining batch text data, and determining a feature word set of each text data in the batch text data; for the feature word set of each text data, determining the weight of each feature word in the feature word set; sorting the batch text data according to the weight of the feature word in the feature word set; and performing clustering calculation on the batch text data based on the sorting result. The method rearranges the order of text clustering using feature weights, and can preferentially cluster text data containing rich information into topic classes, and then perform text clustering based on the clustered topic classes, which can improve the accuracy of clustering.

The present invention discloses a method for clustering unknown protocol texts based on reinforcement learning. The method is based on the static parsing technology route based on network tracking in protocol reverse engineering, adopts the reinforcement learning algorithm DQN to self-learn the classification of hexadecimal network traffic texts, uses CNN for feature extraction, and combines one-dimensional action vectors as inputs to calculate Q values. At the same time, the discreteness and randomness of the traffic texts are combined to simplify the experience playback steps in the reinforcement learning framework, taking into account both the reduction of the amount of calculation and the simplification of the structure. The present invention realizes the source classification of network traffic texts using unknown protocols from multiple different application sources, and has a high classification accuracy and good stability.

The invention provides a text clustering method and device, computer equipment and a medium. The text clustering method comprises the steps of obtaining a first score according to feature complexity of text data in a data set; obtaining a second score according to the semantic complexity of the text data; classifying the data set based on the first score and the second score to obtain a target data set; and carrying out clustering on the target data set. Text data in a data set is classified according to feature complexity and semantic complexity of the text data to obtain a target data set, and then the target data set is clustered. Through the method provided by the invention, the clustering task can be simplified, and the clustering efficiency and accuracy can be improved.

Embodiments of the present application provide a text processing method, device, equipment, storage medium and program product. The method comprises: obtaining a target text; processing the target text based on a vector dictionary to obtain a target vector; and processing the target vector based on a target task model to obtain a text processing result. In the embodiments of the present application, the target text is processed by the pre-constructed vector dictionary to obtain the target vector, and then the target vector is processed by the target task model. Since the target vector includes a semantic feature vector and a local difference vector, the accuracy of text processing can be improved. Moreover, the target task model is any one of a text classification model, a text clustering model and a sentiment analysis model in the call center customer service field. Compared with the use of a general large model, the efficiency of text processing can be improved.

The invention discloses a Chinese handwritten form detection method and system based on a text kernel and text clustering. The method comprises the following steps: acquiring a Chinese handwritten form text image; extracting Chinese handwritten text feature maps, and adjusting the number of channels to unify the number of channels of each feature map; the deep layer features are gradually restored to a higher resolution ratio through up-sampling operation, and the deep layer features are fused with shallow layer features of corresponding scales; dynamically fusing feature maps of different scales; learning similarity vectors among text pixels, and aggregating the text pixels to corresponding text kernels; using the predicted similarity vector to guide the aggregation of text pixels; and adding the text region loss, the text kernel loss and the text clustering loss into model training, and carrying out iteration by utilizing a gradient descent algorithm until the maximum number of iterations is reached or the model is converged. According to the technical scheme, the text kernel prediction technology and the dynamic pixel clustering technology are fused, and the text detection precision and robustness are effectively improved.

This application relates to the field of computer technology and discloses a method, apparatus, device, and storage medium for cross-page test question aggregation. The method includes acquiring multiple test question images, each image including test question elements; extracting text blocks from the test question images and grouping text blocks belonging to the same test question and the same test question element into a text cluster, obtaining a text cluster set; searching for a baseline text cluster and multiple candidate text clusters in the text cluster set, where candidate text clusters are those related to the baseline text cluster; generating label codes for each text cluster in the baseline and candidate text clusters based on the test question element to which the text cluster belongs, the text blocks within the text cluster, and the position of the text blocks in the test question images; and searching for text clusters belonging to the same test question as the baseline text cluster in the candidate text clusters based on the label codes, obtaining a test question text cluster set. The method of this application can achieve automatic clustering of test question elements with high clustering accuracy.

The application provides a keyword reverse propagation algorithm based on a citation network structure, and comprises the following steps: a spring charge model is established, force-directed layout processing is performed, and a force-directed layout graph is established; a keyword propagation model is established by using a reverse propagation algorithm, and a keyword weight change contrast curve is obtained; a citation network model is constructed by using the force-directed layout graph; iterative calculation is performed on the force-directed layout graph until the energy state in the force-directed layout graph reaches a minimum value; while the citation network is iteratively calculated, the keyword weight in the citation network model is adjusted, and a converged citation network layout graph is calculated. The application has the beneficial effects that the entire network is taken as a main body, keywords owned by cited documents are selected at a certain probability and are propagated backward along the network to the citing documents, the weight of the keywords is increased while the text clustering idea is retained, and clear visual data effects are obtained through force-directed layout.

The application relates to a text clustering method based on an improved whale optimization algorithm, and belongs to the fields of big data mining and machine learning. The method comprises the following steps: S1: obtaining scientific research text data from a scientific research text database by using Spark, and performing data cleaning; S2: performing word segmentation, stop word removal, feature selection, vectorization and dimension reduction operations on the text data after data cleaning, so that unstructured text data is converted into structured numerical data; and S3: calculating initial clustering centers by using a K-means algorithm, optimizing the clustering centers by using an improved whale optimization algorithm, and outputting clustering results. The global optimization capability of the improved whale optimization algorithm is utilized, the problem that current text clustering algorithms are prone to falling into local optimization is solved, the accuracy of clustering of scientific research text data is improved, the data dimension is reduced, and the clustering effect is enhanced.

The embodiment of the invention provides a text classification method, electronic equipment, a storage medium and a program product. The method comprises the following steps: processing a to-be-classified text according to a lexical element division rule to obtain a lexical element sequence; according to the lexical element sequence, adopting a pre-trained unsupervised topic model to obtain a topic distribution data set corresponding to the to-be-classified text; adopting a pre-trained unsupervised clustering model to obtain a clustering distribution data set corresponding to the to-be-classified text; splicing the topic distribution data set and the clustering distribution data set to obtain a text hidden topic; obtaining a plurality of text clusters and current cluster hidden topics corresponding to the text clusters, and calculating the similarity between the text hidden topics and the current cluster hidden topics; and determining a target text cluster from the plurality of text clusters, and adding the to-be-classified text to the target text cluster. Through a cluster classification mechanism of subject distribution and cluster distribution conjoint analysis, the accuracy and result stability of text classification are improved.

The application provides a cross-modal retrieval method based on prototype regularization learning, and belongs to the field of artificial intelligence and multi-modal information processing, and comprises the following steps: extracting visual features and text features through a visual encoder and a text encoder and calculating a basic contrast loss; dividing visual clusters and text clusters through a clustering algorithm and calculating visual prototypes and text prototypes, selecting text features most similar to the visual prototypes as anchor points, and constructing cross-modal prototypes; calculating a prototype-level discriminant loss according to the visual prototypes and the text prototypes, calculating an instance-level discriminant loss according to the visual clusters and the text clusters, and calculating a prototype projection loss according to the cross-modal semantic prototypes; constructing a joint optimization objective function by combining all the losses, training the visual encoder and the text encoder, and performing cross-modal retrieval after the training is completed. The application solves the problems of excessive intra-class variance and excessively high inter-class similarity caused by appearance bias in the existing cross-modal retrieval method, and causes the problems of insufficient retrieval accuracy and robustness.

The embodiment of the invention provides an artificial intelligence-driven public place risk dynamic early warning method and device, equipment and a medium. The method comprises the steps of collecting text release information associated with a target place, and performing text clustering on the text release information to obtain a target clustering result; analyzing sub-class clusters in the target clustering result to obtain related feature information; performing topic identification on the target clustering result according to the related feature information to obtain an initial topic and related text information; performing association analysis according to the related text information and the target place to obtain a target topic and associated text information; performing behavior detection on the associated text information to obtain a target behavior type; when the target behavior type meets a preset type, collecting latest text information associated with the target place; performing popularity analysis on the target topic according to the latest text information and the related text information to obtain a target popularity type; and performing risk early warning on the target place according to the target popularity type and the target behavior type to obtain a target early warning result.

The application provides a text clustering method, a text clustering device, an electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. An initial clustering center is selected from a plurality of original sentence embedding vectors, an initial problem text class is constructed based on the initial clustering center, clustering processing is performed on all original sentence embedding vectors based on the initial clustering center, a plurality of intermediate problem text classes are obtained, a clustering center is identified for each intermediate problem text class, a text clustering center is obtained, inter-class dispersion between the intermediate problem text classes and intra-class dispersion of each intermediate problem text class are calculated according to the text clustering center, the intermediate problem text classes are adjusted in text according to the inter-class dispersion and the intra-class dispersion, a target problem text class is obtained, semantic analysis is performed on all original sentence embedding vectors of the target problem text class, a standard problem text is obtained, and standard reply content is determined according to the standard problem text, so that the accuracy of problem answering can be improved.

Implementations generally relate to text clustering with heuristic and multi-metric control. In some implementations, a method includes receiving an electronic source document containing text. The method further includes dividing the text into text units, encoding the text units, and transforming the text units into numerical values. The method further includes generating a graph of the text units based on the numeric values, where the graph includes nodes corresponding to the text units and edges corresponding to pairs of the text units. The method further includes ordering the text units into text clusters based on the graph of the text units. The method further includes generating an electronic target document that presents the text clusters based on one or more preference heuristics.

The invention provides a text clustering processing method, a text classification method and a related device. The method comprises the following steps: clustering first texts in a first text set into a plurality of text clusters; performing text coding on the first text in each text cluster to obtain a coding result of the text cluster; determining intra-cluster compactness of the text clusters based on the coding result of each text cluster; based on the coding result of the first text cluster and the coding result of the second text cluster, determining an inter-cluster separation degree between the first text cluster and the second text cluster; the first text cluster and the second text cluster are any two text clusters in the plurality of text clusters; and determining a first clustering index value of the first text set based on the intra-cluster compactness and the inter-cluster separation degree. Through the method and the device, the accuracy of text clustering processing can be improved.

The present invention relates to a cross-border ethnic text clustering method and device that integrates domain knowledge graphs, and belongs to the field of natural language processing technology. Cross-border ethnic domain texts have their own unique cultural domain vocabulary. In order to solve the problem that the cultural background of cross-border ethnic cultural texts is missing and the cultural differences between different ethnic groups are large, resulting in poor text clustering effect, the present invention proposes a cross-border ethnic cultural text clustering method that integrates domain knowledge graphs, which mainly includes a cross-border ethnic cultural text data preprocessing part, a cross-border ethnic text data feature extraction part, and a cross-border ethnic cultural text clustering structure. According to the functional modularization of these three parts, a cross-border ethnic cultural text clustering device that integrates domain knowledge graphs is made. Compared with the general text clustering method, the present invention effectively alleviates the problem of poor cross-border ethnic cultural text clustering effect.

The application relates to a clustering-based text supervision checking method and device and a computer device. Feature word sets corresponding to a plurality of question description texts in a question list are obtained by respectively extracting feature words from the question description texts. The word frequency of each feature word in the question description text, the first inverse document frequency in all question description texts and the second inverse document frequency in all checked units are respectively calculated to obtain a feature word weight set of the question description text of each checked unit. A plurality of question description text clusters and a plurality of initial high-correlation-degree feature word sets corresponding to the question description text clusters are obtained through clustering. Finally, the initial high-correlation-degree feature word set is optimized according to a chi-square statistic to obtain an accurate high-correlation-degree feature word set. The method saves the calculation resources while ensuring the accuracy of the results by optimizing the feature word weight calculation, adopting a clustering algorithm for preliminary clustering and optimizing the clustering results through chi-square statistics.

The application belongs to the technical field of data mining, and specifically discloses a cross-field technology fusion opportunity finding method based on science and technology fusion interaction, which comprises the following steps: obtaining scientific paper and technology patent data, identifying cross-field scientific fusion and technology fusion based on semantic anchor points and a similarity threshold; using a trained deep text clustering model to respectively perform deep clustering on scientific fusion and technology fusion, and obtaining corresponding clustering sets; obtaining text representation corresponding to each cluster, filtering through a threshold, and establishing association between two types of clusters through bipartite graph maximum weight matching to identify shared topics and exclusive topics; using a dynamic time warping method to align the time series of scientific and technological trends, judging topics with scientific convergence preceding technological convergence, and identifying technology fusion opportunities with growth potential based on the development trend of scientific papers. The application can deeply mine cross-field science and technology semantics, improve clustering accuracy, quantify science and technology association, and dynamically find forward-looking technology fusion opportunities.

Methods for management of alarm text data for received alarms, which can involve processing the alarm text data for clustering to generate processed alarm text data; clustering the processed alarm text data into first clusters through executing a first clustering algorithm, the first clustering algorithm configured to be facilitated via accelerated processing; for the first clusters not meeting a metric, executing a second clustering algorithm on the first clusters to form second clusters that are representative of subgroups of the first clusters with improved homogeneity, the second clustering algorithm being different from the first clustering algorithm; executing a tagging process on the second clusters and ones of the first clusters that meet the metric to tag the second clusters and the ones of the first clusters that meet the metric into groups to form tagged cluster groups; and providing the tagged cluster groups to be displayed on the user interface.

The embodiment of the invention provides a text clustering method and device, and the method comprises the steps: obtaining a plurality of to-be-clustered texts, and calculating the text similarity between the to-be-clustered texts; determining a noise text in the plurality of to-be-clustered texts according to the text similarity, and deleting the noise text in the plurality of to-be-clustered texts to obtain a to-be-clustered text set; based on to-be-clustered texts in the to-be-clustered text set and the text similarity between the to-be-clustered texts, constructing a target text connected graph corresponding to the to-be-clustered text set; and determining a central text corresponding to the target text connected graph, deleting an edge text in the target text connected graph based on the central text, and determining a target text set according to a deletion result.

The invention relates to the field of natural language processing, in particular to an event text hot word extraction method based on similarity adjacent matrix clustering, which comprises the following steps: acquiring a keyword set of each event text; calculating an improved Jaccard similarity value of any two event texts, and constructing a text similarity adjacent matrix; converting the text similarity adjacent matrix into a distance matrix, and dividing all texts into K text clusters based on the distance matrix; removing the K text clusters with abnormal text quantity by using a quartile method to obtain normal text clusters; word weight calculation is conducted on each text in the normal text cluster again to obtain a normal keyword set, the final comprehensive score of each keyword is calculated based on the normal keyword set, candidate keywords in the cluster are reordered according to the final comprehensive score, and the candidate keywords ranked in the front serve as final hot words. The hot word extraction accuracy and efficiency can be improved.

The application discloses a text clustering method and device and electronic equipment. It is related to the field of financial technology or other fields, and the method comprises the following steps: obtaining a plurality of digital vectors of a to-be-processed text, wherein each digital vector corresponds to part of the to-be-processed text; determining a first distance threshold and a second distance threshold based on the plurality of digital vectors, wherein the first distance threshold is the maximum limit value of the clustering range, and the second distance threshold is the minimum limit value of the clustering range; performing first clustering processing on the plurality of digital vectors based on the first distance threshold and the second distance threshold to obtain a clustering result; obtaining the number of clusters in the clustering result; performing second clustering processing on the plurality of digital vectors based on the number of clusters to obtain a target centroid vector corresponding to each cluster, wherein the target centroid vector represents the characteristics of the cluster corresponding to the target centroid vector. The application solves the technical problem of poor text clustering effect caused by the fact that the number of clusters cannot be accurately determined in the prior art.

The application discloses a kind of method and system for commodity title clustering, specifically related to text clustering technical field, for solving the technical scheme of general data is generally aimed at existing disclosure technology, there is also scheme is aimed at such data, but due to different application scenarios, such scheme is not completely applicable problem, its method includes data crawling, commodity title normalization, semantic vector conversion, similarity analysis, clustering analysis and search recommendation of similar goods, system is composed of hardware and software, software includes crawler module, processing module, semantic vector module, similarity calculation module, clustering module and recommendation module, hardware includes CPU, memory bank, storage and GPU;It is through the competitive product analysis of online commodity transaction information data, obtains the similarity of commodity title and carries out clustering analysis and search recommendation to the similarity of commodity title, to improve the accuracy of clustering and search recommendation.

This application provides a text clustering method, a text clustering device, and a text clustering system. In this solution, the Word2vec model is combined with the TF-IDF algorithm to represent word vectors, enhancing the distinction between different texts. This approach not only takes advantage of word vectors but also incorporates the influence of words on text. The word vectors represented by the combination of the two serve as the input of the WMD algorithm, which is used as a similarity measurement algorithm in text clustering to improve the accuracy of text clustering.

The invention discloses a literature metering method and device based on a knowledge graph, and the method comprises the steps: carrying out the statistics of the word frequency of each subject term in a literature data set, determining an inverse text frequency index, and obtaining an inverse text frequency weight value of each subject term; carrying out descending sorting on the inverse text frequency weight value of each subject term to obtain an inverse text frequency weight value after descending sorting; obtaining a text feature value according to a preset screening condition and the inverse text frequency weight value after descending; and clustering the subject terms by adopting a clustering algorithm and a cosine distance function to obtain a text core subject. According to the literature metering method based on the knowledge graph, the word frequency matrix is constructed through co-word analysis of high-frequency words, correlation between keywords is mined, the distance between subject terms is calculated through text clustering, a research subject is clarified, and a text core subject and research hotspots in related fields are effectively determined.

The invention provides a short text clustering method based on adaptive optimal transmission and three-level robust representation, which comprises the following steps of: 1, coding an input original short text based on a pre-training language model in a server carrying a GPU (Graphics Processing Unit) to obtain a semantic representation of the original short text, and performing multi-type data enhancement processing on the semantic representation to obtain a semantic representation of the original short text; obtaining an enhanced sample representation; 2, performing pseudo label distribution on enhanced sample representation based on an optimal transmission algorithm; 3, learning and calculating a corresponding loss function based on the pseudo label and three-level robust representation, and updating model parameters of the loss function in a CPU (Central Processing Unit) of a server; 4, iteratively executing the steps 1-3, and outputting a target clustering result when a clustering result meets a convergence condition; the problems of data sparseness and class imbalance in short text clustering can be effectively relieved, and the stability and precision of a clustering result are improved.

The invention discloses a data marking method and device and a machine readable storage medium, and relates to the technical field of computers. The method comprises the steps of performing text clustering on a text vector corresponding to data to be marked based on a preset clustering number to obtain a text clustering result; when the text clustering result meets a preset clustering condition, performing text marking on the to-be-marked data based on the text clustering result to obtain a first mark; inputting the to-be-marked data into a preset entity recognition model to obtain an entity; performing entity marking on the to-be-marked data based on the entity to obtain a second mark; combining the entities, and determining a corresponding entity combination vector; performing entity clustering on the entity combination vector, and performing entity combination marking on the to-be-marked data based on an entity clustering result to obtain a third mark; and based on the first mark, the second mark and the third mark, generating a mark of the to-be-marked data. The advantages of text clustering, entity recognition and entity clustering are combined, and the accuracy and efficiency of data marking are improved.

The invention relates to the technical field of computer vision and intelligent information processing, in particular to a machining drawing information extraction method based on YOLO + OCR, which comprises the following steps: sequentially carrying out graying processing, Gaussian filtering noise reduction and Canny edge detection on a machining drawing, and extracting and screening drawing frames and drawing areas; detecting a text group in the drawing by adopting a YOLO-OBB algorithm; and performing discrete classification prediction on the rotation angle of the detected text group image. According to the method, text pixel loss caused by direct zooming is avoided through drawing preprocessing, the 360-degree text direction problem is solved by utilizing a YOLO-OBB algorithm and a four-classification neural network model, PMI information is aligned and recognized in combination with a special symbol model, global parameters are extracted through natural language processing, and finally errors are monitored and corrected. The problems of text pixel loss, insufficient direction processing, poor symbol and text association and the like in the prior art are solved.

The invention provides a method for automatically generating an intelligent safety risk control template, which belongs to the technical field of template automatic generation and comprises the following steps of: 1, executing a template matching and auditing queue admission process automatically generated by the intelligent safety risk control template; step 2, carrying out text clustering based on multi-dimensional features; step 3, carrying out cluster text sampling; 4, template extraction driven by a large model is carried out; 5, performing template to-be-audited and alarm notification; step 6, after template to-be-audited and alarm notification is carried out, manual auditing and template effectiveness are executed; and step 7, after manual auditing and the template take effect, performing auditing queue message processing and template multiplexing. By fusing the text clustering technology and the text understanding ability of a large language model, an intelligent security risk control strategy for automatically generating a template is provided, and the goals of ensuring violation interception, efficiently generating the template and having no interference on customer experience are achieved.