141 results about "Text cluster" patented technology

A computer-implemented method for generating a fixed-size N-dimensional vector representation for a given document is disclosed. The method comprises extracting text-portions from a plurality of documents, embedding the extracted text-portions into fixed-sized K-dimensional text-portion vectors, clustering the text-portion vectors into N clusters C_1, C_2, . . . , C_N, generating an N-dimensional document vector E(D) for a document D by (i) associating its nth coordinate value E(D)_n to the nth cluster C_n, and (ii) if not previously done for the document D extracting text-portions from the document D and embedding the extracted text-portions into K-dimensional text-portion vectors, and (iii) setting the values E(D)_n based on similarity matching score values between the text-portion vectors of the document D and text-portions vectors of the N clusters.

Apparatuses, systems, and techniques generating one or more cluster performance evaluation metrics that allow for evaluation of the performance of an unsupervised natural language processing clustering algorithms to be used with unlabeled data. At least one embodiment pertains to methods of generating one or more cluster performance evaluation metrics based, at least in part, on one or more vectors generated by one or more neural networks to indicate a relationship among members of one or more clusters of data generated using the one or more data clustering algorithms, according to various novel techniques described herein.

The invention aims to provide a work order text processing method and equipment, and the method comprises the steps: obtaining a preset number of work order texts related to complaints, carrying out the data preprocessing of the preset number of work order texts, and obtaining a preset number of preprocessed work order texts, the preset number comprises at least two; text clustering is conducted on the preset number of preprocessed work order texts, a comprehensive text cluster corresponding to each appeal in different appeals is obtained, and each comprehensive text cluster comprises at least one preprocessed work order text; performing keyword extraction operation and subject term generation operation on the comprehensive text cluster corresponding to each appeal to obtain keywords and subject terms of the comprehensive text cluster corresponding to each appeal; according to the method, the keyword and the subject term of the comprehensive text cluster corresponding to each appeal are displayed in a visual form, so that important information in the work order text about the complaint is simply and efficiently mined, and the keyword and the subject term are displayed in the visual form.

The embodiment of the invention belongs to the field of artificial intelligence and text hierarchical clustering, and relates to a text clustering method, which comprises the following steps of: obtaining credit review text data, performing multi-layer clustering processing on the credit review text data to obtain a clustering topic set and a clustering text set, inputting the clustering topic set and the clustering text set into a preset large language model, and generating a credit title set, the credit title set comprising a plurality of levels of credit titles, and performing clustering verification on the credit titles of all levels in the credit title set to obtain a credit clustering relation. The invention further provides a text clustering device, computer equipment and a storage medium. According to the method, the manual intervention requirement is reduced through automatic clustering level judgment, meanwhile, the clustering title is automatically generated through the large language model, the labor amount of manual naming is reduced, and the accuracy of clustering and title generation is ensured through clustering verification.

Implementations generally relate to text clustering with heuristic and multi-metric control. In some implementations, a method includes receiving an electronic source document containing text. The method further includes dividing the text into text units, encoding the text units, and transforming the text units into numerical values. The method further includes generating a graph of the text units based on the numeric values, where the graph includes nodes corresponding to the text units and edges corresponding to pairs of the text units. The method further includes ordering the text units into text clusters based on the graph of the text units. The method further includes generating an electronic target document that presents the text clusters based on one or more preference heuristics.

The invention provides a text clustering system for open source security intelligence. The text clustering system comprises a data acquisition module, a preprocessing module, a feature extraction module, a similarity calculation module, a clustering initialization module, an iterative clustering module, an evaluation module and a result output module. The system collects a target text, removes noise, calculates a word frequency and an inverse document frequency to construct a feature vector, calculates similarity after normalization processing, randomly selects an initial clustering center, iteratively updates the clustering center until the clustering center is stable, evaluates a clustering effect, outputs a clustering result and generates a topic tag. According to the invention, the analysis efficiency and accuracy of open source security intelligence can be improved, and security threats can be quickly identified and responded.

The invention discloses an event determination method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: clustering a plurality of obtained texts to obtain a plurality of candidate text clusters; in the multiple candidate text clusters, the candidate text clusters with the number of texts meeting a first number condition are determined as target text clusters, and events associated with the multiple texts in the target text clusters belong to target type events; determining weights of the plurality of texts in the target text cluster based on the interaction information of the plurality of texts in the target text cluster; based on the multiple texts in the target text cluster and the weights of the multiple texts in the target text cluster, an event description text is generated, the semantics of the event description text is the same as the semantics of the multiple texts in the target text cluster, and the event description text is used for describing target type events associated with the multiple texts in the target text cluster. According to the method, computing resources can be saved, and the interpretability of the event determination method can be improved.

The invention discloses a text clustering method and device, a storage medium and a product, and relates to the technical field of natural language processing, and the text clustering method comprises the following steps: obtaining a to-be-clustered text; performing text clustering on the to-be-clustered text according to a plurality of preset clustering parameters to obtain text clustering results corresponding to the plurality of preset clustering parameters; inputting the text clustering results corresponding to the plurality of preset clustering parameters into a pre-trained text clustering quality evaluation model to obtain quality evaluation results corresponding to the plurality of preset clustering parameters; and determining a target text clustering result according to the quality evaluation index values corresponding to the plurality of preset clustering parameters. According to the scheme, iterative optimization of the text clustering result can be realized, so that the quality of the text clustering result is improved.

The present application provides a text clustering method, apparatus, electronic device and computer-readable storage medium. Based on anchor words corresponding to the full-text semantics of each training text, anchor word model features corresponding to the anchor words are obtained to avoid the introduction of additional noisy features. Then, a first clustering result and a second clustering result of each training text are obtained based on the anchor word model features, and a self-training objective function of multiple training texts is determined based on each first clustering result and each second clustering result, as well as a self-training target value of the self-training objective function. Finally, the text clustering model is updated based on the self-training target value until the text clustering model converges, and the converged text clustering model is applied to text clustering. After continuous training until the convergence of the text clustering model, the accuracy and stability of the text clustering model are continuously improved, and the accuracy of text clustering is avoided from being affected by additional noisy features.

The present invention discloses a method and device for text data clustering, and relates to the field of computer technology. A specific implementation of the method includes: obtaining batch text data, and determining a feature word set of each text data in the batch text data; for the feature word set of each text data, determining the weight of each feature word in the feature word set; sorting the batch text data according to the weight of the feature word in the feature word set; and performing clustering calculation on the batch text data based on the sorting result. The method rearranges the order of text clustering using feature weights, and can preferentially cluster text data containing rich information into topic classes, and then perform text clustering based on the clustered topic classes, which can improve the accuracy of clustering.

The invention discloses a synthetic data comprehensive evaluation method based on a large model, and the method comprises the steps: obtaining target synthetic data of the large model, and obtaining target image data and target text data; inputting the target image data into an image machine learning model to obtain an image standard quality grade, inputting the target text data into a text machine learning model to obtain a text standard quality grade, and obtaining a first quality grade; obtaining the number of image clusters and the number of text clusters by utilizing clustering analysis, and obtaining a second quality grade; obtaining similar historical synthetic data and similar data utility, and obtaining a third quality grade according to the similar data utility; according to the first quality grade, the second quality grade and the third quality grade, obtaining a comprehensive quality grade of the target synthetic data; quality evaluation is carried out on the synthetic data from multiple aspects, the comprehensiveness and accuracy of the evaluation result are guaranteed, similar data utility is added into the evaluation system of the synthetic data, and the data utility of the synthetic data is guaranteed.

The present invention discloses a method for clustering unknown protocol texts based on reinforcement learning. The method is based on the static parsing technology route based on network tracking in protocol reverse engineering, adopts the reinforcement learning algorithm DQN to self-learn the classification of hexadecimal network traffic texts, uses CNN for feature extraction, and combines one-dimensional action vectors as inputs to calculate Q values. At the same time, the discreteness and randomness of the traffic texts are combined to simplify the experience playback steps in the reinforcement learning framework, taking into account both the reduction of the amount of calculation and the simplification of the structure. The present invention realizes the source classification of network traffic texts using unknown protocols from multiple different application sources, and has a high classification accuracy and good stability.

The embodiment of the present invention discloses a method, device, equipment and storage medium for extracting text labels. The method comprises: obtaining each text of the label to be extracted, and vectorizing each of the texts to obtain a text vector corresponding to the corresponding text; clustering each of the text vectors to obtain at least one text clustering result; extracting keywords from each of the text clustering results to obtain each label candidate word corresponding to each of the text clustering results; and determining the text label of each of the texts according to each of the label candidate words corresponding to each of the text clustering results. Through the above technical solution, the automatic extraction of text labels is realized, and the accuracy and comprehensiveness of text label extraction and the scalability of the label extraction method are improved.

The invention provides a text clustering method and device, computer equipment and a medium. The text clustering method comprises the steps of obtaining a first score according to feature complexity of text data in a data set; obtaining a second score according to the semantic complexity of the text data; classifying the data set based on the first score and the second score to obtain a target data set; and carrying out clustering on the target data set. Text data in a data set is classified according to feature complexity and semantic complexity of the text data to obtain a target data set, and then the target data set is clustered. Through the method provided by the invention, the clustering task can be simplified, and the clustering efficiency and accuracy can be improved.

Embodiments of the present application provide a text processing method, device, equipment, storage medium and program product. The method comprises: obtaining a target text; processing the target text based on a vector dictionary to obtain a target vector; and processing the target vector based on a target task model to obtain a text processing result. In the embodiments of the present application, the target text is processed by the pre-constructed vector dictionary to obtain the target vector, and then the target vector is processed by the target task model. Since the target vector includes a semantic feature vector and a local difference vector, the accuracy of text processing can be improved. Moreover, the target task model is any one of a text classification model, a text clustering model and a sentiment analysis model in the call center customer service field. Compared with the use of a general large model, the efficiency of text processing can be improved.

The invention discloses a Chinese handwritten form detection method and system based on a text kernel and text clustering. The method comprises the following steps: acquiring a Chinese handwritten form text image; extracting Chinese handwritten text feature maps, and adjusting the number of channels to unify the number of channels of each feature map; the deep layer features are gradually restored to a higher resolution ratio through up-sampling operation, and the deep layer features are fused with shallow layer features of corresponding scales; dynamically fusing feature maps of different scales; learning similarity vectors among text pixels, and aggregating the text pixels to corresponding text kernels; using the predicted similarity vector to guide the aggregation of text pixels; and adding the text region loss, the text kernel loss and the text clustering loss into model training, and carrying out iteration by utilizing a gradient descent algorithm until the maximum number of iterations is reached or the model is converged. According to the technical scheme, the text kernel prediction technology and the dynamic pixel clustering technology are fused, and the text detection precision and robustness are effectively improved.

This application relates to the field of computer technology and discloses a method, apparatus, device, and storage medium for cross-page test question aggregation. The method includes acquiring multiple test question images, each image including test question elements; extracting text blocks from the test question images and grouping text blocks belonging to the same test question and the same test question element into a text cluster, obtaining a text cluster set; searching for a baseline text cluster and multiple candidate text clusters in the text cluster set, where candidate text clusters are those related to the baseline text cluster; generating label codes for each text cluster in the baseline and candidate text clusters based on the test question element to which the text cluster belongs, the text blocks within the text cluster, and the position of the text blocks in the test question images; and searching for text clusters belonging to the same test question as the baseline text cluster in the candidate text clusters based on the label codes, obtaining a test question text cluster set. The method of this application can achieve automatic clustering of test question elements with high clustering accuracy.

The application provides a keyword reverse propagation algorithm based on a citation network structure, and comprises the following steps: a spring charge model is established, force-directed layout processing is performed, and a force-directed layout graph is established; a keyword propagation model is established by using a reverse propagation algorithm, and a keyword weight change contrast curve is obtained; a citation network model is constructed by using the force-directed layout graph; iterative calculation is performed on the force-directed layout graph until the energy state in the force-directed layout graph reaches a minimum value; while the citation network is iteratively calculated, the keyword weight in the citation network model is adjusted, and a converged citation network layout graph is calculated. The application has the beneficial effects that the entire network is taken as a main body, keywords owned by cited documents are selected at a certain probability and are propagated backward along the network to the citing documents, the weight of the keywords is increased while the text clustering idea is retained, and clear visual data effects are obtained through force-directed layout.

The application relates to a text clustering method based on an improved whale optimization algorithm, and belongs to the fields of big data mining and machine learning. The method comprises the following steps: S1: obtaining scientific research text data from a scientific research text database by using Spark, and performing data cleaning; S2: performing word segmentation, stop word removal, feature selection, vectorization and dimension reduction operations on the text data after data cleaning, so that unstructured text data is converted into structured numerical data; and S3: calculating initial clustering centers by using a K-means algorithm, optimizing the clustering centers by using an improved whale optimization algorithm, and outputting clustering results. The global optimization capability of the improved whale optimization algorithm is utilized, the problem that current text clustering algorithms are prone to falling into local optimization is solved, the accuracy of clustering of scientific research text data is improved, the data dimension is reduced, and the clustering effect is enhanced.

The embodiment of the invention provides a text classification method, electronic equipment, a storage medium and a program product. The method comprises the following steps: processing a to-be-classified text according to a lexical element division rule to obtain a lexical element sequence; according to the lexical element sequence, adopting a pre-trained unsupervised topic model to obtain a topic distribution data set corresponding to the to-be-classified text; adopting a pre-trained unsupervised clustering model to obtain a clustering distribution data set corresponding to the to-be-classified text; splicing the topic distribution data set and the clustering distribution data set to obtain a text hidden topic; obtaining a plurality of text clusters and current cluster hidden topics corresponding to the text clusters, and calculating the similarity between the text hidden topics and the current cluster hidden topics; and determining a target text cluster from the plurality of text clusters, and adding the to-be-classified text to the target text cluster. Through a cluster classification mechanism of subject distribution and cluster distribution conjoint analysis, the accuracy and result stability of text classification are improved.

The application provides a cross-modal retrieval method based on prototype regularization learning, and belongs to the field of artificial intelligence and multi-modal information processing, and comprises the following steps: extracting visual features and text features through a visual encoder and a text encoder and calculating a basic contrast loss; dividing visual clusters and text clusters through a clustering algorithm and calculating visual prototypes and text prototypes, selecting text features most similar to the visual prototypes as anchor points, and constructing cross-modal prototypes; calculating a prototype-level discriminant loss according to the visual prototypes and the text prototypes, calculating an instance-level discriminant loss according to the visual clusters and the text clusters, and calculating a prototype projection loss according to the cross-modal semantic prototypes; constructing a joint optimization objective function by combining all the losses, training the visual encoder and the text encoder, and performing cross-modal retrieval after the training is completed. The application solves the problems of excessive intra-class variance and excessively high inter-class similarity caused by appearance bias in the existing cross-modal retrieval method, and causes the problems of insufficient retrieval accuracy and robustness.

The embodiment of the invention provides an artificial intelligence-driven public place risk dynamic early warning method and device, equipment and a medium. The method comprises the steps of collecting text release information associated with a target place, and performing text clustering on the text release information to obtain a target clustering result; analyzing sub-class clusters in the target clustering result to obtain related feature information; performing topic identification on the target clustering result according to the related feature information to obtain an initial topic and related text information; performing association analysis according to the related text information and the target place to obtain a target topic and associated text information; performing behavior detection on the associated text information to obtain a target behavior type; when the target behavior type meets a preset type, collecting latest text information associated with the target place; performing popularity analysis on the target topic according to the latest text information and the related text information to obtain a target popularity type; and performing risk early warning on the target place according to the target popularity type and the target behavior type to obtain a target early warning result.

The application provides a text clustering method, a text clustering device, an electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. An initial clustering center is selected from a plurality of original sentence embedding vectors, an initial problem text class is constructed based on the initial clustering center, clustering processing is performed on all original sentence embedding vectors based on the initial clustering center, a plurality of intermediate problem text classes are obtained, a clustering center is identified for each intermediate problem text class, a text clustering center is obtained, inter-class dispersion between the intermediate problem text classes and intra-class dispersion of each intermediate problem text class are calculated according to the text clustering center, the intermediate problem text classes are adjusted in text according to the inter-class dispersion and the intra-class dispersion, a target problem text class is obtained, semantic analysis is performed on all original sentence embedding vectors of the target problem text class, a standard problem text is obtained, and standard reply content is determined according to the standard problem text, so that the accuracy of problem answering can be improved.

Implementations generally relate to text clustering with heuristic and multi-metric control. In some implementations, a method includes receiving an electronic source document containing text. The method further includes dividing the text into text units, encoding the text units, and transforming the text units into numerical values. The method further includes generating a graph of the text units based on the numeric values, where the graph includes nodes corresponding to the text units and edges corresponding to pairs of the text units. The method further includes ordering the text units into text clusters based on the graph of the text units. The method further includes generating an electronic target document that presents the text clusters based on one or more preference heuristics.

The invention provides a text clustering processing method, a text classification method and a related device. The method comprises the following steps: clustering first texts in a first text set into a plurality of text clusters; performing text coding on the first text in each text cluster to obtain a coding result of the text cluster; determining intra-cluster compactness of the text clusters based on the coding result of each text cluster; based on the coding result of the first text cluster and the coding result of the second text cluster, determining an inter-cluster separation degree between the first text cluster and the second text cluster; the first text cluster and the second text cluster are any two text clusters in the plurality of text clusters; and determining a first clustering index value of the first text set based on the intra-cluster compactness and the inter-cluster separation degree. Through the method and the device, the accuracy of text clustering processing can be improved.

The present invention relates to a cross-border ethnic text clustering method and device that integrates domain knowledge graphs, and belongs to the field of natural language processing technology. Cross-border ethnic domain texts have their own unique cultural domain vocabulary. In order to solve the problem that the cultural background of cross-border ethnic cultural texts is missing and the cultural differences between different ethnic groups are large, resulting in poor text clustering effect, the present invention proposes a cross-border ethnic cultural text clustering method that integrates domain knowledge graphs, which mainly includes a cross-border ethnic cultural text data preprocessing part, a cross-border ethnic text data feature extraction part, and a cross-border ethnic cultural text clustering structure. According to the functional modularization of these three parts, a cross-border ethnic cultural text clustering device that integrates domain knowledge graphs is made. Compared with the general text clustering method, the present invention effectively alleviates the problem of poor cross-border ethnic cultural text clustering effect.

The application relates to a clustering-based text supervision checking method and device and a computer device. Feature word sets corresponding to a plurality of question description texts in a question list are obtained by respectively extracting feature words from the question description texts. The word frequency of each feature word in the question description text, the first inverse document frequency in all question description texts and the second inverse document frequency in all checked units are respectively calculated to obtain a feature word weight set of the question description text of each checked unit. A plurality of question description text clusters and a plurality of initial high-correlation-degree feature word sets corresponding to the question description text clusters are obtained through clustering. Finally, the initial high-correlation-degree feature word set is optimized according to a chi-square statistic to obtain an accurate high-correlation-degree feature word set. The method saves the calculation resources while ensuring the accuracy of the results by optimizing the feature word weight calculation, adopting a clustering algorithm for preliminary clustering and optimizing the clustering results through chi-square statistics.

The application belongs to the technical field of data mining, and specifically discloses a cross-field technology fusion opportunity finding method based on science and technology fusion interaction, which comprises the following steps: obtaining scientific paper and technology patent data, identifying cross-field scientific fusion and technology fusion based on semantic anchor points and a similarity threshold; using a trained deep text clustering model to respectively perform deep clustering on scientific fusion and technology fusion, and obtaining corresponding clustering sets; obtaining text representation corresponding to each cluster, filtering through a threshold, and establishing association between two types of clusters through bipartite graph maximum weight matching to identify shared topics and exclusive topics; using a dynamic time warping method to align the time series of scientific and technological trends, judging topics with scientific convergence preceding technological convergence, and identifying technology fusion opportunities with growth potential based on the development trend of scientific papers. The application can deeply mine cross-field science and technology semantics, improve clustering accuracy, quantify science and technology association, and dynamically find forward-looking technology fusion opportunities.

Methods for management of alarm text data for received alarms, which can involve processing the alarm text data for clustering to generate processed alarm text data; clustering the processed alarm text data into first clusters through executing a first clustering algorithm, the first clustering algorithm configured to be facilitated via accelerated processing; for the first clusters not meeting a metric, executing a second clustering algorithm on the first clusters to form second clusters that are representative of subgroups of the first clusters with improved homogeneity, the second clustering algorithm being different from the first clustering algorithm; executing a tagging process on the second clusters and ones of the first clusters that meet the metric to tag the second clusters and the ones of the first clusters that meet the metric into groups to form tagged cluster groups; and providing the tagged cluster groups to be displayed on the user interface.

The present invention provides a text clustering analysis system and method, which relates to the technical field of computer natural language processing. The target sentence is obtained through a feature representation module, and the text data set in the target sentence is converted into a primary text vector data set and a secondary text vector data set; a first feature representation and a first text distribution probability value are generated according to the primary text vector data set, and a second feature representation and a second text distribution probability are generated according to the secondary text vector data set to generate a feature data group; the clustering analysis module determines the straight-line distance between the high-dimensional vector and the frequency vector according to the feature data group, and combines the preset classification criteria to determine the target sample cluster; the evaluation and optimization module performs quality analysis on the target sample cluster, and outputs the optimal sample cluster according to the quality analysis result. The technical effect of small clustering analysis error and high accuracy is achieved.