Text hotspot clustering method based on large model

Abstract

The invention provides a text hotspot clustering method based on a large language model. The method comprises the following steps: obtaining abstract output of a text from to-be-processed text data in a summarizing manner by utilizing the large language model; dividing the text abstract according to fields based on a large language model to obtain classification tags; performing element extraction on the text abstract based on a large language model to obtain element information; taking the text abstract as input, and using a vector embedding model to obtain semantic vector output; original text attributes, text abstracts, classification tags, element information and semantic vectors are stored in a vector library; a retrieval condition is formed according to business requirements, and a text candidate set is recalled from the vector library; and according to the text candidate set, obtaining a clustering result by utilizing a stage optimal clustering algorithm, taking the text abstract set of each cluster as input, obtaining a hotspot abstract summary based on a large language model, and integrating into a final hotspot result. According to the method, the efficiency and accuracy of large-scale dynamic text data analysis can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text hotspot clustering method based on a large model. Background Art

[0002] Text hotspot discovery is mainly based on unsupervised clustering algorithms. In large-scale text data analysis scenarios, discovering hidden patterns and regularities in data helps save data analysis costs and improve data analysis efficiency. The core concept of clustering algorithms is to group data points into several clusters so that the data points in the same cluster have high similarity, while the data points in different clusters have low similarity. Currently, the widely used clustering algorithms mainly include K-means, hierarchical clustering, spectral clustering, DBSCAN, etc. These methods are greatly affected by the settings of algorithm-related parameters. The multiple clustering results of the same batch of data are inconsistent due to the influence of random initial cluster centers. It is difficult to obtain stable and effective hotspot results for big data text analysis scenarios with fast dynamic changes.

[0003] To this end, the present invention provides a text hotspot clustering method based on a large model, proposes a stage optimal clustering algorithm, can obtain stable hotspot clustering results, realizes text multi-classification hotspot aggregation and generates hotspot title summaries based on the text classification and summary capabilities of a large language model, and is more conducive to the analysis and research of large-scale text data. Summary of the invention

[0004] The present invention aims to at least solve the technical problem in the prior art of how to cluster stable hot spot results in multiple fields and generate hot spot summary descriptions in large-scale dynamic text data analysis scenarios to improve the efficiency of large-scale text data analysis.

[0005] To this end, the present invention provides a text hotspot clustering method based on a large model.

[0006] The text hotspot clustering method based on a large model proposed by the present invention includes:

[0007] Use a large language model to summarize the text data to be processed to obtain a summary output of the text;

[0008] Based on the large language model, the text summary is divided into fields to obtain classification labels;

[0009] Extract elements from text summaries based on a large language model to obtain element information;

[0010] Take the text summary as input and use the vector embedding model to get the semantic vector output;

[0011] Store original text attributes, text summaries, classification labels, feature information and semantic vectors into the vector library;

[0012] Recall text candidate sets from the vector library based on search conditions formed according to business requirements;

[0013] According to the text candidate set, the clustering result is obtained by using the stage optimal clustering algorithm. The text summary set of each cluster is taken as input, and the hot spot summary is obtained based on the large language model. The summary is integrated into the final hot spot result to complete the text hot spot aggregation.

[0014] The text hotspot clustering method based on a large model according to the above technical solution of the present invention may also have the following additional technical features:

[0015] In the above technical solution, the process of using a vector embedding model to obtain a semantic vector output using a text summary as input includes:

[0016] The vector embedding model is used to calculate the vector dense layer retrieval score, sparse retrieval score and multi-level vector retrieval score of the text summary respectively, and the hybrid retrieval score is obtained as the semantic vector representation based on the weighted summation method.

[0017] In the above technical solution, the method for calculating the vector dense layer retrieval score includes:

[0018] Each character of the to-be-matched text q and the to-be-recalled text p is converted into a hidden layer vector by the text encoder, and the vector encoding of the first special character [CLS] of each text is regularized to represent the vector features of the entire input text:

[0019] e q =norm(H q [0])

[0020] e p =norm(H p [0])

[0021] Among them, H q Represents the vector set after the text sequence to be matched is transformed by the encoder, H q [0] represents the vector encoding of the special character [CLS] in the text to be matched, e q represents the output of the text to be matched after the vector regularization operation; norm(·) represents the regularization operation; H p H represents the vector set of the text sequence to be recalled after being transformed by the encoder, p [0] represents the vector encoding of the special characters [CLS] in the text to be recalled, e p Represents the output of the text to be recalled after the vector regularization operation;

[0022] Take the dot product of the two vectors to get the vector dense layer retrieval score:

[0023] s dense = <e p ,e q >

[0024] Among them, s dense represents the vector dense layer retrieval score, and <·> represents the dot product.

[0025] In the above technical solution, the method for calculating the sparse search score includes:

[0026] The importance weight of the character t in the sequence of the to-be-matched text q It is expressed as:

[0027]

[0028] The importance weight of character t in the sequence of the to-be-recalled text p It is expressed as:

[0029]

[0030] Among them, W lex ∈R d×1 represents the hidden layer weight distribution matrix of sparse query; Relu(·) represents the activation function; H q [i] represents the vector representation of the i-th character of the text to be matched; H p [j] indicates the jth text to be recalled

[0031] A vector representation of characters;

[0032] The sparse retrieval score between the text to be matched and the text to be recalled is:

[0033] s lex =∑ t∈q∩p (w qt *w pt )

[0034] Among them, s lex Represents the sparse retrieval score.

[0035] In the above technical solution, the method for calculating the multi-level vector retrieval score includes:

[0036] Based on the hidden layer output, the entire to-be-matched text q and the to-be-recalled text p are vectorized using an intermediate matrix that can be used for parameter learning:

[0037]

[0038] Among them, E qRepresents the vector representation of the text q to be matched; E p represents the vector representation of the text p to be recalled; norm(·) represents the regularization operation; represents a trainable multi-level vector query matrix;

[0039] The multi-level vector retrieval score between the text to be matched and the text to be recalled is:

[0040]

[0041] Where N represents the total number of characters in the text to be matched; i represents the number of characters in the text to be matched; M represents the total number of characters in the text to be recalled; j represents the number of characters in the text to be recalled; E q [i] represents the vector representation of the i-th character of the text q to be matched; E p [j] represents the jth character of the text p to be recalled.

[0042] In the above technical solution, the mixed search score is obtained as a semantic vector representation based on the weighted sum method, including:

[0043] The weighted sum of the vector dense layer retrieval score, sparse retrieval score, and multi-level vector retrieval score is obtained

[0044] Mixed search score:

[0045] s rank =w1.s dense +w2.s lex +w3.s mul

[0046] Among them, s rank represents the mixed retrieval score; w1 represents the weight of the vector dense layer retrieval score; s dense

[0047] represents the vector dense layer retrieval score; w2 represents the weight of the sparse retrieval score; s lex represents the sparse retrieval score; w3 represents the weight of the multi-level vector retrieval score; s mul Represents the multi-level vector retrieval score.

[0048] In the above technical solution, the clustering result is obtained by using the stage optimal clustering algorithm, and the text summary set of each cluster is used as input, and the hot spot summary is obtained based on the large language model, which is integrated into the final hot spot result to complete the text hot spot aggregation, including:

[0049] The text collection recalled from the vector library is used to construct a semantic vector matrix in index order, and the vector similarity between two texts is calculated using cosine similarity and matrix parallel operation to construct a similarity query matrix;

[0050] Based on the stage-optimal clustering algorithm, it focuses on the most valuable clusters in the current processed text collection.

[0051] In the above technical solution, the text set recalled from the vector library is used to construct a semantic vector matrix in index order, and the vector similarity between two texts is calculated by cosine similarity and matrix parallel operation to construct a similarity query matrix, including:

[0052] After embedding n texts, we get n k-dimensional vectors, which are constructed into a matrix format and expressed as:

[0053] M=[v1,v2,…,v x ,…,v n ]

[0054] Where M represents the semantic vector matrix; v x Represents the semantic vector corresponding to the x-th text, 1≤x≤n;

[0055] The similarity between two vectors in the vector matrix M is calculated using cosine similarity to obtain the similarity query matrix S, which is expressed as:

[0056]

[0057] Among them, s xy Represents the cosine similarity value between the semantic vector corresponding to the x-th text and the semantic vector corresponding to the y-th text, 1≤x≤n, 1≤y≤n.

[0058] In the above technical solution, the method for calculating the similarity between two vectors in the vector matrix M using cosine similarity is:

[0059]

[0060] Among them, v xg Represents vector v x The value of the gth dimension; v yg Represents vector v y The value of the g-th dimension.

[0061] In the above technical solution, the phase-based optimal clustering algorithm focuses on the most valuable clusters in the currently processed text collection, including:

[0062] Construct n texts into an array L to be clustered according to the index:

[0063] L=[d1,d2,…,d x ,…,d n ]

[0064] Among them, d xIndicates the index information corresponding to the xth text;

[0065] Sequentially cluster the text d in the array L to be clustered x As the cluster center, query the similarity value between the cluster center and other texts from the similarity query matrix S, and filter out the texts with similarity values ​​greater than the similarity association threshold θ and store them in the cluster array cluster x :

[0066] cluster x =[d x ,d x1 ,d x2 ,…]

[0067] Among them, cluster x Indicates that the text x is the cluster array of cluster centers, text d x1 ,d x2 With text d x The cosine similarity value of is greater than or equal to the similarity association threshold θ;

[0068] After clustering each text, we get the stage clustering set C, which is expressed as:

[0069] C=[cluser1,cluster2,…,cluster x ,…,cluster n ]

[0070] Count the number of clusters in each cluster array in the cluster set C in the stage, and filter out the cluster array cluster corresponding to the maximum value xmax As the optimal clustering result of this stage; if the maximum number of clusters is greater than or equal to the minimum cluster threshold, the cluster array cluster corresponding to the current maximum value is xmax Store the clustering result array, update the array to be clustered L, and store the clustering array cluster xmax The text index contained in is removed from L;

[0071] The updated array to be clustered continues to be clustered until the array to be clustered L is empty, or the number of clusters of the optimal clustering result is less than the minimum cluster threshold.

[0072] In summary, due to the adoption of the above technical features, the beneficial effects of the present invention are:

[0073] The present invention is based on large-model text processing and a stage-optimal hotspot clustering algorithm, which enables better performance in large-scale dynamic text data analysis.

[0074] Specifically, the method of the present invention introduces a semantic big model, proposes a stage-optimal clustering algorithm to focus on the dynamic large-scale text data analysis scenario, and improves the stability and value of hot spot clustering. Based on the big language model, complete the label classification, entity element extraction, and summary of text data, obtain the effective structured information of text data and remove the redundant parts, support the subsequent multi-angle data retrieval and corresponding hot spot analysis, so that the feature expression of the original text is clearer and the subsequent clustering effect is improved; based on the vector embedding model, the text summary generated by the big model is used as input to obtain a high-dimensional semantic vector feature expression, so that the vector distance of the same type of text in the feature space is closer, and the vector distance of different types of text is farther, which improves the clustering effect. The vector similarity query matrix is ​​calculated using cosine similarity and matrix operation, which saves computing time and reduces redundant calculations; based on the stage-optimal clustering algorithm, focus on the most valuable clusters in the current processing text set, which is in line with the large-scale dynamic text data analysis scenario and improves the recall ability of high-heat hot spots; based on the big language model, the summary text set of each cluster after clustering is used as input to obtain the summary of each hot spot set, which improves the analysis efficiency of analysts. At the same time, the semantic large model component and the vector embedding model in this method are used in the form of plug-ins and can be directly replaced. If the effects of the two are improved, the hotspot clustering effect of this method for text data analysis scenarios will also be improved accordingly.

[0075] Additional aspects and advantages of the present invention will become apparent from the following description or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0077] Figure 1 It is a flowchart of text structure preprocessing in a text hotspot clustering method based on a large model according to an embodiment of the present invention;

[0078] Figure 2 The present invention is a flowchart of text hotspot aggregation in a large model-based text hotspot clustering method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0079] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0080] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0081] Refer to the following Figure 1 to Figure 2 To describe the text hotspot clustering method based on a large model provided according to some embodiments of the present invention.

[0082] Some embodiments of the present application provide a text hotspot clustering method based on a large model.

[0083] The first embodiment of the present invention proposes a text hotspot clustering method based on a large model, including the following steps S1-S7.

[0084] S1. Use a large language model to summarize the text data to be processed to obtain a summary output of the text;

[0085] S2, based on the large language model, divide the text summary into fields to obtain classification labels;

[0086] S3, extracting elements from the text summary based on the large language model to obtain element information;

[0087] S4, using the text summary as input and using the vector embedding model to obtain the semantic vector output;

[0088] S5, storing original text attributes, text summary, classification label, element information and semantic vector into the vector library;

[0089] S6. Recalling a text candidate set from the vector library by forming search conditions according to business requirements;

[0090] S7. Based on the text candidate set, the clustering result is obtained by using the stage optimal clustering algorithm. The text summary set of each cluster is used as input, and the hot spot summary is obtained based on the large language model. The hot spot summary is integrated into the final hot spot result to complete the text hot spot aggregation.

[0091] Steps S1-S7 can be divided into two major processes, wherein steps S1-S5 are text structure preprocessing, and steps S6-S7 are text hotspot aggregation. In addition, the present disclosure uses the QWEN-72B semantic large model as an example for explanation of the large language model. It is understandable that those skilled in the art can also select other suitable large language models for application according to the content of the present disclosure.

[0092] Figure 1 The flowchart of text structure preprocessing is shown in FIG. 1 , which mainly includes prompt engineering and hybrid retrieval. Figure 1As shown in the figure, the text data to be processed is queried from the text database, and based on the QWEN-72B semantic large model, the summary output of the text is generated through the summary prompt project, and the redundant information of the text is removed to realize the feature enhancement of the text; the generated text summary is used as input, and the classification label of the text is obtained by using the text field classification prompt project, and the analysis dimension is expanded, so that the hot spot analysis of the text can be carried out according to different fields in the future; the generated text summary is used as input, and the entity element extraction prompt project is used to extract the time, place, participating subject and other element information contained in the text summary, and expand the hot spot analysis dimension; the text summary is converted into a high-dimensional semantic vector in the feature space through the BGE-M3 vector embedding model, so that the vector distance between the same type of text is closer, and the distance between different types of text is farther, so as to enhance the hot spot clustering effect; the original text and its attribute information, summary (text summary), classification label, time, place, main participants, semantic vector, etc. that have been produced are stored in the ES vector library to complete the text structured preprocessing.

[0093] Among them, text structured preprocessing can be divided into the prompt engineering represented by steps S1-S3 and the hybrid retrieval represented by steps S4-S5.

[0094] The prompt project uses a large language model that has been trained using pre-training and prompt learning. Task elements such as task context information, clear task goals or problems, precautions and restrictions, and specified output formats are combined in natural language to form the model's prompt input. The appropriateness of the prompt input can enhance the model's output results.

[0095] Hybrid retrieval includes using the BGE-M3 vector embedding model to calculate the vector dense layer retrieval score, sparse retrieval score and multi-level vector retrieval score of the text summary respectively, and obtaining the hybrid retrieval score as a semantic vector representation based on the weighted summation method.

[0096] Specifically, the method for calculating the vector dense layer retrieval score includes:

[0097] Each character of the to-be-matched text q and the to-be-recalled text p is converted into a hidden layer vector by the text encoder, and the vector encoding of the first special character [CLS] of each text is regularized to represent the vector features of the entire input text:

[0098] e q =norm(H q [0])

[0099] e p =norm(H p [0])

[0100] Among them, H qRepresents the vector set after the text sequence to be matched is transformed by the encoder, H q [0] represents the vector encoding of the special character [CLS] in the text to be matched, e q represents the output of the text to be matched after the vector regularization operation; norm(·) represents the regularization operation; H p H represents the vector set of the text sequence to be recalled after being transformed by the encoder, p [0] represents the vector encoding of the special characters [CLS] in the text to be recalled, e p Represents the output of the text to be recalled after the vector regularization operation;

[0101] Take the dot product of the two vectors to get the vector dense layer retrieval score:

[0102] s dense = <e p ,e q >

[0103] Among them, s dense represents the vector dense layer retrieval score, and <·> represents the dot product.

[0104] The calculation method of the sparse search score includes:

[0105] The importance weight of the character t in the sequence of the to-be-matched text q It is expressed as:

[0106]

[0107] The importance weight of character t in the sequence of the to-be-recalled text p It is expressed as:

[0108]

[0109] Among them, W lex ∈R d×1 represents the hidden layer weight distribution matrix of sparse query; Relu(·) represents the activation function; H q [i] represents the vector representation of the i-th character of the text to be matched; H p [j] represents the vector of the jth character of the text to be recalled.

[0110] The sparse retrieval score between the text to be matched and the text to be recalled is:

[0111] s 1ex =∑ t∈q∩p (w qt *w pt )

[0112] Among them, s lex Represents the sparse retrieval score.

[0113] The calculation method of the multi-level vector retrieval score includes:

[0114] Based on the hidden layer output, the entire to-be-matched text q and the to-be-recalled text p are vectorized using an intermediate matrix that can be used for parameter learning:

[0115]

[0116] Among them, E q Represents the vector representation of the text q to be matched; E p represents the vector representation of the text p to be recalled; norm(·) represents the regularization operation; represents a trainable multi-level vector query matrix;

[0117] The multi-level vector retrieval score between the text to be matched and the text to be recalled is:

[0118]

[0119] Where N represents the total number of characters in the text to be matched; i represents the number of characters in the text to be matched; M represents the total number of characters in the text to be recalled; j represents the number of characters in the text to be recalled; E q [i] represents the vector representation of the text q to be matched; E p [j] represents the vector representation of the text p to be recalled; T represents the transposed matrix.

[0120] In a specific embodiment, after the above calculation, the vector dense layer retrieval score, the sparse retrieval score, and the multi-level vector retrieval score are weighted and summed to obtain a hybrid retrieval score:

[0121] s rank =w1.s dense +w2.s lex +w3.s mul

[0122] Among them, s rank represents the mixed retrieval score; w1 represents the weight of the vector dense layer retrieval score; s dense represents the vector dense layer retrieval score; w2 represents the weight of the sparse retrieval score; s lex represents the sparse retrieval score; w3 represents the weight of the multi-level vector retrieval score; s mul Represents the multi-level vector retrieval score.

[0123] It is understandable that, in some cases, mixed search scores can also be obtained by direct summation or other methods. The above weighted summation is not the only way to combine vector dense layer search scores, sparse search scores, and multi-level vector search scores.

[0124] Figure 2 The flowchart of text hotspot aggregation is shown in Figure 1, which mainly includes two parts: similarity calculation and phase optimal clustering algorithm. Figure 2 As shown in the figure, query statements are generated according to business analysis requirements to recall text candidate sets from the ES vector library; semantic vector matrices are constructed from the recalled text sets in index order, and the vector similarities between two texts are calculated using cosine similarity and matrix parallel operations to construct a similarity query matrix. The subsequent clustering process only needs to query the similarity and no further calculation is required, saving computational cost; the index array of the text to be clustered is initialized according to the recalled text set; the optimal clustering algorithm is executed and the clustering results are output; based on the QWEN-72B large language model, the summary text sets corresponding to each cluster are taken as input in turn through the hotspot summary prompt project to generate summary summaries of each hotspot, which are integrated into the final hotspot results to complete text hotspot aggregation.

[0125] In some implementations, step S7 includes the following steps S71-S72.

[0126] S71. Construct a semantic vector matrix from the text set recalled from the vector library in index order, calculate the vector similarity between two texts using cosine similarity and matrix parallel operation, and construct a similarity query matrix.

[0127] S72. Based on the stage-optimal clustering algorithm, focus on the most valuable clusters in the currently processed text collection.

[0128] In a specific embodiment, step S71 includes:

[0129] S711. After performing vector embedding on n texts, the n k-dimensional vectors are constructed into a matrix format, which is expressed as:

[0130] M=[v1,v2,…,v x ,…,v n ]

[0131] Where M represents the semantic vector matrix; v x Represents the semantic vector corresponding to the x-th text, 1≤x≤n.

[0132] S712, using cosine similarity to calculate the similarity between two vectors in the vector matrix M to obtain a similarity query matrix S, which is expressed as:

[0133]

[0134] Among them, s xyRepresents the cosine similarity value between the semantic vector corresponding to the x-th text and the semantic vector corresponding to the y-th text, 1≤x≤n, 1≤y≤n.

[0135] Specifically, the method for calculating the similarity between two vectors in the vector matrix M using cosine similarity is:

[0136]

[0137] Among them, v xg Represents vector v x The value of the gth dimension; v yg Represents vector v y The value of the g-th dimension.

[0138] In a specific embodiment, step S72 includes:

[0139] S721, initialization: construct n texts into an array L to be clustered according to the index method:

[0140] L=[d1,d2,…,d x ,…,d n ]

[0141] Among them, d x Represents the index information corresponding to the x-th text; in this step, an empty array H can also be defined to store the final clustering results.

[0142] S722, sequentially cluster the text d in the array L to be clustered x As the cluster center, query the similarity value between the cluster center and other texts from the similarity query matrix S, and filter out the texts with similarity values ​​greater than the similarity association threshold θ and store them in the cluster array cluster x :

[0143] cluster x =[d x ,d x1 ,d x2 ,…]

[0144] Among them, cluster x Indicates that the text x is the cluster array of cluster centers, text d x1 ,d x2 With text d x The cosine similarity value of is greater than or equal to the similarity association threshold θ;

[0145] S723, after clustering each text, a stage clustering set C is obtained, which is expressed as:

[0146] C=[cluser1,cluster2,…,clusterx ,…,cluster n ]

[0147] Count the number of clusters in each cluster array in the cluster set C in the stage, and filter out the cluster array cluster corresponding to the maximum value xmax As the optimal clustering result of this stage; if the maximum number of clusters is greater than or equal to the minimum cluster threshold, the cluster array cluster corresponding to the current maximum value is xmax Store the clustering result array H, update the array to be clustered L, and store the clustering array cluster xmax The text index contained in is removed from L;

[0148] S724, repeat the above steps S722-S723 until the array L to be clustered is empty, or the number of clusters of the optimal clustering result is less than the minimum cluster threshold.

[0149] In a specific embodiment, the similarity association threshold θ is set to 0.6. xy ≥θ means that text x and text y belong to the same hotspot cluster; set the minimum cluster threshold c=6, if the number of clusters is greater than c, it is a valid hotspot.

[0150] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0151] Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A text hotspot clustering method based on a large model, characterized in that: include: Use a large language model to summarize the text data to be processed to obtain a summary output of the text; Based on the large language model, the text summary is divided into fields to obtain classification labels; Extract elements from text summaries based on a large language model to obtain element information; Take the text summary as input and use the vector embedding model to get the semantic vector output; Store original text attributes, text summaries, classification labels, feature information and semantic vectors into the vector library; Recall text candidate sets from the vector library based on search conditions formed according to business requirements; According to the text candidate set, the clustering result is obtained by using the stage optimal clustering algorithm. The text summary set of each cluster is taken as input, and the hot spot summary is obtained based on the large language model. The summary is integrated into the final hot spot result to complete the text hot spot aggregation.

2. The text hotspot clustering method based on a large model according to claim 1, characterized in that: The method of using a vector embedding model to obtain a semantic vector output by taking a text summary as input includes: The vector embedding model is used to calculate the vector dense layer retrieval score, sparse retrieval score and multi-level vector retrieval score of the text summary respectively, and the hybrid retrieval score is obtained as the semantic vector representation based on the weighted summation method.

3. The text hotspot clustering method based on a large model according to claim 2, characterized in that: The method for calculating the vector dense layer retrieval score includes: Each character of the to-be-matched text q and the to-be-recalled text p is converted into a hidden layer vector by the text encoder, and the vector encoding of the first special character [CLS] of each text is regularized to represent the vector features of the entire input text: e p =norm(H p [0]) e p =norm(H p [0]) Among them, H p Represents the vector set after the text sequence to be matched is transformed by the encoder, H q [0] represents the vector encoding of the special character [CLS] of the text to be matched, e q represents the output of the text to be matched after the vector regularization operation; norm(·) represents the regularization operation; H p H represents the vector set of the text sequence to be recalled after being transformed by the encoder, p [0] represents the vector encoding of the special characters [CLS] in the text to be recalled, e p Represents the output of the text to be recalled after the vector regularization operation; Take the dot product of the two vectors to get the vector dense layer retrieval score: s dense = <e p ,e q > Among them, s dense represents the vector dense layer retrieval score, and <·> represents the dot product.

4. The text hotspot clustering method based on a large model according to claim 3 is characterized in that: The calculation method of the sparse search score includes: The importance weight w of the character t in the sequence of the to-be-matched text q qt It is expressed as: The importance weight w of the character t in the to-be-recalled text p in the sequence pt It is expressed as: Among them, W lex ∈R d×1 represents the hidden layer weight distribution matrix of sparse query; Relu(·) represents the activation function; H q [i] represents the vector representation of the i-th character of the text to be matched; H p [j] represents the vector representation of the jth character of the text to be recalled; The sparse retrieval score between the text to be matched and the text to be recalled is: s lex =∑ t∈q∩p (w pt *w pt ) Among them, s lex Represents the sparse retrieval score.

5. The text hotspot clustering method based on a large model according to claim 4 is characterized in that: The calculation method of the multi-level vector retrieval score includes: Based on the hidden layer output, the entire to-be-matched text q and the to-be-recalled text p are vectorized using an intermediate matrix that can be used for parameter learning: Among them, E q Represents the vector representation of the text q to be matched; E p represents the vector representation of the text p to be recalled; norm(·) represents the regularization operation; represents a trainable multi-level vector query matrix; The multi-level vector retrieval score between the text to be matched and the text to be recalled is: Where N represents the total number of characters in the text to be matched; i represents the number of characters in the text to be matched; M represents the total number of characters in the text to be recalled; j represents the number of characters in the text to be recalled; E q [i] represents the vector representation of the i-th character of the text q to be matched; E p [j] represents the vector representation of the jth character of the text p to be recalled; T represents the transposed matrix.

6. The text hotspot clustering method based on a large model according to claim 5 is characterized in that: The method of obtaining the mixed search score as a semantic vector representation based on weighted summation includes: The weighted sum of the vector dense layer retrieval score, sparse retrieval score, and multi-level vector retrieval score is used to obtain the hybrid retrieval score: s rank =w1.s dense +w2.s lex +w3.s mul Among them, s rank represents the mixed retrieval score; w1 represents the weight of the vector dense layer retrieval score; w2 represents the weight of the sparse retrieval score; w3 represents the weight of the multi-level vector retrieval score.

7. The text hotspot clustering method based on a large model according to claim 1 is characterized in that: The clustering result is obtained by using the optimal clustering algorithm in the stage, and the text summary set of each cluster is used as input. The hotspot summary is obtained based on the large language model, and the hotspot summary is integrated into the final hotspot result to complete the text hotspot aggregation, including: The text collection recalled from the vector library is used to construct a semantic vector matrix in index order, and the vector similarity between two texts is calculated using cosine similarity and matrix parallel operation to construct a similarity query matrix; Based on the stage-optimal clustering algorithm, it focuses on the most valuable clusters in the current processed text collection.

8. The text hotspot clustering method based on a large model according to claim 7 is characterized in that: The text set recalled from the vector library is constructed into a semantic vector matrix according to the index order, and the vector similarity between two texts is calculated by using cosine similarity and matrix parallel operation to construct a similarity query matrix, including: After embedding n texts, we get n k-dimensional vectors, which are constructed into a matrix format and expressed as: M=[v1,v2,…,v x ,…,v n ] Where M represents the semantic vector matrix; v x Represents the semantic vector corresponding to the x-th text, 1≤x≤n; The similarity between two vectors in the vector matrix M is calculated using cosine similarity to obtain the similarity query matrix S, which is expressed as: Among them, s xy Represents the cosine similarity value between the semantic vector corresponding to the x-th text and the semantic vector corresponding to the y-th text, 1≤x≤n, 1≤y≤n.

9. The text hotspot clustering method based on a large model according to claim 8, characterized in that: The method for calculating the similarity between two vectors in the vector matrix M using cosine similarity is as follows: Among them, v xg Represents vector v x The value of the gth dimension; v yg Represents vector v y The value of the g-th dimension.

10. The text hotspot clustering method based on a large model according to claim 8, characterized in that: The stage-based optimal clustering algorithm focuses on the most valuable clusters in the current processed text collection, including: Construct n texts into an array L to be clustered according to the index: L=[d1,d2,…,d x ,…,d n ] Among them, d x Indicates the index information corresponding to the xth text; Sequentially cluster the text d in the array L to be clustered x As the cluster center, query the similarity value between the cluster center and other texts from the similarity query matrix S, and filter out the texts with similarity values ​​greater than the similarity association threshold θ and store them in the cluster array cluster x : cluster x =[d x ,d x1 ,d x2 ,…] Among them, cluster x Indicates that the text x is the cluster array of cluster centers, text d x1 d x2 With text d x The cosine similarity value of is greater than or equal to the similarity association threshold θ; After clustering each text, we get the stage clustering set C, which is expressed as: C=[cluser1,cluster2,…,cluster x ,…,cluster n ] Count the number of clusters in each cluster array in the cluster set C in the stage, and filter out the cluster array cluster corresponding to the maximum value xmax As the optimal clustering result of this stage; if the maximum number of clusters is greater than or equal to the minimum cluster threshold, the cluster array cluster corresponding to the current maximum value is xmax Store the clustering result array, update the array to be clustered L, and store the clustering array cluster xmax The text index contained in is removed from L; The updated array to be clustered continues to be clustered until the array to be clustered L is empty, or the number of clusters of the optimal clustering result is less than the minimum cluster threshold.

Images