Financial news theme modeling method based on multi-level comparative learning

Abstract

The invention provides a financial news topic modeling method based on multi-level comparative learning, which applies a comparative learning fine tuning task to the financial news topic modeling method, adopts a multi-level comparative learning method, and adjusts vector representation generated by a pre-training language model to obtain a multi-level comparative learning model. According to the method, the generated text vector representation is more suitable for the topic modeling task, and the topic modeling is carried out through the community discovery algorithm, so that the number of topics does not need to be appointed in advance, various hyper-parameters do not need to be set in advance, the application is more convenient, and the influence of the setting of the hyper-parameters on the topic modeling result is also avoided. Through experimental tests of the inventor, compared with an existing topic modeling method, when the method provided by the invention is applied to a financial news text data set, the two indexes of topic consistency and topic diversity of the topic obtained through modeling are improved to a great extent.

Description

Technical Field

[0001] The present invention belongs to the technical field of text topic mining, and in particular relates to a financial news topic modeling method based on multi-level contrastive learning. Background Art

[0002] With the development of the internet and the economy, people have access to a vast amount of financial news online. Some of this news is crucial for investors in specific financial sectors or researchers in related fields. However, due to the sheer volume of financial news, it's impossible for these individuals to access all of it. Furthermore, these individuals are often more interested in news in specific areas, often encompassing more than just the sections listed on news platforms. Furthermore, some news that interests these individuals is focused on specific events, which may appear across multiple sections on news platforms. Topic modeling can be used to categorize financial news into topics at varying granularity levels. A topic is a collection of news from a specific niche or event, helping them better understand financial events occurring within a specific timeframe.

[0003] The most widely recognized classic topic model in academia and industry is the LDA topic model, proposed in 2003. The LDA topic model is a probabilistic generative topic model that assumes that text generation begins by probabilistically selecting a topic from a topic distribution, then probabilistically selecting words from a topic word distribution, and then generating each word individually to form a document. The unique properties of the Dirichlet distribution allow this model to be mathematically interpretably approximated and inferred. Over the past two decades, this model has been continuously refined. Numerous experts and scholars have proposed generative topic models based on the LDA topic model, such as the dynamic LDA topic model and the hierarchical LDA topic model, for application in diverse fields and adaptable to different corpora.

[0004] BERTopic is a topic model based on a pre-trained large language model proposed by Maarten Grootendorst. The specific approach is to convert documents into vectors using a large language model, reduce the dimensionality of the vectors, cluster them, and then extract the words with the highest relative frequency in each category as topic representations. Each cluster naturally becomes a topic. BERTopic improves some of the shortcomings of the LDA topic model. For example, the number of topics is no longer a hyperparameter that needs to be set, but is naturally given according to the number of clusters; it can have better representation for low-frequency words; it is no longer a bag-of-words model, and word order is also taken into consideration. However, BERTopic also has some defects. For example, the use of traditional clustering methods often requires setting multiple hyperparameters, and the setting of hyperparameters will seriously affect the results of topic modeling. In addition, the traditional pre-trained language model is used for vectorized representation of news, and it is not optimized for the specific task of topic modeling.

[0005] However, existing topic modeling methods for financial news still have some shortcomings. The main manifestations are that traditional methods require researchers to specify the number of topics in advance, and some clustering-based methods require too many pre-specified hyperparameters, which is time-consuming and labor-intensive for researchers; and methods based on pre-trained language models do not optimize the specific task of topic modeling when using pre-trained language models to vectorize financial news. Therefore, the matching degree between the generated vectorized representation and the topic modeling options is still insufficient. For example, using existing pre-trained language models to vectorize financial news text datasets and using indicators such as topic consistency and topic diversity to evaluate the generated vectorized representation, it is found that these indicators are still not ideal. Summary of the Invention

[0006] This invention addresses the aforementioned issues. It uses a community discovery algorithm for topic modeling, avoiding the need to predefine the number of topics and numerous hyperparameter settings. It innovatively improves document vector generation for topic modeling through a multi-level contrastive learning fine-tuning task. Specifically, the invention employs the following technical solutions:

[0007] The present invention provides a financial news topic modeling method based on multi-level contrastive learning, which has the following technical features: step S1, respectively performing text preprocessing and word segmentation processing on the financial news text required for topic modeling and the news text used for training the contrastive learning network to obtain two corresponding processed data sets; step S2, using the data set corresponding to the news text used for training the contrastive learning network to fine-tune the pre-trained language model; step S3, using the fine-tuned pre-trained language model to generate a vector representation of the pre-processed financial news document; step S4, establishing a document network based on the vector representation, and using a community discovery algorithm to perform community discovery on the document network to obtain multiple communities, each of which is a topic; step S5, using a word frequency-inverse document frequency algorithm in each of the topics to obtain the most representative topic words in the topic to represent the main content or events of the topic.

[0008] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have such technical features, wherein step S1 includes the following sub-steps: step S1-1, obtaining the news text used to train the contrastive learning network, including financial news text and non-financial news text with section labels; step S1-2, performing data cleaning on the financial news text and the news text respectively, including converting them into a unified format, removing stop words and performing word segmentation processing, to obtain two data sets.

[0009] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have such technical features, wherein step S2 includes the following sub-steps: step S2-1, constructing a document matrix as the contrastive learning dataset based on the news text dataset used to train the contrastive learning network, wherein the document matrix includes multiple samples, each of the samples includes a financial news document as an anchor point, a financial news document in the same section as the anchor point, a financial news document in a different section from the anchor point, and a non-financial news document, wherein the document contains the corresponding subject words; step S2-2, inputting the document matrix into the pre-trained language model, and fine-tuning the pre-trained language model based on the fine-tuning method of multi-level contrastive learning.

[0010] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have the following technical features: wherein, in step S2-2, the optimization objectives of the fine-tuning include: minimizing the vector distance between the vector representation of the anchor point and the vector representation of the financial news documents in the same section; maximizing the vector distance between the vector representation of the anchor point and the vector representation of the financial news documents in different sections, and ensuring that the vector distance is no greater than the vector distance between the vector representation of the financial news document and the vector representation of the non-financial news document.

[0011] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have the following technical features, wherein the loss function corresponding to the optimization objective is:

[0012] +α·max(0,sim(z i ,z l )-sim(z i ,z k )+δ)

[0013] Where z i 、z j 、z k 、z l They are the anchor point, financial news documents in the same sector as the anchor point, financial news documents in different sectors from the anchor point, and non-financial news documents. The sim() function is a function that calculates the cosine similarity of two vectors. λ is the penalty term for strong negative samples, τ is the temperature scaling factor, δ is the hyperparameter used to control the minimum vector distance, and α is the weight of the second loss.

[0014] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have such technical features, wherein step S4 includes the following sub-steps: step S4-1, for all the financial news texts that need to be topic modeled, calculating the vector similarity between the corresponding documents; step S4-2, establishing a document network based on the vector similarity, wherein the nodes of the document network are the documents, and the edge between two of the nodes represents that the vector similarity between the two nodes is greater than a predetermined threshold; step S4-3, using the community discovery algorithm to perform community discovery on the document network, and the discovered community is used as the topic.

[0015] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have the following technical features: wherein, in step S4-1, the vector similarity is vector cosine similarity, and the similarity measurement function is:

[0016]

[0017] Where, v i ·v j Represents document v i With document v j The dot product between |v i | and |v j | respectively document v i and v j The norm of the document similarity matrix S∈R is constructed by calculating the similarity between all document pairs. n×n , where each element S ij Represents document d i With document d j The similarity between them, n is the number of documents. In step S4-2, the edge is represented as:

[0018] ε={(d i ,d j )|Sim(d i ,d j )>θ}

[0019] Where ε is the edge set in the document network, Sim(d i ,d j ) indicates that the document pair (d i ,d j ), θ is a predetermined threshold. In step S4-3, the community discovery algorithm is the Leiden community discovery algorithm.

[0020] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have such a technical feature, the method also includes: step S6, according to the research needs of the topic, dynamically modeling the topic through a constant topic scheme to study the topic evolution trend, or dynamically modeling the topic through a variable topic scheme to capture the emergence of new topics.

[0021] The financial news topic modeling method based on multi-level contrastive learning provided by the present invention may also have such technical features, wherein step S6 includes the following sub-steps: step S6-1, for the case where it is desired to discover the topic evolution trend, the constant topic scheme includes: reading in all the financial news texts that need to be dynamically topic modeled at one time, establishing a static topic model based on the read financial news texts, and then representing the different topics in each time window in chronological order according to the divided time windows; step S6-2, for the case where it is desired to capture the emergence of new topics, the variable topic scheme includes: establishing the connection between the topics between the time windows by tracking the topics of the same financial news text in the two topic models, and if there is a topic that does not contain the financial news text in the previous time window, it is identified as a new topic.

[0022] Functions and effects of the invention

[0023] According to the financial news topic modeling method based on multi-level contrastive learning provided by the present invention, the financial news topic modeling method based on multi-level contrastive learning applies the contrastive learning fine-tuning task to the topic modeling method of financial news, adopts the multi-level contrastive learning method, adjusts the vector representation generated by the pre-trained language model, so that the generated text vector representation is more suitable for the topic modeling task, and, since the method performs topic modeling through the community discovery algorithm, it does not need to pre-specify the number of topics, nor does it need to pre-set numerous hyperparameters, which makes the application more convenient and avoids the influence of the setting of hyperparameters on the topic modeling results. After experimental testing by the inventors, compared with the existing topic modeling methods, when the method provided by the present invention is applied to the financial news text data set, the topics modeled have a considerable improvement in both topic consistency and topic diversity. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic diagram of a financial news topic modeling method based on multi-level contrastive learning in an embodiment of the present invention;

[0025] Figure 2 is a flowchart of a method for topic modeling of financial news based on multi-level contrastive learning in an embodiment of the present invention;

[0026] Figure 3 is an example diagram of a document matrix in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the technical means, creative features, objectives and effects of the present invention easier to understand, the following is a detailed description of the financial news topic modeling method based on multi-level comparative learning of the present invention in conjunction with embodiments and drawings.

[0028] <Example>

[0029] Figure 1 is a schematic diagram of the financial news topic modeling method based on multi-level contrastive learning in this embodiment. Figure 2 4 is a flowchart of the financial news topic modeling method based on multi-level contrastive learning in this embodiment.

[0030] like Figure 1 and Figure 2 As shown, the financial news topic modeling method based on multi-level contrastive learning in this embodiment mainly includes three stages: text vector generation, topic discovery, and topic representation. In this embodiment, it is divided into the following steps:

[0031] Step S1 : performing text preprocessing and word segmentation processing on the financial news text required for topic modeling and the news text used for training the contrastive learning network model, respectively, to obtain two corresponding processed data sets.

[0032] Step S2: fine-tune the pre-trained language model using the data set corresponding to the news text used to train the contrastive learning network.

[0033] Step S3: Using the fine-tuned pre-trained language model, a vector representation of the pre-processed financial news document that needs to be subject to topic modeling is generated.

[0034] Step S4: Establish a document network based on the vector representation, and use a community discovery algorithm to perform community discovery on the document network to obtain multiple communities, each of which is a topic.

[0035] Step S5: In each topic, a word frequency-inverse document frequency algorithm is used to obtain the most representative topic words in the topic to represent the main content or event of the topic.

[0036] Step S6: Implement dynamic topic modeling based on the topic and its representative keywords.

[0037] The above steps will be described in detail below.

[0038] Step S1: perform text preprocessing and word segmentation processing on the financial news text required for topic modeling and the news text used to train the contrastive learning network, respectively, to obtain two corresponding data sets.

[0039] Specifically, step S1 includes the following sub-steps:

[0040] Step S1-1, obtaining news texts for training the contrastive learning network, including financial news texts with sector labels and news texts in non-financial fields, where sector labels can be, for example, gold, foreign exchange, futures, stocks, etc.; non-financial news texts can be, for example, people's livelihood news, science and technology news texts, etc.

[0041] Step S1-2, respectively, performs data cleaning on the financial news texts that need to be subject to topic modeling and the news texts used to train the contrastive learning network model, including converting these two types of news texts into a unified format, removing stop words (including meaningless modal particles, etc.), and performing word segmentation processing on the news texts after format conversion and stop word removal, to obtain two types of subject words corresponding to the financial news texts that need to be subject to topic modeling and the news texts used to train the contrastive learning network model, and construct two corresponding data sets based on these subject words.

[0042] Step S2: fine-tune the pre-trained language model (contrastive learning network) using the data set corresponding to the news text used to train the contrastive learning network.

[0043] Specifically, step S2 includes the following sub-steps:

[0044] Step S2-1: construct a document matrix required for contrastive learning based on the data set corresponding to the news text used to train the contrastive learning network. The document matrix includes multiple samples provided to the pre-trained language model for fine-tuning.

[0045] Each sample in the document matrix includes a financial news document as an anchor point, a financial news document in the same sector as the anchor point, a financial news document in a different sector from the anchor point, and a non-financial news document, wherein the documents contain corresponding keywords.

[0046] Figure 3 This is an example diagram of the document matrix in this embodiment.

[0047] like Figure 3 As shown, for example, a document matrix is ​​constructed containing four columns and multiple rows, where each row represents a sample provided to the pre-trained language model for fine-tuning. Taking the first row in the document matrix as an example, the News in the first column A,1 Represents the first document in the financial sector A, which serves as the anchor point; News A,k1+1 Represents the k1+1th document in the financial sector A, which is a financial news document in the same sector as the anchor point; News B,2k2+1 Represents the 2k2+1th document in the financial sector B, which is a financial news document in a different sector from the anchor point; New Tech,1represents the first document in the science and technology (non-financial) news, and so on. This means that the document matrix in this example contains samples of financial news from four financial sectors and two non-financial news sectors, science and technology and society. In actual use, the number of financial sectors can be adjusted as needed.

[0048] In step S2-2, the document matrix is ​​input into the pre-trained language model, and the pre-trained language model is fine-tuned based on a fine-tuning method of multi-level contrastive learning.

[0049] The optimization goal is to minimize the vector distance between the anchor point's vector representation and the vector representation of news documents in the same section, and maximize the vector distance between the anchor point's vector representation and the vector representation of news documents in different sections, while ensuring that this vector distance is no greater than the vector distance between the vector representation of financial news documents and the vector representation of non-financial news documents. Vector distance can be measured, for example, using cosine similarity.

[0050] The above optimization objective can be expressed as the following loss function:

[0051]

[0052] Where z i 、z j 、z k 、z l They are respectively the vector representations of the anchor point, financial news documents in the same sub-field (i.e., the same section as the anchor point), financial news documents in different sub-fields of the same field (i.e., different sections from the anchor point), and news documents in different fields (i.e., non-financial). The sim() function is a function that calculates the cosine similarity of two vectors. λ is a penalty term for strong negative samples, which is used to control the importance of strong negative samples and should be a constant greater than 1. τ is a temperature scaling factor, which is used to control the "smoothness" of the similarity distribution. An additional contrast loss term is introduced in the second term to ensure that the similarity between documents in different fields should be greater than the similarity between documents in different sub-fields of the same field. δ is a margin hyperparameter used to control the minimum distance and should be a constant greater than 0. α is the weight of the second loss, which is used to balance the losses of the two parts.

[0053] The pre-trained language model may adopt a model in the prior art, for example, a BERT-wwm model.

[0054] Step S3: Using the fine-tuned pre-trained language model, a vector representation of the pre-processed financial news document that needs to be subject to topic modeling is generated.

[0055] Specifically, for a collection of financial news documents that require topic modeling Each document di The document vector is calculated by fine-tuning the pre-trained language model based on the aforementioned multi-level contrastive learning fine-tuning method. The vector is calculated using v i ∈R d Represented as , d is the dimension of the vector space.

[0056] Step S4: Establish a document network based on the vector representation, and use a community discovery algorithm to perform community discovery on the document network to obtain multiple communities, each of which is a topic.

[0057] Specifically, step S4 includes the following sub-steps:

[0058] Step S4-1: For all financial news texts that require topic modeling, calculate the vector similarity between corresponding documents.

[0059] Among them, the vector similarity can be vector cosine similarity, and the similarity measurement function is as follows:

[0060]

[0061] Where, v i ·v j Represents document v i With document v j The dot product between |v i | and |v j | respectively document v i and v j By calculating the similarity between all document pairs, a document similarity matrix S∈R is constructed. n×n , where each element S ij Represents document d i With document d j The similarity between them is n, where n is the number of documents.

[0062] Step S4-2: Establish a document network based on the calculated document vector similarity.

[0063] In the document network, each document d i is regarded as a node, and the weight of the edge between nodes is determined by its similarity value Sim(d i ,d j ). If the similarity between two documents is higher than a preset threshold θ, an edge will be connected between the nodes corresponding to the two documents. The edge of the document similarity network can be expressed as:

[0064] ε={(d i ,d j )|Sim(d i ,d j )>θ}

[0065] Where ε is the edge set in the document network, Sim(d i ,d j ) indicates that the document pair (d i ,d j ) between them.

[0066] In step S4-3, a Leiden community discovery algorithm is used to perform community discovery on the formed document network, and the discovered communities are used as topics.

[0067] Among them, for the established network in is a document set, ε is a weighted edge set, and community discovery is performed on the document similarity network of the aforementioned fused entity information with reference to the Leiden community discovery algorithm.

[0068] The Leiden community discovery algorithm optimizes as follows: Initially, each node belongs to a separate community. In each subsequent iteration, each node is reassigned to a neighboring community to maximize local modularity. Specifically, nodes decide whether to move to a different community based on the strength of their connections to neighboring nodes. This algorithm attempts to reduce the boundaries within communities and increase the number of edges within them, thereby improving modularity. The goal of optimization is to achieve a final community structure that maximizes modularity through multiple rounds of local adjustments.

[0069] After each optimization, the community discovery algorithm refines the community structure and further optimizes community partitioning using a multi-resolution strategy to accommodate communities of varying sizes and densities. Specifically, in a document network, different communities may have varying sizes and densities: some contain many nodes and are densely connected by edges, while others are smaller and more sparsely connected. For larger, edge-dense communities, the algorithm optimizes at a higher resolution. This means that at higher resolutions, the algorithm focuses more on the compactness within the community and the density of edges, fine-tuning the community affiliation of nodes to further refine the larger community structure. For smaller or sparser communities, the algorithm uses a lower resolution. This strategy helps avoid over-segmenting small communities and prevents overfitting caused by excessive community partitioning. At lower resolutions, the algorithm focuses more on optimizing the large-scale community structure rather than focusing on local details, thereby improving the algorithm's robustness. The optimization process continues iteratively until further improvement in modularity is impossible or the maximum number of iterations is reached.

[0070] Ultimately, the community discovery algorithm returns an optimized community partitioning result, assigning nodes (i.e., individual keywords) in the document network to different communities (i.e., topics). In other words, through the community discovery algorithm, each news document is assigned a unique community label. Using the community label as the topic label completes the topic assignment task of the topic model.

[0071] The modularity involved in the above algorithm is an indicator to measure the quality of community division in a graph. It calculates the difference between the density of edges within a community and the expected value when randomly assigned. The calculation formula of modularity Q is:

[0072]

[0073] Where A ij is an element of the adjacency matrix, indicating whether there is an edge between node i and node j; γ is the resolution, which determines the tightness of the connection within the community; k i and k j represents the degree of node i and node j, that is, the number of their respective connected edges; m is the total number of edges in the graph; c i and c j Represents the communities to which nodes i and j belong respectively; δ(c i ,c j ) is the Kroneckerdelta function, when c i =c j The value is 1 when it is set, otherwise it is 0.

[0074] In step S5, the term frequency-inverse document frequency (TF-IDF) algorithm is used in each topic to obtain the most representative topic words in the topic to represent the main content or event of the corresponding topic. Since the TF-IDF algorithm is an existing technology, step S5 will not be described in detail.

[0075] Step S6: Dynamic topic modeling is performed based on the topic and its representative keywords. According to the research needs of the topic, a "constant topic" scheme or a "variable topic" scheme can be selected to perform dynamic topic modeling.

[0076] Specifically, step S6 includes the following sub-steps:

[0077] Step S6-1: In the case of wanting to discover the evolution trend of a topic, the topic is dynamically modeled and represented through a "constant topic" solution.

[0078] Specifically, this approach simultaneously reads all news text required for dynamic topic modeling. It first builds a static topic model based on the read-in news text, then represents the different topics within each time window according to the time window division. This allows the representation of the same topic over consecutive time windows to capture the topic's evolution. This approach enables the study of topic evolution trends at a low computational cost. For example, the sentiment expressed in the gold market topic from a year ago differs from that of the current one, and the displayed keywords are also different. However, the disadvantage is that the number of topics is immutable, making it less sensitive to changes.

[0079] Step S6-2: In order to capture the emergence of new topics, the topic is dynamically modeled and represented through a "changing topic" solution.

[0080] Specifically, this solution performs static topic modeling on news within each of two time windows. By tracking the topics that the same news article belongs to in both topic models, it establishes connections between topics in the time windows. If a topic does not contain news from the previous time window, it is considered a new topic. This solution addresses the limitation of the immutable number of topics in the evolution of dynamic topic models, while also ensuring a smooth transition of topics between different time windows.

[0081] Through the above method, a fine-tuned pre-trained language model can be used to generate text vector representations that are more suitable for topic modeling tasks, thereby better mining topics in financial news texts, and two methods of dynamic topic modeling and representation are provided.

[0082] The inventors applied the above method to a dataset of financial news text to test it. They also applied several existing topic modeling methods to the same dataset to compare their effectiveness with the method provided in the examples. These existing methods were based on the topic classification model LDA-15, LDA-3, and the topic modeling tool Bertopic. The method in the examples is abbreviated as STFTC.

[0083] For the above three existing methods and the method of the embodiment, the effects of their topic modeling are evaluated from two aspects: topic consistency and topic diversity. Among them, topic consistency is measured by the Normalized Pointwise Mutual Information (NPMI) indicator; topic diversity is measured by the Diversity indicator.

[0084] Table 1 below shows a summary of the test results.

[0085] Table 1 Test data of various topic modeling methods

[0086] Methods\Evaluation Indicators NPMI-top5 NPMI-top3 Diversity LDA-15 0.061 0.103 0.73 LDA-3 0.056 0.023 0.80 Bertopic 0.149 0.154 0.78 STFTC 0.220 0.221 0.81

[0087] From the comparison of the test data in Table 1, it can be seen that compared with the existing methods, the topic modeling method of the embodiment achieves a significant improvement in the index of topic consistency and also achieves a certain degree of improvement in the index of topic diversity. Both indicators are significantly better than the other three existing methods.

[0088] Functions and Effects of the Embodiments

[0089] According to the financial news topic modeling method based on multi-level contrastive learning provided by this embodiment, the contrastive learning fine-tuning task is applied to the topic modeling method of financial news. The multi-level contrastive learning method is adopted to adjust the vector representation generated by the pre-trained language model so that the generated text vector representation is more suitable for the topic modeling task. Moreover, since this method performs topic modeling through the community discovery algorithm, it does not need to pre-specify the number of topics or pre-set numerous hyperparameters, which makes it more convenient to apply and avoids the influence of the setting of hyperparameters on the topic modeling results. According to the experimental tests of the inventors, compared with the existing topic modeling methods, when the method provided by the embodiment is applied to the financial news text dataset, the topics obtained by modeling have a considerable improvement in both topic consistency and topic diversity, which proves the superiority of the method of this embodiment.

[0090] In an embodiment, each sample provided to the pre-trained language model for fine-tuning includes an anchor point, documents in the same section as the anchor point, documents in a different section from the anchor point, and non-financial documents. Combined with the above-mentioned objective function, the pre-trained language model is optimized and trained for the specific task of topic modeling, so that the fine-tuned model can output a more suitable vector representation.

[0091] Furthermore, the community discovery algorithm optimizes the community structure based on modularity. The algorithm does not require prior knowledge and can adaptively identify the number of communities (topics). Therefore, it has good versatility and can efficiently process large-scale data, thereby accurately and efficiently modeling topics.

[0092] Furthermore, the embodiment also provides two methods of dynamic modeling of topics. According to the research needs of the topic, the constant topic scheme can be used to obtain the topic evolution trend, and its computational cost is low; the variable topic scheme can also be used to capture the emergence of new topics, thereby providing a useful tool for relevant researchers.

[0093] The above embodiments are merely illustrative of specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention as claimed. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A financial news topic modeling method based on multi-level contrastive learning, characterized by: The following steps are involved: Step S1, performing text preprocessing and word segmentation processing on the financial news text required for topic modeling and the news text used to train the contrastive learning network, respectively, to obtain two corresponding processed data sets; Step S2, fine-tuning the pre-trained language model using the data set corresponding to the news text used to train the contrastive learning network; Step S3, generating a vector representation of the pre-processed financial news document using the fine-tuned pre-trained language model; Step S4, establishing a document network based on the vector representation, and performing community discovery on the document network using a community discovery algorithm to obtain multiple communities, each of which serves as a topic; Step S5: Using a word frequency-inverse document frequency algorithm in each of the topics, obtain the most representative topic words in the topic to represent the main content or event of the topic.

2. The method for financial news topic modeling based on multi-level contrastive learning according to claim 1, characterized in that: in, Step S1 includes the following sub-steps: Step S1-1, obtaining the news text for training the contrastive learning network, including financial news text and non-financial news text with sector labels; Step S1-2, performing data cleaning on the financial news text and the news text respectively, including converting them into a unified format, removing stop words and performing word segmentation processing, to obtain two data sets.

3. The financial news topic modeling method based on multi-level contrastive learning according to claim 1, Its characteristics are: in, Step S2 includes the following sub-steps: Step S2-1: constructing a document matrix as the contrastive learning dataset based on a news text dataset used to train a contrastive learning network, wherein the document matrix includes multiple samples, each of which includes a financial news document serving as an anchor point, a financial news document in the same sector as the anchor point, a financial news document in a different sector from the anchor point, and a non-financial news document, wherein the documents contain the corresponding keyword; Step S2-2: input the document matrix into the pre-trained language model, and fine-tune the pre-trained language model based on a fine-tuning method of multi-level contrastive learning.

4. The method for financial news topic modeling based on multi-level contrastive learning according to claim 3, Its characteristics are: in, In step S2-2, the optimization objectives of the fine-tuning include: Minimizing the vector distance between the vector representation of the anchor point and the vector representation of the financial news documents in the same section; The vector distance between the vector representation of the anchor point and the vector representations of the financial news documents in different sectors is maximized, and the vector distance is ensured to be no greater than the vector distance between the vector representation of the financial news document and the vector representation of the non-financial news document.

5. The method for financial news topic modeling based on multi-level contrastive learning according to claim 4 is characterized in that: in, The loss function corresponding to the optimization objective is: Where z i 、z j 、z k 、z l They are the anchor point, financial news documents in the same sector as the anchor point, financial news documents in different sectors from the anchor point, and non-financial news documents. The sim() function is a function that calculates the cosine similarity of two vectors. λ is the penalty term for strong negative samples, τ is the temperature scaling factor, δ is the hyperparameter used to control the minimum vector distance, and α is the weight of the second loss.

6. The method for financial news topic modeling based on multi-level contrastive learning according to claim 1, characterized in that: in, Step S4 includes the following sub-steps: Step S4-1, for all the financial news texts that need to be subject to topic modeling, calculate the vector similarity between the corresponding documents; Step S4-2: establishing a document network based on the vector similarity, wherein the nodes of the document network are the documents, and an edge between two nodes represents that the vector similarity between the two nodes is greater than a predetermined threshold; Step S4-3: Use the community discovery algorithm to perform community discovery on the document network, and use the discovered community as the topic.

7. The method for financial news topic modeling based on multi-level contrastive learning according to claim 6, characterized in that: in, In step S4-1, the vector similarity is vector cosine similarity, and the similarity measurement function is: Where, v i ·v j Represents document v i With document v j The dot product between |v i | and |v j | respectively document v i and v j The norm of the document similarity matrix S∈R is constructed by calculating the similarity between all document pairs. n×n , where each element S ij Represents document d i With document d j The similarity between them, n is the number of documents, In step S4-2, the edge is represented as: ε={(d i ,d j )∣Sim(d i ,d j )>θ} Where ε is the edge set in the document network, Sim(d i ,d j ) indicates that the document pair (d i ,d j ), θ is the predetermined threshold, In step S4-3, the community discovery algorithm is the Leiden community discovery algorithm.

8. The method for financial news topic modeling based on multi-level contrastive learning according to claim 1, characterized in that: Also includes: Step S6, according to the research needs of the subject, dynamically model the subject through a constant subject scheme to study the subject evolution trend, or dynamically model the subject through a variable subject scheme to capture the emergence of new subjects.

9. The method for financial news topic modeling based on multi-level contrastive learning according to claim 8, Its characteristics are: in, Step S6 includes the following sub-steps: Step S6-1: In the case of discovering topic evolution trends, the common topic solution includes: reading all the financial news texts that require dynamic topic modeling at once, building a static topic model based on the read financial news texts, and then representing the different topics within each time window in chronological order according to the divided time windows; In step S6-2, in the case where it is desired to capture the emergence of a new topic, the topic change scheme includes: establishing a connection between the topics between the time windows by tracking the topics to which the same financial news text belongs in the two topic models; if a topic does not contain the financial news text in the previous time window, it is identified as a new topic.

Images