Large model financial data synthesis method and system based on text correlation

Abstract

The invention provides a large model financial data synthesis method and system based on text correlation. Firstly, a financial field document is combined with a financial characteristic deep optimization vector generation process to perform de-duplication, low-dimensional vector representation is converted, a corpus graph is constructed, then nodes and edges of the corpus graph are redefined according to uniqueness of financial data, and the corpus graph is elaborately constructed. Then, documents are connected in series based on defined node priorities and a greedy algorithm, financial risk assessment and market trend analysis factors are introduced, a path search strategy is dynamically adjusted according to the relevance and importance of financial data, the most relevant documents are preferentially connected in series, and financial risk assessment and market trend analysis are achieved. And forming a context with highly coherent and smooth semantics as a large-model financial pre-training corpus.

Description

Technical Field

[0001] The present invention relates to the technical field of financial data models, and in particular to a method and system for synthesizing large-model financial data based on text relevance. Background Art

[0002] Pretraining of large language models (LLMs) is a key technical method that has emerged in the field of natural language processing (NLP) in recent years. The core idea of ​​this technology is to pretrain a general deep learning model on large-scale unlabeled text data, allowing the model to learn the general laws and representations of language through self-supervision, and then apply this pretrained model to subsequent fine-tuning tasks.

[0003] When pre-training existing large language models, they generally randomly splice several short texts. The texts are unrelated to each other and lack internal logical connections and coherence. As a result, it is difficult for the large language model to obtain effective context information during the pre-training stage, which often affects the training effect of the large language model. Especially in specific fields such as finance, when the corpus is small and the distribution is relatively concentrated, the large language model is likely to fit incorrect features from multiple, accidental noise co-occurrences. These erroneous features may mislead the output results of the large language model, causing deviations when processing related tasks, reducing accuracy and reliability.

[0004] Although existing research has achieved certain results in solving the problem of model training corpus, there are still some shortcomings. For example, when processing code-related pre-training corpus, the InternLM2[1] large language model maintains code integrity and context consistency by sorting the dependencies of code repository files, but this research only targets code-related corpus. Invention 1[2] can align large language models to specific task information in a specific domain with a small number of data samples by sorting instances within the domain, but this invention is limited to the model alignment stage and is not suitable for large-scale large language model pre-training corpus.

[0005] [1]Cai Z,Cao M,Chen H,et al.Internlm2 technical report[J].arXivpreprint arXiv:2403.17297,2024;

[0006] [2] Song Yan, Liu Chang, Tian Yuanhe, et al. A medical imaging report generation method based on large model domain migration: CN117174240A[P]. 2023.12.05. Summary of the invention

[0007] The technical problems to be solved by the technical solution of the present invention are: 1. When the existing large language model is pre-trained, several short texts are generally randomly spliced, and the texts are unrelated to each other and cannot provide effective context information, which often affects the training effect of the large language model. 2. In specific fields such as finance, the corpus is small and concentrated, and the large language model is likely to fit the wrong features from multiple and accidental noise co-occurrences.

[0008] In view of the shortcomings of the prior art that the pre-training corpus is lacking, resulting in poor pre-training effect of large language models, or the processing of corpus is limited to the model alignment stage, which is not suitable for large-scale pre-training corpus of large models, the technical solution of the present invention provides a large model financial data synthesis method based on text relevance, including the following steps:

[0009] Collect large-scale text data related to the financial field to establish a total corpus, set the screening ratio according to the specific corpus screening mechanism in the financial field, clean the text in the total corpus to remove noise, special symbols and meaningless content, give priority to text related to the financial field, preset the loss function through comparative learning to obtain similar positive sample financial document groups and dissimilar negative sample financial document groups, adjust the weight of documents with preset financial terms or financial events in the negative sample financial document group, balance the contribution of financial text classification and risk assessment according to the preset multi-task learning total loss function to obtain a pre-vectorized document library, convert the financial documents in the pre-vectorized document library into low-dimensional vector representations, and build a special vectorized financial corpus;

[0010] The cosine similarity of any two document vectors in the vectorized financial corpus is calculated according to the cosine similarity formula. The similarity threshold is adjusted in real time according to the characteristics of financial documents and changes in data distribution. The vectorized financial corpus is deduplicated to obtain the deduplicated vectorized financial corpus.

[0011] According to any document vector, the feature matrix corresponding to the node is obtained. According to the cosine similarity between any document vectors, the relationship matrix is ​​obtained and the threshold is set. The relationship matrix is ​​binarized to obtain the adjacency matrix. The edge weight is determined according to the correlation formula, the influence corresponding to any document vector and the cosine similarity between any document vectors. The distance index formula is defined according to the edge weight to obtain the distance index, and the financial corpus is constructed.

[0012] Calculate the risk assessment value based on the standard deviation of the return of the financial asset portfolio involved in any document vector, determine the corresponding factor weight coefficient based on the quantitative assessment of the financial market trend, industry development trend, and macroeconomic indicator change trend of any document vector, multiply the indicator values ​​corresponding to the financial market trend, industry development trend, and macroeconomic indicator change trend by the corresponding factor weight coefficient, combine and sum them, and obtain the market trend relevance of any document vector, calculate the risk assessment value, market trend relevance, and the average value of the distance index between any document vector and other any document vectors based on the standard deviation of the return, determine the priority of any vector document, and traverse the optimization path according to the priority to obtain the optimal path;

[0013] According to the optimal path, documents involving the same financial event or related financial concepts are preferentially selected, and the corpus is truncated or supplemented in order to be spliced ​​according to the input length required by the large language model to construct a context with more closely related semantics to obtain a pre-training dataset, and the pre-training dataset is extracted as the input of the large language model for training.

[0014] Preferably, the large-scale text data includes financial news, research reports, financial statements, etc.

[0015] Preferably, the term ratio threshold is preset according to the characteristics of the financial document and the change of data distribution, and the similarity threshold is adjusted in real time according to the comparison between the ratio of the number of professional terms in the document vector and the total number of words in the document vector and the term ratio threshold.

[0016] Preferably, the loss function adopts contrast loss, and the loss function formula is:

[0017]

[0018] Among them, the positive sample pair (D1, D2) is similar financial documents, the negative sample pair (D1, D3) is dissimilar financial documents, sim(D1, D2) represents the similarity between documents D1 and D2, sim(D1, D3) represents the similarity between documents D1 and D2, and t is the temperature parameter.

[0019] Preferably, the formula for adjusting the weight of the document with the preset financial term or financial event in the negative sample financial document group is:

[0020] v adjusted =ω original +ω term ×f term (D)+ω event ×f event (D)

[0021] where ω original is the original weight, ω termis the weight of financial terms, ω event is the weight of financial events, f term (D) is the frequency of occurrence of financial terms in the document, f event (D) Frequency of occurrence of financial events in documents.

[0022] Preferably, the data deduplication further includes:

[0023] For any two document vectors whose cosine similarity is higher than the high similarity threshold in the similarity threshold, the financial concept set and entity relationship set are extracted through the large language model to define the difference index formula and the semantic difference threshold, and it is determined whether the difference between any two document vectors calculated according to the difference index formula is greater than the semantic difference threshold. If so, the any two document vectors have important semantic differences, and the corresponding any two document vectors are retained.

[0024] Preferably, the optimization path is traversed using a greedy algorithm or a genetic algorithm according to the priority.

[0025] The technical solution of the present invention also provides a large model financial data synthesis system based on text relevance, which adopts the large model financial data synthesis method based on text relevance as described above, including:

[0026] A text vectorization module, used to convert the financial documents in the pre-vectorized document library into low-dimensional vector representations, construct a special vectorized financial corpus to facilitate subsequent cosine similarity calculations, and perform data deduplication on the vectorized financial corpus to obtain a deduplicated vectorized financial corpus;

[0027] A corpus graph construction module, used for constructing a financial corpus graph according to the arbitrary document vector;

[0028] The path search module is used to calculate the risk assessment value, market trend relevance and the average distance index between any document vector and any other document vector based on the return standard deviation, determine the priority of any vector document, and traverse the optimized path according to the priority to obtain the optimal path to ensure that the documents are arranged according to relevance;

[0029] The data synthesis module is used to preferentially select documents involving the same financial event or related financial concepts according to the optimal path, and sequentially truncate or supplement the corpus according to the input length required by the large language model to construct a context with more closely related semantics to obtain a pre-training data set;

[0030] The pre-training data extraction interface is used to extract pre-training data sets as input for large language models for training to enhance the performance of the model in the financial field.

[0031] The technical solution of the present invention proposes a large-model financial data synthesis method and system based on text relevance, introduces a large-model data synthesis method based on text relevance, synthesizes large-model pre-training corpus in a specific order, improves the large-model pre-training effect, and solves the shortcomings of the existing technology.

[0032] First, the financial documents are deduplicated by combining the financial characteristics with the deep optimization vector generation process, converted into low-dimensional vector representation and constructed into a corpus. Then, the nodes and edges of the corpus are redefined based on the uniqueness of financial data to carefully construct the corpus. Then, the documents are connected in series using the defined node priority and the greedy algorithm, and financial risk assessment and market trend analysis factors are introduced. The path search strategy is dynamically adjusted according to the relevance and importance of financial data, and the most relevant documents are connected in series first, forming a highly coherent and fluent context as a large-model financial pre-training corpus. Compared with traditional methods, this method can better guide the model to learn financial knowledge, improve the processing ability of specific financial tasks, accelerate model convergence and reduce instability.

[0033] In addition, the system part of the technical solution of the present invention creates an efficient and automated platform. The platform can accurately and automatically execute the data synthesis process based on text relevance, covering key steps such as building corpus graphs, accurately sorting corpus, and efficiently generating pre-training data, providing strong support for pre-training of large models in the financial field. The technical solution of the present invention focuses on the corpus in the financial field and innovatively introduces a large model pre-training data synthesis method based on text relevance. In the process of data synthesis, the disordered method is abandoned, the logical order of curriculum learning (Curriculum Learning) is cleverly integrated, and the data synthesis method is scientifically planned to lay a solid foundation for the construction of large model pre-training corpus. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of a method for synthesizing large-model financial pre-training data provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a construction diagram provided for an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of a generation path provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of pre-training data splicing provided in an embodiment of the present invention;

[0038] Figure 5 Schematic diagram of a large-model financial pre-training data synthesis system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0040] The embodiment of the present invention proposes an innovative method to optimize the performance of large language models by enhancing the quality of pre-training data. The core of the method is to introduce a data synthesis strategy based on text relevance. By reasonably arranging the order of pre-training corpus, this method can significantly improve the training effect of large language models. Specifically, Figure 1 As shown, an embodiment of the present invention provides a large model financial data synthesis method based on text correlation, comprising the following steps:

[0041] Collect large-scale text data related to the financial field, including financial news, research reports, financial statements, etc., clean the text, remove noise, special symbols and meaningless content, and conduct data collection and preprocessing. According to the specific corpus screening mechanism in the financial field, give priority to texts related to the financial field and build a special financial corpus.

[0042] When converting documents into low-dimensional vector representations for text vectorization, we combine the characteristics of the financial field, deeply optimize the vector generation process through a specific corpus screening mechanism and a fine-tuning method tailored to the characteristics of financial texts, so that the resulting embeddings can more accurately reflect the connotation of financial texts.

[0043] Specifically, the embodiment of the present invention constructs a semantic vector model in combination with the characteristics of the financial field. First, in the pre-training stage, the embodiment of the present invention introduces a corpus screening mechanism specific to the financial field. Based on the RetroMAE pre-training algorithm, large-scale corpora related to the financial field are preferentially selected. Assume that the total corpus is C and the financial corpus is C. finance , we can set a screening ratio r to select |C from the total corpus finance |=r×|C| to ensure that the model can better learn the characteristic patterns of financial texts during the pre-training process.

[0044] Secondly, when fine-tuning using the hard negative sample mining method, targeted adjustments are made according to the characteristics of financial texts. Specifically, let the positive sample pair be (D1, D2) and the negative sample pair be (D1, D3), where D1 and D2 are similar financial documents, and D1 and D3 are dissimilar financial documents. The model can learn to distinguish between positive sample pairs and negative sample pairs through contrastive learning. The loss function can use contrastive loss: Where sim(D1,D2) represents the similarity between documents D1 and D2, and t is the temperature parameter. For important information such as financial terms and financial events, increase their weight in hard negative samples. Specifically, for documents containing financial terms or financial events, their weight in negative samples can be adjusted to ω adjusted =ω original +ω term ×f term (D)+ω event ×f event (D), where ω original is the original weight, ω term is the weight of financial terms, ω event is the weight of financial events, f term (D) is the frequency of occurrence of financial terms in the document, f event (D) The frequency of occurrence of financial events in the document. This allows the model to focus more on key content in the financial field during fine-tuning.

[0045] Finally, we introduce specific tasks in the financial field: financial text classification and risk assessment, and conduct multi-task learning to further improve the performance of the model in the financial field. Suppose the loss function of the financial text classification task is L classification , the loss function of the risk assessment task is L risk The total loss function of multi-task learning can be L total =αL contrastive +βL classification +γL risk , where α, β, and γ are weight coefficients used to balance the contributions of different tasks.

[0046] Through the above optimization measures, the embodiment of the present invention can more accurately convert financial documents into low-dimensional vector representations, so that the obtained embeddings more accurately reflect the connotation of financial texts, and provide a better data foundation for subsequent pre-training of large language models in the financial field.

[0047] De-duplicate the document vectors A and B based on the cosine similarity between embeddings:

[0048]

[0049] Filter the corpus with similarity greater than the threshold (α).

[0050] On this basis, the embodiment of the present invention improves the data deduplication process. The fixed threshold filtering method has certain limitations. The present invention introduces a dynamic threshold adjustment mechanism to adjust the similarity threshold α in real time according to the characteristics of financial documents and changes in data distribution. Suppose the number of professional terms in document D is N terms (D), the total number of words in the document is Nwords (D), the proportion of professional terms is P terms (D) = N terms (D) / N words (D). When P terms (D)>T professional (where T professional is a pre-set threshold for the proportion of highly specialized terms), the document is considered to be a highly specialized and term-intensive document. When processing different types of financial documents, for highly specialized and term-intensive documents, the threshold α is appropriately lowered according to the term ratio threshold to retain more unique information; for more general financial news documents, the threshold α can be increased to more strictly remove similar content.

[0051] At the same time, in order to further improve the deduplication effect, the present invention also combines a large language model for financial entity recognition. similar , T similar For document pairs with a high similarity threshold set by cosine, the financial concepts and entity relationships are extracted through the large language model to determine whether they have important differences in semantics. If they are highly similar but have obvious differences in financial semantics, the two documents are retained to avoid accidentally deleting valuable information.

[0052] Specifically, for documents A and B, the financial concepts and entity relationship sets related to them are K A and K B , define the semantic difference index where |K A ΔK B | represents the number of elements of the symmetric difference of two sets, |K A ∪K B | represents the number of elements in the union of two sets. semantic (A,B)>T diff (where T diff is a pre-set semantic difference threshold), it is considered that documents A and B are significantly different in financial semantics, and these two documents are retained.

[0053] Through these measures, the present invention can perform data deduplication more intelligently and efficiently, and provide high-quality refined data for pre-training of large language models in the financial field.

[0054] The corpus graph is constructed based on document embedding. The nodes represent different documents, while the edges reflect the correlation between these documents. In addition, the distance between nodes is determined by indicators that are more in line with financial reality, such as considering the relevance and influence of financial events, so that the corpus graph can better reflect the knowledge structure in the financial field.

[0055] For the constructed corpus graph G = (V, E), V (|V| = n) and E are the sets of nodes and edges respectively. is the feature matrix, i.e., the embedding of n nodes. Based on the vector similarity, the relationship matrix R is obtained, and by setting the threshold τ, the relationship matrix is ​​binarized to obtain the adjacency matrix A of the graph:

[0056]

[0057] First, the weights of edges are adjusted by considering the relevance and influence of financial events so that the corpus graph can better reflect the knowledge structure in the financial field.

[0058] For financial documents A and B, the edge weight W AB It can be expressed as:

[0059] W AB =f(C AB ,I A ,I B ,sim AB )

[0060] The correlation is Where N common It is the sum of comprehensive factors such as the number of financial terms that appear in both document A and document B, the number of the same financial institutions or events involved, etc.; N total is the total number of all these considerations in document A and document B. A and I B are the influence of A and B respectively, I A =α×S A +β×P A +γ×R A , where S A is the source authority score of document A, which can be evaluated and assigned based on factors such as the reputation and professionalism of the source organization; P A is the spread score of document A, which can be determined by counting the number of document readings, forwardings, and other indicators; R A is the citation score of document A, that is, the number of citations of this document by other documents. α, β, γ are weight coefficients, and α+β+γ=1. AB is the similarity between the embeddings of document A and document B, which can be calculated using methods such as cosine similarity, such as f is a comprehensive function, which can be set as Among them, λ1, λ2, λ3 are weight coefficients, and λ1+λ2+λ3=1.

[0061] This formula considers relevance, influence, and embedding similarity to determine the weight of the edge. Documents with high relevance, influence, and embedding similarity will have smaller distances between them, which means they are more closely connected in the corpus. By adjusting the weight coefficient, the importance of these three factors can be balanced according to different application scenarios and requirements.

[0062] Finally, the distance metric is defined according to the weight formula: Thus, when the correlation C AB High, Impact I A and I B Large, embedding similarity sim AB When high, W AB The value will be larger, and d AB The value of will be smaller, which means that document A and document B are closer in the corpus.

[0063] Overall construction Figure 2 The example is as follows, where Doc i represents the i-th article obtained in the original document order.

[0064] Next, based on the financial corpus, the document concatenation path is found to obtain an ordered document sequence and generate the optimal path.

[0065] In the implementation process, financial risk assessment and market trend analysis factors are introduced. For document D, let the standard deviation of the return of the financial asset portfolio involved in the document be σ D , through the formula To calculate the risk assessment value. As for the market trend correlation (T D ), then comprehensively consider multiple factors such as financial market trends, industry development trends, and macroeconomic indicator change trends. After quantitatively evaluating each factor, determine the corresponding weight coefficient, and then multiply the value of each market trend indicator by the weight coefficient and sum them up to obtain the market trend relevance of the document.

[0066] Determine the document priority based on financial risk assessment, market trend analysis results, and the distance between documents. Suppose the document priority is where d D is the average distance between document D and other documents, ω, η, ζ are weight coefficients, and ω+η+ζ = 1. Documents that are related to current financial hot spots and high-risk areas and are closer are given higher priority.

[0067] The priority of setting documents does not seem to be in order from easy to difficult, but in fact it is still an innovative embodiment of the course learning concept. From the perspective of knowledge importance, high-priority documents are very important for the model to understand the core and key knowledge in the financial field. These high-priority documents often contain the most valuable information for model training, which helps the model establish an understanding framework of important events and key knowledge in the financial field. Learning this information first can provide better guidance for subsequent learning, just like learning the core theorems and methods of the subject first, and then solving complex practical problems.

[0068] Finally, based on the document priority, a greedy algorithm is used to select the document with the highest priority as the next node to join the path each time. Alternatively, a genetic algorithm is used to optimize the path selection through continuous iterative genetic operations. Each document is thrown away after use to avoid duplication. After each graph is completely traversed, the remaining nodes can be randomly sampled to traverse the next graph. The path weights between nodes in different graphs are counted as 1000. Through these innovative measures, a coherent, smooth and most conducive to model learning financial domain knowledge context is provided for large model pre-training. Examples of generated paths are as follows: Figure 3 shown.

[0069] Document sequence splicing corpus based on optimal path to perform large model pre-training data splicing.

[0070] When splicing, the input length required by the language model is cut or supplemented (here 4096 is taken). In this way, a context with more closely related semantics can be constructed to provide more valuable learning materials for the large language model. For example, for corpus in the financial field, documents involving the same financial event or related financial concepts are spliced ​​first, so that the model can better understand the internal logic and correlation of the financial field during the learning process. The example of pre-training data splicing is as follows: Figure 4 shown.

[0071] Finally, based on the concatenated corpus, the large language model pre-training process is performed. In this process, the loss function of the large language model is set to L. For a sample (x, y) in the given concatenated corpus, where x is the input text and y is the corresponding target output, the predicted output of the large language model is The loss is calculated using the cross entropy loss function: During the training process, the model parameters are continuously updated through the optimization algorithm to minimize the loss function.

[0072] Compared with traditional methods, the pre-training process of the present invention is more targeted and efficient. Due to the higher semantic relevance of the corpus, the model can converge faster and reduce the instability problem in the training process. At the same time, for specific financial tasks (such as financial text classification, risk assessment, etc.), the model can better extract and learn professional knowledge in the financial field, and improve the processing ability and accuracy of these tasks.

[0073] like Figure 5 As shown, an embodiment of the present invention further provides a large model financial data synthesis system based on text relevance, which adopts the above-mentioned large model financial data synthesis method based on text relevance, and the system includes:

[0074] Computer-readable storage medium: used to store executable program codes and other necessary data structures, such as corpus files.

[0075] Processor: Responsible for executing program instructions stored on computer-readable storage media and handling various computing tasks such as text vectorization, corpus graph construction, path search, etc.

[0076] Input / output interface: supports interaction with external devices, including receiving user input (such as configuration parameters, initial corpus, etc.) and outputting processing results (such as sorted corpus, etc.).

[0077] Network interface: allows the system to communicate with other systems or external data sources to facilitate the acquisition or update of the corpus.

[0078] Executable program: A set of program instructions for implementing a text-correlation-based financial data synthesis method.

[0079] Functional modules include:

[0080] Text vectorization module: responsible for converting raw text data into vector representation to facilitate subsequent similarity calculations and deduplicate documents.

[0081] Corpus construction module: Build a corpus based on text vectors, each node represents a document, and the edges between nodes are weighted according to the distance of the text vectors.

[0082] Path search module: Based on the relevant algorithm, it searches for an optimal path in the corpus to ensure that documents are arranged according to relevance.

[0083] Data synthesis module: sort documents according to the optimal path found to generate a high-quality dataset for pre-training.

[0084] Pre-training data extraction interface: used to extract sorted data sets to pre-train large language models to enhance the performance of the model in the financial field.

[0085] The system part of the present invention aims to provide an efficient and automated platform for pre-training data synthesis of large models in the financial field. The system can automatically perform the data synthesis process based on text relevance, including key steps such as building corpus graphs, sorting corpus, and generating pre-training data.

[0086] The advantages of the embodiments of the present invention are as follows:

[0087] 1. Innovatively process the large model pre-training corpus and introduce a large model data synthesis method based on text relevance.

[0088] 2. Based on the idea of ​​course learning, synthesize the pre-training corpus of large models in the financial field to improve the pre-training effect of large models.

[0089] 3. Combining the characteristics of the financial field, the pre-trained corpus is constructed into a corpus graph based on text vectors, and large-scale corpus sorting is achieved.

Claims

1. A large-model financial data synthesis method based on text relevance, characterized in that: The following steps are involved: Collect large-scale text data related to the financial field to establish a total corpus, set the screening ratio according to the specific corpus screening mechanism in the financial field, clean the text in the total corpus to remove noise, special symbols and meaningless content, give priority to text related to the financial field, preset the loss function through comparative learning to obtain similar positive sample financial document groups and dissimilar negative sample financial document groups, adjust the weight of documents with preset financial terms or financial events in the negative sample financial document group, balance the contribution of financial text classification and risk assessment according to the preset multi-task learning total loss function to obtain a pre-vectorized document library, convert the financial documents in the pre-vectorized document library into low-dimensional vector representations, and build a special vectorized financial corpus; The cosine similarity of any two document vectors in the vectorized financial corpus is calculated according to the cosine similarity formula. The similarity threshold is adjusted in real time according to the characteristics of financial documents and changes in data distribution. The vectorized financial corpus is deduplicated to obtain the deduplicated vectorized financial corpus. According to any document vector, the feature matrix corresponding to the node is obtained. According to the cosine similarity between any document vectors, the relationship matrix is ​​obtained and the threshold is set. The relationship matrix is ​​binarized to obtain the adjacency matrix. The edge weight is determined according to the correlation formula, the influence corresponding to any document vector and the cosine similarity between any document vectors. The distance index formula is defined according to the edge weight to obtain the distance index, and the financial corpus is constructed. Calculate the risk assessment value based on the standard deviation of the return of the financial asset portfolio involved in any document vector, determine the corresponding factor weight coefficient based on the quantitative assessment of the financial market trend, industry development trend, and macroeconomic indicator change trend of any document vector, multiply the indicator values ​​corresponding to the financial market trend, industry development trend, and macroeconomic indicator change trend by the corresponding factor weight coefficient, combine and sum them, and obtain the market trend relevance of any document vector, calculate the risk assessment value, market trend relevance, and the average value of the distance index between any document vector and other any document vectors based on the standard deviation of the return, determine the priority of any vector document, and traverse the optimization path according to the priority to obtain the optimal path; According to the optimal path, documents involving the same financial event or related financial concepts are preferentially selected, and the corpus is truncated or supplemented in order to be spliced ​​according to the input length required by the large language model to construct a context with more closely related semantics to obtain a pre-training dataset, and the pre-training dataset is extracted as the input of the large language model for training.

2. A large-model financial data synthesis method based on text correlation as claimed in claim 1, characterized in that: The large-scale text data includes financial news, research reports, and financial statements.

3. A large-model financial data synthesis method based on text correlation as claimed in claim 1, characterized in that: The term ratio threshold is preset according to the characteristics of financial documents and changes in data distribution, and the similarity threshold is adjusted in real time according to the comparison between the ratio of the number of professional terms in the document vector and the total number of words in the document vector and the term ratio threshold.

4. A large-model financial data synthesis method based on text correlation as claimed in claim 1, characterized in that: The loss function adopts contrast loss, and the loss function formula is: Among them, the positive sample pair (D1, D2) is similar financial documents, the negative sample pair (D1, D3) is dissimilar financial documents, sim(D1, D2) represents the similarity between documents D1 and D2, sim(D1, D3) represents the similarity between documents D1 and D2, and t is the temperature parameter.

5. A large-model financial data synthesis method based on text correlation as claimed in claim 1, characterized in that: The formula for adjusting the weight of the document with preset financial terms or financial events in the negative sample financial document group is: oh adjusted =ω original +oh term ×f term (D)+ω event ×f event (D) where ω original is the original weight, ω term is the weight of financial terms, ω event is the weight of financial events, f term (D) is the frequency of occurrence of financial terms in the document, f event (D) Frequency of occurrence of financial events in documents.

6. A method for synthesizing large-model financial data based on text correlation as claimed in claim 1, characterized in that: The data deduplication also includes: For any two document vectors whose cosine similarity is higher than the high similarity threshold in the similarity threshold, the financial concept set and entity relationship set are extracted through the large language model to define the difference index formula and the semantic difference threshold, and it is determined whether the difference between any two document vectors calculated according to the difference index formula is greater than the semantic difference threshold. If so, the any two document vectors have important semantic differences, and the corresponding any two document vectors are retained.

7. A method for synthesizing large-scale financial data based on text correlation as claimed in claim 1, characterized in that: The greedy algorithm or genetic algorithm is used to traverse the optimization path according to the priority.

8. A large-model financial data synthesis system based on text relevance, characterized in that: The method for synthesizing large-model financial data based on text correlation as claimed in claim 1 comprises: A text vectorization module, used to convert the financial documents in the pre-vectorized document library into low-dimensional vector representations, construct a special vectorized financial corpus to facilitate subsequent cosine similarity calculations, and perform data deduplication on the vectorized financial corpus to obtain a deduplicated vectorized financial corpus; A corpus graph construction module, used for constructing a financial corpus graph according to the arbitrary document vector; The path search module is used to calculate the risk assessment value, market trend relevance and the average distance index between any document vector and any other document vector based on the return standard deviation, determine the priority of any vector document, and traverse the optimized path according to the priority to obtain the optimal path to ensure that the documents are arranged according to relevance; The data synthesis module is used to preferentially select documents involving the same financial event or related financial concepts according to the optimal path, and sequentially truncate or supplement the corpus according to the input length required by the large language model to construct a context with more closely related semantics to obtain a pre-training data set; The pre-training data extraction interface is used to extract pre-training data sets as input for large language models for training to enhance the performance of the model in the financial field.

Images