Chinese financial multi-task large model based on adaptive semantic space learning

Abstract

The invention provides a Chinese financial multi-task large model based on adaptive semantic space learning, and the method specifically comprises the following steps: S1, carrying out the adaptive selection of LoRA experts and data thereof, and obtaining the data of the LoRA experts; s2, collecting Chinese financial fine tuning data from the financial field; and S3, constructing a multi-task data set based on the LoRA expert data and the Chinese finance fine tuning data, and training a Chinese finance multi-task large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly relates to a Chinese financial multi-task large model based on adaptive semantic space learning. Background Art

[0002] Large language models have shown amazing strength in various tasks in the field of natural language processing. With their powerful knowledge storage and context understanding capabilities, they have been widely used in many professional fields such as finance and law. In the process of building large language models, the instruction fine-tuning technology plays a core role. It uses supervised datasets to fine-tune the model, helping the model to make a style transition from simple text continuation to complex task answering. However, restricted by the data volume and computing resources in specific fields, instruction fine-tuning often needs to rely on parameter-efficient fine-tuning strategies. By only updating or adding a small number of parameters to the base model, the model's response ability to instructions can be significantly improved. The low-rank adaptation algorithm realizes modular transformation and comprehensive performance improvement by introducing a decomposable low-parameter bypass matrix into the original model, and has the simple "plug-and-play" feature.

[0003] Currently, one of the research hotspots in the field of natural language processing is to use a mixture of experts (MoE) model to address the challenges of multi-task processing. By integrating multiple expert models and leveraging the expertise of each model in sub-problems, the overall performance can be improved when dealing with complex tasks. Zadouri first proposed combining the LoRA method with the MoE framework, using a token-level soft routing mechanism to weight-fuse the outputs of each expert model, effectively improving the model performance. Subsequently, researchers further proposed the LoRA MoE method, which divides the expert models into two groups: one group focuses on handling general world knowledge, and the other group is dedicated to learning new tasks encountered during instruction fine-tuning. This design aims to enhance the model's ability to handle downstream tasks while retaining the world knowledge accumulated by the large model. These multi-expert strategies have shown advantages in dealing with different problems, but their token-level processing and the fixity of the expert and routing strategies after training limit the model's ability to quickly adapt to new instructions or scenarios.

[0004] To solve this problem, researchers proposed the LoraRetriever method, which draws on the retrieval idea. It takes the average of the 12 data embeddings in each expert model as the LoRA's own embedding, and uses the embedding of the input problem to retrieve the most matching LoRA, thus effectively making up for the deficiencies of multi-expert models in dynamic addition and deletion, and further improving the performance and generalization ability of the hybrid training model.

[0005] Although the above methods are excellent in enhancing the dynamic adaptability of adapters such as LoRA, they often rely on task types to manually partition the training data when training LoRA, without fully considering the intrinsic connections of the data in the semantic space. This processing method may lead to biases in the embeddings of LoRA experts, thus affecting the overall performance of the system. Summary of the Invention

[0006] The present invention is made to solve the above problems, and the purpose is to provide a Chinese financial multi-task large model based on adaptive semantic space learning.

[0007] The present invention provides a Chinese financial multi-task large model based on adaptive semantic space learning, having the following features, specifically including the following steps: S1, adaptively select LoRA experts and their data to obtain LoRA expert data; S2, collect Chinese financial fine-tuning data from the financial field; S3, construct a multi-task data set based on the LoRA expert data and the Chinese financial fine-tuning data, and train a Chinese financial multi-task large language model.

[0008] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it may also have the following features: Among them, in step S1, when performing the adaptive selection of LoRA experts, optimize the data partition to avoid potential conflicts between tasks, and ensure that for each input, the expert who is best at handling this type of problem can be matched. Specifically, it includes the following sub-steps: S1-1, add new instructions to each sub-task in each task set of LoRA experts for instruction expansion, so as to enhance the generalization ability of the system to diverse instructions; S1-2, use the sentence encoder Emb(·) to splice each instruction and its input data for encoding, obtain the embedding vectors of all data in the same semantic space, and obtain the data in the multi-task data set.

[0009] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it may also have the following features: Among them, step S3 includes the following sub-steps: Among them, S3-1, use the K-Means clustering method to perform semantic space clustering on the data in the multi-task data set, so as to optimize the data partition of the tasks, and finally form six categories of clusters. The clustering process can be represented by the following formula: Among them, X represents the set of all data points, C i is the set of data points in the i-th cluster, and μ i is the centroid of the i-th cluster. For each LoRA expert, select the centroid of its cluster as the semantic embedding of the expert. The centroid of each cluster is the average position of the semantic embeddings of all points in the cluster. The calculation formula is as follows: Among them, C i is the set of data points in the i-th cluster, and |C i | represents the number of elements in the set C i . μ i represents the mean of all points in cluster i, that is, the centroid of this cluster. Whenever there is user input, the system finds the expert closest to the semantic embedding of the user input through the following formula for response:

[0010] Emb(x) is the embedding vector of the user input, Emb(e) is the semantic embedding vector of the expert, and e * is the selected expert. Through this matching method, the system can find the expert most suitable for the training task in the semantic space.

[0011] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it may also have the following features: Among them, step S3 further includes the following sub-steps: S3-2, design an adaptively adjusted A-DBSCAN algorithm based on the DBSCAN algorithm, perform nested clustering on the data in each cluster, dynamically adjust the connection number of different regions according to the data density, and screen out a small amount of data; S3-3, use the small amount of screened data to perform preliminary LoRA fine-tuning on the Chinese financial multi-task large language model, and at the same time consider the conflict situation between the data and the knowledge of the large model itself, adaptively supplement the necessary data points not selected in step S3-2, make the fine-tuning data distribution smoother, and enhance the diversity of data selection. According to the score difference of the Chinese financial multi-task large language model for the unselected data before and after preliminary training, design two scoring mechanisms to evaluate the value of each data for the current Chinese financial multi-task large language model, screen out 4000 pieces of data in each cluster, accounting for about 10% of the total data volume in total, and use six types of data after adaptive semantic distribution smoothing to train six different LoRA expert models respectively to adapt to different financial domain tasks. When dealing with specific financial problems, by calculating the similarity between the representation of the problem in the same semantic space and the representations of the six LoRA experts, automatically select the most suitable LoRA expert to answer.

[0012] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it may also have the following features: Among them, step S3-2 further includes the following sub-steps: S3-2-1, the A-DBSCAN algorithm evaluates the local density of data points in the semantic space through the distance calculation framework of the K-nearest neighbor algorithm. The K-nearest neighbor algorithm is KNN. The local density of each data point is the reciprocal of the average distance to its k nearest neighbors. Its mathematical expression is: Among them, d(x i , x ij ) represents the data point xi and its j-th nearest neighbor x ij the distance between; S3-2-2, the KNN algorithm sorts the data points according to the calculated local density values to form a priority queue, and preferentially processes the data points with higher local density. In each iteration, the point with the highest local density in the queue is selected as the starting point, and a cluster is generated around this point. In the adaptive process, the KNN algorithm uses the median of the KNN distances of all data points in the queue to define the neighborhood radius ε, and heuristically sets the initial value of the number of neighborhood nodes MinPts to: where ρ max represents the global maximum local density, that is, the local density of the first data point in the initial priority queue. After each cluster is formed, MinPts is updated according to the following formula to adapt to the current local density environment: Here, ρ current represents the local density of the first point in the current priority queue. This dynamic adjustment strategy enables the algorithm to more flexibly adapt to data distributions with different densities, improving the accuracy and efficiency of clustering. In each cluster, downsampling and upsampling are respectively performed on high-density regions and low-density regions. Each cluster selects nearly 2000 fine-tuning data, and the discrete data in the extremely low-density region is regarded as noise.

[0013] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it can also have the following feature: Among them, the two scoring mechanisms in step S3-3 include a difference scoring mechanism and a ratio scoring mechanism, and the difference scoring mechanism and the ratio scoring mechanism are respectively defined as:

[0014] Score diff (x) = LLM Raw (x) - LLM LoRA (x)

[0015]

[0016] Score llm (x) = Score diff (x) + Score prop (x)

[0017] where LLM Raw (x) and LLM LoRA (x) are calculated by the Rouge method. In addition, in order to ensure the quality of the newly selected data and the coverage of the entire clustering cluster, inspired by the MMR formula, we designed a utility function for the training data:

[0018]

[0019] where μ represents the clustering center point, dnew Represents the data points to be added, D selected Represents the set of data points that have been selected. λ1, λ2, and λ3 are three adjustable weight parameters used to adjust the contributions of similarity, diversity, and the scores of the Chinese financial multi-task large language model to the final utility value of the data points.

[0020] In the Chinese financial multi-task large model based on adaptive semantic space learning provided by the present invention, it may also have the following features: Among them, the following sub-steps are further included: S4. To verify the effectiveness of the LoRA adaptive selection algorithm, the six LoRA experts after clustering are tested one by one on the CFLEB dataset and the FinEval dataset.

[0021] Functions and effects of the invention

[0022] According to the Chinese financial multi-task large model based on adaptive semantic space learning involved in the present invention, the present invention proposes an Adaptive Semantic Space Learning (ASSL) framework, which realizes the adaptive selection of LoRA experts and their data, and trains a Chinese financial multi-task large language model "Yintong". It fully considers the internal relationship of data in the semantic space, avoids the deviation of the embedding of LoRA experts. After testing, in the experiment of CFLEB, we used 10% of the total data to achieve test scores similar to those of the full-data fine-tuning model, and even significantly exceeded the full-data fine-tuning model in the FinNSP2 index. At the same time, the model that performs full-data fine-tuning on each cluster has a certain improvement compared with the full-data fine-tuning model on the CFLEB dataset. On the untrained FinEval evaluation benchmark, we tested the generalization ability of the model and its comprehension ability of financial domain knowledge. The accuracy rate of the trained model decreased slightly in mathematical calculations and multiple-choice questions of qualification exams, but increased significantly in multiple-choice questions of economic and financial types. The average score of FinEval also increased to a certain extent compared with the original model. Description of the drawings

[0023] Figure 1 Is the learning framework of the adaptive semantic space in the embodiment of the present invention;

[0024] Figure 2 Is the flow diagram of the Chinese financial multi-task large model based on adaptive semantic space learning in the embodiment of the present invention;

[0025] Figure 3 Is the adaptive semantic space data redistribution process in the embodiment of the present invention; and

[0026] Figure 4This is the selection of experts for each type of task in the CFLEB dataset in this embodiment. Detailed implementation manners

[0027] In order to make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the following embodiments will specifically elaborate on the Chinese financial multi-task large model based on adaptive semantic space learning of the present invention in conjunction with the accompanying drawings.

[0028] Figure 1 This is the learning framework of the adaptive semantic space in the embodiment of the present invention. Figure 2 This is the process schematic diagram of the Chinese financial multi-task large model based on adaptive semantic space learning in the embodiment of the present invention.

[0029] As Figure 1 shown, in this embodiment, an adaptive semantic space learning framework ASSL is proposed, which realizes the adaptive selection of LoRA experts and their data, and collects 220,000 Chinese financial fine-tuning data from 23 different sources in the financial field to construct a multi-task dataset, and trains a Chinese financial multi-task large language model named "SilverSight". The process schematic diagram is as Figure 2 shown.

[0030] The Chinese financial multi-task large model based on adaptive semantic space learning in this embodiment specifically includes the following steps:

[0031] S1. Perform adaptive selection on LoRA experts and their data to obtain LoRA expert data.

[0032] When performing the adaptive selection of LoRA experts, optimize the data division to avoid potential conflicts between tasks, and ensure that for each input, the expert most proficient in handling this type of problem can be matched. Step S1 specifically includes the following sub-steps:

[0033] S1-1. Add new instructions to each sub-task in the task sets of LoRA experts for instruction expansion, so as to enhance the generalization ability of the system to diverse instructions.

[0034] S1-2. Use the sentence encoder Emb(·) to encode the concatenation of each instruction and its input data to obtain the embedding vectors of all data in the same semantic space, and obtain the data in the multi-task dataset.

[0035] S2. Collect Chinese financial fine-tuning data from the financial field.

[0036] S3. Based on the LoRA expert data and the Chinese financial fine-tuning data, construct a multi-task dataset and train a Chinese financial multi-task large language model.

[0037] Step S3 includes the following sub-steps:

[0038] S3-1. Use the K-Means clustering method to perform semantic space clustering on the data in the multi-task dataset, so as to optimize the data partitioning of the tasks, and finally form six categories of clustering. The clustering process can be represented by the following formula:

[0039]

[0040] Among them, X represents the set of all data points, C i is the set of data points in the i-th cluster, and μ i is the centroid of the i-th cluster.

[0041] For each LoRA expert, select the centroid of its cluster as the semantic embedding of the expert. The centroid of each cluster is the average position of the semantic embeddings of all points in the cluster. The calculation formula is as follows:

[0042]

[0043] Among them, C i is the set of data points in the i-th cluster, and |C i | represents the number of elements in the set C i , μ i represents the mean of all points in cluster i, that is, the centroid of the cluster. Whenever there is a user input, the system finds the expert closest to the user input's semantic embedding through the following formula for response:

[0044]

[0045] Emb(x) is the embedding vector of the user input, Emb(e) is the semantic embedding vector of the expert, and e * is the selected expert. Through this matching method, the system can find the expert most matching the training task in the semantic space.

[0046] Figure 3 is the adaptive semantic space data redistribution process in the embodiment of the present invention.

[0047] Clustering the multi-task supervised data can isolate conflicting tasks and aggregate mutually enhancing tasks, but it will cause problems such as unbalanced data ratio and inconsistent instructions. Therefore, we design a two-stage data redistribution operation for each cluster, which can achieve the effect comparable to fine-tuning on the full dataset while performing effective fine-tuning on a small dataset. The specific process is as Figure 3 shown.

[0048] S3-2. In the first stage, to address the issue of data imbalance in the clustering clusters, an adaptive adjusted A-DBSCAN algorithm is designed based on the DBSCAN algorithm. The data in each cluster is nested-clustered, and the number of connections in different regions is dynamically adjusted according to the data density to filter out a small amount of data.

[0049] Step S3-2 specifically includes the following sub-steps:

[0050] S3-2-1. The A-DBSCAN algorithm evaluates the local density of data points in the semantic space through the distance calculation framework of the K-Nearest Neighbor (KNN) algorithm. The K-Nearest Neighbor algorithm is KNN. The local density of each data point is the reciprocal of the average distance to its k nearest neighbors, and its mathematical expression is:

[0051]

[0052] where d(x i , x ij ) represents the distance between the data point x i and its jth nearest neighbor x ij .

[0053] S3-2-2. The KNN algorithm sorts the data points according to the calculated local density values to form a priority queue, and preferentially processes the data points with higher local density. In each iteration, the point with the highest local density in the queue is selected as the starting point, and a cluster is generated around this point.

[0054] In the adaptive process, the KNN algorithm defines the neighborhood radius ε using the median of the KNN distances of all data points in the queue, and heuristically sets the initial value of the neighborhood node number MinPts to:

[0055]

[0056] where ρ max represents the global maximum local density, that is, the local density of the first data point in the initial priority queue. After each cluster is formed, MinPts is updated according to the following formula to adapt to the current local density environment:

[0057]

[0058] Here, ρ currentRepresents the local density of the first point in the current priority queue. This dynamic adjustment strategy enables the algorithm to more flexibly adapt to data distributions with different densities, improving the accuracy and efficiency of clustering. In each cluster, downsampling and upsampling are respectively performed on high-density regions and low-density regions. Approximately 2,000 fine-tuning data are selected for each cluster, and discrete data in extremely low-density regions are regarded as noise.

[0059] S3-3, The second stage. Use the small amount of data screened to perform preliminary LoRA fine-tuning on the Chinese financial multi-task large language model. At the same time, consider the conflict situation between the data and the knowledge of the large model itself, and adaptively supplement the necessary data points not selected in step S3-2 to make the fine-tuning data distribution smoother and enhance the diversity of data selection. According to the score differences of the Chinese financial multi-task large language model for the unselected data before and after the preliminary training, design two scoring mechanisms to evaluate the value of each data point for the current Chinese financial multi-task large language model.

[0060] The two scoring mechanisms include the difference scoring mechanism and the proportional scoring mechanism.

[0061] The difference scoring mechanism and the proportional scoring mechanism are respectively defined as:

[0062] Score diff (x) = LLM Raw (x) - LLM LoRA (x)

[0063]

[0064] Score llm (x) = Score diff (x) + Score prop (x)

[0065] Among them, LLM Raw (x) and LLM LoRA (x) are calculated by the Rouge method. In addition, to ensure the quality of the newly selected data and the coverage of the entire cluster, inspired by the MMR formula, we design a utility function for the training data:

[0066]

[0067] Among them, μ represents the cluster center point, d new represents the data point to be added, D selected represents the set of data points that have been selected, and λ1, λ2, and λ3 are three adjustable weight parameters used to adjust the contributions of similarity, diversity, and the score of the Chinese financial multi-task large language model to the final utility value of the data point.

[0068] 4000 pieces of data are screened out from each cluster, accounting for about 10% of the total data volume. Six different LoRA expert models are trained respectively using six types of data after adaptive semantic distribution smoothing to adapt to different financial domain tasks. When dealing with specific financial problems, by calculating the similarity between the representation of the problem in the same semantic space and the representations of the six LoRA experts, the most suitable LoRA expert is automatically selected to answer, realizing an efficient, adaptive and clearly divided financial multi-task large model system.

[0069] S4. To verify the effectiveness of the LoRA adaptive selection algorithm, the six LoRA experts after clustering are tested one by one on the CFLEB dataset and the FinEval dataset.

[0070] Table 1

[0071]

[0072]

[0073] Table 2

[0074] FinEval Accounting Certificate Economy Finance AVG Qwen-7B-Chat 44.5 53.6 52.1 51.5 50.5 ChatGPT 45.2 55.1 61.6 59.3 55.0 Qwen-1.5-7B-Chat 69.5 71.3 62.8 65.6 67.8 GPT-4 59.3 70.4 74.5 71.0 68.6 <![CDATA[SilverSight LoRA-0 > 68.9 72.2 64.7 65.9 68.3 <![CDATA[SilverSight LoRA-1 > 66.9 70.7 59.4 62 65.3 <![CDATA[SilverSight LoRA-2 > 62 65.6 58.5 62.6 62.6 <![CDATA[SilverSight LoRA-3 > 67.5 69.8 67.1 67.2 68 <![CDATA[SilverSight LoRA-4 > 68.5 70 63.8 66.2 67.5 <![CDATA[SilverSight LoRA-5 > 68.2 71.3 63.3 65.6 67.5 SilverSight(our,10%) 67.9 70 66.7 67.9 68.3

[0075] Table 1 is the evaluation result of CFLEB in Table 1. Table 2 is the evaluation result of FinEval.

[0076] As shown in Table 1 and Table 2, in the experiment of CFLEB, we achieved test scores similar to those of the full-data fine-tuning model using 10% of the total data, and even significantly exceeded the full-data fine-tuning model in the FinNSP2 metric. At the same time, the model that performs full-data fine-tuning on each cluster has a certain improvement compared to the full-data fine-tuning model on the CFLEB dataset, fully verifying the effectiveness of the ASSL framework. On the untrained FinEval evaluation benchmark, we examined the generalization ability of the model and the comprehension ability of financial domain knowledge. The accuracy rate of the trained model decreased slightly in mathematical calculations and multiple-choice questions in qualification exams, but increased significantly in multiple-choice questions of economic and financial types. The average score of FinEval also had a certain improvement compared to the original model.

[0077] Figure 4 It is the selection situation of experts for each type of task in the CFLEB dataset in this embodiment.

[0078] The experimental results show that the performance of the LoRA adaptive selection algorithm on each task is similar to that of the single LoRA expert with the best performance on that task, proving that most of the time this LoRA adaptive selection algorithm can select the most suitable LoRA expert to answer questions. At the same time Figure 4It not only proves the effectiveness of the LoRA adaptive selection algorithm, but also proves that the simultaneous use of the clustering method and the LoRA adaptive selection algorithm can ensure the consistency and coherence of the semantic space, ultimately improving the comprehensive performance of the system.

[0079] Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A Chinese financial multi-task large model based on adaptive semantic space learning, characterized in that, Specifically, it includes the following steps: S1. Adaptively select LoRA experts and their data to obtain LoRA expert data; S2. Collect Chinese financial fine-tuning data from the financial field; S3. Based on the LoRA expert data and the Chinese financial fine-tuning data, construct a multi-task dataset and train a Chinese financial multi-task large language model.

2. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 1, wherein: Among them, In step S1, when performing the adaptive selection of LoRA experts, optimize the data division to avoid potential conflicts between tasks and ensure that for each input, the expert most proficient in handling such problems can be matched. Specifically, it includes the following sub-steps: S1-1. Add new instructions to each sub-task in the task sets of the LoRA experts for instruction expansion, thereby enhancing the system's generalization ability for diverse instructions; S1-2, concatenate each instruction with its input data and use the sentence encoder Emb(·) to encode them, obtaining the embedding vectors of all data in the same semantic space and getting the data in the multi-task dataset.

3. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 1, wherein: Among them, Step S3 includes the following sub-steps: Among them, S3-1. Use the K-Means clustering method to perform semantic space clustering on the data in the multi-task dataset, thereby optimizing the data division of the tasks. Finally, six categories of clusters are formed. The clustering process can be represented by the following formula: where X represents the set of all data points, and C i is the set of data points in the i-th cluster, and μ i is the centroid of the i-th cluster For each LoRA expert, select the centroid of its cluster as the semantic embedding of the expert. The centroid of each cluster is the average position of the semantic embeddings of all points in the cluster. The calculation formula is as follows: Among them, C i is the set of data points in the i-th cluster, and |C i | represents the number of elements in the set C i . μ i represents the mean of all points in cluster i, that is, the centroid of this cluster. Whenever there is user input, the system finds the expert closest to the semantic embedding of the user input for response through the following formula: Emb(x) is the embedding vector input by the user, Emb(e) is the semantic embedding vector of the expert, and e * is the selected expert. Through this matching method, the system can find the expert that best matches the training task in the semantic space.

4. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 3, wherein: Among them, Step S3 further includes the following sub-steps: S3-2. Design an adaptively adjusted A-DBSCAN algorithm based on the DBSCAN algorithm to perform nested clustering on the data in each cluster, dynamically adjust the number of connections in different regions according to the data density, and screen out a small amount of data; S3-3. Use the screened small amount of data to perform preliminary LoRA fine-tuning on the Chinese financial multi-task large language model. At the same time, consider the conflict situation between the data and the knowledge of the large model itself, and adaptively supplement the necessary data points not selected in step S3-2 to make the fine-tuning data distribution smoother and enhance the diversity of data selection. According to the score difference of the Chinese financial multi-task large language model for the unselected data before and after preliminary training, design two scoring mechanisms to evaluate the value of each data for the current Chinese financial multi-task large language model. Screen out 4000 pieces of data in each cluster, accounting for about 10% of the total data volume. Use the six categories of data after adaptive semantic distribution smoothing to train six different LoRA expert models to adapt to different financial field tasks. When dealing with specific financial problems, automatically select the most suitable LoRA expert to answer by calculating the similarity between the representation of the problem in the same semantic space and the representations of the six LoRA experts.

5. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 4, wherein: Among them, The step S3-2 further includes the following sub-steps: S3-2-1. The A-DBSCAN algorithm evaluates the local density of data points in the semantic space through the distance calculation framework of the K-nearest neighbor algorithm. The K-nearest neighbor algorithm is KNN. The local density of each data point is the reciprocal of the average distance to its k nearest neighbors, and its mathematical expression is: where d(x i , x ij ) represents the distance between the data point x i and its j-th nearest neighbor x ij ; S3-2-2. The KNN algorithm sorts the data points according to the calculated local density values to form a priority queue, and preferentially processes data points with higher local density. In each iteration, the point with the highest local density in the queue is selected as the starting point, and a cluster is generated around this point. In the adaptive process, the KNN algorithm defines the neighborhood radius ε using the median of the KNN distances of all data points in the queue, and heuristically sets the initial value of the number of neighborhood nodes MinPts to: where ρ max represents the global maximum local density, that is, the local density of the first data point in the initial priority queue. After each cluster is formed, MinPts is updated according to the following formula to adapt to the current local density environment: Here, ρ current represents the local density of the first point in the current priority queue. This dynamic adjustment strategy enables the algorithm to more flexibly adapt to data distributions with different densities, improving the accuracy and efficiency of clustering. Downsampling and upsampling are respectively performed on high-density and low-density regions in each cluster. Nearly 2,000 fine-tuning data are selected for each cluster, and the discrete data in the extremely low-density region are regarded as noise.

6. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 5, wherein: Among them, The two scoring mechanisms in the step S3-3 include a difference scoring mechanism and a ratio scoring mechanism. The difference scoring mechanism and the ratio scoring mechanism are respectively defined as: Score diff (x) = LLM Raw (x) - LLM LoRA (x) Score llm (x) = Score diff (x) + Score prop (x) Among them, LLM Raw (x) and LLM LoRA (x) are calculated by the Rouge method. In addition, to ensure the quality of the newly selected data and the coverage of the entire clustering cluster, inspired by the MMR formula, we designed a utility function for the training data: Among them, μ represents the clustering center point, and d new represents the data point to be added, and D selected represents the set of data points that have been selected. λ1, λ2, and λ3 are three adjustable weight parameters used to adjust the contributions of similarity, diversity, and the score of the Chinese financial multi-task large language model to the final utility value of the data point.

7. The Chinese financial multi-task large model based on adaptive semantic space learning according to claim 1, wherein: Among them, It further includes the following sub-steps: S4. To verify the effectiveness of the LoRA adaptive selection algorithm, the six LoRA experts after clustering are tested one by one on the CFLEB dataset and the FinEval dataset.

Images