Reinforcement learning buying and selling point prediction model training method based on historical experience

Abstract

The invention discloses a historical experience-based reinforcement learning buying and selling point prediction model training method, which is used for the financial field and comprises the following steps of: obtaining financial transaction data, and preprocessing to obtain a simplified time-sharing graph; on the basis of the simplified time-sharing diagram, a band theory analysis framework is utilized to construct a supervision fine tuning data set; performing pre-fine tuning on the large model according to the supervised fine tuning data set to obtain an enhanced large model; utilizing a reward model and reinforcement learning near-end strategy optimization to score and rearrange the reinforced large model, and performing fine tuning on the supervision fine tuning data set; and training the initial large model according to the fine-tuned supervised fine-tuning data set to obtain a buying and selling point prediction model. The invention provides a training framework for learning financial knowledge and transaction experience from a historical experience feedback technology by applying reinforcement learning, the band analysis and purchase point prediction capabilities of the model are improved, and a more competitive transaction strategy can be realized in a quickly changing financial market.

Description

Technical Field

[0001] The present invention relates to the financial field, and in particular, to a training method for a reinforcement learning buying and selling point prediction model based on historical experience. Background Art

[0002] With the rapid development of artificial intelligence technology, the application of large language models (LLMs) in the financial field has gradually matured, especially in market forecasting, risk management and trading strategy development, which has become a hot research topic. Large models can significantly improve the prediction accuracy of market trends through deep learning of multi-dimensional data such as historical market data, news sentiment and macroeconomic events. In addition, with the advancement of multimodal large model technology, these models can learn the correlation and complementarity between different modal data through training of massive data. This enables them to effectively process large amounts of unstructured data in the financial field, provide more accurate analysis and prediction, and thus gain a competitive advantage in the dynamically changing financial environment.

[0003] At present, the market mainly relies on manual trading for band analysis and buying point prediction. Manual trading refers to the process in which traders execute transactions manually based on their own judgment and decision-making. This trading method is in sharp contrast to automated trading, which relies on computer programs to automatically execute transactions according to preset algorithms. This model has certain limitations, and trading decisions are often affected by personal emotions, fatigue and other factors, which may lead to irrational decisions. In general, manual trading is a trading method that emphasizes subjective judgment and is suitable for traders with rich experience and high self-discipline, but its effectiveness may be limited in a rapidly changing market environment.

[0004] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0005] In order to overcome the above problems, the present invention aims to propose a reinforcement learning buy and sell point prediction model training method based on historical experience, with the aim of solving the problem of enhancing intraday trading and buy and sell point prediction capabilities.

[0006] To this end, the specific technical solution adopted by the present invention is as follows:

[0007] A method for training a buying and selling point prediction model based on reinforcement learning of historical experience, the method comprising the following steps:

[0008] S1. Acquire financial transaction data, pre-process the financial transaction data, and obtain a simplified time-sharing chart;

[0009] S2. Based on the simplified time-sharing graph, using the band theory analysis framework, a supervised fine-tuning dataset is constructed;

[0010] S3. Pre-fine-tune the multimodal large model according to the supervised fine-tuning dataset to obtain an enhanced multimodal large model;

[0011] S4. Use the reward model and reinforcement learning proximal policy optimization to score and rearrange the reinforced multimodal large model, and fine-tune the supervised fine-tuning dataset based on the scoring and rearrangement results;

[0012] S5. According to the fine-tuned supervised fine-tuning dataset, the multimodal large model is trained to obtain the buy and sell point prediction model.

[0013] Optionally, obtaining financial transaction data, preprocessing the financial transaction data, and obtaining a simplified time-sharing chart includes the following steps:

[0014] S11. Obtaining financial transaction data;

[0015] S12. Identify erroneous values ​​or outliers in financial transaction data and clean them;

[0016] S13, converting the cleaned financial transaction data to obtain a time-sharing chart;

[0017] S14. Optimize the fluctuations in the time-sharing chart using an optimization algorithm to obtain a simplified time-sharing chart.

[0018] Optionally, optimizing the fluctuations in the time-sharing chart using an optimization algorithm to obtain a simplified time-sharing chart includes the following steps:

[0019] S141, identifying all extreme points in the time-sharing chart curve;

[0020] S142, calculating the distance between adjacent extreme value points;

[0021] S143, determine whether the distance between adjacent extreme value points is greater than a preset threshold, if so, retain the adjacent extreme value points, otherwise, merge the adjacent extreme value points, and repeat steps S11-S12 to calculate the distance of subsequent adjacent extreme value points until the distance between all adjacent extreme value points is greater than the preset threshold.

[0022] Optionally, based on the simplified time-sharing chart, using the band theory analysis framework, constructing a supervised fine-tuning dataset includes the following steps:

[0023] S21, split the band theory analysis framework and construct corresponding prompt words from multiple dimensions;

[0024] S22, inputting the constructed prompt words and the corresponding simplified time-sharing diagram images into the large language model to obtain multiple output results;

[0025] S23, integrating the multiple output results and inputting them into the large language model again to obtain the final analysis process and buying point prediction conclusion;

[0026] S24. Package the prompt words, simplified time-sharing graph images, and the output results of the large language model into a supervised fine-tuning dataset.

[0027] Optionally, the band theory analysis framework is split and corresponding prompt words are constructed from multiple dimensions, including:

[0028] The band theory analysis framework is divided into all-day data, the band where the buying point is located, and the preset time before the band. Corresponding prompt words are constructed from the three dimensions of overall market trend, buying point time and band identification.

[0029] Optionally, the multimodal large model is pre-fine-tuned according to the supervised fine-tuning dataset, and the enhanced multimodal large model includes:

[0030] S31, import the supervised fine-tuning dataset into the multimodal large model;

[0031] S32. Use the multimodal big model to effectively fuse the visual information and text data in the supervised fine-tuning dataset to obtain an enhanced multimodal big model.

[0032] Optionally, using the multimodal large model to effectively fuse the visual information and text data in the supervised fine-tuning dataset to obtain an enhanced multimodal large model includes the following steps:

[0033] S321, using a visual encoder to encode the input visual data image to obtain a visual data code, and converting the visual data code through a linear layer to obtain a linear visual code;

[0034] S322, converting the query data into a linear query code using an embedding layer;

[0035] S323, concatenating the linear visual code and the linear query code to obtain a combined feature;

[0036] S324. Use the language model in combination with the combined features to generate an enhanced multimodal large model.

[0037] Optionally, the expression of the enhanced multimodal large model is:

[0038]

[0039] Where, X a represents the final output result; f represents the language model; H v represents linear visual encoding; H q represents linear query encoding; W represents linear layer, Z v represents visual data encoding; concat represents concatenation operation; g represents visual encoder; X vRepresents a visual data image.

[0040] Optionally, using the reward model and reinforcement learning proximal policy optimization, scoring and re-ranking the reinforced multimodal large model, and fine-tuning the supervised fine-tuning dataset according to the scoring and re-ranking results includes the following steps:

[0041] S41, collect various band pictures to the picture library;

[0042] S42. Based on the output of the enhanced multimodal large model, the re-ranking model is used to retrieve the paragraph of the knowledge base corresponding to the answer to obtain a similarity score;

[0043] S43, according to the simplified time-sharing diagram, matching various bands in the image library, outputting the matching degree of each band type, and obtaining the matching degree weight;

[0044] S44, re-score the similarity scores based on the matching weights, and re-rank them according to the new scores;

[0045] S45, calculating the loss function according to the ranking, obtaining a difference result, and optimizing the reward model according to the difference result;

[0046] S46. Fine-tune the supervised fine-tuning dataset using reward model and reinforcement learning proximal policy optimization.

[0047] Optionally, the loss function is calculated as:

[0048]

[0049] In the formula, σ(.) represents the logistic function; loss(θ) represents the loss function; r θ (x, y) represents the reward score output by the reward model for a given prompt x and a generated response y; x represents the given prompt; y w represents the superior generation result; y1 represents the inferior generation result; D represents the result dataset compared by human annotators; E represents the expected value; Indicates normalization for all comparisons.

[0050] Compared with the prior art, this application has the following beneficial effects:

[0051] 1. This invention proposes a training technology for a buy / sell point prediction model based on historical experience by designing a comprehensive pipeline based on a large language model, which realizes the gradual learning of the model from general capabilities to field specialization, and from general capabilities to in-depth understanding and processing in specific fields. Through the band optimization algorithm, short-term fluctuations that have no actual trend significance are effectively shielded. This design ensures that the buy / sell point prediction model can filter out short-term interference caused by noise when identifying market trends.

[0052] 2. The present invention designs an analysis framework based on the band theory, aiming to enhance the knowledge reserve and professional ability of the big model in the financial field. The framework conducts in-depth analysis of the input time-sharing chart from multiple dimensions, including the full-day data, the band where the buying point is located, and the period before the band. This multi-dimensional analysis method can comprehensively summarize information and generate a more complete band analysis. At the same time, it avoids the high cost of manual labeling.

[0053] 3. In view of the relative lack of financial knowledge and trading experience contained in the data set, in order to enhance the professional quality of the buy and sell point prediction model, the present invention will introduce a reward model. By integrating books and trading diaries related to band theory, the buy and sell point prediction model will be given a richer financial background, thereby improving the predictive ability of the buy and sell point prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above characteristics, features and advantages of the present invention and their implementation methods and methods will become more clearly understood in conjunction with the following description of the embodiments, which are described in detail in conjunction with the accompanying drawings. Herein, a schematic diagram is shown:

[0055] Figure 1 is a flowchart of a method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to an embodiment of the present invention;

[0056] Figure 2 It is a schematic diagram of the LLaVa model structure in a historical experience-based reinforcement learning buying and selling point prediction model training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] According to an embodiment of the present invention, a method for training a buying and selling point prediction model based on reinforcement learning based on historical experience is provided.

[0059] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1-Figure 2 As shown, according to the historical experience-based reinforcement learning buying and selling point prediction model training method of an embodiment of the present invention, the method includes the following steps:

[0060] S1. Obtain financial transaction data, pre-process the financial transaction data, and obtain a simplified time-sharing chart.

[0061] Preferably, obtaining financial transaction data, preprocessing the financial transaction data, and obtaining a simplified time-sharing chart includes the following steps:

[0062] S11. Obtaining financial transaction data;

[0063] S12. Identify erroneous values ​​or outliers in financial transaction data and clean them;

[0064] S13, converting the cleaned financial transaction data to obtain a time-sharing chart;

[0065] S14. Optimize the fluctuations in the time-sharing chart using an optimization algorithm to obtain a simplified time-sharing chart.

[0066] Preferably, the method of optimizing the fluctuations in the time-sharing diagram by using an optimization algorithm to obtain a simplified time-sharing diagram comprises the following steps:

[0067] S141, identifying all extreme points in the time-sharing chart curve;

[0068] S142, calculating the distance between adjacent extreme value points;

[0069] S143, determine whether the distance between adjacent extreme value points is greater than a preset threshold, if so, retain the adjacent extreme value points, otherwise, merge the adjacent extreme value points, and repeat steps S11-S12 to calculate the distance of subsequent adjacent extreme value points until the distance between all adjacent extreme value points is greater than the preset threshold.

[0070] It should be explained that the data preprocessing stage includes data crawling, cleaning, conversion, and noise reduction. Among them, when cleaning the data, it is necessary to identify and correct the wrong values ​​or abnormal values ​​in the data, convert the data, reduce the small fluctuations in the time-sharing chart through the optimization algorithm, and finally generate a time-sharing chart with a simplified curve.

[0071] Small fluctuations in the time-sharing chart are usually caused by short-term noise or irrational trading behavior in the market. Such fluctuations do not reflect the real trend of stock prices, but are more of a short-term fluctuation in market sentiment or the immediate reaction of some traders. The present invention mainly optimizes the time-sharing chart through the following steps:

[0072] 1. Identify extreme points: First, the algorithm identifies all extreme points in the curve, which are important nodes for determining trend changes.

[0073] 2. Calculate the distance: Next, the algorithm calculates the distance between adjacent extreme points. If this distance is less than the preset threshold, the algorithm merges these extreme points to eliminate the noise caused by short-term fluctuations.

[0074] 3. Iterative processing: Finally, the algorithm repeats the second step until the distance between all adjacent extreme points is greater than the threshold. Through this iterative process, the algorithm ensures that the final curve can more clearly show the overall trend of the market.

[0075] S2. Based on the streamlined time-sharing chart, a supervised fine-tuning dataset is constructed using the band theory analysis framework.

[0076] Preferably, based on the simplified time-sharing chart and using the band theory analysis framework, constructing a supervised fine-tuning dataset includes the following steps:

[0077] S21, split the band theory analysis framework and construct corresponding prompt words from multiple dimensions;

[0078] S22, inputting the constructed prompt words and the corresponding simplified time-sharing diagram images into the large language model to obtain multiple output results;

[0079] S23, integrating the multiple output results and inputting them into the large language model again to obtain the final analysis process and buying point prediction conclusion;

[0080] S24. Package the prompt words, simplified time-sharing graph images, and the output results of the large language model into a supervised fine-tuning dataset.

[0081] Preferably, the band theory analysis framework is split and corresponding prompt words are constructed from multiple dimensions, including:

[0082] The band theory analysis framework is divided into three parts: all-day data, the band where the buying point is located, and the period before the band. Corresponding prompt words are constructed from the three dimensions of overall market trend, a period of time near the buying point, and band identification.

[0083] It needs to be explained that large models usually rely on massive amounts of general data for training, but lack targeted financial data, which makes them limited in understanding complex financial concepts and performing sophisticated technical analysis. To meet this challenge, the current technical solution is to introduce professional data sets. However, there is a lack of training data sets for professional analysis of time-sharing charts on the market. In order to solve this problem, the present invention designs an analysis framework based on band theory. The band theory analysis framework contains basic financial knowledge to enhance the analysis capabilities of large language models (such as GPT4o). The study ultimately generated an SFT (supervised fine-tuning) data set that includes "instructions-pictures-analysis process and buying point prediction conclusions". The specific steps are as follows:

[0084] 1. Construct prompt words: Split the band theory analysis framework into three parts, and construct corresponding prompt words from the three dimensions of overall market trend, a period of time near the buying point, and band identification. This division can ensure that the model pays attention to information at different levels during analysis.

[0085] 2. Input model: Input the constructed prompt words and the corresponding simplified time-sharing graph images into a large language model (such as GPT4o) to generate three output results. These output results will include a detailed analysis process.

[0086] 3. Combine the analysis results: After integrating the three output results, input them into the large language model (such as GPT4o) again, prompting the model to generate the final analysis process and buying point prediction conclusion. In this way, it can ensure that information from different dimensions is comprehensively considered, thereby improving the accuracy of the prediction.

[0087] Finally, the prompt words, simplified time-sharing chart images, and the output results of large language models (such as GPT4o) are packaged into a data set for fine-tuning the buy and sell point prediction model. This method not only improves the buy and sell point prediction model's understanding and application of the band theory, but also provides a more accurate basis for buying point prediction.

[0088] S3. Pre-fine-tune the multimodal large model according to the supervised fine-tuning dataset to obtain an enhanced multimodal large model.

[0089] Preferably, the multimodal large model is pre-fine-tuned according to the supervised fine-tuning dataset to obtain an enhanced multimodal large model including:

[0090] S31, import the supervised fine-tuning dataset into the multimodal large model;

[0091] S32. Use the multimodal big model to effectively fuse the visual information and text data in the supervised fine-tuning dataset to obtain an enhanced multimodal big model.

[0092] Preferably, using a multimodal large model to effectively fuse visual information and text data in a supervised fine-tuning dataset to obtain an enhanced multimodal large model includes the following steps:

[0093] S321, using a visual encoder to encode the input visual data image to obtain a visual data code, and then converting the visual data code through a linear layer to obtain a linear visual code;

[0094] S322, converting the query data into a linear query code using an embedding layer;

[0095] S323, concatenating the linear visual code and the linear query code to obtain a combined feature;

[0096] S324. Use the language model in combination with the combined features to generate an enhanced multimodal large model.

[0097] Preferably, the expression of the enhanced multimodal large model is:

[0098]

[0099] Where, X a represents the final output result; f represents the language model; H v represents linear visual encoding; H q represents linear query encoding; W represents linear layer, Z v represents visual data encoding; concat represents concatenation operation; g represents visual encoder; X v Represents a visual data image.

[0100] It should be explained that since the analysis ability of the multimodal large model is not very strong compared with the large language model (such as GPT4o), we added a step to fine-tune the large model using the dataset generated by the large language model (such as GPT4o) to improve the analysis level of the large model. Figure 2 As shown in the figure (Vision Encoder represents the visual decoder, LanguageModel represents the language model, ProjectionW represents the linear layer, Image represents the image, Language Instruction represents the language instruction, and Language Response represents the language output), the structure of the large model (LLaVa) is shown, in which Vicuna is used as the language model and ViT-L / 14 pre-trained by CLIP (pre-trained in the joint embedding space of images and text) is used as the visual encoder. The model projects the visual features into the feature space of word embedding through a simple linear layer W. The specific workflow is as follows:

[0101] 1. First, use the visual encoder g to convert the input visual data X v Coded as Z v , and then transform it into H through a linear layer W v , for subsequent processing.

[0102] 2. Then, use the embedding layer to embed the query data X q Convert to H q , to ensure that they operate in the same feature space.

[0103] 3. Finally, H v and H q Concatenate and input the combined features into the language model fφ(.) to generate the final output X a .

[0104] Through this process, LLaVa (large model) can effectively integrate visual information and text data, thereby improving its application in financial technical analysis. The formula is:

[0105]

[0106] Where, X a represents the final output result; f represents the language model; H v represents linear visual encoding; H q represents linear query encoding; W represents linear layer, Z v represents visual data encoding; concat represents concatenation operation; g represents visual encoder; X v Represents a visual data image.

[0107] S4. Use the reward model and reinforcement learning proximal policy optimization to score and rearrange the reinforced multimodal large model, and fine-tune the supervised fine-tuning dataset based on the scoring and rearrangement results.

[0108] Preferably, using the reward model and reinforcement learning proximal strategy optimization, scoring and rearranging the reinforced multimodal large model, and fine-tuning the supervised fine-tuning dataset according to the scoring and rearrangement results includes the following steps:

[0109] S41, collect various band pictures to the picture library;

[0110] S42. Based on the output of the enhanced multimodal large model, the re-ranking model is used to retrieve the paragraph of the knowledge base corresponding to the answer to obtain a similarity score;

[0111] S43, according to the simplified time-sharing diagram, matching various bands in the image library, outputting the matching degree of each band type, and obtaining the matching degree weight;

[0112] S44, re-score the similarity scores based on the matching weights, and re-rank them according to the new scores;

[0113] S45, calculating the loss function according to the ranking, obtaining a difference result, and optimizing the reward model according to the difference result;

[0114] S46. Fine-tune the supervised fine-tuning dataset using reward model and reinforcement learning proximal policy optimization.

[0115] Preferably, the loss function is calculated as:

[0116]

[0117] In the formula, σ(.) represents the logistic function, which is used to map the difference to the interval [0,1], and is used to represent the probability that a result is considered better by the annotator; loss(θ) represents the loss function, which is used to measure the difference between the model prediction output and the actual target value; r θ (x, y) represents the reward score output by the reward model for a given prompt x and a generated response y; x represents the given prompt; y w represents the superior generation result; y1 represents the inferior generation result; D represents the result dataset compared by human annotators; E represents the expected value; Indicates normalization for all comparisons.

[0118] It should be explained that, given the relative lack of financial knowledge and trading experience in the dataset, this paper introduces a reward model to draw on its ideas for evaluating the analysis process of the multimodal large model. The goal is to enable the large model to learn an excellent analysis process so that it can make more accurate judgments in future decisions. To this end, a scorer will be designed to score and rearrange the output of the large model to optimize its performance. The specific steps are as follows:

[0119] 1. Save the standard band images in the training document to the image library for image verification. The various bands in the image library are hand-drawn bands by traders and obtained from the training document.

[0120] 2. For the n answers output by the multimodal large model, use the bge-reranker deep learning model to retrieve the paragraphs in the knowledge base corresponding to each answer and return the similarity score. bge-reranker is a reranking model specifically used for information retrieval and natural language processing, developed by the Beijing Zhiyuan Artificial Intelligence Research Institute. It reranks the candidate results based on the results of the preliminary retrieval to improve the relevance and quality of the final retrieval results.

[0121] 3. Use the time-sharing chart to match the bands in the image library, and output the matching degree of each band type as the weight.

[0122] 4. Use the matching degree in step 3 as the weight, adjust the scores of the n answers in step 2 to get new scores, and re-sort them according to the new scores.

[0123] 5. Calculate the loss based on the ranking and optimize the reward model.

[0124] 6. Fine-tune the SFT (supervised fine-tuning) model using the reward model and PPO strategy (reinforcement learning proximal policy optimization).

[0125] The loss formula is as follows. This study uses Pairwise Ranking Loss to compare K answers pairwise, where x represents the instruction and y represents the answer.

[0126] The calculation formula of the loss function is:

[0127]

[0128] In the formula, σ(.) represents the logistic function, which is used to map the difference to the interval [0,1], and is used to represent the probability that a result is considered better by the annotator; loss(θ) represents the loss function, which is used to measure the difference between the model prediction output and the actual target value; r θ (x, y) represents the reward score output by the reward model for a given prompt x and a generated response y; x represents the given prompt; y w represents the superior generation result; y1 represents the inferior generation result; D represents the result dataset compared by human annotators; E represents the expected value; Indicates normalization for all comparisons.

[0129] S5. According to the fine-tuned supervised fine-tuning dataset, the multimodal large model is trained to obtain the buy and sell point prediction model.

[0130] It needs to be explained that according to the iterative update of the supervised fine-tuning dataset, the multimodal large model is further fine-tuned to obtain a reinforcement learning buying and selling point prediction model based on historical experience.

[0131] In summary, with the help of the above technical solution of the present invention, the present invention proposes a training technology of a reinforcement learning buy and sell point prediction model based on historical experience by designing a comprehensive pipeline based on a large language model, which realizes the gradual learning of the model from general capabilities to field specialization, and the in-depth understanding and processing from general capabilities to specific fields. Through the band optimization algorithm, short-term fluctuations that have no actual trend significance are effectively shielded. This design ensures that the buy and sell point prediction model can filter out short-term interference caused by noise when identifying market trends; the present invention designs an analysis framework based on band theory, aiming to enhance the knowledge reserve and professional ability of the large model in the financial field. The framework conducts in-depth analysis of the input time-sharing chart from multiple dimensions, including all-day data, the band where the buying point is located, and the period before the band. This multi-dimensional analysis method can comprehensively summarize information and generate a more complete band analysis. At the same time, the high cost of manual labeling is avoided. In view of the relative scarcity of financial knowledge and trading experience contained in the data set, in order to enhance the professional quality of the buy and sell point prediction model, the present invention will introduce a reward model, and by integrating books and trading diaries related to band theory, give the buy and sell point prediction model a richer financial background, thereby improving the predictive ability of the buy and sell point prediction model.

[0132] Although the present invention has been disclosed as above with preferred embodiments, the embodiments are merely examples for the convenience of description and are not intended to limit the present invention. Those skilled in the art may make several changes and modifications without departing from the spirit and scope of the present invention. The scope of protection claimed by the present invention shall be based on the claims.

Claims

1. A method for training a buying and selling point prediction model based on reinforcement learning based on historical experience, comprising the following steps: S1. Acquire financial transaction data, pre-process the financial transaction data, and obtain a simplified time-sharing chart; S2. Based on the simplified time-sharing graph, using the band theory analysis framework, a supervised fine-tuning dataset is constructed; S3. Pre-fine-tune the multimodal large model according to the supervised fine-tuning dataset to obtain an enhanced multimodal large model; S4. Use the reward model and reinforcement learning proximal policy optimization to score and rearrange the reinforced multimodal large model, and fine-tune the supervised fine-tuning dataset based on the scoring and rearrangement results; S5. According to the fine-tuned supervised fine-tuning dataset, the multimodal large model is trained to obtain the buy and sell point prediction model.

2. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 1, characterized in that: The obtaining of financial transaction data, preprocessing of the financial transaction data, and obtaining a simplified time-sharing chart comprises the following steps: S11. Obtaining financial transaction data; S12. Identify erroneous values ​​or outliers in financial transaction data and clean them; S13, converting the cleaned financial transaction data to obtain a time-sharing chart; S14. Optimize the fluctuations in the time-sharing chart using an optimization algorithm to obtain a simplified time-sharing chart.

3. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 2, characterized in that: The method of optimizing the fluctuations in the time-sharing chart by using an optimization algorithm to obtain a simplified time-sharing chart includes the following steps: S141, identifying all extreme points in the time-sharing chart curve; S142, calculating the distance between adjacent extreme value points; S143, determine whether the distance between adjacent extreme value points is greater than a preset threshold, if so, retain the adjacent extreme value points, otherwise, merge the adjacent extreme value points, and repeat steps S11-S12 to calculate the distance of subsequent adjacent extreme value points until the distance between all adjacent extreme value points is greater than the preset threshold.

4. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 1, characterized in that: The construction of a supervised fine-tuning dataset based on a simplified time-sharing chart and using a band theory analysis framework includes the following steps: S21, split the band theory analysis framework and construct corresponding prompt words from multiple dimensions; S22, inputting the constructed prompt words and the corresponding simplified time-sharing graph images into the large language model to obtain multiple output results; S23, integrating the multiple output results and inputting them into the large language model again to obtain the final analysis process and buying point prediction conclusion; S24. Package the prompt words, simplified time-sharing graph images, and the output results of the large language model into a supervised fine-tuning dataset.

5. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 4, characterized in that: The above-mentioned splitting of the band theory analysis framework and constructing corresponding prompt words from multiple dimensions include: The band theory analysis framework is divided into all-day data, the band where the buying point is located, and the preset time before the band. Corresponding prompt words are constructed from the three dimensions of overall market trend, buying point time and band identification.

6. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 1, characterized in that: The multimodal large model is pre-fine-tuned according to the supervised fine-tuning dataset to obtain an enhanced multimodal large model including: S31, import the supervised fine-tuning dataset into the multimodal large model; S32. Use the multimodal big model to effectively fuse the visual information and text data in the supervised fine-tuning dataset to obtain an enhanced multimodal big model.

7. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 6, characterized in that: The method of using the multimodal large model to effectively fuse the visual information and text data in the supervised fine-tuning dataset to obtain an enhanced multimodal large model includes the following steps: S321, using a visual encoder to encode the input visual data image to obtain a visual data code, and converting the visual data code through a linear layer to obtain a linear visual code; S322, converting the query data into a linear query code using an embedding layer; S323, concatenating the linear visual code and the linear query code to obtain a combined feature; S324. Use the language model in combination with the combined features to generate an enhanced multimodal large model.

8. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 7, characterized in that: The expression of the enhanced multimodal large model is: Where, X a represents the final output result; f represents the language model; H v represents linear visual encoding; H q represents linear query encoding; W represents linear layer; Z v Represents visual data encoding; concat represents concatenation operation; g represents visual encoder; X v Represents a visual data image.

9. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 1, characterized in that: The method of using the reward model and the reinforcement learning proximal strategy optimization to score and rearrange the reinforced multimodal large model, and fine-tuning the supervised fine-tuning dataset according to the scoring and rearrangement results includes the following steps: S41, collect various band pictures to the picture library; S42. Based on the output of the enhanced multimodal large model, the re-ranking model is used to retrieve the paragraph of the knowledge base corresponding to the answer to obtain a similarity score; S43, according to the simplified time-sharing diagram, matching various bands in the image library, outputting the matching degree of each band type, and obtaining the matching degree weight; S44, re-score the similarity scores based on the matching weights, and re-rank them according to the new scores; S45, calculating the loss function according to the ranking, obtaining a difference result, and optimizing the reward model according to the difference result; S46. Fine-tune the supervised fine-tuning dataset using reward model and reinforcement learning proximal policy optimization.

10. The method for training a buying and selling point prediction model based on reinforcement learning based on historical experience according to claim 9, characterized in that: The calculation formula of the loss function is: In the formula, σ(.) represents the logistic function; loss(θ) represents the loss function; r θ (x, y) represents the reward score output by the reward model for a given prompt x and a generated response y; x represents the given prompt; y w represents the superior generation result; y1 represents the inferior generation result; D represents the result dataset compared by human annotators; E represents the expected value; Indicates normalization for all comparisons.

Images