Data language model training method and device, equipment, medium and program product

Abstract

The invention relates to a data language model training method and device, equipment, a medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a standard vocabulary set, first financial training data and an initial financial data language model in the financial field; inputting the first financial training data into the initial financial data language model to obtain a prediction vocabulary; and performing model fine tuning on the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model. According to the method and the device, the model fine tuning is performed on the initial financial data language model, and the accuracy of the model parameters of the target financial data language model can be improved through the model fine tuning, so that the data processing precision of the target financial data language model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data language model training method, device, equipment, medium and program product. Background Art

[0002] With the continuous diversification of banking products, more and more financial data need to be processed. In order to meet the needs of large amounts of data processing, financial data language models can be used to process financial data.

[0003] In the related art, the initial basic language model is trained according to the training data to obtain the financial data language model. However, the data processing accuracy of the financial data language model in the related art is low. Summary of the invention

[0004] Based on this, it is necessary to provide a data language model training method, device, equipment, medium and program product that can improve the data processing accuracy of the financial data language model in response to the above technical problems.

[0005] In a first aspect, the present application provides a data language model training method, comprising:

[0006] Obtaining a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model;

[0007] Inputting the first financial training data into the initial financial data language model to obtain a predicted vocabulary;

[0008] The initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model.

[0009] In one embodiment, fine-tuning the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model includes:

[0010] Determine a target standard vocabulary that matches the predicted vocabulary from the standard vocabulary set;

[0011] According to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and the updated initial financial data language model is evaluated to obtain an evaluation result;

[0012] The steps of adjusting the parameters of the initial financial data language model and evaluating the model are iteratively performed according to the evaluation result until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model.

[0013] In one embodiment, the standard vocabulary set includes a plurality of standard vocabulary, and obtaining the standard vocabulary set in the financial field includes:

[0014] Acquire multiple initial standard vocabularies in the financial field;

[0015] A first data processing is performed on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0016] In one embodiment, obtaining first financial training data includes:

[0017] Obtaining initial financial data in the financial field;

[0018] The initial financial data is subjected to a second data processing to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0019] In one embodiment, obtaining an initial financial data language model includes:

[0020] Obtain second financial training data;

[0021] The initial basic language model is pre-trained according to the second financial training data to obtain an initial financial data language model.

[0022] In one embodiment, the method further comprises:

[0023] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0024] In a second aspect, the present application also provides a data language model training device, comprising:

[0025] An acquisition module, used to acquire a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model;

[0026] A prediction module, used for inputting the first financial training data into the initial financial data language model to obtain a prediction vocabulary;

[0027] The fine-tuning module is used to fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model.

[0028] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of the first aspect.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method of the first aspect when the computer program is executed by a processor.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the method of the first aspect when executed by a processor.

[0031] The above-mentioned data language model training method, device, computer equipment, storage medium and program product obtain a standard vocabulary set, a first financial training data and an initial financial data language model in the financial field; input the first financial training data into the initial financial data language model to obtain a predicted vocabulary; fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model. After obtaining the initial financial data language model, the embodiment of the present application uses the first financial training data and the initial financial data language model to obtain a predicted vocabulary, and then fine-tunes the initial financial data language model according to the standard vocabulary set and the predicted vocabulary. Compared with the traditional technology of only training the initial basic language model according to the training data to obtain the financial data language model, the embodiment of the present application can improve the accuracy of the model parameters of the target financial data language model through model fine-tuning, thereby improving the accuracy of data processing of the target financial data language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] FIG1 is an application environment diagram of a data language model training method in one embodiment of the present application;

[0034] FIG2 is a flow chart of a data language model training method in one embodiment of the present application;

[0035] FIG3 is a flow chart of a data language model training method in another embodiment of the present application;

[0036] FIG4 is a flow chart of a data language model training method in another embodiment of the present application;

[0037] FIG5 is a flow chart of a data language model training method in another embodiment of the present application;

[0038] FIG6 is a flow chart of a data language model training method in another embodiment of the present application;

[0039] FIG7 is a block diagram of an exemplary implementation structure of a data language model training method in another embodiment of the present application;

[0040] FIG8 is a schematic diagram of the structure of a data language model training device in one embodiment of the present application;

[0041] FIG. 9 is a diagram showing the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The data language model training method, apparatus, device, medium and program product provided in the embodiments of the present application can be applied to application scenarios of data language model training; of course, they can also be applied to other scenarios, which is not limited in the embodiments of the present application.

[0044] It should be noted that, for the convenience of explanation, the following embodiments are described by taking the data language model training method of the embodiment of the present application as an example of applying the financial data language model training of a bank. It should be understood that when the data language model training method of the embodiment of the present application is applied to other scenarios, its implementation principle and technical effect are similar.

[0045] FIG1 is a schematic diagram of the implementation environment of the data language model training method provided in an embodiment of the present application. As shown in FIG1 , the implementation environment of the embodiment of the present application may include: a terminal 101 and a server 102; wherein the terminal 101 can communicate with the server 102 through a network. The terminal 101 may be a terminal corresponding to a banking system. Among them, the terminal 101 may include but is not limited to various personal computers, laptops, smart phones and tablet computers. The server 102 may be implemented with an independent server or a server cluster consisting of multiple servers. The data storage system may store the financial training data that the server 101 needs to process. The data storage system may be integrated on the server 102, or may be placed on a cloud or other network server. Exemplarily, the server 102 in the embodiment of the present application may use the data language model training method of the embodiment of the present application to train the corresponding financial data language model.

[0046] In the related art, the financial data language model is obtained by training the initial basic language model according to the training data. These models are limited by the coverage of the data set and the financial field knowledge and have poor universality. Therefore, this method of training the initial basic language model to obtain the financial data language model may lead to the problem of low accuracy in financial data processing.

[0047] The data language model training method, apparatus, device, medium and program product provided in the embodiments of the present application obtain a standard vocabulary set, first financial training data and an initial financial data language model in the financial field; use the first financial training data and the initial financial data language model to obtain a predicted vocabulary, and then fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary. Through model fine-tuning, the accuracy of the model parameters of the target financial data language model can be improved, thereby improving the accuracy of data processing of the target financial data language model.

[0048] In one embodiment, FIG2 is a flow chart of a data language model training method in an embodiment of the present application. In the embodiment of the present application, the method is applied to the server in FIG1 as an example for explanation. As shown in FIG2, the method in the embodiment of the present application may include the following steps:

[0049] Step S201: Acquire a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model.

[0050] In the financial field, there are many types of machine learning services, including keyword recognition, entity recognition, sentiment analysis, etc. For each type of service, the server can obtain a standard vocabulary set corresponding to the service type, which can include common instructions, keywords, terms, etc. corresponding to the service type in the financial field. For example, for the keyword recognition service type, the standard vocabulary set in the financial field is the commonly used keywords in the financial field, such as "buy", "sell", "limit up", "limit down", etc.; for another example, for the sentiment analysis service type, the standard vocabulary set in the financial field is the emotions or attitudes in the instructions in the financial field, such as positive, negative, neutral, etc.

[0051] Exemplarily, the first financial training data involved in the embodiments of the present application may include financial text segments, financial product names, and financial operation data, etc. Among them, financial text segments refer to text content related to the financial field, which may include news reports, market analysis, economic data, etc.; financial product names refer to the names of various investment products or tools in the financial market, which may include stocks, bonds, funds, futures, options, etc.; financial operations refer to various text data related to finance performed during the operation and management process, which may include capital flow data, income and expenditure data, financial statement data, etc.

[0052] Exemplarily, the initial financial data language model involved in the embodiments of the present application can be obtained by pre-training an initial basic language model, and the initial basic language model can include a big language (GPT-3, Generative Pre-trained Transformer), a pre-trained language model (BERT, Bidirectional Encoder Representations from Transformers), and an autoencoding language model (XLNet, eXtreme Language Understanding Network); further, the pre-training can adopt a deep neural network model based on the Transformer architecture, and use the training method of the autoregressive language model to pre-train the initial basic language model.

[0053] The following is an exemplary description of the manner in which the server acquires the standard vocabulary set in the financial field, the first financial training data, and the initial financial data language model.

[0054] For example, for a standard vocabulary set in the financial field, the server can execute a crawler program to crawl financial-related online dictionaries, a terminology list of a financial institution, or a standard terminology set of a professional organization. For the first financial training data, the server can obtain rich data resources from the database of a financial institution, including financial news, company reports, financial data, etc.

[0055] In one possible implementation, the server may obtain the latest financial news articles or historical financial data from the bank database at preset intervals (such as daily, weekly or monthly) to obtain a standard vocabulary set in the financial field, the first financial training data and an initial financial data language model.

[0056] In another possible implementation, upon receiving a trigger request, the server may obtain the required financial news articles or historical data from the bank database according to the target requirements, so as to obtain a standard vocabulary set in the financial field, the first financial training data and the initial financial data language model, and return them to the requester; illustratively, the trigger request may include but is not limited to user requests, conditional triggers, etc. Among them, the conditional trigger can be used to indicate that when a certain event occurs or reaches a certain preset threshold, a request is triggered to obtain relevant financial data. Exemplarily, the target requirements may include but are not limited to at least one of the following: user personalized requirements, real-time data requirements, etc. Among them, user personalized requirements refer to obtaining financial news articles or data related to user needs based on user preferences or personalized settings.

[0057] Of course, the server can also obtain the standard vocabulary set in the financial field, the first financial training data, and the initial financial data language model through other means.

[0058] Step S202: input the first financial training data into the initial financial data language model to obtain predicted vocabulary.

[0059] Exemplarily, the predicted vocabulary in the embodiments of the present application refers to the suggestions, explanations, predictions or other relevant information related to the first financial training data output by the initial financial data language model based on the first financial training data. The predicted vocabulary may include finance-related terms, entity names, operating instructions, market trends, etc.; among them, finance-related terms refer to specific words or terms used in the financial field, which may include stocks, bonds, interest rates, balance sheets, price-earnings ratios, etc.; entity names refer to entities with specific identities or attributes in the financial field, which may include company names, institution names, etc.; market trends refer to changing trends in prices, trading volumes, volatility, etc. in the financial market, which may include rising stock markets, falling gold prices, sideways trading, etc.

[0060] In this embodiment, the server can input the acquired first financial training data into the initial financial data language model for prediction, thereby obtaining predicted vocabulary. Furthermore, according to the different business types to which the first financial training data belongs, the obtained predicted vocabulary will also be different. For example, for the business type of keyword recognition, the first financial training data is input into the initial financial data language model, and the predicted vocabulary obtained may be keywords commonly used in the financial field, such as "buy", "sell", "limit up", "limit down", etc.; for another example, for the business type of sentiment analysis results, the first financial training data is input into the initial financial data language model, and the predicted vocabulary obtained may be emotions or attitudes in the financial text, such as positive, negative, neutral, etc.; for another example, for the business type of entity recognition, the first financial training data is input into the initial financial data language model, and the predicted vocabulary obtained may be entities such as company names, stock codes, fund codes, etc. recognized in the financial text.

[0061] Step S203: fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model.

[0062] Exemplarily, the model fine-tuning involved in the embodiments of the present application refers to fine-tuning the parameters of the model for specific business types and fields based on the trained initial financial data language model.

[0063] Exemplarily, the target financial data language model involved in the embodiments of the present application refers to a model obtained by training an initial financial data language model according to first financial training data, fine-tuning the parameters of the model, and updating it.

[0064] In this embodiment, the server can calculate the difference between the target standard vocabulary and the predicted vocabulary in the standard vocabulary set, and according to the difference between the two, use a preset algorithm to adjust the parameters of the initial financial data language model to obtain the target financial data language model, wherein the preset algorithm may include a back propagation algorithm, a gradient descent method, etc.

[0065] For example, for the business type of sentiment analysis, a standard vocabulary set is embedded in the initial financial data language model, and the similarity is calculated between the target standard vocabulary in the standard vocabulary set and the predicted vocabulary of the initial financial data language model, so as to fine-tune the parameters of the model; wherein, the standard vocabulary set in the sentiment analysis business type may include the polarity of sentiment, such as positive, negative, neutral, etc.; for another example, for the business type of entity recognition, a standard vocabulary set is embedded in the initial financial data language model, and the similarity is calculated between the target standard vocabulary in the standard vocabulary set and the predicted vocabulary of the initial financial data language model, so as to fine-tune the parameters of the model; wherein, the standard vocabulary set in the entity recognition business type may include financial texts that may be recognized, such as company names, stock codes, fund codes and other entities.

[0066] In the above data language model training method, by obtaining a standard vocabulary set in the financial field, the first financial training data and the initial financial data language model, the first financial training data is input into the initial financial data language model to obtain a predicted vocabulary; the initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model. It can be seen that in the embodiment of the present application, when obtaining a standard vocabulary set in the financial field, the first financial training data and the initial financial data language model, the initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary. Through model fine-tuning, the accuracy of the model parameters of the target financial data language model can be improved, thereby improving the accuracy of data processing of the target financial data language model.

[0067] Based on the embodiment shown in FIG2 , FIG3 is a flow chart of a data language model training method in another embodiment of the present application. On the basis of the above embodiment, the present application introduces and explains the step S203 involved in the above embodiment in which the initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model. As shown in FIG3 , the above step S203 includes S301, S302 and S303:

[0068] S301: Determine a target standard vocabulary that matches the predicted vocabulary from a standard vocabulary set.

[0069] Exemplarily, the target standard vocabulary involved in the embodiments of the present application can be used to indicate a plurality of vocabulary items that match the predicted vocabulary output by the initial financial data language model and the standard vocabulary set of the initial financial data language model.

[0070] In this embodiment, the server may compare and analyze the predicted vocabulary output by the initial financial data language model with the standard vocabulary set to determine which vocabulary items in the standard vocabulary set match or are similar to the predicted vocabulary.

[0071] Exemplarily, the server may use text matching or text similarity methods to calculate the similarity between the vocabulary in the standard vocabulary set and the predicted vocabulary, and determine the vocabulary in the standard vocabulary set that has the highest match or similarity to the predicted vocabulary as the target standard vocabulary, wherein the text similarity measurement method may include cosine similarity, edit distance, etc.

[0072] For example, for entity recognition business types, assume that the standard vocabulary set contains multiple company information lists, which may include company names and company locations, where company names include company 1, company 2, and company 3; company locations include location A, location B, and location C. Assume there is a sentence: "Company 1 held an annual meeting at location B." The goal is to identify the company name and company location from the sentence. Assume that the predicted vocabulary includes company 1, company 2, and location B, compare the predicted company name and location vocabulary with the company name and location in the standard vocabulary set, measure the similarity or matching degree between them, and determine which vocabulary items in the standard vocabulary set match the predicted vocabulary.

[0073] S302. According to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and a model evaluation is performed on the updated initial financial data language model to obtain an evaluation result.

[0074] Exemplarily, the updated initial financial data language model involved in the embodiments of the present application can be used to indicate the model after fine-tuning the parameters of the initial financial data language model.

[0075] Exemplarily, the parameters of the initial financial data language model involved in the embodiments of the present application may include learning rate, number of training iterations, batch size, etc., which are used to characterize the overall performance of the model, wherein the learning rate refers to a hyperparameter used to control the update speed of the model parameters during the training process; the batch size refers to the number of samples used to update the model parameters in each training iteration.

[0076] Exemplarily, the model evaluation involved in the embodiments of the present application is used to indicate the use of evaluation indicators to evaluate the model and judge the performance of the model under the current data set; wherein the evaluation indicators include but are not limited to at least one of the following: accuracy, recall rate, F1 value, etc.

[0077] In this embodiment, the server can calculate the difference between the predicted vocabulary and the target standard vocabulary through a preset calculation method, and use the back propagation algorithm to calculate the gradient based on the difference between the two to determine the adjustment direction of the parameters, and use optimization algorithms such as gradient descent to update the weights and biases in the model based on the calculated parameter gradients to obtain an updated initial financial data language model, and further, merge the updated initial financial data language model with the initial financial data language model; wherein the preset calculation methods include mean square error, loss function, word vector similarity, etc.

[0078] The following is an exemplary description of a difference calculation method for adjusting the initial financial data language model parameters by the server according to the difference between the target standard vocabulary and the predicted vocabulary.

[0079] In a possible implementation, for the business type of entity recognition, assuming that the target standard vocabulary of the company name is company A, and the result of the predicted vocabulary is company AB, the difference between the predicted vocabulary and the target standard vocabulary is calculated by the mean square error. If the mean square error is large, it means that the gap between the predicted vocabulary and the target standard vocabulary is large. The learning rate can be reduced to update the model parameters with a smaller step size to improve the convergence and stability of the model. On the contrary, if the mean square error is small, the learning rate can be appropriately increased to accelerate the convergence of the model. The initial financial data language model is adjusted by adjusting the learning rate, batch size and other parameters, such as increasing the company name samples in the training data, and performing appropriate model optimization, so as to obtain an updated initial financial data language model. Furthermore, a model fusion method based on weighted average is adopted to automatically adjust the weights of different models, and the updated initial financial data language model is evaluated according to the evaluation indicators such as accuracy, recall rate, F1 value, etc., so as to obtain the evaluation result.

[0080] In another possible implementation, for the business type of entity recognition, assuming that the target standard vocabulary of the company name is Company AA, and the result of the predicted vocabulary is Company AB, an appropriate loss function (such as cross entropy loss) is used to measure the difference between the predicted vocabulary and the target standard vocabulary. Specifically, the target standard vocabulary is encoded and converted into a corresponding vector representation, and then the loss value is obtained according to the loss function. The gradient is calculated by the back propagation algorithm to determine the adjustment direction of the parameters to minimize the loss function, and an updated initial financial data language model can be obtained. Furthermore, a model fusion method based on weighted average is adopted to automatically adjust the weights of different models, and the updated initial financial data language model is evaluated according to evaluation indicators such as accuracy, recall rate, F1 value, etc., to obtain an evaluation result.

[0081] S303 , iteratively executing the steps of adjusting the parameters of the initial financial data language model and model evaluation according to the evaluation result, until the evaluation result of the updated initial financial data language model reaches a preset convergence condition, so as to obtain a target financial data language model.

[0082] Exemplarily, the preset convergence condition involved in the embodiments of the present application means that during the model training process, when a certain predefined condition is met, the model is considered to have reached a convergence state, wherein the preset convergence condition includes a threshold for training error, a limit on the number of training rounds, and performance indicators on the validation set reaching a certain level.

[0083] In this embodiment, the server can adjust the parameters of the initial financial data language model according to the evaluation results. After each parameter adjustment, the model performance is re-evaluated and it is determined whether the preset convergence conditions are met. If the convergence conditions are met, the training is stopped to obtain the target financial data language model; otherwise, the next round of iteration is performed. For example, for the sentiment analysis business type, the model is evaluated according to the evaluation indicators such as accuracy, recall rate, F1 value, etc., and the evaluation results, such as accuracy, are obtained. If the convergence conditions are met, the training is stopped to obtain the target financial data language model. If the accuracy does not reach the preset accuracy threshold, according to the model evaluation results, analyze which aspects can be improved and adjusted, for example, adjust the network structure, adjust the number of hidden units, adjust the learning rate or regularization parameters, etc., update the parameters of the model, and retrain and evaluate until the convergence conditions are met to obtain the target financial data language model.

[0084] In this embodiment, a target standard vocabulary matching the predicted vocabulary is determined from a standard vocabulary set, and according to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and the updated initial financial data language model is evaluated to obtain an evaluation result. By selecting a suitable method to calculate the difference between the model prediction and the true label, the performance of the model can be accurately evaluated; according to the evaluation result, the steps of adjusting the parameters of the initial financial data language model and the model evaluation are iteratively performed until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model; by iteratively adjusting the model and continuously performing parameter adjustment, training and evaluation according to the preset convergence condition, the accuracy of the model parameters of the target financial data language model can be improved, thereby improving the accuracy of data processing of the target financial data language model.

[0085] Based on the embodiment shown in FIG2 , FIG4 is a flow chart of a data language model training method in another embodiment of the present application. As shown in FIG4 , based on the above embodiment, the present application introduces and explains the step S201 involved in the above embodiment to obtain a standard vocabulary set in the financial field. As shown in FIG4 , the above step S201 includes S401 and S402:

[0086] S401. Acquire multiple initial standard vocabularies in the financial field.

[0087] Exemplarily, the initial standard vocabulary involved in the embodiments of the present application may include financial terms, names of financial products, identifiers of financial institutions, etc., wherein financial terms may include professional terms such as stocks, bonds, futures, etc. used to describe financial transactions and investment activities; financial product names may include deposit accounts, insurance products, wealth management products, etc.; identifiers of financial institutions include stock exchange codes, bank account numbers, etc.

[0088] In this embodiment, the server may collect a plurality of initial standard vocabularies in the financial field from databases of financial institutions, financial newspapers and financial news.

[0089] S402, performing a first data processing on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0090] Exemplarily, the data augmentation processing involved in the embodiments of the present application may include synonym replacement, sentence reorganization, vocabulary insertion, etc.

[0091] Exemplarily, the data preprocessing involved in the embodiments of the present application can be used to instruct the cleaning and processing of the initial standard vocabulary in the financial field, which may include removing noise and non-text information, performing vocabulary segmentation and labeling, etc., wherein removing noise and non-text information may include removing punctuation marks and special characters, removing stop words, processing abbreviations and acronyms, etc.

[0092] In this embodiment, the server can perform corresponding data augmentation processing and data preprocessing on the initial standard vocabulary collected from the financial field to obtain multiple standard vocabulary; for example, assuming that the server has collected a set of initial standard vocabulary in the financial field, such as: "stock", "trading", "investment", "profit", "risk", "analysis", "asset", "liability", "report", and "economy"; data augmentation technology can be used to perform synonym replacement, suffix addition, prefix addition and other operations on the initial standard vocabulary, for example, replacing "stock" with "shares" and "trading" with "trading behavior"; adding the suffix "rate" to "profit" to generate "profit rate"; adding the prefix "uncertainty" to "risk" to generate "uncertainty risk", etc.

[0093] In an embodiment of the present application, multiple initial standard vocabularies are obtained in the financial field, and further, data augmentation and data preprocessing are performed on the multiple initial standard vocabularies to obtain multiple standard vocabularies. Data augmentation and data preprocessing help to improve the coverage and generalization ability of the model, thereby better handling natural language processing tasks in the financial field.

[0094] Based on the embodiment shown in FIG2 , FIG5 is a flow chart of a data language model training method in another embodiment of the present application. As shown in FIG5 , based on the above embodiment, the present application introduces and explains the acquisition of the first financial training data in step S201 involved in the above embodiment. As shown in FIG5 , the above step S201 includes S501 and S502:

[0095] S501. Acquire initial financial data in the financial field.

[0096] Exemplarily, the initial financial data involved in the embodiments of the present application may include financial product names, stock market data, transaction data, etc., which are used to indicate the initial input data for fine-tuning the initial financial data language model; among which the stock market data may include stock indexes, market capitalization, market transaction data, etc., and the transaction data may include stock prices, trading volumes, transaction amounts, transaction times, etc.

[0097] In this embodiment, the server may obtain initial financial data through channels such as professional databases of financial institutions, professional knowledge of experts in the financial field, public financial data, and financial news;

[0098] S502: Perform a second data processing on the initial financial data to obtain first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0099] In this embodiment, the server can perform data preprocessing and data augmentation processing on the initial financial data obtained from professional databases of financial institutions, professional knowledge of experts in the financial field, public financial data, financial news and other channels to obtain the first financial training data; for example, assuming that the server collects a set of initial financial data in the financial field, including stock market data, transaction data, financial product names, etc.; preprocessing the data includes removing duplicate data, processing missing values ​​and outliers, and for stock market data and transaction data, time series processing, such as time alignment and resampling, can be performed to analyze time correlation. Augmentation processing is performed on the data, and through data augmentation technology, the synonym library in the financial field is used to replace the name of the financial product with the name of its similar product; for example, "stock" is replaced with "shares" or "equity"; the names of financial products are combined to generate a new product name; "stock" and "fund" are combined to generate "stock fund"; for transaction data, numerical transformation, such as logarithmization and standardization, can be performed to better apply to model training; further, according to the above operations, the first financial training data is obtained.

[0100] In an embodiment of the present application, the server can generate diversified first financial training data by acquiring initial financial data in the financial field and performing preprocessing and data augmentation processing techniques on these data, including standardized financial product names, augmented names, and processed stock market data and transaction data; this helps to improve the richness and coverage of the data and the generalization ability of the model, so that it can be better applied to training and analysis tasks in the financial field.

[0101] Based on the embodiment shown in FIG2 , FIG6 is a flow chart of a data language model training method in another embodiment of the present application. As shown in FIG6 , based on the above embodiment, the present application introduces and explains the acquisition of the initial financial data language model in step S201 involved in the above embodiment. As shown in FIG6 , the above step S201 includes S601 and S602:

[0102] S601. Obtain second financial training data.

[0103] Exemplarily, the second financial training data involved in the embodiments of the present application may include a large amount of text data in the financial field, such as bank log information, business documents, account information, etc., which is used to indicate input data for pre-training the initial basic language model.

[0104] In this embodiment, the server can obtain initial financial data through the bank's front-end system, back-end system, financial news and other channels, wherein the initial financial data may include but is not limited to mobile banking log information, business documents, account information and other large amounts of text data in the financial field. Further, the initial financial data is preprocessed to obtain second financial training data, and the data preprocessing includes but is not limited to removing noise and non-text information, performing vocabulary segmentation and annotation, etc. For example, the server obtains a text data set containing financial news, and the original text is that company A released a financial report, and the net profit in the third quarter increased by 20% year-on-year; the financial news data can be preprocessed to obtain the preprocessed text, and company A released a financial report, and the net profit in the third quarter increased by 20% year-on-year.

[0105] S602: Pre-train the initial basic language model according to the second financial training data to obtain an initial financial data language model.

[0106] Exemplarily, the initial basic language model involved in the embodiments of the present application is used to indicate a natural language processing model trained based on deep learning technology, wherein the initial basic language model may include GPT-3, BERT, XLNet, T5, etc.

[0107] In this embodiment, the server may pre-train the initial basic language model according to the second financial training data to obtain the initial financial data language model. For example, after obtaining the second financial training data, a deep neural network model based on a transformer architecture is used, and the initial basic language model is pre-trained using the training method of an autoregressive language model to obtain the initial financial data language model.

[0108] In an embodiment of the present application, the server can generate diversified second financial training data by obtaining second financial training data and preprocessing the data, including processed log information, standardized business documents, account information and other large amounts of text data in the financial field; through multiple rounds of iterations and pre-training of large-scale data, the initial basic language model can learn rich financial language knowledge and general semantic representation capabilities.

[0109] Based on the embodiment shown in FIG. 2 , in the embodiment of the present application, the initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary in step S203 involved in the above embodiment to obtain the target financial data language model. The above step S203 includes:

[0110] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0111] Exemplarily, the model compression involved in the embodiments of the present application refers to compressing the target financial language model using a preset compression technology, wherein the compression technology may include pruning, quantization, distillation, etc.

[0112] In the embodiment of the present application, the server can use a preset compression technology to perform model compression processing on the updated target financial data language model, thereby obtaining a compressed target financial data language model. For example, assuming that there is a target financial data language model for financial risk prediction, which contains multiple hidden layers and a large number of parameters, pruning technology can be applied to reduce the number of parameters in the model. According to the evaluation results of the importance of the parameters, a pruning threshold is set, and the parameters exceeding the threshold will be retained, while the parameters below the threshold will be pruned, thereby achieving the purpose of model compression.

[0113] In the embodiment of the present application, by using a preset compression technology to perform model compression processing on the target financial data language model, the scale of the target financial data language model can be reduced, thereby reducing the storage requirements and computing costs of the model.

[0114] In one embodiment, a data language model training method is provided, which is used for the server 102 in FIG. 1 , referring to FIG. 7 , and includes the following steps:

[0115] S701. The server obtains second financial training data.

[0116] S702: The server pre-trains the initial basic language model according to the second financial training data to obtain an initial financial data language model.

[0117] S703: The server obtains a plurality of initial standard vocabularies in the financial field.

[0118] S704. The server performs a first data processing on the multiple initial standard vocabularies to obtain multiple standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0119] S705. The server obtains initial financial data in the financial field.

[0120] S706. The server performs second data processing on the initial financial data to obtain first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0121] S707: The server inputs the first financial training data into the initial financial data language model to obtain predicted vocabulary.

[0122] S708: The server determines a target standard vocabulary that matches the predicted vocabulary from the standard vocabulary set.

[0123] S709: The server adjusts the parameters of the initial financial data language model according to the difference between the target standard vocabulary and the predicted vocabulary to obtain an updated initial financial data language model, and performs model evaluation on the updated initial financial data language model to obtain an evaluation result.

[0124] S710. The server iteratively performs the steps of adjusting the parameters of the initial financial data language model and model evaluation according to the evaluation result, until the evaluation result of the updated initial financial data language model reaches a preset convergence condition, so as to obtain a target financial data language model.

[0125] S711. The server performs model compression processing on the target financial data language model to obtain a compressed target financial data language model.

[0126] Exemplarily, the embodiment shown in FIG. 7 is described with reference to a banking scenario.

[0127] Taking the case of a bank's pending financial business A, the bank server obtains the second financial training data X, pre-trains the initial basic language model according to the second financial training data X, and obtains the initial financial data language model M; the bank server obtains multiple initial standard vocabularies in the financial field, performs the first data processing on the multiple initial standard vocabularies, and obtains multiple standard vocabularies Y, wherein the first data processing includes: data augmentation processing, and / or data preprocessing. Next, the bank server obtains the initial financial data Z in the financial field related to the pending financial business A, performs the second data processing on the initial financial data Z, and obtains the first financial training data W; wherein the second data processing includes: data augmentation processing, and / or data preprocessing. Afterwards, the bank server inputs the first financial training data W into the initial financial data language model M to obtain the predicted vocabulary B; further, the bank server determines the target standard vocabulary C that matches the predicted vocabulary B from the standard vocabulary set Y by means of word vector similarity, calculates the difference between the target standard vocabulary C and the predicted vocabulary B by means of mean square error, adjusts the parameters of the initial financial data language model M according to the difference between the target standard vocabulary C and the predicted vocabulary B, so as to obtain an updated initial financial data language model, and performs model evaluation on the updated initial financial data language model to obtain an evaluation result; afterward, the bank server iteratively performs the steps of adjusting the parameters of the initial financial data language model M and model evaluation according to the evaluation result, until the evaluation result of the updated initial financial data language model M reaches a preset convergence condition, so as to obtain the target financial data language model N. At the same time, the bank server performs model compression processing on the target financial data language model N to obtain a compressed target financial data language model.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiment of the present application also provides a data language model training device for implementing the data language model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data language model training device embodiments provided below can refer to the limitations on the data language model training method above, and will not be repeated here.

[0130] In one embodiment, FIG8 is a schematic diagram of the structure of a data language model training device in an embodiment of the present application. The data language model training device provided in the embodiment of the present application can be applied to a server. As shown in FIG8, the data language model training device in the embodiment of the present application includes: an acquisition module 10, a prediction module 11 and a fine-tuning module 12, wherein:

[0131] The acquisition module 10 is used to acquire a standard vocabulary set in the financial field, first financial training data and an initial financial data language model.

[0132] The prediction module 11 is used to input the first financial training data into the initial financial data language model to obtain a prediction vocabulary.

[0133] The fine-tuning module 12 is used to fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model.

[0134] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0135] In one embodiment, the fine-tuning module 12 includes: a determination unit, an adjustment unit and an update unit, wherein:

[0136] The determination unit is used to determine a target standard vocabulary matching the predicted vocabulary from the standard vocabulary set.

[0137] The adjustment unit is used to adjust the parameters of the initial financial data language model according to the difference between the target standard vocabulary and the predicted vocabulary to obtain an updated initial financial data language model, and perform model evaluation on the updated initial financial data language model to obtain an evaluation result.

[0138] An updating unit is used to perform the steps of adjusting the parameters of the initial financial data language model and model evaluation according to the evaluation result, until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model.

[0139] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0140] In one embodiment, the acquisition module 10 is used to:

[0141] Access to multiple initial standard vocabularies in the financial sector.

[0142] A first data processing is performed on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0143] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0144] In one embodiment, the acquisition module 10 is used to:

[0145] Obtain initial financial data in the financial field.

[0146] The initial financial data is subjected to a second data processing to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0147] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0148] In one embodiment, the acquisition module 10 is used to:

[0149] Obtain second financial training data;

[0150] The initial basic language model is pre-trained according to the second financial training data to obtain an initial financial data language model.

[0151] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0152] In one embodiment, the fine-tuning module 12 is further used for:

[0153] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0154] The data language model training device provided in this embodiment can execute the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0155] Each module in the above-mentioned data language model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0156] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG9. The computer device may include a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data language model training method is implemented.

[0157] Those skilled in the art will understand that the structure shown in FIG. 9 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0159] Obtaining a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model;

[0160] Inputting the first financial training data into the initial financial data language model to obtain a predicted vocabulary;

[0161] The initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model.

[0162] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0163] Determine a target standard vocabulary that matches the predicted vocabulary from the standard vocabulary set;

[0164] According to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and the updated initial financial data language model is evaluated to obtain an evaluation result;

[0165] The steps of adjusting the parameters of the initial financial data language model and evaluating the model are iteratively performed according to the evaluation result until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model.

[0166] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0167] Acquire multiple initial standard vocabularies in the financial field;

[0168] A first data processing is performed on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0169] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0170] Obtaining initial financial data in the financial field;

[0171] The initial financial data is subjected to a second data processing to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] Obtain second financial training data;

[0174] The initial basic language model is pre-trained according to the second financial training data to obtain an initial financial data language model.

[0175] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0176] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0177] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0178] Obtaining a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model;

[0179] Inputting the first financial training data into the initial financial data language model to obtain a predicted vocabulary;

[0180] The initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] Determine a target standard vocabulary that matches the predicted vocabulary from the standard vocabulary set;

[0183] According to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and the updated initial financial data language model is evaluated to obtain an evaluation result;

[0184] The steps of adjusting the parameters of the initial financial data language model and evaluating the model are iteratively performed according to the evaluation result until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0186] Acquire multiple initial standard vocabularies in the financial field;

[0187] A first data processing is performed on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0188] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0189] Obtaining initial financial data in the financial field;

[0190] The initial financial data is subjected to a second data processing to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0191] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0192] Obtain second financial training data;

[0193] The initial basic language model is pre-trained according to the second financial training data to obtain an initial financial data language model.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0196] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0197] Obtaining a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model;

[0198] Inputting the first financial training data into the initial financial data language model to obtain a predicted vocabulary;

[0199] The initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain the target financial data language model.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] Determine a target standard vocabulary that matches the predicted vocabulary from the standard vocabulary set;

[0202] According to the difference between the target standard vocabulary and the predicted vocabulary, the parameters of the initial financial data language model are adjusted to obtain an updated initial financial data language model, and the updated initial financial data language model is evaluated to obtain an evaluation result;

[0203] The steps of adjusting the parameters of the initial financial data language model and evaluating the model are iteratively performed according to the evaluation result until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain a target financial data language model.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0205] Acquire multiple initial standard vocabularies in the financial field;

[0206] A first data processing is performed on a plurality of initial standard vocabularies to obtain a plurality of standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

[0207] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0208] Obtaining initial financial data in the financial field;

[0209] The initial financial data is subjected to a second data processing to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0211] Obtain second financial training data;

[0212] The initial basic language model is pre-trained according to the second financial training data to obtain an initial financial data language model.

[0213] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0214] The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

[0215] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0216] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0217] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0218] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data language model training method, characterized in that: The method comprises: Obtaining a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model; Inputting the first financial training data into the initial financial data language model to obtain a predicted vocabulary; The initial financial data language model is fine-tuned according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model.

2. The method according to claim 1, characterized in that The fine-tuning of the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model includes: Determining a target standard vocabulary matching the predicted vocabulary from the standard vocabulary set; According to the difference between the target standard vocabulary and the predicted vocabulary, adjusting the parameters of the initial financial data language model to obtain an updated initial financial data language model, and performing model evaluation on the updated initial financial data language model to obtain an evaluation result; The steps of adjusting the parameters of the initial financial data language model and the model evaluation are iteratively performed according to the evaluation result until the evaluation result of the updated initial financial data language model reaches a preset convergence condition to obtain the target financial data language model.

3. The method according to claim 1 or 2, characterized in that: The standard vocabulary set includes a plurality of standard vocabulary, and the step of obtaining the standard vocabulary set in the financial field includes: Acquire multiple initial standard vocabularies in the financial field; Performing a first data processing on the multiple initial standard vocabularies to obtain the multiple standard vocabularies; wherein the first data processing includes: data augmentation processing and / or data preprocessing.

4. The method according to claim 1 or 2, characterized in that: Acquiring the first financial training data includes: Obtaining initial financial data in the financial field; Performing a second data processing on the initial financial data to obtain the first financial training data; wherein the second data processing includes: data augmentation processing and / or data preprocessing.

5. The method according to claim 1 or 2, characterized in that: Acquiring the initial financial data language model includes: Obtain second financial training data; The initial basic language model is pre-trained according to the second financial training data to obtain the initial financial data language model.

6. The method according to claim 1 or 2, characterized in that: The method further comprises: The target financial data language model is subjected to model compression processing to obtain a compressed target financial data language model.

7. A data language model training device, characterized in that: The device comprises: An acquisition module, used to acquire a standard vocabulary set in the financial field, first financial training data, and an initial financial data language model; A prediction module, configured to input the first financial training data into the initial financial data language model to obtain a prediction vocabulary; A fine-tuning module is used to fine-tune the initial financial data language model according to the standard vocabulary set and the predicted vocabulary to obtain a target financial data language model.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images