Financial system and method based on multi-dimensional data analysis

Abstract

The invention discloses a financial analysis system, and the system comprises a query input module which is used for collecting questions of a user; the query input module is connected with the data model module, the data model module is used for receiving and analyzing natural language questions from the query input module, and the data model module retrieves structured data and unstructured data and outputs results; the data model module is connected with the fine tuning module; the fine tuning module is used for correcting an output result of the data model module; the fine tuning module is connected with the display module which is used for displaying the comprehensive analysis report. According to the method using the financial analysis system, structured data and unstructured data are combined for comprehensive analysis, more comprehensive and in-depth company financial condition analysis can be provided, and the integrity and accuracy of an analysis result are ensured by performing unified processing on different types of data; and financial conditions and operation results of companies are comprehensively reflected.

Description

Technical Field

[0001] The present invention relates to the field of intelligent financial technology, and in particular to a financial system and method based on multi-dimensional data analysis. Background Art

[0002] In the context of big data, the scale of data is getting larger and larger, and the forms of data are becoming more and more diverse, resulting in increasing requirements for data application. For the enterprise side, the rational use and analysis of data can contribute value to the development of corporate financial work.

[0003] In the prior art, commonly used financial analysis tools include Excel, QuickBooks, Tableau, Power BI, SAP, and Oracle Financial Services. However, these tools mainly process structured data, such as numbers and tables, and mainly rely on the processing of structured data, lacking effective use of unstructured data. Moreover, existing financial analysis tools usually require users to have certain professional knowledge, are complex to operate, and are difficult to adapt to the rapidly changing market environment. Therefore, there is an urgent need for a financial system that can integrate structured and unstructured data, simplify operations, and perform intelligent analysis to improve the efficiency and accuracy of financial data analysis. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of the above-mentioned background technology and to propose a financial system and method based on multi-dimensional data analysis.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A financial analysis system comprises: a query input module, the query input module is used to collect user questions; the query input module is connected to a data model module, the data model module is used to receive natural language questions from the query input module and analyze them, and the data model module retrieves structured data and unstructured data respectively, and outputs results; the data model module is connected to a fine-tuning module, the fine-tuning module is used to correct the output results of the data model module; the fine-tuning module is connected to a display module, and the display module is used to display a comprehensive analysis report.

[0007] A method of using a financial analysis system, comprising:

[0008] S1. The user inputs a query in the query input module;

[0009] S2, use the data model module to process natural language questions;

[0010] S3, optimize natural language questions through fine-tuning modules;

[0011] S4. Combine structured data and unstructured data to generate a comprehensive analysis report and output it through the display module.

[0012] As a further solution of the present invention: in step S2, the step of using the data model module includes:

[0013] S21. Use Text to SQL technology to process natural language questions for structured data in the database;

[0014] S22. For the unstructured data in the database, the RAG algorithm is used to analyze and retrieve natural language questions and unstructured data such as documents, reports, emails, etc. stored in the database.

[0015] As a further solution of the present invention: in step S21, the specific steps of processing the natural language question include:

[0016] S211, Natural Language Processing: The system classifies and processes the natural language questions input by users, including word segmentation, part-of-speech tagging, dependency syntax analysis, entity recognition, and semantic parsing;

[0017] S212, keyword extraction: extract key information from the parsed natural language question, including time, indicators and operations;

[0018] S213, intent recognition: determining the intent of the user's query, such as query, filter, and sort;

[0019] S214, slot filling: filling key information into predefined slots according to the intent recognition result;

[0020] S215, parameter filling: filling the specific parameters obtained by semantic analysis into the matching SQL template;

[0021] S216, SQL generation: Generate corresponding SQL query statements according to the extracted key information and database schema;

[0022] S217, SQL execution: After optimizing the generated SQL statement, the optimized SQL statement is sent to the corresponding database for execution to obtain the query result.

[0023] As a further solution of the present invention: in step S22, the steps of the RAG algorithm include:

[0024] S221, converting the natural language question input by the user into a vector representation, recorded as a question vector;

[0025] S222. According to the question vector, retrieve the document most relevant to the question vector from the unstructured data in the database;

[0026] S223, converting the retrieved relevant documents into document vectors, and performing vector fusion with the question vector;

[0027] S224. Combine the question vector and the document vector and use the generative model to generate the final answer.

[0028] As a further solution of the present invention: in step S3, the optimization step includes:

[0029] S31. Collect and process data for specific tasks;

[0030] S32, select a pre-trained model suitable for a specific task and perform fine-tuning training on it;

[0031] S33. Evaluate the performance of the fine-tuned model on the validation set and optimize it.

[0032] As a further solution of the present invention: in step S4, the specific steps include:

[0033] S41. Report integration: Integrate the numerical answers and charts of structured data with the financial analysis reports generated by unstructured data;

[0034] S42, PDF generation: generating a PDF file from the integrated report content;

[0035] S43, file output: sending the PDF file to the user-specified email address or system interface download link, and storing it in the system;

[0036] S44. Comprehensive analysis: Combine the specific numerical answers and charts generated by structured data with the analysis report generated by unstructured data.

[0037] As a further solution of the present invention: in the step S44, the specific steps include:

[0038] S441. Data comparison: Compare the results of structured data and unstructured data to find consistency and differences;

[0039] S442, In-depth analysis: Based on the numerical values ​​and charts of structured data, in-depth analysis of the company's financial status is carried out, combined with the analysis report of unstructured data to provide comprehensive financial analysis;

[0040] S443. Report presentation: presenting the results of the comprehensive analysis to the user.

[0041] Beneficial effects of the present invention:

[0042] (1) In the present invention, structured data and unstructured data are combined for comprehensive analysis, which can provide a more comprehensive and in-depth analysis of the company's financial status. By uniformly processing different types of data, the integrity and accuracy of the analysis results are ensured, which fully reflects the company's financial status and operating results.

[0043] (2) In the present invention, the fine-tuning module is used to optimize the processing of natural language questions, making the output results more detailed and accurate. Combined with the comprehensive analysis of structured data and unstructured data, it not only improves the efficiency of information retrieval and analysis, but also improves the quality and reliability of financial analysis reports, providing more comprehensive and accurate financial insights. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below in conjunction with the accompanying drawings.

[0045] Figure 1 It is a structural schematic diagram of the financial analysis system of the present invention;

[0046] Figure 2 is a flow chart of a method of using a financial analysis system according to the present invention;

[0047] Figure 3 It is a flow chart of processing natural language questions using a data model module in S2 of the present invention;

[0048] Figure 4 It is a schematic diagram of the process of optimizing the natural language problem by fine-tuning the module in S3 of the present invention;

[0049] Figure 5 It is a flow chart of the present invention combining structured data and unstructured data in S4 to generate a comprehensive analysis report and output it through a display module;

[0050] FIG6 is a diagram showing the effect of the intelligent question-answering method combining structured data and unstructured data in the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] See also Figure 1As shown, the present invention is a financial analysis system, including a query input module, which is used to collect user questions. Users input natural language questions through the system interface. User questions can involve different fields and topics. In this embodiment, the input natural language questions are financial.

[0053] The query input module is connected to the data model module. The data model module is used to receive and analyze natural language questions from the query input module. The data model module can also retrieve structured data and unstructured data separately. After processing structured data, numerical answers and charts are generated. After processing unstructured data, financial analysis reports are generated. The data model module combines the numerical answers and charts generated after processing structured data (i.e., output results) with the financial analysis report generated after processing unstructured data (i.e., output results) to generate a comprehensive analysis report. The comprehensive analysis report can perform comprehensive financial analysis.

[0054] The data model module is connected to the fine-tuning module, and the fine-tuning module is used to correct the output results of the data model module.

[0055] The fine-tuning module is connected to the display module, and the display module is used to display the comprehensive analysis report.

[0056] See also Figure 2 - As shown in FIG6 , a method of using a financial analysis system includes:

[0057] S1. The user inputs a query in the query input module.

[0058] Users can input natural language questions through the system interface in the query input module. The natural language questions can involve specific numerical values, chart generation, or financial analysis reports.

[0059] S2. Use the data model module to process natural language problems.

[0060] The specific steps include:

[0061] S21. Use Text to SQL technology to process natural language questions for structured data in the database.

[0062] Specifically, in the database, structured data needs to be cleaned and converted. Data cleaning includes missing value processing, duplicate value processing, and outlier processing, and data conversion includes data type conversion, encoding conversion, feature engineering processing, and data processing.

[0063] Missing value processing: fill in missing values ​​(such as mean filling, median filling) or delete records with serious missing values.

[0064] Duplicate value handling: Identify and remove duplicate records.

[0065] Outlier processing: detect and process outliers. Common methods include box plot method, standard deviation method, etc.

[0066] Data type conversion: Ensure that all data types are consistent, such as converting a string date to a date type.

[0067] Encoding conversion: convert categorical data into numerical encoding, such as one-hot encoding, which is also called one-bit effective encoding. Its main method is to use an n-bit state register to encode n states. At any time, only one bit is valid, that is, only one bit is 1, and the rest are 0.

[0068] Feature engineering processing includes feature selection and feature construction. Feature selection: select features related to the analysis task and eliminate irrelevant or redundant features; feature construction: construct new features as needed, such as the extraction of time features (year, month, day).

[0069] Data processing includes normalization and standardization. Normalization: scaling the data to a specific range, such as [0,1]. The normalization formula (minimum-maximum normalization) is: Where x is the original data, x min is the minimum value in the data, x max is the maximum value in the data, x norm The data is normalized; standardization: convert the data into a standard normal distribution with a mean of 0 and a standard deviation of 1. The standardization formula is: Among them, x represents the original data, μ represents the mean of the data, σ represents the standard deviation of the data, and x' represents the standardized data.

[0070] The specific steps for processing natural language problems include:

[0071] S211, Natural Language Processing: The system classifies and processes the natural language questions input by the user, including word segmentation, part-of-speech tagging, dependency syntax analysis, entity recognition and semantic parsing.

[0072] Specifically, word segmentation is to divide the input sentence into words or phrases; part-of-speech tagging is to mark the part of speech of each word or phrase, such as noun, verb, adjective; dependency syntax analysis is to analyze the grammatical structure of the sentence and determine the dependency relationship between words; entity recognition is to identify the key entities in the sentence; semantic parsing is to convert the user's natural language questions into the logic and meaning behind them.

[0073] For example, the user enters in step S1: the revenue of a logistics company in Jiangsu Province in the first half of 2023.

[0074] The word segmentation results are: ["2023","first half of the year","Jiangsu Province","logistics company","of","income","situation"].

[0075] Part-of-speech tagging results: ["2023 / time","first half of the year / time","Jiangsu Province / place","logistics company / entity","of / particle","income / noun","situation / noun"].

[0076] Result of dependency syntax analysis: income (subject) <-- situation (predicate).

[0077] S212, keyword extraction: extract key information from the parsed natural language questions, including time, indicators and operations.

[0078] S213, intent recognition: determine the user's query intent, such as query, filter, and sort.

[0079] In this embodiment, the intent recognition result is: querying income status.

[0080] S214, slot filling: fill key information into predefined slots according to the intent recognition results.

[0081] In this embodiment, the slot filling results are: time slot - first half of 2023, location slot - Jiangsu Province, entity slot - logistics company, query content slot - income situation.

[0082] S215, parameter filling: filling the specific parameters obtained by semantic analysis into the matching SQL template.

[0083] SQL template is a predefined framework of SQL query statements, for example: SELECT{index}FROM{table name}WHERE{condition}.

[0084] In this embodiment, the matching SQL template is: SELECT revenue FROM logistics WHERE time = {time} AND location = {location} AND company = {entity}.

[0085] SQL statement after filling in parameters: SELECT revenue FROM logistics WHERE time = 'first half of 2023' AND location = 'Jiangsu Province' AND company = 'Logistics Company'.

[0086] S216, SQL generation: Generate corresponding SQL query statements based on the extracted key information and database schema.

[0087] S217, SQL execution: After optimizing the generated SQL statement, the optimized SQL statement is sent to the corresponding database for execution to obtain the query result. According to the SQL query result, a numerical answer and a corresponding chart (such as a line chart, a bar chart, a pie chart) are generated.

[0088] Among them, optimizing the generated SQL statements can ensure their execution efficiency.

[0089] In this embodiment, the optimized SQL statement is: SELECT revenue FROM logistics WHERE time = 'first half of 2023' AND location = 'Jiangsu Province' AND company = 'Logistics Company'.

[0090] Query results: The revenue of Jiangsu Logistics Company in the first half of 2023 was 20 million yuan.

[0091] S22. For the unstructured data in the database, the RAG algorithm is used to analyze and retrieve natural language questions and unstructured data such as documents, reports, emails, etc. stored in the database.

[0092] The RAG algorithm is a model that combines retrieval and generation techniques for unstructured data retrieval and generation. This technology significantly improves the effect of unstructured data processing by first retrieving relevant documents and then generating answers. The steps of the RAG algorithm include:

[0093] S221. Convert the natural language question input by the user into a vector representation, recorded as a question vector.

[0094] Example: The user's query "Revenue of logistics companies in Jiangsu Province in the first half of 2023" is converted into a vector representation.

[0095] In the process of converting questions into vectors, embedding generation is used, which is to use pre-trained language models (such as BERT and RoBERTa) to convert the input natural language questions into vector representations of fixed dimensions. In addition, the generated embedding vectors need to be standardized, i.e., vector normalization, to ensure that they are suitable for subsequent vector retrieval steps.

[0096] Furthermore, vector standardization is also called normalization, which is the process of converting a vector into a vector of length 1. The purpose of standardization is to make the scale of the vector consistent, thereby improving the accuracy of retrieval.

[0097] Vector normalization formula: For a non-zero vector Its normalized vector for: in For vector Model.

[0098] For example, the user inputs in step S1: What are the main driving factors of the global economic growth in 2024? In this step, this question is transformed into a vector representation.

[0099] This step utilizes the embedding technology in deep learning to transform the text into a vector for similarity calculation.

[0100] S222. Retrieve the document most relevant to the question vector from the unstructured data in the database according to the question vector.

[0101] Specifically, it is necessary to process the unstructured data, and the processing of unstructured data includes:

[0102] Data cleaning, and data cleaning includes:

[0103] Text cleaning: Remove special characters, punctuation marks, HTML tags, etc.

[0104] Case conversion: Uniformly convert to lowercase to avoid affecting subsequent processing due to different cases.

[0105] Word segmentation and part-of-speech tagging. Word segmentation: Segment the text into words or phrases. Common tools for Chinese word segmentation such as Jieba can be used, and for English word segmentation, NLTK, etc. can be used; Part-of-speech tagging: Tag the part of speech of each word, such as noun, verb, adjective, etc.

[0106] Stop word removal: Remove common meaningless words, such as "de", "le", "and", "the", etc.

[0107] Stem extraction and lemmatization. Stem extraction: Restore the word to its stem form; Lemmatization: Restore the word to its original form.

[0108] Vectorization processing, and vectorization processing includes:

[0109] Bag of words model: Convert the text into a word frequency vector, recording the number of times each word appears in the text.

[0110] TF-IDF: Calculate the term frequency-inverse document frequency, convert the text into a weighted word frequency vector, and emphasize important words.

[0111] TF-IDF value calculation;

[0112] TF value calculation: Where TF t,d represents the term frequency of term t in document d, n t,d represents the number of times term t appears in document d, ∑ k n K,d is the total number of times all words appear in document d.

[0113] IDF calculation: where N is the total number of documents in the document set, and df t is the number of documents containing the word t.

[0114] Calculation of TF-IDF value: TF-IDF t,d = TF t,d × IDF t .

[0115] Word embedding: Use a pre-trained word vector model to convert words into dense vectors.

[0116] Sentence vector: Use a pre-trained sentence vector model (such as BERT, GPT) to convert an entire sentence or paragraph into a vector.

[0117] The following is an example:

[0118] Data cleaning:

[0119] Example text: This is a test text! It contains special characters.

[0120] Cleaned text: This is a test text contains special characters

[0121] Case conversion: Chinese generally does not require case conversion, but for the sake of uniformity, the English part can still be processed.

[0122] Cleaned text: This is a test text contains special characters.

[0123] Word segmentation and part-of-speech tagging:

[0124] Example text: This is an example of Chinese word segmentation.

[0125] Word segmentation result: ["This is", "a", "Chinese", "word segmentation", "example"]

[0126] Part-of-speech tagging result: ["This is / pronoun", "a / quantifier", "Chinese / noun", "word segmentation / verb", "example / noun"]

[0127] Stop word removal: Remove common meaningless words such as "de", "le", "he", etc.

[0128] After stop word removal: ["This is", "a", "Chinese", "word segmentation", "example"]

[0129] Stemming and lemmatization: Since Chinese does not require this step, it will not be shown here.

[0130] Vectorization processing:

[0131] Bag of Words Model: Convert text into word frequency vectors, recording the number of times each word appears in the text.

[0132] A collection of example texts: ["This is a test text.", "This text is another example."]

[0133] Output result {bag-of-words model vector: [[1 01 1 01 1 01 0] [01 1 01 01 1 01]]

[0136] Bag of Words model feature names:

[0137] ['one','example','text','test','is','of','this','another','this']

[0138] TF-IDF: Calculates term frequency-inverse document frequency, converts text into weighted term frequency vectors, and emphasizes important words.

[0139] A collection of example texts: ["This is a test text.", "This text is another example."]

[0140] Result output {TF-IDF vector: [[0.0.0.0.4690.0.61 0.61 0.0.] [0.0.61 0.0.0.61 0.0.0.61 0.]]

[0143] TF-IDF feature name:

[0144] ['one','example','text','test','is','of','this','another','this']

[0145] The process of retrieving the documents most relevant to the question vector includes vector retrieval and document ranking.

[0146] Vector retrieval: Use vector retrieval tools (such as FAISS, Annoy, etc.) to perform similarity searches in the document library based on the query vector to find the most relevant documents. Vector retrieval uses the distance in the vector space (Euclidean distance) to measure the relevance of the document to the query.

[0147] Among them, Euclidean distance is the straight-line distance between two points in Euclidean space. It is often used to measure the difference or similarity between two points or vectors.

[0148] Formula: For two points in n-dimensional space P = (P1, P2....P n ) and Q=(Q1,Q2....Q n), the Euclidean distance d(P,Q) between them is defined as:

[0149]

[0150] Document ranking: Sort the retrieved documents according to their similarity scores and select the documents with the highest scores as the final result. This step ensures that the most relevant documents are selected for the generation phase.

[0151] Example: The system finds and sorts some documents in the document library, such as "Financial Report of Jiangsu Logistics Company in the First Half of 2023" and "Market Analysis of Logistics Industry in the First Half of 2023".

[0152] Search results: Document 1: "Financial Report of Jiangsu Logistics Company in the First Half of 2023"; Document 2: "Market Analysis of Logistics Industry in the First Half of 2023".

[0153] S223: Convert the retrieved relevant documents into document vectors, and perform vector fusion with the question vector.

[0154] The relevant documents are converted into document vectors through embedding generation, that is, the retrieved documents are converted into vector representations using the same pre-trained language model as the question vector. The embedding vector of each document represents the semantic information of the document.

[0155] Vector fusion: The document vector is fused with the question vector for use in the generative model. The fused vector represents the combination of the user's query and the retrieved document content. This step usually uses a weighted average or attention mechanism to combine the query and document information to provide richer contextual information.

[0156] Example: The retrieved documents "Financial Report of Jiangsu Logistics Company in the First Half of 2023" and "Market Analysis of Logistics Industry in the First Half of 2023" are converted into vector representations and fused with the user query vector.

[0157] S224. Combine the question vector and the document vector and use the generative model to generate the final answer.

[0158] Specifically, you need to select a suitable generative model (such as GPT-3, BART), which can generate natural language answers based on the input.

[0159] Model input: The fused query vector and document vector are passed as input to the generative model. The generative model generates answers based on the input vectors, and the answer content is usually related to the retrieved documents and the semantics of the query.

[0160] Answer generation: The generative model outputs an answer, which can be a sentence, a paragraph, or a longer text, depending on the complexity of the user query and the content of the document.

[0161] Example: The generative model generates an answer based on the input: "In the first half of 2023, the revenue of logistics companies in Jiangsu Province increased by 15%, mainly benefiting from the development of e-commerce and the upgrading of infrastructure."

[0162] Generated result: The answer is "In the first half of 2023, the revenue of logistics companies in Jiangsu Province increased by 15%, mainly benefiting from the development of e-commerce and the upgrading of infrastructure."

[0163] Answer optimization: Optimize the generated answers to ensure their accuracy and relevance. Specifically, it includes:

[0164] Grammatical and semantic checking: The generated answers are grammatically and semantically checked to ensure that the language expression of the answers is fluent and the meaning is clear.

[0165] Content verification: Verify whether the generated answer is consistent with the retrieved document content and user query to ensure the accuracy of the answer.

[0166] Answer adjustment: Fine-tune the answer as needed, remove redundant information or add necessary details to improve the quality of the answer.

[0167] Example: Check the generated answers for grammar and semantics to ensure language fluency and verify that the answers are consistent with the content of the "Jiangsu Logistics Company 2023 First Half Financial Report" and "Logistics Industry 2023 First Half Market Analysis".

[0168] Optimization results: The optimized answers are fluent in language, accurate in content, and consistent with the document content.

[0169] Answer output: The final answer is returned to the user to meet the user's query requirements. The answer can be displayed in text form on the user interface or delivered to the user through other means.

[0170] Example: The system displays the final answer "In the first half of 2023, the revenue of logistics companies in Jiangsu Province increased by 15%, mainly benefiting from the development of e-commerce and the upgrading of infrastructure." on the user interface.

[0171] Output result: The answer the user sees on the interface is "In the first half of 2023, the revenue of logistics companies in Jiangsu Province increased by 15%, mainly benefiting from the development of e-commerce and the upgrading of infrastructure.", and there is also a link to download the PDF file.

[0172] S3. Optimize natural language problems through fine-tuning modules, so that the model can better adapt to the needs of specific tasks and improve performance.

[0173] Specifically, specific tasks include but are not limited to online customer service, intelligent assistants, education and training, and corporate consulting.

[0174] The optimization steps include:

[0175] S31. Collect and process data for specific tasks, including but not limited to public data sets, internal enterprise data, and user-generated content.

[0176] Data collection refers to the process of obtaining relevant data from multiple sources. These sources can include public datasets, internal enterprise data, and user-generated content. The diversity and quality of data are key to ensuring model performance.

[0177] Example: Collect data about customer service conversations from multiple public datasets and internal enterprise data to train a question-answering model.

[0178] Data processing includes data cleaning, data labeling, and data segmentation.

[0179] Data cleaning refers to cleaning the collected data, removing noise and irrelevant content, and ensuring the quality and consistency of the data. Specific steps include removing duplicate data, correcting errors, processing missing values, and standardizing data formats.

[0180] Example: De-duplicate conversation records in conversation data, correct incorrect labels, and handle missing conversation content.

[0181] Data labeling refers to labeling data according to task requirements. For example, for classification tasks, each data sample needs to be labeled with a category label; for question-answering tasks, question and answer pairs need to be labeled; for named entity recognition tasks, entity boundaries and types need to be labeled.

[0182] Example: Annotate the collected customer service conversation data, where each conversation includes annotations of questions and answers.

[0183] Data partitioning refers to the process of dividing a dataset into training, validation, and test sets. It is usually divided in proportion, such as 80% for training, 10% for validation, and 10% for testing. The purpose of partitioning is to ensure the independence of model training, validation, and testing to prevent overfitting.

[0184] Example: Use 80% of the labeled conversation data for training, 10% for validation, and 10% for testing.

[0185] S32. Select a pre-trained model suitable for a specific task and perform fine-tuning training on it.

[0186] Model selection refers to selecting a pre-trained model that is suitable for the target task. Commonly used pre-trained models include BERT, GPT, RoBERTa, and T5. When selecting a model, factors such as the characteristics of the task, data size, and computing resources need to be considered.

[0187] Example: Choose the BERT model because it performs well in text classification and question answering tasks.

[0188] Model initialization: Model initialization refers to the process of loading the parameters of a pre-trained model and preparing for fine-tuning tasks. The parameters of the initialized model usually come from the pre-training results of a large-scale corpus, which provide the model with rich language knowledge and semantic information.

[0189] Example: Load a pre-trained BERT model and prepare for fine-tuning training.

[0190] Fine-tuning training refers to the process of further training a pre-trained model on a specific task dataset. Specifically, it includes:

[0191] Loss function definition: Define an appropriate loss function based on the task requirements. For example, the classification task uses cross entropy loss, and the regression task uses mean square error loss. The choice of loss function directly affects the optimization direction and effect of the model.

[0192] Loss function formula:

[0193] Mean square error function formula: Where yi is the true value, is the predicted value.

[0194]

[0195] Cross entropy loss function: The cross entropy loss function is used to evaluate the difference between the prediction results of the classification model and the true label, which is particularly suitable for multi-classification tasks. Formula

[0196] Where C is the total number of categories, y i is the true label.

[0197] Example: Define the cross entropy loss function for question answering tasks.

[0198] Optimizer selection: Choose a suitable optimizer (such as Adam, SGD, etc.) to update the model parameters. The choice of optimizer and parameter setting (such as learning rate, weight decay, etc.) have an important impact on the convergence speed and stability of the training process.

[0199] Example: Select the Adam optimizer and set appropriate learning rate and weight decay parameters.

[0200] Adam Optimizer: Adam is an adaptive learning rate optimization algorithm based on first-order and second-order momentum.

[0201] The formula is as follows:

[0202] m t =β1m t-1 +(1-β1)g t

[0203]

[0204] Where θ is the model parameter, η is the learning rate, β1β2 is the decay rate of the momentum term, and g t is the current gradient, m t is the first-order momentum estimate, v t is the second-order momentum estimate, and ∈ is the minimum value that prevents division by zero.

[0205] Model training: Use the defined loss function and optimizer to iteratively train the model on the task dataset. Usually batch training is used to gradually optimize the model parameters. During the training process, it is necessary to monitor the changes in loss values ​​and evaluation indicators to prevent overfitting and underfitting.

[0206] Lora fine-tuning technology; Lora (Low-Rank Adaptation) fine-tuning is an efficient parameter fine-tuning method specifically for large models. By adding a small number of parameters to the original model, Lora fine-tuning can significantly reduce computing costs and storage requirements while maintaining model performance.

[0207] Lora fine-tuning involves the following formula:

[0208] Parameter matrix decomposition: The original parameter matrix W is decomposed into two low-rank matrices A and B, where W ≈ A × B.

[0209] W′=W+ΔW=W+A×B

[0210] Gradient update: During training, only the low-rank matrices A and B are updated with gradients, while the original parameter matrix W remains unchanged:

[0211]

[0212] Among them, η is the learning rate and L is the loss function.

[0213] Merge update: The updated low-rank matrices A and B are multiplied with the original parameter matrix WWW to form the final parameter matrix:

[0214] W′=W+A×B

[0215] Example: Train a BERT model on a customer service conversation dataset and monitor the training loss and validation set accuracy.

[0216] S33. Evaluate the performance of the fine-tuned model on the validation set and optimize it.

[0217] Model evaluation refers to the process of evaluating the performance of a fine-tuned model on a validation set. Specifically, it includes:

[0218] Evaluation indicator selection: Select appropriate evaluation indicators according to task requirements, such as accuracy, F1 value, AUC, etc. The selection of evaluation indicators should be able to fully reflect the performance of the model on the task.

[0219] Accuracy formula and F1 value formula:

[0220] Accuracy:

[0221]

[0222] F1 value formula:

[0223]

[0224] in:

[0225] ·

[0226] ·

[0227] Example: Select accuracy and F1 value as evaluation indicators.

[0228] Model validation: Run the fine-tuned model on the validation set, calculate the evaluation indicators, and evaluate the performance of the model. The validation results are used to guide the adjustment and optimization of the model.

[0229] Example: Evaluate the model's accuracy and F1 score on the validation set to determine the model's performance.

[0230] Hyperparameter tuning;

[0231] Based on the validation results, the model's hyperparameters are tuned to further improve model performance. Common hyperparameter tuning methods include grid search, random search, and Bayesian optimization.

[0232] Example: Optimizing hyperparameters such as learning rate and batch size via grid search.

[0233] Model optimization specifically includes:

[0234] Model deployment: refers to the process of applying the fine-tuned model to the actual environment. Specifically includes:

[0235] Model saving: Save the fine-tuned model parameters and structure as a file for subsequent loading and use. The saved model file usually includes information such as model weights, model configuration, and vocabulary.

[0236] Example: Save a fine-tuned BERT model and its configuration file.

[0237] Deployment environment configuration: Configure the hardware and software environment required for model deployment, such as servers, containers, dependent libraries, etc. The deployment environment needs to ensure the efficient operation and stability of the model.

[0238] Example: Configure a server and necessary software environment such as TensorFlow or PyTorch.

[0239] Model loading and reasoning: Load the saved model in the deployment environment, process the input data in the actual task, perform reasoning and generate output. The reasoning process needs to consider performance indicators such as response time and concurrent request processing.

[0240] Example: Load a model and process a customer service conversation to generate appropriate responses.

[0241] Continuous monitoring and optimization: Continuous monitoring and optimization refers to the process of continuously monitoring the performance of the model after it is deployed, and optimizing and updating it as needed. Specifically, it includes:

[0242] Performance monitoring: Monitor the performance of the model in actual applications, including indicators such as response time and accuracy. Performance monitoring helps to discover and solve problems in a timely manner and ensure the reliability and effectiveness of the model.

[0243] Example: Monitoring model response time and answer accuracy in a customer service system.

[0244] User feedback: Collect user feedback to understand the performance and problems of the model in actual use. User feedback is an important basis for model improvement and can help identify and solve potential problems.

[0245] Example: Collect feedback from customers and support agents to evaluate the quality of the model's responses.

[0246] Model update: Update and optimize the model based on monitoring data and user feedback to maintain high performance and stability. Model update can include retraining, parameter adjustment, and the introduction of new data sets.

[0247] Example: Regularly update and retrain the BERT model based on feedback and new data to improve its performance.

[0248] S4. Combine structured data and unstructured data to generate a comprehensive analysis report and output it through the display module.

[0249] Specifically include:

[0250] S41. Report integration: Integrate the numerical answers and charts of structured data with the financial analysis reports generated from unstructured data.

[0251] Example: Integrate revenue values ​​and charts from SQL queries and market analysis from generative models to form a complete financial report.

[0252] S42. PDF generation: Generate a PDF file from the integrated report content.

[0253] Example: Use the PDF generation tool to generate a PDF financial report containing numerical values, charts, and detailed analysis.

[0254] S43, file output: the PDF file is sent to the user-specified email address or system interface download link, and stored in the system for subsequent access.

[0255] Example: A PDF report containing a detailed financial analysis is generated and sent to the user via email while being archived in the system.

[0256] S44. Comprehensive analysis: Combine the specific numerical answers and charts generated by structured data with the analysis reports generated by unstructured data to provide a more comprehensive company financial situation. Specifically include:

[0257] S441. Data comparison: Compare the results of structured data and unstructured data to find out the consistency and differences.

[0258] Example: Compare SQL query results with the report generated by document retrieval to verify data consistency.

[0259] S442. In-depth analysis: Based on the numerical values ​​and charts of structured data, in-depth analysis of the company's financial status is carried out, combined with analysis reports of unstructured data to provide comprehensive financial analysis.

[0260] Example: Combine revenue growth data with market analysis to provide forecasts and recommendations for the company's future development.

[0261] S443. Report presentation: Display the results of comprehensive analysis to users to help them better understand the company's financial status and future trends.

[0262] Example: Present detailed financial data, charts, and market analysis in reports to help users make decisions.

[0263] In this application, combining structured data and unstructured data for comprehensive analysis can provide a more comprehensive and in-depth analysis of the company's financial status. By uniformly processing different types of data, the completeness and accuracy of the analysis results can be ensured, fully reflecting the company's financial status and operating results.

[0264] It can be understood that by optimizing the processing of natural language questions through the fine-tuning module, the result output is more detailed and accurate. Combined with the comprehensive analysis of structured data and unstructured data, it not only improves the efficiency of information retrieval and analysis, but also improves the quality and reliability of financial analysis reports, providing more comprehensive and accurate financial insights.

[0265] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope of the claims of the present invention.

Claims

1. A financial analysis system, characterized in that: include: A query input module, wherein the query input module is used to collect user questions; The query input module is connected to the data model module, the data model module is used to receive the natural language question from the query input module and analyze it, and the data model module retrieves the structured data and the unstructured data respectively and outputs the result; The data model module is connected to a fine-tuning module, and the fine-tuning module is used to correct the output result of the data model module; The fine-tuning module is connected to the display module, and the display module is used to display the comprehensive analysis report.

2. A method of using the financial analysis system of claim 1, characterized in that: include: S1. The user inputs a query in the query input module; S2, use the data model module to process natural language questions; S3, optimize natural language questions through fine-tuning modules; S4. Combine structured data and unstructured data to generate a comprehensive analysis report and output it through the display module.

3. The method according to claim 2, characterized in that: In step S2, the step of using the data model module includes: S21. Use Text to SQL technology to process natural language questions for structured data in the database; S22. For the unstructured data in the database, the RAG algorithm is used to analyze and retrieve natural language questions and unstructured data such as documents, reports, emails, etc. stored in the database.

4. The method according to claim 3, characterized in that: In step S21, the specific steps of processing the natural language question include: S211, Natural Language Processing: The system classifies and processes the natural language questions input by users, including word segmentation, part-of-speech tagging, dependency syntax analysis, entity recognition, and semantic parsing; S212, keyword extraction: extract key information from the parsed natural language question, including time, indicators and operations; S213, intent recognition: determining the intent of the user's query, such as query, filter, and sort; S214, slot filling: filling key information into predefined slots according to the intent recognition result; S215, parameter filling: filling the specific parameters obtained by semantic analysis into the matching SQL template; S216, SQL generation: Generate corresponding SQL query statements according to the extracted key information and database schema; S217, SQL execution: After optimizing the generated SQL statement, the optimized SQL statement is sent to the corresponding database for execution to obtain the query result.

5. The method according to claim 3, characterized in that: In step S22, the steps of the RAG algorithm include: S221, converting the natural language question input by the user into a vector representation, recorded as a question vector; S222. According to the question vector, retrieve the document most relevant to the question vector from the unstructured data in the database; S223, converting the retrieved relevant documents into document vectors, and performing vector fusion with the question vector; S224. Combine the question vector and the document vector and use the generative model to generate the final answer.

6. The method according to claim 2, characterized in that: In step S3, the optimization step includes: S31. Collect and process data for specific tasks; S32, select a pre-trained model suitable for a specific task and perform fine-tuning training on it; S33. Evaluate the performance of the fine-tuned model on the validation set and optimize it.

7. The method according to claim 2, characterized in that: In step S4, the specific steps include: S41. Report integration: Integrate the numerical answers and charts of structured data with the financial analysis reports generated by unstructured data; S42, PDF generation: generating a PDF file from the integrated report content; S43, file output: sending the PDF file to the user-specified email address or system interface download link, and storing it in the system; S44. Comprehensive analysis: Combine the specific numerical answers and charts generated by structured data with the analysis report generated by unstructured data.

8. The method according to claim 7, characterized in that: In step S44, the specific steps include: S441. Data comparison: Compare the results of structured data and unstructured data to find consistency and differences; S442, In-depth analysis: Based on the numerical values ​​and charts of structured data, in-depth analysis of the company's financial status is carried out, combined with the analysis report of unstructured data to provide comprehensive financial analysis; S443. Report presentation: presenting the results of the comprehensive analysis to the user.

Images