Financial field intelligent data analysis platform based on large language model

Abstract

The application discloses a kind of intelligent data analysis platform in financial field based on large language model, the platform includes user input processing layer, core analysis engine, data processing layer and result display layer, in user input processing layer fine user input processing and intention recognition, improve the understanding accuracy of user demand;In core analysis engine, combined vector retrieval and keyword retrieval are used in hybrid retrieval strategy, and the result ordering is optimized using Rerank model, so as to enhance the relevance and comprehensiveness of query result. Meanwhile, in data processing layer and result display layer, through standardized data processing process and diversified display mode, the consistency, quality and display effect of data are ensured, and the needs of different users are met.

Description

A smart data analysis platform for the fiscal field based on a large language model Technical Field

[0001] This invention relates to the fields of computer software and financial data analysis technology, and mainly to an intelligent data analysis platform for the financial field based on a large language model. Background Technology

[0002] The fiscal sector involves a wide variety of data, including budgets, expenditures, revenues, and fiscal indicators. With the development of big data and artificial intelligence technologies, the demand for intelligent data analysis from governments and enterprises is constantly increasing. However, the large volume and complex structure of fiscal data make it difficult for traditional data analysis to effectively process and extract useful information.

[0003] For example, Chinese patent CN109062874B discloses a method, terminal device, and medium for acquiring financial data. The method includes: acquiring a pre-published text to be analyzed; converting the text format of the text to be analyzed from PDF to DOC format using a preset text conversion tool; acquiring the text code corresponding to the text to be analyzed based on the DOC format text; the text code includes various types of page tags; finding table tags in the page tags and locating the tables in the text to be analyzed based on the text position of the table tags; extracting the values ​​of each field associated with the table and the table description information; and outputting the table description information and each field value to a pre-created text document so that the business system can process the text document and acquire the financial data associated with the text to be analyzed. This reduces the difficulty of acquiring enterprise financial data and achieves multi-dimensional acquisition of financial data. However, the above invention has shortcomings. User input processing is not refined enough, which may lead to inaccurate intent recognition; the question classification and retrieval strategies are relatively simple and may not fully meet complex query needs; data processing and display lack standardization, affecting data consistency and display effects; and data quality management is not comprehensive enough, which may lead to data inconsistencies and quality problems.

[0004] Therefore, there is an urgent need for an intelligent data analysis platform in the fiscal field that provides more comprehensive and accurate fiscal data analysis services. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this application provides an intelligent data analysis platform for the financial field based on a large language model.

[0006] The technical solution of this application is as follows:

[0007] A financial intelligent data analysis platform based on a large language model, the platform comprising a user input processing layer, a core analysis engine, a data processing layer, and a result display layer, wherein:

[0008] The user input processing layer is used to perform preliminary processing on the user's natural language input data to obtain the pre-processed input data;

[0009] The core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intent recognition module, and a JSON calibration module. The question classification module uses a question classification model to classify questions and extract keywords from the initially processed input data, obtaining question type labels and keywords. The hybrid retrieval knowledge base query module uses a hybrid retrieval strategy combining vector retrieval and keyword retrieval to obtain query results from the initially processed input data, question type labels, and keywords. These query results are detailed descriptions of fiscal indicators, including definitions, calculation methods, applicable scope, and policy basis. The intent recognition module performs natural language processing on the initially processed input data, combines the query results with additional contextual intent analysis and demand mapping, and obtains the JSON structure of the user's requested fiscal indicator data, including fiscal indicators, analysis dimensions, time range, and geographical scope. The JSON calibration module is used to normalize the JSON structure of the user's requested fiscal indicator data, obtaining a standardized fiscal data request structure.

[0010] The data processing layer obtains a standardized fiscal data request structure through the API and outputs a standardized fiscal indicator dataset, including indicator values, related metadata, and quality tags.

[0011] The results display layer is used to present the fiscal indicator dataset in the form of text descriptions, tables, and statistical charts.

[0012] Preferably, the preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are not useful for analysis, and marking the part of speech of each word remaining after removal.

[0013] Preferably, the problem classification model is a large language model based on the Transformer architecture, including a shared encoder, a classification head, and an extraction head. The pre-processed input data is used as the input to the shared encoder, which captures contextual information in the input data through a self-attention mechanism and outputs corresponding feature representations. The feature representations are used as the input to the classification head and the extraction head to output problem type labels and keywords.

[0014] The large language model is pre-trained using a financial domain database, and additional training is performed using financial sample data pre-annotated by experts during the pre-training process to obtain a fully trained problem classification model.

[0015] Preferably, the vector retrieval in the hybrid retrieval strategy includes text embedding and approximate nearest neighbor retrieval, specifically:

[0016] The text embedding uses an embedding model to capture the semantic information of the pre-processed input data to obtain a high-dimensional vector. The embedding model is either a BERT model or a Word2Vec model. The high-dimensional vector is then subjected to an approximate nearest neighbor search using the HNSW or IVF algorithm to obtain preliminary relevant records. These preliminary relevant records are then weighted according to the question type labels corresponding to the pre-processed input data, and preliminary relevant records that are irrelevant to the question type labels are removed to obtain a vector retrieval relevant record dataset.

[0017] Preferably, the keyword retrieval in the hybrid retrieval strategy utilizes inverted index technology and synonym expansion for precise matching and fuzzy matching. Specifically, it involves building an index containing keywords, identifying and expanding the synonyms of the keywords, and using the expanded keywords to perform retrieval in the inverted index to obtain a keyword retrieval dataset.

[0018] Preferably, the hybrid retrieval strategy further includes initially merging the vector retrieval related record dataset and the keyword retrieval dataset to obtain a merged dataset, and using the Rerank model to sort the merged dataset. The Rerank model scores and sorts the dataset based on vector similarity and keyword matching degree to obtain the query results.

[0019] Preferably, the specific steps for obtaining the JSON structure of user demand financial indicator data are as follows:

[0020] Named entity recognition technology is used to identify important entities in the initially processed input data. Predefined rules and patterns are used to identify the relationships between important entities to obtain data after natural language processing.

[0021] In-depth analysis of natural language processing data is conducted. Specifically, based on the natural language processing data, the query results are combined with contextual information for further understanding. Through the reasoning capabilities of open-source large language models, key elements are identified from the natural language processing data, and the data type and request method desired by the user are inferred. The key elements, the data type of the user's financial indicators, and the request method of the user's financial indicators are combined to obtain the intent analysis results of the user's financial indicators.

[0022] The intent analysis results are mapped to specific fields, including financial indicators, analysis dimensions, time range, and geographic range, and a user demand financial indicator data structure conforming to JSON format is generated based on the mapping results.

[0023] Preferably, the JSON structure of the user's requested fiscal indicator data is standardized to obtain a standardized fiscal data request structure, specifically as follows:

[0024] The JSON structure of the received user demand financial indicator data is validated by combining a rule engine and machine learning. The rule engine is used to define validation rules to check the existence of fields and the validity of values, while machine learning is used to train the model to identify abnormal data or data that does not conform to the specifications, and obtain the JSON structure of the successfully validated user demand financial indicator data.

[0025] The specific formats and subject systems of government financial statements at different levels are collected in advance as DSL specifications, and corresponding conversion rules are defined. Based on the DSL specifications of financial indicator data required by users, the predefined conversion rules are applied to convert the standard JSON structure into DSL format.

[0026] Preferably, the verification rules include: determining whether the fields of fiscal indicators, analysis dimensions, time range, and geographic range exist; determining whether the values ​​of fiscal indicators and analysis dimensions conform to predefined specifications; determining whether the time range format is correct and whether the start time is less than the end time; and determining whether the geographic range conforms to the corresponding geographic specifications.

[0027] Preferably, the data processing layer obtains a standardized fiscal data request structure through an API and outputs a standardized fiscal indicator dataset, including indicator values, related metadata, and quality tags, specifically:

[0028] The data processing layer acquires and processes user-required financial indicator data through standardized API interfaces and integrates data sources for data transformation.

[0029] The data processing layer stores data through Redis and has a preset caching mechanism;

[0030] The data processing layer is designed with a metadata model for storing metadata, which includes the definition, data source, and unit of the indicators in the user demand financial indicator data, and performs metadata management.

[0031] The data processing layer also saves data snapshots at different points in time through version control and automatically marks problematic data for data quality assessment.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] 1. This invention provides an intelligent data analysis platform for the financial field based on a large language model. By performing detailed word segmentation, removing useless words and part-of-speech tags on natural language input data, and combining the natural language processing technology of the intent recognition module, it accurately extracts user needs and generates a standardized JSON structure, thereby improving the accuracy of intent judgment and the precision of data requests.

[0034] 2. This invention provides an intelligent data analysis platform for the fiscal field based on a large language model. By combining vector retrieval and keyword retrieval and optimizing the ranking of retrieval results through the Rerank model, it significantly improves the relevance and comprehensiveness of query results and enhances the accuracy of fiscal data retrieval.

[0035] 3. This invention provides an intelligent data analysis platform for the financial field based on a large language model. It obtains data through standardized API interfaces, uses Redis caching mechanisms, performs metadata management and data quality assessment, ensuring data consistency and high quality. It presents data in various display formats such as text, tables, and statistical charts, meeting the needs of different users and improving the data display effect. Attached Figure Description

[0036] Figure 1 is a platform structure block diagram of an embodiment of the present invention. Detailed Implementation

[0037] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0038] This invention provides the following technical solution: an intelligent data analysis platform for the financial field based on a large language model.

[0039] Example 1

[0040] This embodiment provides an intelligent data analysis platform for the financial field based on a large language model. The platform includes a user input processing layer, a core analysis engine, a data processing layer, and a result display layer, wherein:

[0041] S1, User Input Processing Layer;

[0042] The user input processing layer is used to perform preliminary processing on the user's natural language input data to obtain the pre-processed input data;

[0043] Preferably, the preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are not useful for analysis, and marking the part of speech of each word remaining after removal;

[0044] S2, the core analysis engine;

[0045] The core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intent recognition module, and a JSON calibration module;

[0046] S21, Problem Classification Module;

[0047] The problem classification module uses a problem classification model to classify problems and extract keywords from the pre-processed input data, obtaining problem type labels and keywords, wherein:

[0048] The problem classification model is a large language model based on the Transformer architecture, specifically BERT or GPT. This model includes a shared encoder, a classification head, and an extraction head. The pre-processed input data is used as input to the shared encoder, which processes the input data and generates context-aware representations. This shared encoder includes multiple self-attention layers (such as those in Transformer) and feedforward neural network layers, capable of capturing complex dependencies and contextual information in the text. It outputs a set of context-rich feature representations, which are multi-dimensional vectors. Each vector corresponds to a token (word or subword) in the input data. These feature representations encompass an understanding of the entire input data and also include information about each token within its context. The output feature representation is then used as input to the classification head and the extraction head.

[0049] The classification head includes fully connected layers and a softmax activation function. It receives token-level representations from the shared encoder, aggregates all token-level representations to obtain a global representation. Specifically, it uses pooling operations, including average pooling or max pooling, to aggregate the token-level representations into a fixed-length vector. If the large language model is a BERT model, it directly uses the [CLS]-labeled representation, where the [CLS]-labeled output represents the global information of the entire sequence. This global representation is then processed by one or more fully connected layers and a softmax activation function to obtain the predicted probability of the question category.

[0050] The extraction head includes a sequence labeling layer, which is a Conditional Random Field (CRF) layer or a fully connected layer. In this embodiment, since the CRF layer helps to capture the relationship between entity tags and improve the accuracy of entity recognition, the sequence labeling layer is a CRF layer. The extraction head receives the token level representation output by the shared encoder. The level representation of each token is used to determine whether the token is a keyword or key entity and which entity category it belongs to, and generates a label for each token.

[0051] The large language model is pre-trained using a fiscal domain database to better identify and handle fiscal-related issues. This fiscal domain database can be constructed by extracting large-scale fiscal domain data from government websites, official websites of fiscal-related international organizations, and academic databases. During the pre-training process, a small number of labeled fiscal data samples are used to further train the large language model. These labeled fiscal data samples are manually labeled, specifically by fiscal domain experts, and contain specific categories or labels of various issues within the fiscal domain. Because the labeling process involves expert knowledge, these data samples are of high quality and can accurately reflect the classification of fiscal domain issues. This improves the model's classification ability with limited data samples, avoids overfitting, and enhances the model's generalization ability.

[0052] Preferably, the problem type labels in this embodiment include budget preparation, execution supervision, final accounts analysis, fund management, policy formulation, and others, wherein:

[0053] The budget preparation mentioned above involves issues such as the formulation, modification, approval, and allocation of the fiscal budget; the execution supervision mentioned above covers issues such as supervision, auditing, compliance checks, and monitoring of fund usage during the budget execution process; the final accounts analysis mentioned above focuses on issues such as the review, summary, analysis, and report preparation of fiscal final accounts; the fund management mentioned above involves issues such as fund use, allocation, settlement, and fund sources and expenditures; the policy formulation mentioned above involves issues such as the formulation, adjustment, and evaluation of fiscal policies and their impact on the economy; and "other" refers to other issues not explicitly categorized but involving the fiscal field.

[0054] Preferably, the keyword entities mentioned in this embodiment include, but are not limited to, budget items, implementing departments, final accounts data, policy names, and funding types;

[0055] S22, Hybrid Search Knowledge Base Query Module;

[0056] The hybrid retrieval knowledge base query module is based on a hybrid retrieval strategy combining vector retrieval and keyword retrieval. It takes the pre-processed input data, question type tags, and keywords as input to the hybrid retrieval knowledge base query module. Specifically:

[0057] S221, Vector retrieval;

[0058] The vector retrieval includes text embedding and approximate nearest neighbor search. The text embedding uses an embedding model to capture the semantic information of the pre-processed input data to obtain a high-dimensional vector. The embedding model is either a BERT model or a Word2Vec model. The high-dimensional vector is then subjected to an approximate nearest neighbor search using the HNSW or IVF algorithm to obtain preliminary relevant records. These preliminary relevant records are weighted according to the question type labels corresponding to the pre-processed input data, and preliminary relevant records that are irrelevant to the question type labels are removed to obtain a vector retrieval relevant record dataset.

[0059] S222, Keyword Search;

[0060] The keyword retrieval utilizes inverted indexing and synonym expansion for precise and fuzzy matching, finding records containing the keywords in a pre-defined knowledge base. This pre-defined knowledge base can be constructed by acquiring data from the aforementioned financial database, specifically as follows:

[0061] Build an index containing keywords, identify synonyms of keywords and expand the keywords to be queried accordingly, use the expanded keywords to search in the inverted index, and obtain a keyword retrieval dataset;

[0062] The vector retrieval related record dataset and the keyword retrieval dataset are initially merged to obtain a merged dataset. The merged dataset is then sorted using the Rerank model, which scores and ranks the datasets based on vector similarity and keyword matching. The query results are specifically detailed descriptions of fiscal indicators, including definitions, calculation methods, scope of application, and policy basis.

[0063] S23, Intent recognition module;

[0064] The intent recognition module utilizes open-source large language models (such as qwen, chatglm, BERT, etc.) to perform natural language processing on the initially processed input data, and combines the query results with additional context for intent analysis and demand mapping to obtain the JSON structure of user demand financial indicator data, wherein:

[0065] S231, Natural Language Processing;

[0066] Named Entity Recognition (NER) technology is used to identify important entities in the pre-processed input data. Specific tools and models used include SpaCy, BERT, and RoBERTa. The important entities include financial indicators, time, and region.

[0067] Using predefined rules and patterns to identify relationships between important entities, we extract relationships and obtain data after natural language processing.

[0068] S232, Intent Analysis;

[0069] In-depth analysis of natural language processing data is performed. Specifically, based on the current input, the query results are combined with additional contextual information for understanding. Through the reasoning capabilities of open-source large language models, key elements are identified from the natural language processing data. The desired data type and request method of the user are inferred and determined. The key elements, the data type of the user's financial indicators, and the request method of the user's financial indicators are combined to obtain the intent analysis results of the user's financial indicators.

[0070] S233, Demand Mapping;

[0071] The intent analysis results are mapped to specific fields, including financial indicators, analysis dimensions, time range, and geographic range, and a user demand financial indicator data structure conforming to JSON format is generated based on the mapping results.

[0072] S24, JSON calibration module;

[0073] The JSON calibration module is used to normalize the JSON structure of user-required financial indicator data. This normalization process includes data validation and structure transformation, wherein:

[0074] The JSON calibration module combines a rule engine and machine learning to validate the JSON structure of the received user-required financial indicator data. The rule engine defines rules to check the existence of fields and the validity of values, while machine learning trains a model to identify anomalous or non-compliant data. Specific validation rules include:

[0075] Ensure that fields such as fiscal indicators, analysis dimensions, time range, and geographic range exist;

[0076] Ensure that the values ​​of fiscal indicators and analytical dimensions conform to predefined specifications (e.g., check if the indicators are in the list of allowed indicators);

[0077] Ensure the time range format is correct and that the start time is less than the end time;

[0078] The geographical scope must conform to the relevant geographical regulations;

[0079] Preferably, return detailed error information in the JSON structure of the user's requested financial indicator data that failed verification, such as missing fields or format errors, so that the user can make timely corrections.

[0080] The JSON calibration module also includes an integrated Domain-Specific Language (DSL) parser, which is used to convert the successfully validated standard JSON structure into a budget item system and financial statement format that is adapted to different levels of government. Specifically, the specific formats and item systems of financial statements of different levels of government are collected in advance as DSL specifications, and corresponding conversion rules are defined. According to the target DSL specification, the predefined conversion rules are applied to convert the standard JSON structure into DSL format.

[0081] S3, Data Processing Layer;

[0082] The data processing layer obtains a standardized fiscal data request structure through an API and outputs a standardized fiscal indicator dataset, including indicator values, related metadata, and quality tags, specifically:

[0083] The data processing layer acquires and processes user-required financial indicator data through standardized API interfaces and integrates data sources for data transformation. It stores data in Redis with a pre-set caching mechanism to improve response speed, and stores metadata through a designed metadata model. This metadata includes information such as the definition, data source, and unit of the indicators in the user-required financial indicator data, and performs metadata management to enhance data interpretability. Furthermore, the data processing layer uses version control to save data snapshots at different points in time and performs data quality assessment to automatically mark problematic data.

[0084] After the above processing, the data processing layer outputs a standardized dataset of fiscal indicators, including indicator values, related metadata, and quality tags;

[0085] S4, Results Display Layer;

[0086] The results display layer is used to present the fiscal indicator dataset through text descriptions, tables, and statistical charts. Specifically:

[0087] The fiscal indicator dataset is formatted, including cleaning, aggregation, and standardization.

[0088] The text description format is presented using natural language processing technology. The result display layer automatically generates concise and clear text descriptions based on the formatted fiscal indicator dataset, summarizing the changing trends and important characteristics of key fiscal indicators such as revenue, expenditure, and deficit.

[0089] The tabular format presentation organizes the formatted fiscal indicator dataset into a structured table, clearly displaying each fiscal indicator and its related metadata, making it easy for users to quickly obtain specific values ​​and make intuitive comparisons.

[0090] The statistical chart formats include chart type recommendations and interactive chart generation. The chart type recommendations combine a Large Language Model (LLM) and a chart analysis knowledge base. The results display layer intelligently recommends the most suitable chart type (such as bar charts, line charts, pie charts, etc.) based on data characteristics and user query intent. The interactive chart generation utilizes visualization libraries such as ECharts. The results display layer generates rich interactive charts that support user operations such as zooming, dragging, and data filtering to explore and analyze data more deeply. Preferably, the results display layer automatically adjusts the chart's color, font, labels, and other parameters to improve readability and aesthetics.

[0091] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

[0092] Those skilled in the art will understand that embodiments of this application can be provided as methods, platforms, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A financial intelligent data analysis platform based on a large language model, characterized in that, The platform comprises a user input processing layer, a core analysis engine, a data processing layer, and a result display layer. Specifically: the user input processing layer performs preliminary processing on the user's natural language input data to obtain pre-processed input data; the core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intent recognition module, and a JSON calibration module. The question classification module uses a question classification model to classify questions and extract keywords from the pre-processed input data, obtaining question type labels and keywords; the hybrid retrieval knowledge base query module uses a hybrid retrieval strategy combining vector retrieval and keyword retrieval to obtain query results using the pre-processed input data, question type labels, and keywords. The query results specifically provide detailed descriptions of fiscal indicators, including definitions, calculation methods, applicable scope, and policy basis; the intent recognition module performs natural language processing on the pre-processed input data and combines the query results as additional context for intent analysis and needs assessment. The process involves several steps: First, a mapping is performed to obtain the JSON structure of the user-requested fiscal indicator data, including fiscal indicators, analysis dimensions, time range, and geographic range. Second, a JSON calibration module is used to standardize the JSON structure of the user-requested fiscal indicator data, obtaining a standardized fiscal data request structure. Specifically, this involves combining a rule engine and machine learning to validate the received JSON structure of the user-requested fiscal indicator data. Pre-collected specified formats and subject systems of government fiscal reports at different levels are used as a DSL standard, and corresponding conversion rules are defined. Based on the DSL standard of the user-requested fiscal indicator data, the predefined conversion rules are applied to convert the validated JSON structure into the DSL format. Third, the data processing layer obtains the standardized fiscal data request structure through an API and outputs a standardized fiscal indicator dataset, including indicator values, related metadata, and quality tags. Fourth, the result display layer presents the fiscal indicator dataset through text descriptions, tables, and statistical charts.

2. The intelligent data analysis platform for the fiscal field based on a large language model according to claim 1, characterized in that, The preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are not useful for analysis, and marking the part of speech of each remaining word after removal.

3. The intelligent data analysis platform for the fiscal field based on a large language model according to claim 2, characterized in that, The problem classification model is a large language model based on the Transformer architecture, including a shared encoder, a classification head, and an extraction head. The pre-processed input data is used as the input to the shared encoder, which captures contextual information in the input data through a self-attention mechanism and outputs corresponding feature representations. These feature representations are then used as the input to the classification head and the extraction head, outputting problem type labels and keywords. The large language model is pre-trained using a financial domain database, and additional training is performed during the pre-training process using financial sample data pre-annotated by experts to obtain a fully trained problem classification model.

4. The intelligent data analysis platform for the financial field based on a large language model according to claim 3, characterized in that, The vector retrieval in the hybrid retrieval strategy includes text embedding and approximate nearest neighbor retrieval. Specifically, the text embedding uses an embedding model to capture the semantic information of the pre-processed input data to obtain a high-dimensional vector. The embedding model is either a BERT model or a Word2Vec model. The high-dimensional vector is then subjected to an approximate nearest neighbor search using the HNSW or IVF algorithm to obtain preliminary relevant records. These preliminary relevant records are then weighted according to the question type labels corresponding to the pre-processed input data, and preliminary relevant records that are irrelevant to the question type labels are removed to obtain a vector retrieval relevant record dataset.

5. The intelligent data analysis platform for the financial field based on a large language model according to claim 4, characterized in that, The keyword retrieval strategy in the hybrid retrieval strategy utilizes inverted index technology and synonym expansion for exact and fuzzy matching. Specifically, it involves building an index containing keywords, identifying and expanding the synonyms of the keywords, and using the expanded keywords to perform retrieval in the inverted index to obtain a keyword retrieval dataset.

6. The intelligent data analysis platform for the financial field based on a large language model according to claim 5, characterized in that, The hybrid retrieval strategy also includes initially merging the vector retrieval related record dataset and the keyword retrieval dataset to obtain a merged dataset. The merged dataset is then sorted using a Rerank model, which scores and ranks the datasets based on vector similarity and keyword matching to obtain the query results.

7. The intelligent data analysis platform for the financial field based on a large language model according to claim 6, characterized in that, The specific steps to obtain the JSON structure of user demand financial indicator data are as follows: use named entity recognition technology to identify important entities in the pre-processed input data, use predefined rules and patterns to identify the relationships between important entities, and obtain the data after natural language processing. In-depth analysis of the natural language processing (NLP) data is performed. Specifically, based on the NLP data, query results are combined with additional contextual information for understanding. Leveraging the reasoning capabilities of open-source large language models, key elements are identified from the NLP data, and the desired data type and request method of the user are inferred. The key elements, the data type of the user's desired financial indicator, and the request method for the user's desired financial indicator data are combined to obtain the intent analysis results of the user's desired financial indicator data. These intent analysis results are then mapped to specific fields, including financial indicators, analysis dimensions, time range, and geographic range. Based on the mapping results, a user's desired financial indicator data structure conforming to JSON format is generated.

8. The intelligent data analysis platform for the financial field based on a large language model according to claim 7, characterized in that, The rules engine is used to define validation rules to check the existence of fields and the validity of values. Machine learning is used to train models to identify abnormal or non-compliant data, and obtain the JSON structure of user demand financial indicator data that has been successfully validated.

9. A financial intelligent data analysis platform based on a large language model according to claim 8, characterized in that, The validation rules include determining whether the fields for fiscal indicators, analysis dimensions, time range, and geographic range exist; determining whether the values ​​of fiscal indicators and analysis dimensions conform to predefined specifications; determining whether the time range format is correct and whether the start time is less than the end time; and determining whether the geographic range conforms to the corresponding geographic specifications.

10. A financial intelligent data analysis platform based on a large language model according to claim 9, characterized in that, The data processing layer obtains a standardized fiscal data request structure through an API and outputs a standardized fiscal indicator dataset, including indicator values, related metadata, and quality markers. Specifically, the data processing layer acquires and processes user-requested fiscal indicator data through a standardized API interface and integrates data sources for data transformation. The data processing layer stores data through Redis and has a pre-set caching mechanism. The data processing layer has a metadata model for storing metadata, which includes the definition, data source, and unit of the indicators in the user-requested fiscal indicator data, and performs metadata management. The data processing layer also saves data snapshots at different points in time through version control and automatically marks problematic data for data quality assessment.

Images