Financial field intelligent data analysis platform based on large language model
Through the intelligent data analysis platform in the financial field based on large language models, the problems of insufficient user input processing and unstandardized data processing are solved, and high accuracy and consistency of fiscal data analysis is achieved, which meets the complex query needs and multiple data Display format.
Patent Information
- Application Number
- CN202411137846.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-19
AI Technical Summary
The existing financial data analysis platform has not been meticulous enough in user input processing, inaccurate intention identification, simple problem classification and retrieval strategies, unable to meet complex query needs, and lack of standardization of data processing and display, which affects data consistency and display effect.
Design an intelligent data analysis platform in the financial field based on large language models, including user input processing layer, core analysis engine, data processing layer and result display layer, and a hybrid search strategy combining natural language processing, vector search and keyword search. Generate a normalized JSON structure, use the API to obtain and process data, and ensure data consistency and quality through Redis cache and metadata management.
It improves the accuracy of intention judgment and the accuracy of data requests, significantly improves the relevance and comprehensiveness of query results, ensures the consistency and high quality of data, and meets the data display needs of different users.
Smart Images

Figure CN119025544A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer software and financial data analysis, and mainly to an intelligent data analysis platform in the financial field based on a large language model. Background Art
[0002] The financial sector involves a wide variety of data, including budgets, expenditures, income, financial indicators, etc. With the development of big data and artificial intelligence technology, the demand for intelligent data analysis by governments and enterprises continues to increase. However, the large amount of data and complex structure in the financial sector make it difficult for traditional data analysis to effectively process and extract useful information.
[0003] For example, the Chinese patent CN109062874B discloses a method, terminal device and medium for acquiring financial data, which includes: acquiring a pre-published text to be analyzed; converting the text format of the text to be analyzed from pdf format to document doc format through a preset text conversion tool; acquiring the text encoding corresponding to the text to be analyzed based on the text to be analyzed in doc format; the text encoding includes multiple types of page tags; searching for table tags in page tags, and locating the table in the text to be analyzed according to the text position to which the table tag belongs; extracting each field value and table description information associated with the table; outputting the table description information and each field value to a pre-created text document, so that the business system can obtain the financial data associated with the text to be analyzed after identifying and processing the text document. The difficulty of acquiring corporate financial data is reduced, and the multi-dimensional acquisition effect of financial data is achieved. However, the user input processing of the above invention is not fine enough, which may lead to inaccurate intention recognition; the problem classification and retrieval strategy are relatively simple, and may not fully meet the complex query requirements; data processing and display lack standardization, affecting data consistency and display effect; data quality management is not comprehensive enough, and there may be data inconsistency and quality problems.
[0004] Therefore, there is an urgent need for an intelligent data analysis platform in the financial field that can provide more comprehensive and accurate financial data analysis services. Summary of the invention
[0005] In order to solve the above-mentioned problems existing in the prior art, the present application provides an intelligent data analysis platform in the financial field based on a large language model.
[0006] The technical solution of this application is as follows:
[0007] A financial intelligent data analysis platform based on a large language model, the platform includes a user input processing layer, a core analysis engine, a data processing layer and a result display layer, wherein:
[0008] The user input processing layer is used to perform preliminary processing on the user's natural language input data to obtain the input data after preliminary processing;
[0009] The core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intention recognition module and a JSON calibration module. The question classification module uses a question classification model to classify questions and extract keywords from the input data after preliminary processing to obtain question type labels and keywords. The hybrid retrieval knowledge base query module uses a hybrid retrieval strategy based on vector retrieval and keyword retrieval to obtain query results using the input data after preliminary processing, question type labels and keywords. The query results are specifically detailed descriptions of fiscal indicators, including definitions, calculation methods, applicable scopes and policy basis. The intention recognition module obtains the JSON structure of user demand fiscal indicator data by performing natural language processing on the input data after preliminary processing, combining the query results with intention analysis and demand mapping as additional context, including fiscal indicators, analysis dimensions, time range and geographical range. The JSON calibration module is used to normalize the JSON structure of user demand fiscal indicator data to obtain a normalized fiscal data request structure.
[0010] The data processing layer obtains the standardized financial data request structure through the API and outputs a standardized financial indicator data set, including indicator values, related metadata and quality tags;
[0011] The result display layer is used to display the financial indicator data set in the form of text description, tables and statistical charts.
[0012] Preferably, the preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are useless for analysis, and marking the part of speech of each word remaining after the removal.
[0013] Preferably, the question classification model is a large language model based on the Transformer architecture, including a shared encoder, a classification head and an extraction head, wherein the input data after preliminary processing is used as the input of the shared encoder, the shared encoder captures the context information in the input data through a self-attention mechanism, outputs the corresponding feature representation, and the feature representation is used as the input of the classification head and the extraction head, and the question type label and keywords are output;
[0014] The large language model is pre-trained using a database in the financial field, and during the pre-training process, additional training is performed using financial sample data pre-labeled by experts to obtain a trained question classification model.
[0015] Preferably, the vector search in the hybrid search strategy includes text embedding and approximate nearest neighbor search, specifically:
[0016] The text embedding uses an Embedding model to capture semantic information of the input data after preliminary processing to obtain a high-dimensional vector, and the Embedding model is a BERT model or a Word2Vec model; an HNSW or IVF algorithm is used to perform an approximate nearest neighbor search on the high-dimensional vector to obtain preliminary relevant records, and the preliminary relevant records are weighted according to the question type labels corresponding to the input data after preliminary processing, and preliminary relevant records that are not related to the question type labels are eliminated to obtain a vector retrieval related record data set.
[0017] Preferably, the keyword search in the hybrid search strategy uses inverted index technology and synonym expansion for exact matching and fuzzy matching, specifically establishing an index containing keywords, identifying synonyms of the keywords and expanding them, using the expanded keywords to search in the inverted index, and obtaining a keyword search data set.
[0018] Preferably, the hybrid search strategy also includes preliminarily merging the vector search related record data set and the keyword search data set to obtain a merged data set, and sorting the merged data set using a Rerank model, wherein the Rerank model performs comprehensive scoring and sorting based on vector similarity and keyword matching degree to obtain query results.
[0019] Preferably, the specific steps of obtaining the JSON structure of the user demand financial indicator data are:
[0020] Use named entity recognition technology to identify important entities in the input data after preliminary processing, use predefined rules and patterns to identify the relationship between important entities, and obtain data after natural language processing;
[0021] Conduct in-depth analysis of the data after natural language processing, specifically, based on the data after natural language processing, use the query results as additional contextual information for understanding, identify key elements from the data after natural language processing through the reasoning ability of the open source large language model, and infer the data type and request method desired by the user, and combine the key elements, the data type of user-demand financial indicator data, and the request method of user-demand financial indicator data to obtain the intention analysis results of user-demand financial indicator data;
[0022] The intent analysis results are mapped to specific fields, including financial indicators, analysis dimensions, time range and geographical range, and a user demand financial indicator data structure conforming to the JSON format is generated based on the mapping results.
[0023] Preferably, the JSON structure of the financial indicator data required by the user is standardized, and the standardized financial data request structure is obtained as follows:
[0024] The rule engine and machine learning are combined to perform data validation on the JSON structure of the received user demand financial indicator data. The rule engine is used to define validation rules to check the existence of fields and the validity of values. Machine learning is used to train the model to identify abnormal data or data that does not conform to the specifications, and obtain the JSON structure of the successfully validated user demand financial indicator data.
[0025] Collect the specific formats and subject systems of government financial statements at different levels in advance as DSL specifications, and define corresponding conversion rules. According to the DSL specifications of user-required financial indicator data, apply predefined conversion rules to convert the standard JSON structure into DSL format.
[0026] Preferably, the validation rules include determining whether the financial indicator, analysis dimension, time range and geographical range fields exist; determining whether the values of the financial indicator and analysis dimension meet predefined specifications; determining whether the time range format is correct and the start time is less than the end time; and determining whether the geographical range meets the corresponding geographical specifications.
[0027] Preferably, the data processing layer obtains a standardized financial data request structure through an API, and outputs a standardized financial indicator data set, including indicator values, related metadata, and quality tags, specifically:
[0028] The data processing layer acquires and processes the financial indicator data required by users by developing standardized API interfaces, and integrates data sources for data conversion;
[0029] The data processing layer stores data through Redis and presets a cache mechanism;
[0030] The data processing layer is designed with a metadata model for storing metadata, the metadata including the definition, data source, and unit of the indicators in the user demand financial indicator data, and performs metadata management;
[0031] The data processing layer also saves data snapshots at different time points through version control, and performs data quality assessment to automatically mark problematic data.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] 1. The present invention provides an intelligent data analysis platform in the financial field based on a large language model. By detailed word segmentation of natural language input data, removal of useless words and part-of-speech tags, and combining the natural language processing technology of the intent recognition module, the platform can accurately extract user needs and generate a standardized JSON structure, thereby improving the accuracy of intent judgment and the precision of data requests;
[0034] 2. The present invention provides an intelligent data analysis platform in the financial field based on a large language model, which combines vector retrieval and keyword retrieval, optimizes the sorting of retrieval results through the Rerank model, significantly improves the relevance and comprehensiveness of query results, and enhances the accuracy of financial data retrieval;
[0035] 3. The present invention provides an intelligent data analysis platform in the financial field based on a large language model. It obtains data through a standardized API interface, uses a Redis cache mechanism, and performs metadata management and data quality assessment, thereby ensuring the consistency and high quality of the data. It presents data in a variety of display forms such as text, tables, and statistical charts, thereby meeting the needs of different users and improving the data display effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a platform structure block diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0038] The present invention provides the following technical solution: an intelligent data analysis platform in the financial field based on a large language model.
[0039] Example 1
[0040] This embodiment provides a financial intelligent data analysis platform based on a large language model, the platform includes a user input processing layer, a core analysis engine, a data processing layer and a result display layer, wherein:
[0041] S1, user input processing layer;
[0042] The user input processing layer is used to perform preliminary processing on the user's natural language input data to obtain the input data after preliminary processing;
[0043] Preferably, the preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are useless for analysis, and marking the part of speech of each word remaining after the removal;
[0044] S2, core analysis engine;
[0045] The core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intent recognition module, and a JSON calibration module;
[0046] S21, question classification module;
[0047] The question classification module uses the question classification model to perform question classification and keyword extraction on the input data after preliminary processing to obtain question type labels and keywords, where:
[0048] The question classification model is a large language model based on the Transformer architecture, which is BERT or GPT, including a shared encoder, a classification head and an extraction head. The preliminarily processed input data is used as the input of the shared encoder. The shared encoder is responsible for processing the input data and generating a context-aware representation, including multiple self-attention layers (such as the self-attention layer in the Transformer) and a feedforward neural network layer, which can capture the complex dependencies and context information in the text and output a set of context-rich feature representations. The feature representation is a multi-dimensional vector, each vector corresponding to a Token (word or subword) in the input data. The above feature representation includes an understanding of the entire input data, and also includes a feature representation corresponding to the information output of each Token in its context. The feature representation is used as the input of the classification head and the extraction head;
[0049] The classification head includes a fully connected layer and a softmax activation function. By receiving the Token-level representation output by the shared encoder, all Token-level representations are aggregated to obtain a global representation. Specifically, the Token-level representation is aggregated into a vector of a fixed length through a pooling operation, including average pooling or maximum pooling. If the large language model is a BERT model, the representation of the [CLS] tag is directly used, and the output of the [CLS] tag represents the global information of the entire sequence. The global representation is passed through one or more fully connected layers and a softmax activation function to obtain the predicted probability of the question category.
[0050] The extraction head includes a sequence annotation layer, which is a conditional random field CRF layer or a fully connected layer. In this embodiment, the sequence annotation layer is a CRF layer because the CRF layer helps to capture the relationship between entity tags and improve the accuracy of entity recognition. The extraction head receives the Token level representation output by the shared encoder. The level representation of each Token is used to determine whether the Token is a keyword or a key entity, and which entity category it belongs to, and generate a label for each Token;
[0051] Pre-training the large language model with a fiscal database to enable it to better identify and handle fiscal-related issues. The fiscal database can be constructed by extracting large-scale fiscal data from government websites, official websites of fiscal-related international organizations, and academic databases. During the pre-training process, a small number of labeled fiscal data samples are used to perform additional training on the large language model. The small number of labeled fiscal data samples are manually labeled, specifically data labeled by experts in the fiscal field, and contain specific categories or labels of various issues in the fiscal field. Since the labeling process involves expert knowledge, these data samples are of high quality and can accurately reflect the classification of fiscal issues, thereby improving the classification ability of the model when data samples are limited, avoiding overfitting, and improving the generalization ability of the model.
[0052] Preferably, the question type labels in this embodiment include budget preparation, execution supervision, final accounting analysis, fund management, policy formulation and others, among which:
[0053] The budget preparation mentioned above involves the formulation, modification, approval and budget allocation of the fiscal budget; the execution supervision mentioned above covers the supervision, auditing, compliance inspection and monitoring of the use of funds during the budget execution process; the final account analysis focuses on the review, summary, analysis and report preparation of the fiscal final account; the fund management involves the use, allocation, settlement, source and expenditure of funds; the policy formulation involves the formulation, adjustment and evaluation of fiscal policies and their impact on the economy; and others are other issues that are not clearly classified but involve the fiscal field;
[0054] Preferably, the keyword entities in this embodiment include but are not limited to budget items, execution departments, final accounting data, policy names and funding types;
[0055] S22, hybrid retrieval knowledge base query module;
[0056] The hybrid retrieval knowledge base query module is based on a hybrid retrieval strategy combining vector retrieval and keyword retrieval, and uses the preliminarily processed input data, question type labels and keywords as inputs of the hybrid retrieval knowledge base query module. Specifically:
[0057] S221, vector search;
[0058] The vector retrieval includes text embedding and approximate nearest neighbor search. The text embedding uses an Embedding model to capture the semantic information of the input data after preliminary processing to obtain a high-dimensional vector. The Embedding model is a BERT model or a Word2Vec model. The HNSW or IVF algorithm is used to perform an approximate nearest neighbor search on the high-dimensional vector to obtain preliminary related records. The preliminary related records are weighted according to the question type labels corresponding to the preliminary processed input data, and preliminary related records that are not related to the question type labels are eliminated to obtain a vector retrieval related record data set.
[0059] S222, keyword search;
[0060] The keyword search uses inverted index technology and synonym expansion to perform exact matching and fuzzy matching to find records containing keywords in a preset knowledge base. The preset knowledge base can be constructed by obtaining data from the above-mentioned financial field database, specifically:
[0061] Create an index containing keywords, identify synonyms of keywords and expand the keywords to be queried accordingly, use the expanded keywords to search in the inverted index, and obtain a keyword search data set;
[0062] Preliminarily merging the vector retrieval related record data set and the keyword retrieval data set to obtain a merged data set, and sorting the merged data set using a Rerank model. The Rerank model performs comprehensive scoring and sorting based on vector similarity and keyword matching degree to obtain a query result. The query result is specifically a detailed description of the fiscal indicator, including definition, calculation method, scope of application and policy basis;
[0063] S23, intention recognition module;
[0064] The intent recognition module uses open source large language models (such as qwen, chatglm, BERT, etc.) to perform natural language processing on the input data after preliminary processing, combines the query results with additional contextual intent analysis and demand mapping, and obtains the JSON structure of the user demand financial indicator data, where:
[0065] S231, Natural Language Processing;
[0066] Use named entity recognition (NER) technology to identify important entities in the input data after preliminary processing. The specific tools and models used include SpaCy, BERT, RoBERTa, etc. The important entities include financial indicators, time, region, etc.;
[0067] Use predefined rules and patterns to identify the relationships between important entities, perform relationship extraction, and obtain data after natural language processing;
[0068] S232, intention analysis;
[0069] Conduct in-depth analysis of data after natural language processing, specifically, understand the query results as additional contextual information based on the current input, identify key elements from the data after natural language processing through the reasoning ability of the open source large language model; and infer the data type and request method that the user wants, and combine the key elements, the data type of user-demand financial indicator data, and the request method of user-demand financial indicator data to obtain the intention analysis results of user-demand financial indicator data;
[0070] S233, requirements mapping;
[0071] Map the intention analysis results to specific fields, including financial indicators, analysis dimensions, time range, and geographical range, and generate a user demand financial indicator data structure in JSON format based on the mapping results;
[0072] S24, JSON calibration module;
[0073] The JSON calibration module is used to normalize the JSON structure of the user's required financial indicator data. The normalization process includes data verification and structure conversion, wherein:
[0074] The JSON calibration module combines the rule engine and machine learning to perform data verification on the JSON structure of the received user demand financial indicator data. The rule engine is used to define rules to check the existence of fields and the validity of values. Machine learning is used to train the model to identify abnormal data or data that does not conform to the specifications. The specific verification rules include:
[0075] Make sure that fields such as financial indicators, analysis dimensions, time range, and geographical range exist;
[0076] Determine that the values of financial indicators and analytical dimensions comply with predefined specifications (e.g., check whether the indicator is in the list of allowed indicators);
[0077] Make sure the time range is in the correct format and that the start time is less than the end time;
[0078] Determine that the geographical scope complies with the corresponding geographical specifications;
[0079] Preferably, detailed error information of the JSON structure of the user demand financial indicator data that failed verification is returned, such as missing fields, format errors, etc., so that the user can modify it in time;
[0080] The JSON calibration module also includes an integrated domain-specific language (DSL) parser for converting the successfully verified standard JSON structure into a budget account system and financial report format adapted to governments at different levels. Specifically, the specific formats and account systems of financial reports of governments at different levels are collected in advance as DSL specifications, and corresponding conversion rules are defined. According to the target DSL specifications, the predefined conversion rules are applied to convert the standard JSON structure into the DSL format;
[0081] S3, data processing layer;
[0082] The data processing layer obtains the standardized financial data request structure through the API and outputs a standardized financial indicator data set, including indicator values, related metadata and quality tags, specifically:
[0083] The data processing layer acquires and processes the financial indicator data required by users by developing standardized API interfaces, and integrates data sources for data conversion; the data processing layer stores data through Redis and presets a cache mechanism to improve response speed, stores metadata by designing a metadata model, and the metadata is the definition of indicators in the financial indicator data required by users, data sources, units and other information, and performs metadata management to enhance data interpretability; the data processing layer also saves data snapshots at different time points through version control, and performs data quality assessment to automatically mark problematic data;
[0084] After the above processing, the data processing layer outputs a standardized financial indicator data set, including indicator values, relevant metadata and quality tags;
[0085] S4, result display layer;
[0086] The result display layer is used to display the financial indicator dataset in the form of text description, tables and statistical charts. Specifically:
[0087] Formatting the fiscal indicator dataset, including cleaning, aggregation and standardization;
[0088] The text description form display utilizes natural language processing technology, and the result display layer automatically generates concise and clear text descriptions based on the formatted fiscal indicator data set, outlining the changing trends and important features of key fiscal indicators such as revenue, expenditure, deficit, etc.;
[0089] The table display organizes the formatted fiscal indicator data set into a structured table, clearly displays various fiscal indicators and their related metadata, and facilitates users to quickly obtain specific values and make intuitive comparisons;
[0090] The statistical chart format includes chart type recommendation and interactive chart generation. The chart type recommendation is achieved by combining the large language model (LLM) and the chart analysis knowledge base. The result display layer intelligently recommends the most suitable chart type (such as bar chart, line chart, pie chart, etc.) according to data characteristics and user query intentions; the interactive chart generation uses visualization libraries such as ECharts, and the result display layer generates rich interactive charts, supporting users to perform operations such as zooming, dragging, and data filtering to explore and analyze data more deeply; preferably, the result display layer automatically adjusts the chart's color, font, label and other parameters to improve readability and aesthetics.
[0091] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
[0092] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, platforms, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0093] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.
Claims
1. An intelligent data analysis platform in the financial field based on a large language model, characterized in that: The platform includes a user input processing layer, a core analysis engine, a data processing layer, and a result display layer, wherein: The user input processing layer is used to perform preliminary processing on the user's natural language input data to obtain the input data after preliminary processing; The core analysis engine includes a question classification module, a hybrid retrieval knowledge base query module, an intention recognition module and a JSON calibration module. The question classification module uses a question classification model to classify questions and extract keywords from the input data after preliminary processing to obtain question type labels and keywords. The hybrid retrieval knowledge base query module uses a hybrid retrieval strategy based on vector retrieval and keyword retrieval to obtain query results using the input data after preliminary processing, question type labels and keywords. The query results are specifically detailed descriptions of fiscal indicators, including definitions, calculation methods, applicable scopes and policy basis. The intention recognition module obtains the JSON structure of user demand fiscal indicator data by performing natural language processing on the input data after preliminary processing, combining the query results with intention analysis and demand mapping as additional context, including fiscal indicators, analysis dimensions, time range and geographical range. The JSON calibration module is used to normalize the JSON structure of user demand fiscal indicator data to obtain a normalized fiscal data request structure. The data processing layer obtains the standardized financial data request structure through the API and outputs a standardized financial indicator data set, including indicator values, related metadata and quality tags; The result display layer is used to display the financial indicator data set in the form of text description, tables and statistical charts.
2. According to claim 1, a financial field intelligent data analysis platform based on a large language model is characterized in that: The preliminary processing includes segmenting the user's natural language input data into words or phrases, removing preset common words that are useless for analysis, and marking the part of speech of each word remaining after the removal.
3. According to claim 2, a financial field intelligent data analysis platform based on a large language model is characterized in that: The question classification model is a large language model based on the Transformer architecture, including a shared encoder, a classification head, and an extraction head. The preliminarily processed input data is used as the input of the shared encoder. The shared encoder captures the context information in the input data through the self-attention mechanism and outputs the corresponding feature representation. The feature representation is used as the input of the classification head and the extraction head, and the question type label and keywords are output. The large language model is pre-trained using a database in the financial field, and during the pre-training process, additional training is performed using financial sample data pre-labeled by experts to obtain a trained question classification model.
4. According to claim 3, a financial field intelligent data analysis platform based on a large language model is characterized in that: The vector retrieval in the hybrid retrieval strategy includes text embedding and approximate nearest neighbor retrieval, specifically: The text embedding uses an Embedding model to capture semantic information of the input data after preliminary processing to obtain a high-dimensional vector, and the Embedding model is a BERT model or a Word2Vec model; an HNSW or IVF algorithm is used to perform an approximate nearest neighbor search on the high-dimensional vector to obtain preliminary relevant records, and the preliminary relevant records are weighted according to the question type labels corresponding to the input data after preliminary processing, and preliminary relevant records that are not related to the question type labels are eliminated to obtain a vector retrieval related record data set.
5. According to claim 4, a financial field intelligent data analysis platform based on a large language model is characterized in that: The keyword search in the hybrid search strategy uses inverted index technology and synonym expansion for exact matching and fuzzy matching. Specifically, it builds an index containing keywords, identifies synonyms of keywords and expands them, uses the expanded keywords to search in the inverted index, and obtains a keyword search data set.
6. According to claim 5, a financial field intelligent data analysis platform based on a large language model is characterized in that: The hybrid search strategy also includes preliminarily merging the vector search related record data set and the keyword search data set to obtain a merged data set, and sorting the merged data set using a Rerank model. The Rerank model comprehensively scores and sorts based on vector similarity and keyword matching degree to obtain query results.
7. The financial intelligent data analysis platform based on a large language model according to claim 6 is characterized in that: The specific steps to obtain the JSON structure of the user's required financial indicator data are as follows: Use named entity recognition technology to identify important entities in the input data after preliminary processing, use predefined rules and patterns to identify the relationship between important entities, and obtain data after natural language processing; Conduct in-depth analysis of the data after natural language processing, specifically, based on the data after natural language processing, use the query results as additional contextual information for understanding, identify key elements from the data after natural language processing through the reasoning ability of the open source large language model, and infer the data type and request method desired by the user, and combine the key elements, the data type of user-demand financial indicator data, and the request method of user-demand financial indicator data to obtain the intention analysis results of user-demand financial indicator data; The intent analysis results are mapped to specific fields, including financial indicators, analysis dimensions, time range and geographical range, and a user demand financial indicator data structure in JSON format is generated based on the mapping results.
8. The financial intelligent data analysis platform based on a large language model according to claim 7 is characterized in that: The JSON structure of the user's required financial indicator data is standardized to obtain the standardized financial data request structure as follows: The rule engine and machine learning are combined to perform data validation on the JSON structure of the received user demand financial indicator data. The rule engine is used to define validation rules to check the existence of fields and the validity of values. Machine learning is used to train the model to identify abnormal data or data that does not conform to the specifications, and obtain the JSON structure of the successfully validated user demand financial indicator data. Collect the specific formats and subject systems of government financial statements at different levels in advance as DSL specifications, and define corresponding conversion rules. According to the DSL specifications of user-required financial indicator data, apply predefined conversion rules to convert the standard JSON structure into DSL format.
9. The financial intelligent data analysis platform based on a large language model according to claim 8 is characterized in that: Validation rules include determining whether the financial indicator, analysis dimension, time range, and geographic range fields exist; determining whether the values of the financial indicator and analysis dimension meet the predefined specifications; determining whether the time range format is correct and the start time is less than the end time; and determining whether the geographic range meets the corresponding geographic specifications.
10. The financial intelligent data analysis platform based on a large language model according to claim 9, characterized in that: The data processing layer obtains the standardized financial data request structure through the API and outputs a standardized financial indicator data set, including indicator values, related metadata, and quality tags. Specifically: The data processing layer acquires and processes the financial indicator data required by users by developing standardized API interfaces, and integrates data sources for data conversion; The data processing layer stores data through Redis and presets a cache mechanism; The data processing layer is designed with a metadata model for storing metadata, the metadata including the definition, data source, and unit of the indicators in the user demand financial indicator data, and performs metadata management; The data processing layer also saves data snapshots at different time points through version control, and performs data quality assessment to automatically mark problematic data.
Citation Information
Patent Citations
Index data analysis method and system based on knowledge graph and natural language
CN116244410A
Government affair service field multi-strategy fusion dialogue method based on knowledge graph
CN116628172A
Data visualization analysis method based on natural language large model
CN117235248A
Intelligent data analysis processing method, device, equipment and medium
CN117785920A
Government affair intelligent response device and method based on intention recognition and large language model
CN118035419A
Cited By
Financial statement automatic analysis system based on large language model
CN120124593A
Method for obtaining visual chart through question and answer mode access based on FineBI and Text2DSL
CN120407669A
Method for service recommendation based on identified user intention, computing device and storage medium
CN120632219A
Method, computing device, and storage medium for business recommendation based on identified user intent
CN120632219B
Financial intelligent question number precision method based on dynamic calculation optimization
CN121071105A
