Enterprise financial knowledge question answering method and system based on large model
Through a large-scale enterprise financial knowledge Q&A system, combined with user identity verification, natural language processing and knowledge base search, the data accuracy and security issues in the enterprise financial knowledge Q&A system are solved, and fast, accurate and secure finance is achieved. Information retrieval meets the professional knowledge needs within the enterprise.
Patent Information
- Application Number
- CN202411118491.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The traditional enterprise financial knowledge Q&A system is difficult to meet the internal professional knowledge needs of the enterprise, and the large model training data comes from public channels, lacking corporate professional and private knowledge, resulting in inaccurate output and insufficient correlation.
Using a large-scale enterprise financial knowledge Q&A method and system, through user authentication, natural language processing, sensitive word screening, legality assessment, context analysis and knowledge base search, the most relevant answers are generated and content security checks are ensured. The accuracy and security of the answers support dynamic updates and user feedback optimization.
It improves the accuracy and relevance of corporate financial knowledge questions and answers, ensures information security and timeliness, meets the professional knowledge needs within the enterprise, and provides more in-depth and accurate financial information retrieval.
Smart Images

Figure CN119046428A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big models, and in particular to a method and system for answering questions about corporate financial knowledge based on big models. Background Art
[0002] Today, corporate financial knowledge question and answer systems can help corporate employees quickly obtain the required financial information in their daily work and improve work efficiency and quality.
[0003] At present, due to the huge and diverse amount of corporate financial knowledge information, traditional retrieval methods can no longer meet the needs of employees. Recently, corporate financial question-answering systems based on large models have become a research hotspot. The training data of general pre-trained large models mainly comes from public channels, lacking corporate professional and private knowledge. Direct use will be difficult to support professional knowledge question-answering within the enterprise. Knowledge expansion can be achieved through retraining, but there are problems such as large amount of data required, high training cost and long cycle.
[0004] The big model combined with the knowledge base can combine the big model capabilities with corporate financial knowledge, which can solve the above problems to a certain extent, improve the accuracy and relevance of the output, and avoid the big model from producing "hallucinations" and generating factually incorrect content; therefore, a corporate financial knowledge question-and-answer method and system based on a big model is needed to solve the above problems. Summary of the invention
[0005] The purpose of the present invention is to provide a method and system for answering questions about corporate financial knowledge based on a large model to solve the problems raised in the above-mentioned background technology.
[0006] To achieve the above object, the present invention provides the following technical solution: a method for answering corporate financial knowledge questions based on a large model, the method comprising the following steps:
[0007] Receive query questions submitted by users and verify the user's identity information to confirm whether the user has the authority to access sensitive financial information; if the verification fails, the process is automatically terminated and the preset prompt message is returned;
[0008] Use natural language processing technology to analyze user questions, match them with the financial knowledge base, and determine whether the question belongs to the financial field; if not, refuse to answer and return the preset refusal information;
[0009] Screen the content of questions that have been judged by type to ensure the compliance of the content of the questions; if sensitive words are found, refuse to answer and return the corresponding prompt information;
[0010] Call a third-party security interface to conduct a legitimacy assessment on questions that have passed the sensitive word screening; if the assessment fails, terminate the process and return the preset rejection information;
[0011] Identify the correlation between questions and integrate the questions based on historical conversation records; retrieve relevant information from the financial knowledge base based on the integrated question content; receive user questions and retrieved knowledge base information, generate multiple possible answers, and select the most relevant answer through a sorting mechanism and return it to the user.
[0012] Preferably, it also includes a security check on the answer content, performing a new round of content security check on the answers generated by the large model to ensure that the answers do not contain sensitive or harmful information; if the check fails, the user is replied according to a preset template.
[0013] Preferably, encryption technology is used to process user information to ensure the security of the verification process; at the same time, multiple identity authentication methods are supported, including but not limited to username and password, biometrics, and third-party authorized login.
[0014] Preferably, the knowledge base includes, but is not limited to, definitions of financial terms, financial statement analysis methods, and interpretations of tax policies, and supports dynamic updates to reflect the latest financial regulations and market changes.
[0015] Preferably, a user feedback mechanism is also included, allowing users to evaluate the answers generated by the large model, and using the evaluation information to optimize the training data of the large model to continuously improve the accuracy of the question-answering system and user experience.
[0016] A corporate financial knowledge question-answering system based on a large model, the system comprising:
[0017] The user identity verification module is used to receive the query questions submitted by the user and verify the user's identity information to confirm whether the user has the authority to access sensitive financial information; if the verification fails, the process is automatically terminated and a preset prompt message is returned;
[0018] The question type judgment module uses natural language processing technology to analyze user questions, match them with the financial knowledge base, and determine whether the question belongs to the financial field. If not, it refuses to answer and returns the preset refusal information;
[0019] The sensitive word screening module contains a preset list of sensitive words, and screens the content of questions that pass type judgment to ensure the compliance of the question content; if sensitive words are found, the answer will be rejected and the corresponding prompt information will be returned;
[0020] The legality verification module conducts a legality assessment on questions that have passed the sensitive word screening by calling a third-party security interface; if the assessment fails, the process is terminated and the preset rejection information is returned;
[0021] Intelligent question-answering processing module, including context analysis submodule, knowledge base retrieval submodule and large model processing submodule:
[0022] The context analysis submodule is used to identify the relevance between questions and integrate the questions with historical conversation records;
[0023] The knowledge base retrieval submodule retrieves relevant information from the financial knowledge base based on the integrated question content;
[0024] The large model processing submodule receives user questions and retrieved knowledge base information, generates multiple possible answers, and selects the most relevant answer through a sorting mechanism to return to the user.
[0025] Preferably, it also includes an answer content security check module, which is used to perform a new round of content security check on the answers generated by the large model to ensure that the answers do not contain sensitive or harmful information; if the check fails, the user is replied according to a preset template.
[0026] Preferably, the user identity authentication module uses encryption technology to process user information to ensure the security of the authentication process; at the same time, it supports multiple identity authentication methods, including but not limited to username and password, biometrics, and third-party authorized login.
[0027] Preferably, the knowledge base search submodule includes but is not limited to financial term definitions, financial statement analysis methods, tax policy interpretations, etc., and supports dynamic updates to reflect the latest financial regulations and market changes.
[0028] Preferably, a user feedback mechanism is also included, allowing users to evaluate the answers generated by the large model, and using the evaluation information to optimize the training data of the large model to continuously improve the accuracy of the question-answering system and user experience.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] The enterprise financial knowledge question-and-answer method and system based on a big model proposed in the present invention can quickly retrieve the information most relevant to the user's query by using a vector knowledge base, thereby improving the efficiency and speed of information retrieval; the big model can understand and process complex financial knowledge, and can provide more accurate and in-depth answers in combination with the content of the local knowledge base; the system can regularly update the knowledge base, combine user feedback information, and add new financial knowledge and data, thereby maintaining the timeliness and knowledge coverage of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of the method of the present invention;
[0032] Figure 2 This is a flow chart of the content safety detection of the present invention;
[0033] Figure 3 The present invention is a knowledge base search flow chart;
[0034] Figure 4 It is a flow chart for iterative updating of the model of the present invention. DETAILED DESCRIPTION
[0035] In order to make the purpose and technical solution of the present invention clearly and completely described, and the advantages more clearly understood, the embodiments of the present invention are further described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, rather than all of the embodiments, and are only used to explain the embodiments of the present invention, and are not used to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] For example, see Figures 1 to 4 The present invention provides a technical solution: a method for answering corporate financial knowledge questions based on a large model, the method comprising the following steps:
[0037] 1. The user submits a query.
[0038] 2. The system verifies the user's identity information and determines whether the user has the authority to conduct financial knowledge Q&A, ensuring that only authorized users can access sensitive financial information.
[0039] 1) If user information verification fails, the system will terminate the process and automatically reply with corresponding prompt information according to the preset template.
[0040] 2) User information verification is successful and the process continues to the next step.
[0041] 3. The system uses natural language processing (NLP) technology to determine the type of question based on the content of the question to determine whether it belongs to the financial field. This technology can analyze the keywords and phrases in the question and match them with the knowledge base in the financial field.
[0042] 1) If it is a financial issue, the process will continue to the next step;
[0043] 2) If the question is not financially related, the system will refuse to answer and terminate the process, and return a preset refusal message, such as prompting the user to ask a financially related question.
[0044] 4. The system pre-defined sensitive word list, which contains all words that may violate laws, regulations or social moral standards. The system screens the content of the question for sensitive words to ensure the compliance of the content of the question.
[0045] 1) If you pass the screening, the process continues to the next step;
[0046] 2) If the answer fails the screening, the system will refuse to answer and terminate the process, while returning the corresponding preset information, such as prompting the user to modify the question to meet the specifications.
[0047] 5. In order to prevent malicious users from submitting illegal or harmful questions, the system calls a third-party security interface to perform a legitimacy check on the assessment questions.
[0048] 1) If the verification passes, the process continues to the next step;
[0049] 2) If the verification fails, the process ends and a preset rejection message is returned, such as prompting the user to comply with laws, regulations and social moral standards.
[0050] 6. The system analyzes the context, identifies the relevance between questions, and constructs a complete conversation scenario. If the question is not independent, the system will combine historical conversation records to reintegrate the question.
[0051] 7. The knowledge base will retrieve financial knowledge information including but not limited to the definition of financial terms, analysis methods of financial statements, and interpretation of tax policies based on the content of the questions.
[0052] 8. The system submits user questions and retrieved knowledge base information to the big model.
[0053] 9. In order to prevent the model from generating answers that may contain sensitive or harmful information, the system conducts a new round of content security checks on the answers of the large model, and transfers them to different processes based on the check results. For a detailed description of the process, see 4.4 Large Model Content Security Detection Process.
[0054] 1) If you pass the security check, continue the process.
[0055] 2) If the security check fails, the system will respond based on the preset template information.
[0056] 10. The large model generates multiple possible answers based on the content of the user's question. In order to provide the most accurate answer, the system sorts the generated answers and returns the most relevant answer to the user as the final result.
[0057] 11. Users will evaluate the answers of large models to help the system continuously optimize and improve service quality and provide a better user experience.
[0058] Example 2, based on Example 1, proposes a knowledge base search process
[0059] 1. In order for the system to understand the meaning of the user's question more accurately, the problem is first processed and standardized to resolve uncertainty.
[0060] 2. In order to better understand the structure of the question, the system segments the standardized questions.
[0061] 3. Since there are often some errors in questions, in order to identify and correct these errors and understand the user's questions more accurately, the system performs text error correction on the questions after word segmentation to improve the accuracy of the text.
[0062] 4. The system generates a search query based on the user's question, searches the knowledge base through the Elasticsearch search engine, and obtains a series of search results. Based on the search results, the system performs the following judgments:
[0063] 1) If the search results meet the requirements, that is, exceed the threshold, the system directly outputs these search results as the final search results.
[0064] 2) If the search result does not meet the requirements, that is, it is lower than the threshold, the system will proceed to the next step.
[0065] 5. The system uses the bge search method to search the knowledge base. Based on the search results, the system performs the following judgments
[0066] 1) If the search results meet the requirements, that is, exceed the threshold, the system directly outputs the search results as the final search results
[0067] 2) If the search results do not meet the requirements, that is, they are below the threshold, the system will proceed to the next step.
[0068] 6. If the thresholds of both search methods are above the threshold, the system compares the results of the two search methods and returns the search result with the highest score. In this way, the system can make full use of the advantages of the two search methods and improve the accuracy and efficiency of the search.
[0069] Embodiment 3
[0070] Based on the second embodiment, the model iteration update is proposed
[0071] In order to maintain the accuracy and relevance of the model, the model is regularly trained in combination with user feedback and new financial data to better meet user needs. The following is the main process:
[0072] 1. Regularly collect user feedback information and generate input-output pairs for model training.
[0073] 2. Collect corporate financial data, including but not limited to financial statements, annual reports, quarterly reports, etc.
[0074] 3. To facilitate model training, clean, deduplicate and preprocess the collected data. Construct input-output pairs for model training.
[0075] 4. Divide the data output from the first two steps to construct training set, validation set and test set. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the performance of the model.
[0076] 5. According to the specific data and tuning goals, set the training speed and performance training strategy that affects the model, including but not limited to parameters such as learning rate and batch size
[0077] 6. After setting the training set to iteratively update the model parameters, in order to make the model better adapt to the new data, the model fine-tuning training operation is performed through continuous iteration and updating.
[0078] 7. Use the test set to evaluate the functionality of the fine-tuned model to ensure that the fine-tuned model achieves the expected results on the new task.
[0079] 1) If the model function meets the expected requirements, the fine-tuning is successful and the process continues to the next step.
[0080] 2) If the model function fails to meet the expected requirements, go to step 2 and perform fine-tuning again until the expected tuning target is met.
[0081] 8. When the model function meets the expected requirements, deploy the tuned large model to the production environment
[0082] In summary, by combining user feedback information and new financial data and regularly training the model, we can continuously improve the performance of the model and better meet user needs.
[0083] Embodiment 4
[0084] Based on Example 3, a large model content security detection process is proposed
[0085] It is necessary and important to conduct security testing on the content generated by large models. The following is an introduction to the process
[0086] 1. Receive the content generated by the large model and perform preprocessing.
[0087] 2. The system pre-defined sensitive word list, which contains all words that may violate laws, regulations or social moral standards. The system performs a preliminary detection of sensitive words in the text content.
[0088] 1) If the test passes, the process continues to call the local interface;
[0089] 2) If the test fails, the system returns the corresponding preset information, such as prompting that the corresponding answer was not found.
[0090] 3. In order to solve the problem of illegal or harmful content generated by large models, the system calls a third-party security interface. The third-party interface is powerful and has comprehensive coverage to evaluate the legality of the text content.
[0091] 1) If the verification is passed, the system returns the large model generated content to the user;
[0092] 2) If the verification fails, the process ends and a preset rejection message is returned, such as a prompt that no corresponding answer was found.
[0093] Embodiment 5
[0094] Based on the fourth embodiment, a corporate financial knowledge question-answering system based on a large model is proposed, the system comprising:
[0095] The user identity verification module is used to receive the query questions submitted by the user and verify the user's identity information to confirm whether the user has the authority to access sensitive financial information; if the verification fails, the process is automatically terminated and a preset prompt message is returned;
[0096] The question type judgment module uses natural language processing technology to analyze user questions, match them with the financial knowledge base, and determine whether the question belongs to the financial field. If not, it refuses to answer and returns the preset refusal information;
[0097] The sensitive word screening module contains a preset list of sensitive words, and screens the content of questions that pass type judgment to ensure the compliance of the question content; if sensitive words are found, the answer will be rejected and the corresponding prompt information will be returned;
[0098] The legality verification module conducts a legality assessment on questions that have passed the sensitive word screening by calling a third-party security interface; if the assessment fails, the process is terminated and the preset rejection information is returned;
[0099] Intelligent question-answering processing module, including context analysis submodule, knowledge base retrieval submodule and large model processing submodule:
[0100] The context analysis submodule is used to identify the relevance between questions and integrate the questions with historical conversation records;
[0101] The knowledge base retrieval submodule retrieves relevant information from the financial knowledge base based on the integrated question content;
[0102] The large model processing submodule receives user questions and retrieved knowledge base information, generates multiple possible answers, and selects the most relevant answer through a sorting mechanism and returns it to the user.
[0103] It also includes an answer content security check module, which is used to perform a new round of content security checks on the answers generated by the large model to ensure that the answers do not contain sensitive or harmful information. If the check fails, the user is replied according to the preset template.
[0104] The user identity authentication module uses encryption technology to process user information to ensure the security of the authentication process; at the same time, it supports multiple identity authentication methods, including but not limited to username and password, biometrics, and third-party authorized login;
[0105] The knowledge base search submodule includes but is not limited to financial term definitions, financial statement analysis methods, tax policy interpretations, etc., and supports dynamic updates to reflect the latest financial regulations and market changes;
[0106] It also includes a user feedback mechanism that allows users to evaluate the answers generated by the big model, and use the evaluation information to optimize the training data of the big model to continuously improve the accuracy of the question-answering system and user experience.
[0107] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A corporate financial knowledge question-answering method based on a large model, characterized by: The method comprises the following steps: Receive query questions submitted by users and verify the user's identity information to confirm whether the user has the authority to access sensitive financial information; if the verification fails, the process is automatically terminated and the preset prompt message is returned; Use natural language processing technology to analyze user questions, match them with the financial knowledge base, and determine whether the question belongs to the financial field; if not, refuse to answer and return the preset refusal information; Screen the content of questions that have been judged by type to ensure the compliance of the content of the questions; if sensitive words are found, refuse to answer and return the corresponding prompt information; Call a third-party security interface to conduct a legitimacy assessment on questions that have passed the sensitive word screening; if the assessment fails, terminate the process and return the preset rejection information; Identify the correlation between questions and integrate the questions based on historical conversation records; retrieve relevant information from the financial knowledge base based on the integrated question content; receive user questions and retrieved knowledge base information, generate multiple possible answers, and select the most relevant answer through a sorting mechanism and return it to the user.
2. The enterprise financial knowledge question-answering method based on a large model according to claim 1, characterized in that: It also includes answer content security checks, which conduct a new round of content security checks on the answers generated by the large model to ensure that the answers do not contain sensitive or harmful information; If the check fails, the user will be replied according to the preset template.
3. The enterprise financial knowledge question-answering method based on a large model according to claim 1 is characterized by: Encryption technology is used to process user information to ensure the security of the verification process; at the same time, multiple identity authentication methods are supported, including but not limited to username and password, biometrics, and third-party authorized login.
4. The enterprise financial knowledge question-answering method based on a large model according to claim 1 is characterized by: The knowledge base includes but is not limited to definitions of financial terms, financial statement analysis methods, and interpretations of tax policies, and supports dynamic updates to reflect the latest financial regulations and market changes.
5. The enterprise financial knowledge question-answering method based on a large model according to claim 1 is characterized by: It also includes a user feedback mechanism that allows users to evaluate the answers generated by the big model, and use the evaluation information to optimize the training data of the big model to continuously improve the accuracy of the question-answering system and user experience.
6. A big model-based enterprise financial knowledge question answering system according to any one of claims 1 to 5, characterized in that: The system comprises: The user identity verification module is used to receive the query questions submitted by the user and verify the user's identity information to confirm whether the user has the authority to access sensitive financial information; if the verification fails, the process is automatically terminated and a preset prompt message is returned; The question type judgment module uses natural language processing technology to analyze user questions, match them with the financial knowledge base, and determine whether the question belongs to the financial field. If not, it refuses to answer and returns the preset refusal information; The sensitive word screening module contains a preset list of sensitive words, and screens the content of questions that pass type judgment to ensure the compliance of the question content; if sensitive words are found, the answer will be rejected and the corresponding prompt information will be returned; The legality verification module conducts a legality assessment on questions that have passed the sensitive word screening by calling a third-party security interface; if the assessment fails, the process is terminated and the preset rejection information is returned; Intelligent question-answering processing module, including context analysis submodule, knowledge base retrieval submodule and large model processing submodule: The context analysis submodule is used to identify the relevance between questions and integrate the questions with historical conversation records; The knowledge base retrieval submodule retrieves relevant information from the financial knowledge base based on the integrated question content; The large model processing submodule receives user questions and retrieved knowledge base information, generates multiple possible answers, and selects the most relevant answer through a sorting mechanism to return to the user.
7. The enterprise financial knowledge question-answering system based on a large model according to claim 6 is characterized by: It also includes an answer content security check module, which is used to perform a new round of content security checks on the answers generated by the large model to ensure that the answers do not contain sensitive or harmful information; If the check fails, the user will be replied according to the preset template.
8. The enterprise financial knowledge question-answering system based on a large model according to claim 6 is characterized by: The user identity authentication module uses encryption technology to process user information to ensure the security of the authentication process; at the same time, it supports multiple identity authentication methods, including but not limited to username and password, biometrics, and third-party authorized login.
9. The enterprise financial knowledge question-answering system based on a large model according to claim 6 is characterized by: The knowledge base retrieval sub-module includes but is not limited to financial term definitions, financial statement analysis methods, tax policy interpretations, etc., and supports dynamic updates to reflect the latest financial regulations and market changes.
10. The enterprise financial knowledge question-answering system based on a large model according to claim 6, characterized in that: It also includes a user feedback mechanism that allows users to evaluate the answers generated by the big model, and use the evaluation information to optimize the training data of the big model to continuously improve the accuracy of the question-answering system and user experience.
Citation Information
Patent Citations
Automatic financial report question and answer method and device based on large model
CN117235233A
Medical question answering system based on large language model
CN117851558A
Vector database retrieval method and device based on large model, terminal and medium
CN118312594A
Cited By
Method for guaranteeing security of large model generation content through alignment mechanism
CN120220696A
