Tool calling method and device, equipment, storage medium and program product

Abstract

The invention discloses a tool calling method and device, equipment, a storage medium and a program product, and relates to the technical field of data retrieval, the method comprises the steps that financial problem information input by a user is received, the financial problem information is used for calling a preset tool list, and the tool list comprises multiple function tools; performing scene identification and data preprocessing on the financial problem information to obtain multi-source effective data; performing cue word construction according to the multi-source effective data and the financial problem information; and based on the constructed big language model cue word, calling the tool list through the big language model, and outputting a corresponding function tool calling result. Scene recognition and data preprocessing are carried out on the financial problem information, and the term understanding capability of the large language model can be enhanced through the constructed large language model cue word, so that the intelligent agent can flexibly cope with various complex financial service scenes while maintaining the universality, and the financial service efficiency is improved. Therefore, the use accuracy of the large language model tool in the financial field is improved.

Description

Technical Field

[0001] The present application relates to the field of data retrieval technology, and in particular to a tool calling method, device, equipment, storage medium and program product. Background Art

[0002] With the development of Large Language Models (LLMs), research has found that integrating external tools into these models can greatly expand the functional boundaries of artificial intelligence agents, thereby opening up new ways to solve more complex practical problems. The tool integration capabilities of LLMs allow them to access a wealth of resources, such as retrieving news, performing complex arithmetic operations, and even using third-party services, so they can be widely and sophisticatedly applied in many fields. Existing LLM tool call services are often universal, and this universality enables them to have good initial performance in various fields. With the help of this tool call function, AI agents that demonstrate expert-level capabilities in professional fields such as workflow automation and intelligent report generation can be shaped.

[0003] Although LLM's general tool calling function demonstrates strong versatility, in specific professional fields, such as the complex environment of financial intelligent question and answer, the vastness of financial vocabulary, the correspondence between colloquial expressions and formal terms in questions, the unique format of financial terms, and the diversity of knowledge structures in various financial scenarios lead to the knowledge and information in the financial field being highly professional and non-universal, which makes it difficult for LLM to understand, resulting in the accuracy of its tool use and the pass rate of parameter recognition being often limited. Summary of the invention

[0004] The main purpose of this application is to provide a tool calling method, device, equipment, storage medium and program product, aiming to solve the technical problem that the intricate knowledge and information in the financial field are highly professional and non-universal, LLM is difficult to understand, resulting in the accuracy of its tool use is often limited.

[0005] To achieve the above purpose, the present application proposes a tool calling method, the method comprising:

[0006] Receiving financial problem information input by a user, wherein the financial problem information is used to call a preset tool list, wherein the tool list includes a plurality of function tools;

[0007] Performing scenario recognition and data preprocessing on the financial problem information to obtain multi-source valid data;

[0008] Constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model;

[0009] Based on the large language model prompt word, the tool list is called through a preset large language model to output a function tool call result corresponding to the financial problem information.

[0010] In one embodiment, the multi-source valid data includes scenario information and financial slot data; the step of performing scenario recognition and data preprocessing on the financial problem information to obtain the multi-source valid data includes:

[0011] Call and verify the financial problem information according to a preset checking strategy;

[0012] When the verification is passed, the financial question information is subjected to scene recognition through a preset intention recognition model to obtain corresponding scene information;

[0013] The scenario information and the financial problem information are extracted using a preset financial entity extraction model to obtain financial slot data.

[0014] In one embodiment, the step of constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model includes:

[0015] Pulling financial knowledge from a preset knowledge base according to the scenario information to obtain financial rules corresponding to the financial knowledge;

[0016] Performing data analysis on the financial slot data according to the financial rules to obtain background knowledge;

[0017] constructing a sample dialogue based on the financial problem information;

[0018] The financial rules, the background knowledge and the example dialogue are filled into a preset prompt word framework to obtain a large language model prompt word.

[0019] In one embodiment, before the step of calling and verifying the financial problem information according to a preset check strategy, the training process of the intent recognition model includes:

[0020] Collect corpus information of different financial scenarios;

[0021] Performing data cleaning on the corpus information to obtain cleaned corpus information;

[0022] Annotating the financial subfield to which the cleaned corpus information belongs to, to obtain annotated data;

[0023] Based on the labeled data, the pre-trained model is fine-tuned through a supervised fine-tuning training strategy to obtain an intent recognition model.

[0024] In one embodiment, the step of calling the tool list based on the large language model prompt word through a preset large language model and outputting the function tool calling result corresponding to the financial problem information includes:

[0025] The financial problem information is called and planned through a preset large language model to determine the calling scenario;

[0026] Based on the large language model prompt word and the calling scenario, performing tool judgment on the tool list to determine the corresponding function category;

[0027] Parameters of the tool list are identified according to the function category to generate a function tool calling result.

[0028] In one embodiment, the step of performing parameter identification on the tool list according to the function category and generating a function tool call result includes:

[0029] The name of the function that extracts the tool list;

[0030] According to the function category and the function name, determining a corresponding function tool from the tool list;

[0031] Performing parameter identification on the function tool to obtain parameter information of the function tool;

[0032] Performing format verification on the parameter information to obtain a verification result;

[0033] When the verification result is in a non-standard format, the parameter information is converted into a json format to generate a function tool call result.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a tool calling device, the device comprising:

[0035] A question acquisition module, used to receive financial question information input by a user, wherein the financial question information is used to call a preset tool list, wherein the tool list includes a plurality of function tools;

[0036] A preprocessing module, used to perform scene recognition and data preprocessing on the financial problem information to obtain multi-source valid data;

[0037] A prompt word module, used for constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model;

[0038] The tool calling module is used to call the tool list based on the large language model prompt word through a preset large language model, and output the function tool calling result corresponding to the financial problem information.

[0039] In addition, to achieve the above objectives, the present application also proposes a tool calling device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the tool calling method described above.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the tool calling method described above are implemented.

[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the tool calling method described above are implemented.

[0042] One or more technical solutions proposed in this application have at least the following technical effects: this application first receives financial problem information input by a user, and the financial problem information is used to call a preset tool list, and the tool list includes a variety of function tools; then the financial problem information is subjected to scene recognition and data preprocessing to obtain multi-source valid data; then prompt words are constructed based on the multi-source valid data and the financial problem information to obtain large language model prompt words; finally, based on the large language model prompt words, the tool list is called through a preset large language model to output the function tool call result corresponding to the financial problem information. Since this application performs scene recognition and data preprocessing on financial problem information, the constructed large language model prompt words can enhance the terminology comprehension ability of the large language model, so that the intelligent agent can flexibly respond to various complex financial business scenarios while maintaining versatility, thereby improving the accuracy of the use of large language model tools in the financial field. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] Figure 1 The overall flow chart of the tool call implementation process provided by this application;

[0046] Figure 2A flowchart of the first embodiment of the tool calling method of the present application is provided;

[0047] Figure 3 A flowchart of the second embodiment of the tool calling method of this application is provided;

[0048] Figure 4 A flowchart of the third embodiment of the tool calling method of this application is provided;

[0049] Figure 5 This is a schematic diagram of the module structure of the tool calling device of the embodiment of the present application;

[0050] Figure 6 A schematic diagram of the device structure of the hardware operating environment involved in the tool calling method in the embodiment of the present application.

[0051] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] It should be noted that the general tool calling function of LLM is difficult to play its real tool calling function in the financial field, which is manifested in the following three aspects:

[0055] 1) Financial terminology is rich and unique, which leads to limitations in LLM’s understanding of terminology. The breadth, freedom and professional depth of financial vocabulary often make it difficult for LLM to distinguish between common vocabulary and professional terminology, which in turn leads to omissions in parameter identification. At the same time, LLM is also powerless to understand the correspondence between colloquial expressions and formal terms in the questions, resulting in inaccurate understanding. For example, "the market" usually refers to the Shanghai Composite Index.

[0056] 2) Complex data elements that appear frequently lead to the complexity of data processing. Frequently appearing data elements and market sector codes usually follow a unique format, which may be composed of letters, numbers, and even punctuation marks according to their specific rules. This is beyond the understanding ability of LLM and makes it difficult to distinguish different products. In addition, the immediacy of financial Q&A places high demands on the precise analysis of time data, which is also the reason why LLM implementation tool calls increase the complexity of understanding and analysis.

[0057] 3) Differences in scenario adaptability. The knowledge structure, data characteristics, and applicable tools of various financial scenarios are diverse. Each scenario has its own unique knowledge and information, requiring highly customized solutions. General LLM tool calls often fail to meet such depth and breadth of differentiated needs.

[0058] Some commonly used optimization technologies in the LLM field, such as Retrieval Enhanced Generation (RAG), can improve the professional capabilities of LLM to a certain extent. However, the ability to use the tool is restricted by the RAG retrieval model capabilities and the completeness of the knowledge base, and at the same time, it places higher requirements on the knowledge understanding ability of LLM itself. Therefore, it is often accompanied by instability and extremely long recognition delays.

[0059] In addition, professional LLM models modified through fine-tuning technology (such as SFT and Lora) can achieve the effect of understanding financial knowledge, but fine-tuning requires a large amount of formatted and labeled data for various scenarios, which is very labor-intensive and reduces the expansion capability, destroying the versatility of tool calls.

[0060] Therefore, in order to solve the above problems, this application constructs a tool calling method that is both universally applicable in the financial field and can adapt to the unique needs of each sub-scenario. The main solution process is as follows Figure 1 As shown, Figure 1 The overall flow chart of the tool calling implementation process provided for this application mainly includes the following modules: pre-processing module, prompt word module and tool calling module. Figure 3 The process of each module can enhance the ability to understand terminology, optimize data processing strategies, improve scenario adaptability, and introduce standardized tool description specifications, which significantly improves the performance and application scope of intelligent tool calls in the financial field and provides a more efficient and accurate solution for the complex environment of financial intelligent question answering. The specific solution is described as follows.

[0061] It should be noted that the execution subject of this embodiment can be a computing service device with scene recognition, prompt word construction and tool calling functions, such as a personal computer, server, etc., or an electronic device capable of realizing the above functions, a tool calling device that executes the tool calling method of this application, etc., and this embodiment does not limit this. The following takes the tool calling device as an example to illustrate this embodiment and the following embodiments.

[0062] Based on this, the embodiment of the present application provides a tool calling method, referring to Figure 2 , Figure 2 A flowchart diagram of the first embodiment of the tool calling method of this application is provided.

[0063] In this embodiment, the tool calling method includes steps S10 to S40:

[0064] Step S10: receiving financial problem information input by a user, wherein the financial problem information is used to call a preset tool list, and the tool list includes a plurality of function tools.

[0065] It should be noted that financial problem information refers to questions related to the financial field raised by users.

[0066] For example, the financial issue information may include the dynamics of the financial market, the operating conditions of financial institutions, the characteristics of various financial instruments and products, etc.

[0067] It should be noted that the tool list is a list of a series of tools that can be called in the LLM tool call service. These tools are usually functions or programs with specific functions (i.e., function tools), such as data query tools, data analysis tools, text processing tools, image recognition tools, etc.

[0068] like Figure 1 As shown, at the beginning of the process, the tool calling device receives the questions input by the user (i.e., financial question information). When the LLM processes the relevant financial question information raised by the user, it needs to call various tools in the tool list to obtain additional information in order to provide more accurate and comprehensive answers.

[0069] Step S20: Perform scenario recognition and data preprocessing on the financial problem information to obtain multi-source valid data.

[0070] It should be noted that multi-source valid data is the valid data that is integrated after pre-processing the financial problem information, including the scenario to which the analyzed problem belongs, financial data, etc.

[0071] Specifically, during the data preprocessing process, the financial scenario to which the financial problem information belongs (such as banking services, securities trading, insurance business, etc.) can be identified, and key elements in the financial problem information (such as transaction subjects, amounts, types, etc.) can also be identified, so as to obtain multi-source valid data. This embodiment does not limit this.

[0072] In this embodiment, by using multi-source valid data, the problem of blurred boundaries between common vocabulary and professional terms in LLM can be solved, and its ability to handle the correspondence between colloquial expressions and formal terms can be enhanced, thereby improving the accuracy and efficiency of tool calls.

[0073] In a feasible implementation, the multi-source valid data includes scenario information and financial slot data; step S20 of this embodiment may include the steps of: calling and verifying the financial problem information according to a preset inspection strategy; when the verification passes, performing scenario recognition on the financial problem information through a preset intention recognition model to obtain corresponding scenario information; and extracting data from the scenario information and the financial problem information through a preset financial entity extraction model to obtain financial slot data.

[0074] Specifically, the preprocessing process is as follows: Figure 1 As shown, it includes three processes: call verification, scene recognition, and financial slot extraction.

[0075] It should be noted that the preset checking strategy is a strategy pre-set in the tool calling device for checking whether the financial problem information is legal.

[0076] Exemplarily, the preset inspection strategy may include necessary inspection processes such as user authority verification and tool format parsing verification, which is not limited in this embodiment.

[0077] For illegal financial problem information, the tool calling device refuses to process and returns the inspection result. For financial problem information that passes the verification, the tool calling device performs the subsequent scene recognition and financial slot extraction process.

[0078] It should be noted that the intent recognition model is a model used to analyze and understand the financial question information input by the user, identify the scenario to which the user's question belongs, and determine the user's purpose. The intent recognition model can be obtained by training with a large amount of text data.

[0079] Through the intent recognition model, the scenario information of financial issue information (such as banking services, securities trading, insurance business, etc.) can be directly identified.

[0080] It should be noted that the financial entity extraction model is a model used to identify and extract specific entity information from financial-related data. Financial slot data is the information of specific entities in financial-related data, such as financial product names, currency types, transaction times, etc.

[0081] Through the financial entity extraction model, it is possible to identify financial slot data of different granularities in financial problem information according to the specifications in different scenarios (for example, in the deposit scenario, issues related to time deposits can be accurate to the monthly time slot, and issues related to notice deposits can be accurate to the daily time slot).

[0082] In this implementation, in view of the high frequency complexity of financial data, dynamic acquisition of relevant scenario information and financial slot data is achieved by calling data preprocessing strategies such as verification, scenario recognition, and financial slot extraction. It can effectively distinguish the codes and formats of different financial products, solving the limitations of LLM in processing complex data; at the same time, it improves the ability to accurately analyze time data, meeting the high requirements of financial Q&A immediacy.

[0083] Step S30: construct prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model.

[0084] It should be noted that the large language model prompt words are the guide words provided to the model when interacting with the large language model.

[0085] The process of constructing the prompt word can be obtained through the above-mentioned scene information, financial slot data and financial problem information; it can also be achieved through a specific prompt word template, which is not limited in this embodiment.

[0086] In this implementation, the large language model prompt words can help the large language model better understand the user's needs and intentions, guide the model's output direction, and thus generate answers or output content that are more in line with expectations.

[0087] In a feasible implementation, step S30 of this embodiment may include the steps of: pulling financial knowledge from a preset knowledge base according to the scenario information to obtain financial rules corresponding to the financial knowledge; performing data analysis on the financial slot data according to the financial rules to obtain background knowledge; constructing an example dialogue according to the financial problem information; filling the financial rules, the background knowledge and the example dialogue into a preset prompt word framework to obtain a large language model prompt word.

[0088] Specifically, the prompt word construction process is as follows: Figure 1 As shown, after receiving the financial problem information and the above-mentioned multi-source valid data, the processes of financial knowledge extraction, example construction, slot data analysis, etc. are first executed in parallel, and then the results are filled into the prompt word framework.

[0089] It should be noted that the financial rules corresponding to financial knowledge are the guidelines formulated in the financial field to regulate and guide financial behavior and maintain financial market order.

[0090] The purpose of financial knowledge extraction is to help LLM extract corresponding financial knowledge from the knowledge base based on the scenario information to which the financial problem information belongs, so that relevant background and rules can be quickly introduced, allowing the model to understand professional data and terminology in the scenario.

[0091] Similarly, based on the identified financial slot data, various data can be converted into background knowledge that can be understood by LLM.

[0092] It should be noted that sample dialogues can be constructed by retrieving similar user questions in historical Q&A. Sample construction allows LLM to quickly learn relevant knowledge, answering processes, special rules, etc. of this scenario, so that the large model can demonstrate highly professional capabilities in various scenarios.

[0093] It should be noted that the prompt word framework is a structure for constructing prompt words. This embodiment can achieve highly customized solutions in different financial scenarios by establishing a universal prompt word framework in the financial field. The framework not only retains the basic process of tool calling, but also has the ability to expand in complex financial scenarios, thereby meeting the unique needs of various financial sub-scenarios. This design enables LLM to flexibly respond to diverse financial scenarios while maintaining universality.

[0094] In this embodiment, the final LLM prompts include example information that helps LLM learn quickly and financial knowledge information that improves LLM professionalism, as well as key data, terms and descriptions for financial problem information. Therefore, this embodiment achieves highly customized scenario adaptability and achieves the accuracy of tool calls and parameter recognition in various scenarios.

[0095] Step S40: Based on the large language model prompt word, the tool list is called through a preset large language model to output a function tool call result corresponding to the financial problem information.

[0096] It should be noted that the large language model is a language-related model trained based on large-scale data and powerful computing power. Through the large language model, a variety of language tasks can be processed, such as answering financial question information, text generation, knowledge question answering, reasoning calculation, reading comprehension, etc. in this embodiment.

[0097] The function tool calling result is the feedback result of the function tool called according to the financial problem information LLM.

[0098] In this embodiment, if Figure 1 As shown in the figure, after the result of calling the function tool is generated through the large language model, the parameters can be post-processed and fed back to the user, and the question-answering process ends. In this way, professional financial knowledge can be cleverly integrated into the understanding mechanism of the large language model (LLM), enhancing the ability and flexibility of the language model so that it can cope with more complex scenarios and needs.

[0099] In the technical solution provided in this embodiment, the tool calling device first receives the question (i.e., financial question information) input by the user. When the LLM processes the relevant financial question information raised by the user, it needs to call various tools in the tool list to obtain additional information to provide a more accurate and comprehensive answer. At this time, the financial question information is subjected to scene recognition and data preprocessing, including call verification, scene recognition, and financial slot extraction processes to obtain multi-source valid data. Then, prompt words are constructed according to the multi-source valid data and the financial question information to obtain large language model prompt words. The large language model prompt words finally formed include example information that helps LLM to learn quickly and financial knowledge information that improves the professionalism of LLM. Finally, based on the large language model prompt words, the tool list is called through the preset large language model to output the function tool call result corresponding to the financial question information. Since this embodiment performs scene recognition and data preprocessing on the financial question information, the constructed large language model prompt words can enhance the terminology understanding ability of the large language model, so that the intelligent agent can flexibly respond to various complex financial business scenarios while maintaining versatility, thereby improving the accuracy of the use of large language model tools in the financial field.

[0100] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 and Figure 3 , Figure 3 A flowchart diagram of the second embodiment of the tool calling method of this application is provided.

[0101] In this example, step S40 includes steps S41 to S43:

[0102] Step S41: Plan the invocation of the financial problem information through a preset large language model to determine the invocation scenario.

[0103] It should be noted that the calling scenario is the scenario in which the financial problem information uses the function tool, such as tool calling, repeated calling, serial calling, repeated and serial calling, and other calling scenarios.

[0104] Step S42: Based on the large language model prompt word and the calling scenario, perform tool judgment on the tool list to determine the corresponding function category.

[0105] Step S43: performing parameter identification on the tool list according to the function category, and generating a function tool calling result.

[0106] It should be noted that the function category is a category obtained by classifying functions according to the characteristics, functions, properties, etc. of function tools.

[0107] In this implementation, based on the financial problem information and the constructed large language model prompt words, LLM is used to complete the steps of call planning, tool judgment, and parameter identification, ultimately realizing the calling capability of function tools in complex financial scenarios.

[0108] In a feasible implementation, step S40 of this example includes the steps of: extracting the function name from the tool list; determining the corresponding function tool from the tool list according to the function category and the function name; performing parameter identification on the function tool to obtain parameter information of the function tool; performing format verification on the parameter information to obtain a verification result; when the verification result is in a non-standard format, converting the parameter information into a json format to generate a function tool call result.

[0109] It should be noted that the function name is the identification name of the function tool. The function name is used to distinguish different function tools.

[0110] It should be noted that parameter information refers to the data passed when the function tool is defined and called. JSON format is a lightweight data exchange format that is concise, easy to read, and easy to parse.

[0111] In this implementation, it can be determined whether the output of LLM can be standardized into a json format. If so, the parameter information is converted into a json format, a function tool call result is generated, and returned to the user.

[0112] In the technical solution provided in this embodiment, by introducing standardized specifications, the performance and application scope of tool calls in the financial field can be significantly improved.

[0113] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 2 and Figure 4 , Figure 4 A flowchart diagram of the third embodiment of the tool calling method of this application is provided.

[0114] Before the step of calling and verifying the financial problem information according to the preset inspection strategy described in this example, the training process of the intent recognition model includes: collecting corpus information of different financial scenarios; performing data cleaning on the corpus information to obtain cleaned corpus information; marking the financial sub-field to which the cleaned corpus information belongs to obtain labeled data; based on the labeled data, fine-tuning the pre-trained model through a supervised fine-tuning training strategy to obtain an intent recognition model.

[0115] It should be noted that corpus information is a large amount of language text data about the financial scenario field.

[0116] Among them, the data cleaning process may include discovering and correcting errors, missing values, duplicate values, inconsistencies, etc. in the data to improve data quality.

[0117] It should be noted that the financial sub-sector refers to the sub-sectors of the financial sector, such as the securities market, banking, insurance, and fund industries.

[0118] It should be noted that the supervised fine-tuning training strategy (Supervised Fine-Tuning, SFT) is a strategy for fine-tuning models through supervised fine-tuning machine learning.

[0119] In the case of supervised fine-tuning, a large amount of labeled data can be used to initially train the model. Then, further fine-tuning training is performed on this basis.

[0120] In the technical solution provided in this embodiment, according to different financial scenarios, corpus from various channels can be collected and cleaned, annotated, and annotated with reasons given to train the intent recognition capability of the large language model in financial scenarios, and fine-tuned through a supervised fine-tuning training strategy to obtain an intent recognition model. Similarly, data extraction model training is similar to intent recognition model training. This allows the large language model to flexibly respond to various complex financial business scenarios through intent recognition models and data extraction models while maintaining versatility.

[0121] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the tool calling method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0122] This application also provides a tool calling device, please refer to Figure 5 , Figure 5 This is a schematic diagram of the module structure of the tool calling device of the embodiment of the present application; the tool calling device includes:

[0123] The question acquisition module 501 is used to receive financial question information input by a user, wherein the financial question information is used to call a preset tool list, and the tool list includes a plurality of function tools;

[0124] A preprocessing module 502 is used to perform scene recognition and data preprocessing on the financial problem information to obtain multi-source valid data;

[0125] A prompt word module 503 is used to construct prompt words according to the multi-source valid data and the financial problem information to obtain a large language model prompt word;

[0126] The tool calling module 504 is used to call the tool list based on the large language model prompt word through a preset large language model, and output the function tool calling result corresponding to the financial problem information.

[0127] In this embodiment, the multi-source valid data includes scenario information and financial slot data; the preprocessing module 502 is also used to call and verify the financial problem information according to a preset inspection strategy; when the verification passes, the financial problem information is subjected to scenario recognition through a preset intention recognition model to obtain corresponding scenario information; the scenario information and the financial problem information are subjected to data extraction through a preset financial entity extraction model to obtain financial slot data.

[0128] Furthermore, the prompt word module 503 is also used to pull financial knowledge from a preset knowledge base according to the scenario information to obtain financial rules corresponding to the financial knowledge; perform data analysis on the financial slot data according to the financial rules to obtain background knowledge; construct an example dialogue according to the financial problem information; and fill the financial rules, the background knowledge and the example dialogue into a preset prompt word framework to obtain a large language model prompt word.

[0129] Furthermore, the tool calling device is also used to collect corpus information of different financial scenarios; perform data cleaning on the corpus information to obtain cleaned corpus information; mark the financial sub-field to which the cleaned corpus information belongs to obtain labeled data; based on the labeled data, fine-tune the pre-trained model through a supervised fine-tuning training strategy to obtain an intent recognition model.

[0130] Furthermore, the tool calling module 504 is also used to plan the calling of the financial problem information through a preset large language model to determine the calling scenario; based on the large language model prompt words and the calling scenario, perform tool judgment on the tool list to determine the corresponding function category; perform parameter identification on the tool list according to the function category to generate a function tool calling result.

[0131] Furthermore, the tool calling module 504 is also used to extract the function name of the tool list; determine the corresponding function tool from the tool list according to the function category and the function name; perform parameter identification on the function tool to obtain parameter information of the function tool; perform format verification on the parameter information to obtain a verification result; when the verification result is in a non-standard format, convert the parameter information into a json format to generate a function tool calling result.

[0132] Other embodiments or specific implementation methods of the tool calling device of the present application can refer to the above-mentioned method embodiments and will not be repeated here.

[0133] The tool calling device provided by the present application adopts the tool calling method in the above embodiment, which can solve the technical problem that the complicated knowledge and information in the financial field are highly professional and non-universal, and LLM is difficult to understand, resulting in the accuracy of its tool use being often limited. Compared with the prior art, the beneficial effects of the tool calling device provided by the present application are the same as the beneficial effects of the tool calling method provided by the above embodiment, and the other technical features in the tool calling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0134] The present application provides a tool calling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the tool calling method in the above-mentioned embodiment one.

[0135] Reference below Figure 6 , Figure 6 The schematic diagram of the device structure of the hardware operating environment involved in the tool calling method in the embodiment of the present application shows a schematic diagram of the structure of the tool calling device suitable for implementing the embodiment of the present application. The tool calling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The tool calling device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0136] like Figure 6As shown, the tool calling device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the tool calling device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the tool calling device to communicate with other devices wirelessly or wired to exchange data. Although the tool calling device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0137] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0138] The tool calling device provided by the present application adopts the tool calling method in the above embodiment, which can solve the technical problem that the complicated knowledge and information in the financial field are highly professional and non-universal, and LLM is difficult to understand, resulting in the accuracy of its tool use being often limited. Compared with the prior art, the beneficial effects of the tool calling device provided by the present application are the same as the beneficial effects of the tool calling method provided by the above embodiment, and the other technical features in the tool calling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0139] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0140] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0141] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the tool calling method in the above-mentioned embodiment.

[0142] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0143] The computer-readable storage medium may be included in the tool calling device; or may exist independently without being assembled into the tool calling device.

[0144] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the tool calling device, the tool calling device: receives financial problem information input by a user, the financial problem information is used to call a preset tool list, and the tool list includes a plurality of function tools; performs scene recognition and data preprocessing on the financial problem information to obtain multi-source valid data; constructs prompt words according to the multi-source valid data and the financial problem information to obtain large language model prompt words; based on the large language model prompt words, calls the tool list through a preset large language model, and outputs the function tool calling result corresponding to the financial problem information.

[0145] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0146] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0148] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned tool calling method, and can solve the technical problem that the intricate knowledge and information in the financial field are highly professional and non-universal, and LLM is difficult to understand, resulting in the accuracy of its tool use being often limited. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the tool calling method provided by the above-mentioned embodiment, and will not be repeated here.

[0149] The present application also provides a computer program product, including a computer program, which implements the steps of the tool calling method as described above when executed by a processor.

[0150] The computer program product provided by this application can solve the technical problem that the complicated knowledge and information in the financial field are highly professional and non-universal, LLM is difficult to understand, and the accuracy of its tool use is often limited. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the tool calling method provided in the above embodiment, which will not be repeated here.

[0151] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A tool calling method, characterized in that: The method includes: Receiving financial problem information input by a user, wherein the financial problem information is used to call a preset tool list, wherein the tool list includes a plurality of function tools; Performing scenario recognition and data preprocessing on the financial problem information to obtain multi-source valid data; Constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model; Based on the large language model prompt word, the tool list is called through a preset large language model to output a function tool call result corresponding to the financial problem information.

2. The method according to claim 1, characterized in that The multi-source valid data includes scenario information and financial slot data; the step of performing scenario recognition and data preprocessing on the financial problem information to obtain the multi-source valid data includes: Call and verify the financial problem information according to a preset checking strategy; When the verification is passed, the financial question information is subjected to scene recognition through a preset intention recognition model to obtain corresponding scene information; The scenario information and the financial problem information are extracted using a preset financial entity extraction model to obtain financial slot data.

3. The method according to claim 2, characterized in that The step of constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model includes: Pulling financial knowledge from a preset knowledge base according to the scenario information to obtain financial rules corresponding to the financial knowledge; Performing data analysis on the financial slot data according to the financial rules to obtain background knowledge; constructing a sample dialogue based on the financial problem information; The financial rules, the background knowledge and the example dialogue are filled into a preset prompt word framework to obtain a large language model prompt word.

4. The method according to claim 2, characterized in that Before the step of calling and verifying the financial problem information according to the preset checking strategy, the training process of the intention recognition model includes: Collect corpus information of different financial scenarios; Performing data cleaning on the corpus information to obtain cleaned corpus information; Annotating the financial subfield to which the cleaned corpus information belongs to, to obtain annotated data; Based on the labeled data, the pre-trained model is fine-tuned through a supervised fine-tuning training strategy to obtain an intent recognition model.

5. The method according to any one of claims 1 to 4, characterized in that The step of calling the tool list based on the large language model prompt word through a preset large language model and outputting the function tool calling result corresponding to the financial problem information includes: The financial problem information is called and planned through a preset large language model to determine the calling scenario; Based on the large language model prompt word and the calling scenario, performing tool judgment on the tool list to determine the corresponding function category; Parameters of the tool list are identified according to the function category to generate a function tool calling result.

6. The method according to claim 5, characterized in that The step of performing parameter identification on the tool list according to the function category and generating a function tool calling result comprises: The name of the function that extracts the tool list; According to the function category and the function name, determining a corresponding function tool from the tool list; Performing parameter identification on the function tool to obtain parameter information of the function tool; Performing format verification on the parameter information to obtain a verification result; When the verification result is in a non-standard format, the parameter information is converted into a json format to generate a function tool call result.

7. A tool calling device, characterized in that: The device comprises: A question acquisition module, used to receive financial question information input by a user, wherein the financial question information is used to call a preset tool list, wherein the tool list includes a plurality of function tools; A preprocessing module, used to perform scene recognition and data preprocessing on the financial problem information to obtain multi-source valid data; A prompt word module, used for constructing prompt words according to the multi-source valid data and the financial problem information to obtain prompt words of a large language model; The tool calling module is used to call the tool list based on the large language model prompt word through a preset large language model, and output the function tool calling result corresponding to the financial problem information.

8. A tool calling device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the tool calling method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the tool calling method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the tool calling method according to any one of claims 1 to 6 are implemented.

Images