Large model optimization method and device based on financial service scene, equipment and medium

Abstract

The embodiment of the invention provides a large model optimization method and device based on a financial service scene, equipment and a medium, and the method comprises the steps: obtaining a first input parameter which comprises a real input request of a user; training the first input parameter based on the small parameter model and the large parameter model, and correspondingly generating a first output result and a second output result; if there is a difference between the first output result and the second output result, screening out a corresponding second input parameter; obtaining a manual annotation result of the second input parameter, and adjusting the small parameter model and the large parameter model according to the manual annotation result; the large-parameter model is adopted to replace manpower for data annotation, the accuracy of the small-parameter model is improved, manual annotation is needed only when the output results of the large-parameter model and the small-parameter model are different, and manpower is effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of model processing technology, and specifically, to a large model optimization method, device, equipment and medium based on financial business scenarios. Background Art

[0002] Since the human-computer interaction in the financial business scenario has a relatively high requirement for the interaction delay, the intelligent customer service system using a large model for interaction will have certain requirements for the number of parameters in the large model in order to meet the requirement of short response time. The number of parameters of the large model currently used by the intelligent customer service system is 7B, which has met the requirement of end-to-end response time of less than 300ms. Since the response time of the large model is proportional to the number of parameters, the larger the number of parameters, the longer the response time. However, the accuracy of the large model is also proportional to the number of parameters. Therefore, if you want to improve the accuracy, only by increasing the number of parameters, it will conflict with the requirement of response time. The current method to improve the accuracy of model training mainly relies on subsequent manual data labeling, which will consume a lot of manpower. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a large model optimization method, device, equipment and medium based on financial business scenarios, so as to solve the problem that the current method for improving the accuracy of model training mainly relies on subsequent manual labeling of data, which consumes a lot of manpower.

[0004] In a first aspect, an embodiment of the present application provides a large model optimization method based on a financial business scenario, including:

[0005] Obtaining a first input parameter, wherein the first input parameter includes a user's actual input request;

[0006] The first input parameter is trained based on the small parameter model and the large parameter model respectively, and a first output result and a second output result are generated correspondingly;

[0007] If there is a difference between the first output result and the second output result, select the corresponding second input parameter;

[0008] Obtain a manual annotation result of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation result.

[0009] In the above implementation process, a first input parameter is obtained, wherein the first input parameter includes a real input request of the user; the first input parameter is trained based on a small parameter model and a large parameter model respectively, and a first output result and a second output result are generated correspondingly; if there is a difference between the first output result and the second output result, the corresponding second input parameter is screened out; the manual annotation result of the second input parameter is obtained, and the small parameter model and the large parameter model are adjusted according to the manual annotation result; the large parameter model is used to replace manpower for data annotation to improve the accuracy of the small parameter model, and manual annotation is required only when the output results of the large parameter model and the small parameter model are different, thereby effectively reducing manpower.

[0010] Furthermore, the training of the first input parameter based on the small parameter model and the large parameter model respectively, and correspondingly generating the first output result and the second output result, includes:

[0011] Inputting a first input parameter into a small parameter model to obtain a first output result;

[0012] Inputting the first input parameter into the large parameter model to obtain a second output result;

[0013] A data list is generated according to the first input parameter, the first output result and the second output result.

[0014] In the above implementation process, the first output result and the second output result of the large model parameters and the small model parameters are generated respectively, so as to compare the first output result and the second output result to know whether the annotation result of the model is correct.

[0015] Furthermore, the obtaining of the manual annotation result of the second input parameter and adjusting the small parameter model and the large parameter model according to the manual annotation result include:

[0016] Compare the first output result and the second output result with the manual annotation results respectively;

[0017] If the first output result of the small parameter model is wrong and the second output result of the large parameter model is correct, the small parameter model is trained and adjusted;

[0018] If the first output result of the small parameter model is wrong and the second output result of the large parameter model is wrong, the small parameter model and the large parameter model are trained and adjusted respectively;

[0019] If the first output result of the small parameter model is correct and the second output result of the large parameter model is wrong, the large parameter model is trained and adjusted.

[0020] In the above implementation process, it is determined whether the output results of the large parameter model and the small parameter model are correct. If the output result of the model is wrong, the model is optimized and trained.

[0021] Furthermore, the large parameter model and the small parameter model have the same model structure, and both models include multiple intelligent agents;

[0022] If the output of the model is wrong, it also includes:

[0023] Intelligent agents that filter models that have errors in their output results.

[0024] In the above implementation process, if the output result of the model is wrong, the intelligent agent of the corresponding model is screened out to better optimize and adjust the model.

[0025] Furthermore, the intelligent agent corresponding to the model with errors in the screening output results includes:

[0026] Compare the output of the user scenario intelligent agent, the user question intelligent agent, and the sales skill intelligent agent to see if there are any differences with the manual annotation results;

[0027] If at least one of the output language of the user scenario intelligent agent, the output language of the user question intelligent agent, and the output language of the sales skill intelligent agent is different from the manual labeling result, the intelligent agent that is different from the manual labeling result is obtained and recorded;

[0028] If the output of the user scenario intelligent agent, the user question intelligent agent, and the sales skill intelligent agent are all the same as the manual labeling results, then check whether the output of the speech selection intelligent agent is different from the manual labeling results;

[0029] If the output language of the speech selection intelligent agent is different from the manual labeling result, record the speech selection intelligent agent.

[0030] In the above implementation process, if the output result of the model is wrong, the intelligent agent of the corresponding model is screened out to better optimize and adjust the model.

[0031] Furthermore, the model is trained and adjusted, including:

[0032] According to the intelligent agent with errors in the output results, filter out the intelligent agent data related to the intelligent agent in the second input parameter;

[0033] Training and adjusting the corresponding intelligent agent of the model based on the intelligent agent data;

[0034] If there are no errors in the output results of each intelligent agent, all intelligent agents of the model are trained and adjusted according to the second input parameter.

[0035] In the above implementation process, based on the intelligent agent of the model with incorrect output results obtained by screening, the data corresponding to the intelligent agent is called to train and adjust the model to enhance the accuracy and efficiency of the training.

[0036] Furthermore, the obtaining of the manual annotation result of the second input parameter includes:

[0037] According to the second input parameter, each intelligent agent of the model is manually labeled to obtain the manual labeling result.

[0038] In the above implementation process, when the output results of the large parameter model and the small parameter model are inconsistent, each intelligent agent of the model is manually labeled to obtain a more accurate labeling result.

[0039] In a second aspect, an embodiment of the present application provides a large model optimization device based on a financial business scenario, including:

[0040] A parameter acquisition module, used to acquire a first input parameter, wherein the first input parameter includes a user's actual input request;

[0041] A model training module, used to train the first input parameter based on the small parameter model and the large parameter model respectively, and generate a first output result and a second output result accordingly;

[0042] A difference screening module, used for screening out the corresponding second input parameter if there is a difference between the first output result and the second output result;

[0043] The model adjustment module is used to obtain the manual annotation result of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation result.

[0044] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0045] A processor, a memory and a bus, wherein the processor is connected to the memory via the bus, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the large model optimization method based on the financial business scenario as described above.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a server, the large model optimization method based on the financial business scenario as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A flowchart of a large model optimization method based on a financial business scenario provided in an embodiment of the present application;

[0049] Figure 2 It is a structural schematic diagram of a large model optimization device based on a financial business scenario provided in an embodiment of the present application;

[0050] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0052] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0053] Please see Figure 1 , Figure 1 A flowchart of a large model optimization method based on a financial business scenario provided in an embodiment of the present application. The large model optimization method based on a financial business scenario includes:

[0054] 100. Obtain a first input parameter, where the first input parameter includes a user's actual input request.

[0055] It is understandable that in financial business scenarios, obtaining the user's real input request can come from a variety of channels, including but not limited to web forms, mobile applications, telephone customer service, intelligent customer service systems, etc.

[0056] 200. Train the first input parameter based on the small parameter model and the large parameter model respectively, and generate a first output result and a second output result accordingly.

[0057] Specifically, the first input parameter is input into the small parameter model to obtain the first output result; the first input parameter is input into the large parameter model to obtain the second output result; and a data list is generated according to the first input parameter, the first output result and the second output result.

[0058] Thus, the first output result and the second output result of the large model parameters and the small model parameters are generated respectively, so as to compare the first output result and the second output result to know whether the annotation result of the model is correct.

[0059] 300. If there is a difference between the first output result and the second output result, select a corresponding second input parameter.

[0060] Specifically, the first output result and the second output result are combined into a data list, and whether there is a difference between the first output result and the second output result can be directly known through the data list. The input parameters corresponding to the output results with differences are screened to form second input parameters, and the second input parameters are manually labeled to determine whether the output error of the large parameter model or the output error of the small parameter model is. Among them, the input and output of each intelligent agent use the model to determine whether to perform manual labeling.

[0061] It can be understood that if the first output result of the large parameter model is consistent with the second output result of the small parameter model, it is judged that the output results of the two models are correct, and the training results of the two models are relatively accurate and do not need to be adjusted.

[0062] 400. Obtain manual annotation results of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation results.

[0063] Specifically, the first output result and the second output result are compared with the manual annotation results respectively; in the first case: if the first output result of the small parameter model is wrong and the second output result of the large parameter model is correct, the small parameter model is trained and adjusted; in the second case: if the first output result of the small parameter model is wrong and the second output result of the large parameter model is wrong, the small parameter model and the large parameter model are trained and adjusted respectively; in the third case: if the first output result of the small parameter model is correct and the second output result of the large parameter model is wrong, the large parameter model is trained and adjusted.

[0064] Therefore, it is judged whether the output results of the large parameter model and the small parameter model are correct. If the output result of the model is wrong, the model is optimized and trained.

[0065] Exemplarily, a small amount of data is first used to simultaneously train a model with a large parameter amount and a model with a small parameter amount, such as a model with a parameter amount of 70B and a model with a parameter amount of 7B, to obtain a large parameter model and a small parameter model; then the 7B model is applied online to obtain more real data, and then the 70B model is used to retrain these real data. If the output result of the 70B model is consistent with the output result of the original 7B model, no manual labeling is required. If the output result of the 70B model is inconsistent with the output result of the original 7B model, manual labeling is performed for review, which can greatly reduce the amount of labeling. If these inconsistent real data are found to be abnormal in the original 7B model after manual labeling, these real data are recorded as bad cases (data points where the model predicts errors), and the 7B model is retrained with these bad cases, so that the accuracy of the 7B model is improved.

[0066] As described above, the embodiment of the present application obtains a first input parameter, wherein the first input parameter includes a real input request of the user; the first input parameter is trained based on a small parameter model and a large parameter model respectively, and a first output result and a second output result are generated correspondingly; if there is a difference between the first output result and the second output result, the corresponding second input parameter is screened out; the manual annotation result of the second input parameter is obtained, and the small parameter model and the large parameter model are adjusted according to the manual annotation result; the large parameter model is used instead of manpower for data annotation to improve the accuracy of the small parameter model, and manual annotation is required only when the output results of the large parameter model and the small parameter model are different, thereby effectively reducing manpower.

[0067] Specifically, based on the above embodiments, the large model optimization method based on financial business scenarios provided in the embodiments of the present application can also be concretized as follows: the model structure of the large parameter model and the small parameter model is the same, and both models include multiple intelligent agents.

[0068] Exemplarily, both the large parameter model and the small parameter model adopt the Llama 3.1 model, and both the large parameter model and the small parameter model include multiple intelligent agents, specifically including a first node agent: a user scenario agent, a user problem agent, and a sales skill agent; a second node agent: a speech selection agent, and the speech for replying to the user is obtained by combining the first node agent and the second node agent.

[0069] For example, the same model has multiple agents, and each agent has different outputs because of different prompt words. The user question agent outputs a normalized user question phrase based on the user request, and the sales skills agent outputs the sales skills needed for this round; the user context agent identifies the context of the current context user, which is generally information that business personnel need to care about. For example, in a financial credit scenario, whether the user expresses that he or she has failed the review is a user context. By querying the knowledge base based on the output of the user question agent and the output of the user context agent, you can query the knowledge to answer the corresponding user questions; by querying the knowledge base based on the output of the sales skills agent and the output of the user context agent, you can query the knowledge of the corresponding sales skills. The last speech selection agent combines the above two types of knowledge to generate a speech to reply to the user.

[0070] If the output result of the model is wrong, it also includes:

[0071] 410. Filter the intelligent agent corresponding to the model with errors in the output results.

[0072] It can be understood that when the output results of the large parameter model and the small parameter model are erroneous, it is necessary to screen the corresponding intelligent agents; that is, if the output result of the large parameter model is erroneous, the intelligent agent corresponding to the large parameter model with erroneous output results is screened; if the output result of the small parameter model is erroneous, the intelligent agent corresponding to the small parameter model with erroneous output results is screened.

[0073] Specifically, based on the above embodiments, the large model optimization method based on the financial business scenario provided in the embodiments of the present application can also be concretized as follows: the intelligent agent corresponding to the model with errors in the screening output results includes:

[0074] 411. Compare the outputs of the user scenario intelligent agent, the user question intelligent agent, and the sales skill intelligent agent to see if there are any differences with the manual labeling results.

[0075] It should be noted that we first determine whether there are any problems with the first node agent: user scenario agent, user question agent and sales skill agent. That is, after the model accepts the user's real input request, the output language of the user scenario agent, user question agent and sales skill agent is compared with the manual annotation results to determine whether there are any differences.

[0076] Optionally, a similarity threshold can be set according to needs to determine whether there is a difference. When the similarity between the output language of the agent of the first node and the manual annotation result does not exceed the similarity threshold, it is determined that there is a difference. When the similarity between the output language of the agent of the first node and the manual annotation result reaches the similarity threshold, it is determined that there is no difference. Exemplarily, the similarity threshold of the embodiment of the present application is 80%.

[0077] Exemplarily, when the similarity between the output language of the agent of the first node and the manual annotation result reaches 80%, it is determined that there is no difference.

[0078] Optionally, judging whether there is a difference can be based on judging the core meanings of the two, and respectively extracting the core content of the output language of the agent of the first node and the core content of the manually labeled result for comparison. If the comparison results of the core contents are consistent, it is judged that there is no difference. If the comparison results of the core contents are inconsistent, it is judged that there is a difference. Further, the comparison process of the core contents can be performed by extracting the keywords of the output language of the agent of the first node and extracting the keywords of the manually labeled result, and comparing the two one by one. If the overlap of the comparison exceeds the overlap threshold, it means that the comparison results of the core contents are consistent. The overlap threshold can be set according to specific needs. Exemplarily, the overlap threshold of the embodiment of the present application is 90%.

[0079] 412. If at least one of the output language of the user scenario intelligent agent, the output language of the user question intelligent agent, and the output language of the sales skill intelligent agent is different from the manual labeling result, obtain and record the intelligent agent that is different from the manual labeling result.

[0080] Exemplarily, the similarity between the output language of the user scenario agent and the manual labeling result is 70%, and the similarity between the output language of the sales skills agent and the manual labeling result is 75%. Then, the output language of both the user scenario agent and the sales skills agent are different from the manual labeling results. The user scenario agent and the sales skills agent are recorded so that the user scenario agent and the sales skills agent in the model can be trained, adjusted and optimized according to the second input parameter later.

[0081] Exemplarily, the overlap degree between the keywords of the core content of the user scenario agent and the keywords of the manually labeled results is 100%, and the overlap degree between the keywords of the core content of the user question agent and the sales skills agent and the keywords of the manually labeled results is 80%. Then it is judged that the output language of the user question agent and the sales skills agent are different from the manual standard results, and the output language of the user scenario agent is not different from the manual labeling results; the user question agent and the sales skills agent are recorded so that the user question agent and the sales skills agent in the model can be trained, adjusted and optimized according to the second input parameter later.

[0082] 413. If the output language of the user scenario intelligent agent, the output language of the user question intelligent agent and the output language of the sales skill intelligent agent are the same as the manual labeling results, then check whether there is a difference between the output language of the control speech selection intelligent agent and the manual labeling results.

[0083] It should be noted that we first determine whether the first node agent: user scenario agent, user problem agent and sales skill agent have any problems. If the agents of the first node do not differ from the manual labeling results, we then determine whether the second node agent: speech selection agent has any problems.

[0084] 414. If the output language of the intelligent agent selected by the speech script is different from the manual labeling result, record the intelligent agent selected by the speech script.

[0085] It can be understood that the judgment method of judging whether there is a difference between the output language of the second node agent and the manual annotation result is the same as that of the first node agent, and finally it is judged whether there is a problem with the speech selection agent. If there is a problem, it is necessary to train, adjust and optimize the speech selection agent in the model according to the second input parameter.

[0086] Specifically, based on the above embodiments, the large model optimization method based on the financial business scenario provided in the embodiments of the present application can also be concretized as follows: training and adjusting the model, including:

[0087] 421. According to the intelligent agent having errors in the output results, filter out the intelligent agent data related to the intelligent agent in the second input parameter.

[0088] Exemplarily, if there is an error in the output result of the user context agent, that is, there is a difference between the output language of the user context agent and the manual annotation result, the user context data related to the user context agent in the second input parameter is screened out.

[0089] 422. Train and adjust the corresponding intelligent agent of the model based on the intelligent agent data.

[0090] Exemplarily, user context data related to the user context agent is screened out from the second input parameter, and the user context data is input into the model to train and adjust the user context agent.

[0091] 423. If there are no errors in the output results of each intelligent agent, all intelligent agents of the model are trained and adjusted according to the second input parameter.

[0092] Optionally, if it is determined that there are no errors in the output results of each agent, that is, there is no difference between the output language of each agent and the manual annotation result, it is determined that there is a problem with the collaborative output results of each agent. In this case, there is no need to screen the second input parameter, and all the data of the second input parameter can be directly used to train and adjust all the agents of the model.

[0093] In the above implementation process, based on the intelligent agent of the model with incorrect output results obtained by screening, the data corresponding to the intelligent agent is called to train and adjust the model to enhance the accuracy and efficiency of the training.

[0094] Specifically, based on the above embodiment, the large model optimization method based on the financial business scenario provided in the embodiment of the present application can also be concretized as follows: the manual labeling result of the second input parameter is obtained, including:

[0095] According to the second input parameter, each intelligent agent of the model is manually labeled to obtain the manual labeling result.

[0096] Specifically, if the first output result of the large parameter model is inconsistent with the second output result of the small parameter model, it is necessary to manually annotate the model, specifically, manually annotate each agent of the model to obtain the manual annotation result of each agent.

[0097] Therefore, when the output results of the large-parameter model and the small-parameter model are inconsistent, more accurate annotation results can be obtained by manually annotating each intelligent agent of the model.

[0098] The above steps are not to be performed in a strict order as described in the numbers, but should be understood as an overall solution.

[0099] In the second aspect, based on the above embodiments, Figure 2 This is a schematic diagram of the structure of a large model optimization device based on a financial business scenario provided in an embodiment of the present application. Figure 2The large model optimization device based on the financial business scenario provided in this embodiment specifically includes: a parameter acquisition module 201, a model training module 202, a difference screening module 203 and a model adjustment module 204.

[0100] Among them, the parameter acquisition module 201 is used to obtain the first input parameter, wherein the first input parameter includes the user's actual input request; the model training module 202 is used to train the first input parameter based on the small parameter model and the large parameter model respectively, and generate the first output result and the second output result accordingly; the difference screening module 203 is used to screen out the corresponding second input parameter if there is a difference between the first output result and the second output result; the model adjustment module 204 is used to obtain the manual annotation result of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation result.

[0101] As described above, the embodiment of the present application obtains a first input parameter, wherein the first input parameter includes a real input request of the user; the first input parameter is trained based on a small parameter model and a large parameter model respectively, and a first output result and a second output result are generated correspondingly; if there is a difference between the first output result and the second output result, the corresponding second input parameter is screened out; the manual annotation result of the second input parameter is obtained, and the small parameter model and the large parameter model are adjusted according to the manual annotation result; the large parameter model is used instead of manpower for data annotation to improve the accuracy of the small parameter model, and manual annotation is required only when the output results of the large parameter model and the small parameter model are different, thereby effectively reducing manpower.

[0102] The large model optimization device based on financial business scenarios provided in the embodiment of the present application can be used to execute the large model optimization method based on financial business scenarios provided in the above embodiment, and has corresponding functions and beneficial effects.

[0103] On the third aspect, the embodiments of the present application also provide an electronic device, which can integrate the large model optimization device based on financial business scenarios provided in the embodiments of the present application. Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 The electronic device includes: an input device 33, an output device 34, a memory 32 and one or more processors 31; the memory 32 is used to store one or more programs; when the one or more programs are executed by the one or more processors 31, the one or more processors 31 implement the large model optimization method based on the financial business scenario provided in the above embodiment. The input device 33, the output device 34, the memory 32 and the processor 31 can be connected by a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0104] The processor 31 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 32, that is, implements the above-mentioned large model optimization method based on financial business scenarios.

[0105] The electronic device provided above can be used to execute the large model optimization method based on financial business scenarios provided in the above embodiment, and has corresponding functions and beneficial effects.

[0106] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the large model optimization method based on the financial business scenario as described above, and can achieve the same beneficial effects.

[0107] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present application is not limited to the large model optimization method based on financial business scenarios as described above, and can also execute related operations in the large model optimization method based on financial business scenarios provided in any embodiment of the present application.

[0108] In a fifth aspect, the embodiments of the present application also provide a computer program product, and the methods described in the various embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the various embodiments of the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM (Open Application Model) or other programmable device.

[0109] The computer program or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0112] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0113] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0115] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A large model optimization method based on financial business scenarios, characterized in that: The method comprises: Obtaining a first input parameter, wherein the first input parameter includes a user's actual input request; The first input parameter is trained based on the small parameter model and the large parameter model respectively, and a first output result and a second output result are generated correspondingly; If there is a difference between the first output result and the second output result, select the corresponding second input parameter; Obtain a manual annotation result of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation result.

2. The large model optimization method based on financial business scenarios according to claim 1 is characterized in that: The first input parameter is trained based on the small parameter model and the large parameter model respectively, and the first output result and the second output result are generated accordingly, including: Inputting a first input parameter into a small parameter model to obtain a first output result; Inputting the first input parameter into the large parameter model to obtain a second output result; A data list is generated according to the first input parameter, the first output result and the second output result.

3. The large model optimization method based on financial business scenarios according to claim 1 is characterized in that: The step of obtaining a manual annotation result of the second input parameter and adjusting the small parameter model and the large parameter model according to the manual annotation result includes: Compare the first output result and the second output result with the manual annotation results respectively; If the first output result of the small parameter model is wrong and the second output result of the large parameter model is correct, the small parameter model is trained and adjusted; If the first output result of the small parameter model is wrong and the second output result of the large parameter model is wrong, the small parameter model and the large parameter model are trained and adjusted respectively; If the first output result of the small parameter model is correct and the second output result of the large parameter model is wrong, the large parameter model is trained and adjusted.

4. The large model optimization method based on financial business scenarios according to claim 3 is characterized in that: The large parameter model and the small parameter model have the same model structure, and both models include multiple intelligent agents; If the output of the model is wrong, it also includes: Intelligent agents that filter models that have errors in their output results.

5. The large model optimization method based on financial business scenarios according to claim 4 is characterized in that: The intelligent agent corresponding to the model whose screening output result has errors includes: Compare the output of the user scenario intelligent agent, the user question intelligent agent, and the sales skill intelligent agent to see if there are any differences with the manual annotation results; If at least one of the output language of the user scenario intelligent agent, the output language of the user question intelligent agent, and the output language of the sales skill intelligent agent is different from the manual labeling result, the intelligent agent that is different from the manual labeling result is obtained and recorded; If the output of the user scenario intelligent agent, the user question intelligent agent, and the sales skill intelligent agent are all the same as the manual labeling results, then check whether the output of the speech selection intelligent agent is different from the manual labeling results; If the output language of the speech selection intelligent agent is different from the manual labeling result, record the speech selection intelligent agent.

6. The large model optimization method based on financial business scenarios according to claim 4 is characterized in that: Train and tune the model, including: According to the intelligent agent with errors in the output results, filter out the intelligent agent data related to the intelligent agent in the second input parameter; Training and adjusting the corresponding intelligent agent of the model based on the intelligent agent data; If there are no errors in the output results of each intelligent agent, all the intelligent agents of the model are trained and adjusted according to the second input parameter.

7. The large model optimization method based on financial business scenarios according to claim 1 is characterized in that: The obtaining of the manual annotation result of the second input parameter includes: According to the second input parameter, each intelligent agent of the model is manually labeled to obtain the manual labeling result.

8. A large model optimization device based on financial business scenarios, characterized in that: include: A parameter acquisition module, used to acquire a first input parameter, wherein the first input parameter includes a user's actual input request; A model training module, used to train the first input parameter based on the small parameter model and the large parameter model respectively, and generate a first output result and a second output result accordingly; A difference screening module, used for screening out the corresponding second input parameter if there is a difference between the first output result and the second output result; The model adjustment module is used to obtain the manual annotation result of the second input parameter, and adjust the small parameter model and the large parameter model according to the manual annotation result.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the processor is connected to the memory via the bus, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the large model optimization method based on the financial business scenario as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a server, implements a large model optimization method based on a financial business scenario as described in any one of claims 1-7.

Images