Answer acquisition method and device, equipment and storage medium
By calculating the score value and confidence of the annotation model, the final answers to insurance consulting questions are generated and selected, and the problems of cumbersome and inefficient answer acquisition process in the prior art are solved, and fast and reliable answer acquisition is achieved.
Patent Information
- Application Number
- CN202411260185.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-06
AI Technical Summary
The process of obtaining the final answers to existing insurance consulting questions is cumbersome, resulting in inefficiency and susceptible to manual intervention, and the inability to provide answers quickly and accurately.
By obtaining the score values of each annotation model, normalizing operations are performed to calculate the confidence. When the confidence meets the preset conditions, annotation tasks are generated, consulting questions are obtained, and the evaluation value is calculated based on the confidence and accuracy rate, and the total evaluation value is selected. The highest answer is the final answer.
No manual intervention is required, which significantly reduces answer acquisition time, improves efficiency, and improves the reliability of answers.
Smart Images

Figure CN119088939A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and financial technology, and in particular to an answer acquisition method, device, equipment and storage medium. Background Art
[0002] There are many consulting questions in the insurance field, because insurance is a complex financial product, which makes it difficult for consumers to fully understand it based on their own knowledge. Therefore, when consumers hold insurance, they will ask many consulting questions.
[0003] However, the process of obtaining the final answers to existing consulting questions is cumbersome, which is not conducive to improving the efficiency of obtaining the final answers. The reason is that the existing technology mainly adopts manual acquisition to obtain the final answers to consulting questions, which consumes a lot of human resources and time resources, increases the time to obtain the final answers to consulting questions, and is easily affected by manual intervention. Therefore, it is not conducive to improving the efficiency of obtaining the final answers. Summary of the invention
[0004] The present invention provides an answer acquisition method, device, computer equipment and storage medium to solve the technical problem that the process of obtaining the final answer to the existing consulting question is cumbersome and is not conducive to improving the efficiency of obtaining the final answer.
[0005] In a first aspect, a method for obtaining an answer is provided, comprising:
[0006] Get the score corresponding to each annotation model;
[0007] Normalizing the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models;
[0008] When the confidence corresponding to each of the annotation models meets a preset condition, obtaining an annotation task and obtaining a consulting question in the annotation task;
[0009] Obtaining each annotation answer generated by each annotation model based on the consultation question;
[0010] Generate evaluation values of each of the annotation models on each of the annotation answers according to the confidences corresponding to each of the annotation models and a predefined generation method;
[0011] The evaluation values of each of the annotation models on each of the annotation answers are added together to obtain a total evaluation value of each of the annotation answers, and the annotation answer with the highest total evaluation value is selected as the final answer to the consultation question.
[0012] Furthermore, the normalizing operation is performed on the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models, including:
[0013] Adding the score corresponding to the annotation model and the score corresponding to other models to generate a total score;
[0014] According to the score sum, the score values corresponding to each of the annotation models are normalized to obtain the confidence level corresponding to each of the annotation models.
[0015] Furthermore, when the confidence corresponding to each of the annotation models meets a preset condition, obtaining the annotation task and obtaining the consulting question in the annotation task include:
[0016] When the confidence corresponding to each of the annotation models is greater than a preset value, obtaining a connection to the insurance platform and obtaining the annotation task sent by the insurance platform;
[0017] The insurance policy issues and claims issues in the labeling task are selected as the consulting issues. The insurance policy issues are issues that customers inquire about insurance policy business, and the claims issues are issues that customers inquire about claims business.
[0018] Furthermore, the obtaining of each annotation answer generated by each annotation model based on the consultation question includes:
[0019] Obtaining a feature vector of the consulting question, and inputting the feature vector into each of the annotation models;
[0020] Obtain each annotation answer generated by each annotation model based on the feature vector.
[0021] Furthermore, generating evaluation values of each annotation model on each annotation answer according to the confidence corresponding to each annotation model and a predefined generation method includes:
[0022] Obtaining the accuracy of each of the annotation models;
[0023] The accuracy rate corresponding to each of the annotation models is multiplied by the confidence level corresponding to each of the annotation models to generate an evaluation value of each of the annotation models on each of the annotation answers.
[0024] Furthermore, the step of adding the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question includes:
[0025] Obtain a record file of the consultation question, and in the record file, obtain an evaluation value of each of the annotation models on each of the annotation answers;
[0026] The evaluation values of each of the annotation models on each of the annotation answers are added together to obtain a total evaluation value of each of the annotation answers. The total evaluation values of each of the annotation answers are sorted using a preset sorting function. Based on the sorting result, the annotation answer with the highest total evaluation value is selected as the final answer to the consultation question.
[0027] Furthermore, after adding the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question, the answer acquisition method includes:
[0028] Obtain a sending instruction, execute the sending instruction, and send the consulting question and the final answer to the insurance platform.
[0029] In a second aspect, a device for obtaining an answer is provided, comprising:
[0030] The first acquisition module is used to obtain the score value corresponding to each annotation model;
[0031] An operation module, used for normalizing the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models;
[0032] A second acquisition module is used to acquire the labeling task and the consulting question in the labeling task when the confidence corresponding to each of the labeling models meets the preset conditions;
[0033] A third acquisition module is used to obtain each annotation answer generated by each annotation model based on the consultation question;
[0034] A generation module, used to generate evaluation values of each of the annotation models on each of the annotation answers according to the confidence corresponding to each of the annotation models and a predefined generation method;
[0035] The fourth acquisition module is used to add up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and select the annotation answer with the highest total evaluation value as the final answer to the consultation question.
[0036] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned answer acquisition method when executing the computer program.
[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned answer acquisition method are implemented.
[0038] The present application provides an answer acquisition method, apparatus, computer equipment and storage medium, which acquire the score value corresponding to each annotation model; perform normalization operation on the score value corresponding to each annotation model to obtain the confidence degree corresponding to each annotation model; when the confidence degree corresponding to each annotation model meets the preset conditions, acquire the annotation task and obtain the consulting question in the annotation task; acquire each annotation answer generated by each annotation model based on the consulting question; generate the evaluation value of each annotation model on each annotation answer according to the confidence degree corresponding to each annotation model and the predefined generation method; add the evaluation value of each annotation model on each annotation answer to obtain the total evaluation value of each annotation answer, and select The annotated answer with the highest total evaluation value is used as the final answer to the consulting question. This has two beneficial effects. On the one hand, the classification value corresponding to the annotated answer is obtained, and the evaluation value corresponding to the annotated answer is determined based on the classification value and the confidence level. The evaluation values of each annotated model on each annotated answer are added together to obtain the total evaluation value of each annotated answer. The annotated answer with the highest total evaluation value is selected as the final answer to the consulting question. Since manual acquisition is not required, the time for acquiring the final answer to the consulting question is reduced, which is beneficial to improving the efficiency of acquiring the final answer to the consulting question. On the other hand, it will not be affected by manual intervention, which is beneficial to improving the reliability of the acquired final answer to the consulting question. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0040] Figure 1 is a schematic diagram of an application environment of a method for obtaining an answer in one embodiment of the present invention;
[0041] Figure 2 A schematic flow chart of a method for obtaining an answer provided by an embodiment of the present invention;
[0042] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S23;
[0043] Figure 4 yes Figure 1 A schematic flow chart of a specific implementation of step S25;
[0044] Figure 5 yes Figure 1 A schematic flow chart of a specific implementation of step S26;
[0045] Figure 6 is a structural schematic diagram of an answer acquisition device in one embodiment of the present invention;
[0046] Figure 7 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;
[0047] Figure 8 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an application environment of an answer acquisition method according to an embodiment of the present invention. The answer acquisition method provided by the embodiment of the present invention can be applied in the following embodiments: Figure 1 In an application environment, a client communicates with a server through a network.
[0050] The server obtains the score corresponding to each annotation model through the client;
[0051] Normalizing the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models;
[0052] When the confidence corresponding to each of the annotation models meets a preset condition, obtaining an annotation task and obtaining a consulting question in the annotation task;
[0053] Obtaining each annotation answer generated by each annotation model based on the consultation question;
[0054] Generate evaluation values of each of the annotation models on each of the annotation answers according to the confidences corresponding to each of the annotation models and a predefined generation method;
[0055] The evaluation values of each of the annotation models on each of the annotation answers are added together to obtain a total evaluation value of each of the annotation answers, and the annotation answer with the highest total evaluation value is selected as the final answer to the consultation question.
[0056] In the scheme implemented by the above-mentioned answer acquisition method, device, equipment and medium, the beneficial effects are in two aspects. On the one hand, the classification value corresponding to the marked answer is obtained, and the evaluation value corresponding to the marked answer is determined based on the classification value and the confidence level; the evaluation values of each of the marked models on each of the marked answers are added to obtain the total evaluation value of each of the marked answers, and the marked answer with the highest total evaluation value is selected as the final answer to the consulting question. Since manual acquisition is not required, the time for obtaining the final answer to the consulting question is reduced, which is conducive to improving the efficiency of obtaining the final answer to the consulting question; on the other hand, it will not be affected by manual intervention, so it is conducive to improving the reliability of the final answer to the consulting question obtained.
[0057] The client may include but is not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices.
[0058] The server can be implemented with an independent task database or a task database cluster composed of multiple task databases. The present invention is described in detail below through specific embodiments.
[0059] See also Figure 2 , Figure 2 A flowchart of a method for obtaining an answer provided by an embodiment of the present invention includes the following steps:
[0060] S21, obtaining the score value corresponding to each annotation model;
[0061] Exemplarily, the score values corresponding to the respective annotation models include:
[0062] The scoring data is obtained, and in the scoring data, the scoring values corresponding to the respective annotation models are obtained.
[0063] The scoring data is obtained by collecting, collating and analyzing information from different sources based on a series of preset standards or indicators. These standards or indicators involve multiple dimensions, such as functionality, ease of use, reliability, cost-effectiveness, etc., depending on the characteristics of the annotation model and the purpose of the evaluation.
[0064] S22, normalizing the score values corresponding to the respective annotation models to obtain the confidence values corresponding to the respective annotation models;
[0065] The step of normalizing the score values corresponding to the respective annotation models to obtain the confidence levels corresponding to the respective annotation models includes:
[0066] Adding the score corresponding to the annotation model and the score corresponding to other models to generate a total score;
[0067] According to the score sum, the score values corresponding to each of the annotation models are normalized to obtain the confidence level corresponding to each of the annotation models.
[0068] According to the score sum, the score values corresponding to each of the annotation models are normalized to obtain the confidence corresponding to each of the annotation models, specifically:
[0069] The score values corresponding to the respective annotation models are divided by the total score to obtain the confidence values corresponding to the respective annotation models.
[0070] For the sake of illustration, an example is given below:
[0071] For example, there are three annotation models, namely, annotation model 1, annotation model 2, and annotation model 3. The score value corresponding to the annotation model 1, the score value corresponding to the annotation model 2, and the score value corresponding to the annotation model 3 are added together to obtain the total score;
[0072] The score corresponding to the annotation model 1 is divided by the total score to obtain the confidence corresponding to the annotation model 1;
[0073] The score corresponding to the annotation model 2 is divided by the total score to obtain the confidence level corresponding to the annotation model 2.
[0074] The score value corresponding to the annotation model 3 is divided by the total score to obtain the confidence level corresponding to the annotation model 3.
[0075] S23, when the confidence corresponding to each of the annotation models meets a preset condition, obtaining an annotation task and obtaining a consulting question in the annotation task;
[0076] Among them, consulting issues include insurance policy issues and claims issues. The insurance policy issues are issues that customers inquire about insurance policy business, and the claims issues are issues that customers inquire about claims business.
[0077] For the sake of illustration, the following are some examples of questions that customers may ask about insurance policy business:
[0078] When customers inquire about insurance policy matters, they often discuss multiple aspects such as the specific content of the policy, coverage, claims process, premium payment, policy changes and renewals, and customer service.
[0079] For example, if I need to apply for compensation in the future, what process should I follow?
[0080] For example, regarding the payment method of the premium, whether automatic deduction is supported and whether there is a grace period.
[0081] For example, if my personal information changes, such as contact information or address, how should I make changes to my insurance policy?
[0082] For the sake of illustration, the following are some examples of questions that customers may ask about insurance policy business:
[0083] When customers inquire about claims services, they usually focus on the specific conditions of claims, required materials, application procedures, processing time, and handling methods in special circumstances.
[0084] For example, you may need to apply for a claim recently. Before applying, I need to confirm which specific claim conditions have been met.
[0085] For example, what necessary documents and materials should I prepare to support my claim application.
[0086] For example, how long does it take to receive a claim decision after submission?
[0087] S24, obtaining each annotation answer generated by each annotation model based on the consultation question;
[0088] The step of obtaining each annotation answer generated by each annotation model based on the consultation question includes:
[0089] Obtaining a feature vector of the consulting question, and inputting the feature vector into each of the annotation models;
[0090] Obtain each annotation answer generated by each annotation model based on the feature vector.
[0091] Exemplarily, obtaining each annotation answer generated by each annotation model based on the feature vector includes:
[0092] Obtain an annotation file, obtain the annotation fields of each annotation model in the annotation file, and obtain the annotation answers generated by each annotation model based on the feature vector in the annotation fields of each annotation model.
[0093] S25, generating evaluation values of each annotation model on each annotation answer according to the confidence corresponding to each annotation model and a predefined generation method;
[0094] S26, adding up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question.
[0095] For the sake of illustration, an example is given below:
[0096] For example, there are 5 annotation models, namely, annotation model 1, annotation model 2, annotation model 3, annotation model 4, and annotation model 5; there are 2 annotation answers, namely, annotation answer 1 and annotation answer 2;
[0097] The evaluation values of the labeled answer 1 are added together by the labeled model 1, the labeled model 2, the labeled model 3, the labeled model 4, and the labeled model 5 to obtain the total evaluation value 1 of the labeled answer 1.
[0098] The evaluation values of the labeled answer 2 of the labeling model 1, the labeling model 2, the labeling model 3, the labeling model 4, and the labeling model 5 are added together to obtain the total evaluation value 2 of the labeled answer 2.
[0099] Between total evaluation value 1 and total evaluation value 2, when total evaluation value 1 is the highest, marked answer 1 is selected as the final answer to the consultation question.
[0100] Between total evaluation value 1 and total evaluation value 2, when total evaluation value 2 is the highest, marked answer 1 is selected as the final answer to the consultation question.
[0101] Exemplarily, after adding the evaluation values of each of the annotation models on each of the annotation answers to obtain the total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question, the answer acquisition method includes:
[0102] In a preset storage device, a storage area corresponding to the labeling task is created, and the labeling task and the final answer are stored in the storage area.
[0103] Wherein, after adding the evaluation values of each of the annotation models on each of the annotation answers to obtain the total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question, the answer acquisition method includes:
[0104] Obtain a sending instruction, execute the sending instruction, and send the consulting question and the final answer to the insurance platform.
[0105] In the embodiments of the present invention, the beneficial effects are in two aspects. On the one hand, the classification value corresponding to the marked answer is obtained, and based on the classification value and the confidence level, the evaluation value corresponding to the marked answer is determined; the evaluation values of each of the marked models on each of the marked answers are added to obtain the total evaluation value of each of the marked answers, and the marked answer with the highest total evaluation value is selected as the final answer to the consulting question. Since manual acquisition is not required, the time for obtaining the final answer to the consulting question is reduced, which is conducive to improving the efficiency of obtaining the final answer to the consulting question; on the other hand, it will not be affected by manual intervention, so it is conducive to improving the reliability of the final answer to the consulting question obtained.
[0106] See also Figure 3 , Figure 3 yes Figure 1 A specific implementation flow diagram of step S23 is described in detail as follows:
[0107] S31, when the confidence corresponding to each of the annotation models is greater than a preset value, obtaining a connection to an insurance platform and obtaining an annotation task sent by the insurance platform;
[0108] S32, selecting the insurance policy questions and the claims questions in the labeling task as the consulting questions, wherein the insurance policy questions are questions that customers ask about insurance policy business, and the claims questions are questions that customers ask about claims business.
[0109] In the embodiment of the present invention, the insurance policy issues and claims issues in the annotation task are selected as the consulting issues, which is conducive to handling the insurance policy issues and claims issues.
[0110] See also Figure 4 , Figure 4 yes Figure 1 A specific implementation flow diagram of step S25 is described in detail as follows:
[0111] S41, obtaining the accuracy corresponding to each of the annotation models;
[0112] Exemplarily, obtaining the accuracy of each of the annotation models includes:
[0113] Using the sample data in the test set to test each of the annotation models, and obtaining the test results of each of the annotation models;
[0114] According to the test results, the accuracy of each of the annotation models is obtained.
[0115] S42, multiplying the accuracy corresponding to each of the annotation models by the confidence corresponding to each of the annotation models, to generate an evaluation value of each of the annotation models on each of the annotation answers.
[0116] For the sake of illustration, an example is given below:
[0117] For example, there are three annotation models, namely, annotation model 1, annotation model 2, and annotation model 3;
[0118] The accuracy rate corresponding to the annotation model 1 is multiplied by the confidence level corresponding to the annotation model 1 to generate an evaluation value of the annotation model 1 on each of the annotation answers.
[0119] The accuracy rate corresponding to the annotation model 2 is multiplied by the confidence level corresponding to the annotation model 2 to generate an evaluation value of the annotation model 2 on each of the annotation answers.
[0120] The accuracy rate corresponding to the annotation model 3 is multiplied by the confidence level corresponding to the annotation model 3 to generate an evaluation value of the annotation model 3 on each of the annotation answers.
[0121] In the embodiment of the present invention, the role of each annotation model can be brought into play through the evaluation value of each annotation model on each annotation answer.
[0122] See also Figure 5 , Figure 5 yes Figure 1 A specific implementation flow diagram of step S26 is described in detail as follows:
[0123] S51, obtaining a record file of the consultation question, and obtaining, in the record file, an evaluation value of each of the annotation models on each of the annotation answers;
[0124] S52, adding up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, sorting the total evaluation values of each of the annotation answers using a preset sorting function, and based on the sorting result, selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question.
[0125] In an embodiment of the present invention, the marked answer with the highest total evaluation value is selected as the final answer to the consulting question. Since it does not need to be obtained manually, the time for obtaining the final answer to the consulting question is reduced, which is conducive to improving the efficiency of obtaining the final answer to the consulting question.
[0126] See also Figure 6 , Figure 6 is a schematic diagram of a structure of an answer acquisition device in one embodiment of the present invention. Figure 6 As shown, the answer acquisition device includes a first acquisition module 101, an operation module 102, a second acquisition module 103, a third acquisition module 104, a generation module 105, and a fourth acquisition module 106. The functional modules are described in detail as follows:
[0127] The first acquisition module 101 is used to obtain the score value corresponding to each annotation model;
[0128] An operation module 102 is used to perform a normalization operation on the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models;
[0129] The second acquisition module 103 is used to acquire the labeling task and the consulting question in the labeling task when the confidence corresponding to each of the labeling models meets the preset conditions;
[0130] The third acquisition module 104 is used to obtain each annotation answer generated by each annotation model based on the consultation question;
[0131] A generating module 105, configured to generate evaluation values of each of the annotation models on each of the annotation answers according to the confidence corresponding to each of the annotation models and a predefined generating method;
[0132] The fourth acquisition module 106 is used to add up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and select the annotation answer with the highest total evaluation value as the final answer to the consultation question.
[0133] In one embodiment, the operation module 102 includes:
[0134] An acquisition subunit is used to add the score value corresponding to the annotation model and the score values corresponding to other models to generate a score sum;
[0135] The operation subunit is used to perform a normalization operation on the score values corresponding to each of the annotation models according to the score sum, so as to obtain the confidence level corresponding to each of the annotation models.
[0136] In one embodiment, the second acquisition module 103 includes:
[0137] A connection subunit, used for obtaining connection with the insurance platform and obtaining the labeling task sent by the insurance platform when the confidence corresponding to each of the labeling models is greater than a preset value;
[0138] The second acquisition sub-unit is used to select the insurance policy issues and claims issues in the labeling task as the consulting issues, wherein the insurance policy issues are issues that customers inquire about insurance policy business, and the claims issues are issues that customers inquire about claims business.
[0139] In one embodiment, the third acquisition module 104 includes:
[0140] An input subunit, used for obtaining a feature vector of the consulting question and inputting the feature vector into each of the annotation models;
[0141] The first generating subunit is used to obtain each annotation answer generated by each annotation model based on the feature vector.
[0142] In one embodiment, the generating module 105 includes:
[0143] A third acquisition subunit is used to obtain the accuracy corresponding to each of the annotation models;
[0144] The second generating subunit is used to multiply the accuracy corresponding to each of the annotation models by the confidence corresponding to each of the annotation models to generate an evaluation value of each of the annotation models on each of the annotation answers.
[0145] In one embodiment, the fourth acquisition module 106 includes:
[0146] A fourth acquisition subunit is used to acquire a record file of the consulting question, and in the record file, acquire an evaluation value of each of the annotation models on each of the annotation answers;
[0147] A selection subunit is used to add up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, sort the total evaluation values of each of the annotation answers through a preset sorting function, and based on the sorting result, select the annotation answer with the highest total evaluation value as the final answer to the consultation question.
[0148] In one embodiment, the answer acquisition device further includes:
[0149] The display module is used to obtain a sending instruction, execute the sending instruction, and send the consulting question and the final answer to the insurance platform.
[0150] In the embodiments of the present invention, the beneficial effects are in two aspects. On the one hand, the classification value corresponding to the marked answer is obtained, and based on the classification value and the confidence level, the evaluation value corresponding to the marked answer is determined; the evaluation values of each of the marked models on each of the marked answers are added to obtain the total evaluation value of each of the marked answers, and the marked answer with the highest total evaluation value is selected as the final answer to the consulting question. Since manual acquisition is not required, the time for obtaining the final answer to the consulting question is reduced, which is conducive to improving the efficiency of obtaining the final answer to the consulting question; on the other hand, it will not be affected by manual intervention, so it is conducive to improving the reliability of the final answer to the consulting question obtained.
[0151] For the specific limitations on the answer acquisition device, please refer to the limitations on the answer acquisition method above, which will not be repeated here.
[0152] Each module in the above-mentioned answer acquisition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.
[0153] See also Figure 7 , Figure 7 is a schematic diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a server. The internal structure diagram thereof may be as shown in FIG. Figure 7 The computer device includes a processor, a memory, a network interface and a database connected through a system bus.
[0154] The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, the function or steps of an answer acquisition method can be implemented on the server side.
[0155] See also Figure 8 , Figure 8 is another structural diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external task database through a network connection. When the computer program is executed by the processor, the functions or steps of an answer acquisition method can be implemented on the client.
[0156] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor.
[0157] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions of the server and the client in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.
[0158] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0159] The above description and accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of a stated sub-sample, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.
[0160] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0161] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.
[0162] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.
Claims
1. A method for obtaining an answer, It is characterized in that include: Get the score corresponding to each annotation model; Normalizing the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models; When the confidence corresponding to each of the annotation models meets a preset condition, obtaining an annotation task and obtaining a consulting question in the annotation task; Obtaining each annotation answer generated by each annotation model based on the consultation question; Generate evaluation values of each of the annotation models on each of the annotation answers according to the confidences corresponding to each of the annotation models and a predefined generation method; The evaluation values of each of the annotation models on each of the annotation answers are added together to obtain a total evaluation value of each of the annotation answers, and the annotation answer with the highest total evaluation value is selected as the final answer to the consultation question.
2. The answer acquisition method according to claim 1, It is characterized in that The normalizing operation is performed on the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models, including: Adding the score corresponding to the annotation model and the score corresponding to other models to generate a total score; According to the score sum, the score values corresponding to each of the annotation models are normalized to obtain the confidence level corresponding to each of the annotation models.
3. The answer acquisition method according to claim 1, It is characterized in that When the confidence corresponding to each of the annotation models meets the preset condition, obtaining the annotation task and obtaining the consulting question in the annotation task include: When the confidence corresponding to each of the annotation models is greater than a preset value, connecting to the insurance platform and obtaining the annotation task sent by the insurance platform; The insurance policy issues and claims issues in the labeling task are selected as the consulting issues. The insurance policy issues are issues that customers inquire about insurance policy business, and the claims issues are issues that customers inquire about claims business.
4. The answer acquisition method according to claim 1, It is characterized in that The obtaining of each annotation answer generated by each annotation model based on the consultation question includes: Obtaining a feature vector of the consulting question, and inputting the feature vector into each of the annotation models; Obtain each annotation answer generated by each annotation model based on the feature vector.
5. The answer acquisition method according to claim 1, It is characterized in that Generating evaluation values of each annotation model on each annotation answer according to the confidence corresponding to each annotation model and a predefined generation method includes: Obtaining the accuracy of each of the annotation models; The accuracy rate corresponding to each of the annotation models is multiplied by the confidence level corresponding to each of the annotation models to generate an evaluation value of each of the annotation models on each of the annotation answers.
6. The answer acquisition method according to claim 1, It is characterized in that The step of adding the evaluation values of the annotation models on the annotation answers to obtain the total evaluation value of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question includes: Obtain a record file of the consultation question, and in the record file, obtain an evaluation value of each of the annotation models on each of the annotation answers; The evaluation values of each of the annotation models on each of the annotation answers are added together to obtain a total evaluation value of each of the annotation answers. The total evaluation values of each of the annotation answers are sorted using a preset sorting function. Based on the sorting result, the annotation answer with the highest total evaluation value is selected as the final answer to the consultation question.
7. The answer acquisition method according to any one of claims 1 to 6, It is characterized in that After adding the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and selecting the annotation answer with the highest total evaluation value as the final answer to the consultation question, the answer acquisition method includes: Obtain a sending instruction, execute the sending instruction, and send the consulting question and the final answer to the insurance platform.
8. An answer acquisition device, It is characterized in that include: The first acquisition module is used to obtain the score value corresponding to each annotation model; An operation module, used for normalizing the score values corresponding to each of the annotation models to obtain the confidence level corresponding to each of the annotation models; A second acquisition module is used to acquire the labeling task and the consulting question in the labeling task when the confidence corresponding to each of the labeling models meets the preset conditions; A third acquisition module is used to obtain each annotation answer generated by each annotation model based on the consultation question; A generation module, used to generate evaluation values of each of the annotation models on each of the annotation answers according to the confidence corresponding to each of the annotation models and a predefined generation method; The fourth acquisition module is used to add up the evaluation values of each of the annotation models on each of the annotation answers to obtain a total evaluation value of each of the annotation answers, and select the annotation answer with the highest total evaluation value as the final answer to the consultation question.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the answer acquisition method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the steps of the answer acquisition method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Multi-model fusion method of medical question-answering system
CN111681765A
Data annotation method and device and electronic equipment
CN113139072A
Intelligent session method and system, computer equipment and storage medium
CN115062129A
Question answering method and system based on multi-algorithm combination, electronic equipment and storage medium
CN117349421A
Voting consensus system based on multiple large model conversations
CN117633174A
