Judgment method and device, electronic equipment and readable storage medium
By extracting and matching the feature vectors in the judgment method and determining them in combination with the feature vectors of the knowledge base, the problem of poor judgment accuracy in the prior art is solved, and more efficient retrieval and judgment results are achieved.
Patent Information
- Application Number
- CN202411055895.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-08
AI Technical Summary
The existing knowledge base-based judgment methods have poor accuracy in the search process, which leads to inaccurate judgment results.
Eigenvector representation is performed by extracting key information in the judgment request, combining the feature vectors in the knowledge base for matching, obtaining the judgment results, and performing supplementary searches and diversity score calculations when necessary to improve retrieval accuracy.
It improves the search accuracy and judgment accuracy, enhances the influence of key features in search, and improves the reliability of judgment results.
Smart Images

Figure CN118916464A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a determination method, device, electronic device and readable storage medium. Background Art
[0002] At present, for the judgment task, that is, for a question, the judgment result is given in combination with the answer result / situation description. An intelligent judgment system using a large model external knowledge base has been proposed, which mainly involves large model technology, vector database technology, retrieval enhancement generation technology, etc. However, in the storage stage of the vector database, the document content is usually divided into fixed sizes and then converted into embedded vectors for storage, and in the retrieval stage after the user asks the question, the full amount of indiscriminate similarity comparison is performed in the vector database. As a result, the applicability of this retrieval method is poor, resulting in poor judgment accuracy. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a determination method, device, electronic device and readable storage medium to solve the problem of poor determination accuracy of the current knowledge base-based determination method.
[0004] In order to solve the above technical problems, this application is implemented as follows:
[0005] In a first aspect, a determination method is provided, comprising:
[0006] Obtaining a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information;
[0007] In response to the determination request, extract key information from the answer and / or the situation description information, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector;
[0008] Matching the first search vector with a feature vector stored in a knowledge base to obtain a first determination result corresponding to the feature vector matching the first search vector;
[0009] A determination result on the answer and / or situation description information is determined based on the first determination result.
[0010] Optionally, the key information includes at least one of the following: at least one key sentence, at least one characteristic word.
[0011] Optionally, if, when matching the first search vector with the feature vectors stored in the knowledge base, no feature vector matching the first search vector is obtained, or the number of feature vectors matching the first search vector obtained is less than a preset number threshold, then after matching the first search vector with the feature vectors stored in the knowledge base, the method further includes:
[0012] Perform at least one of the following to obtain a second search vector: represent the key information by a feature vector; select at least one target key information from the key information, and represent the first question and the at least one target key information by a feature vector, or represent the at least one target key information by a feature vector;
[0013] Matching the second search vector with a feature vector stored in the knowledge base to obtain a second determination result corresponding to the feature vector matching the second search vector;
[0014] According to the second determination result, a determination result of the answer and / or situation description information is determined.
[0015] Optionally, the method further includes:
[0016] Obtaining multiple answers and / or situation description information for a second question and a determination result corresponding to each answer and / or situation description information; the second question is a question stored in the knowledge base;
[0017] Calculate a diversity score for each answer and / or situation description information according to the knowledge base;
[0018] According to the calculated diversity score, the feature vector corresponding to the first answer and / or situation description information and the determination result corresponding to the first answer and / or situation description information are stored in the knowledge base; wherein the first answer and / or situation description information is the answer and / or situation description information whose corresponding diversity score among the multiple answers and / or situation description information exceeds a preset score threshold.
[0019] Optionally, calculating the diversity score of each answer and / or situation description information according to the knowledge base includes:
[0020] According to at least one of a word dimension, a sentence dimension and a semantic dimension, a diversity score of each answer and / or situation description information is calculated based on the knowledge base.
[0021] Optionally, before determining a determination result of the answer and / or the situation description information according to the first determination result, the method further includes:
[0022] Inputting the first question and the answer and / or the situation description information into a pre-trained large language model to obtain a third determination result of the answer and / or the situation description information;
[0023] Wherein, determining the determination result of the answer and / or situation description information according to the first determination result includes:
[0024] A determination result on the answer and / or situation description information is determined based on the first determination result and the third determination result.
[0025] In a second aspect, a determination device is provided, comprising:
[0026] A first acquisition module is used to acquire a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information;
[0027] a processing module, configured to extract key information from the answer and / or situation description information in response to the determination request, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector;
[0028] A first matching module, used to match the first search vector with a feature vector stored in a knowledge base, and obtain a first determination result corresponding to the feature vector matching the first search vector;
[0029] The first determination module is used to determine a determination result of the answer and / or situation description information according to the first determination result.
[0030] In a third aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.
[0031] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0032] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0033] Through the scheme of the embodiment of the present application, in response to a determination request, the key information in the answer to the first question in the determination request and / or the situation description information can be extracted, and the first question and the key information are represented by feature vectors to obtain a first search vector, and the first search vector is matched with the feature vector stored in the knowledge base to obtain a first determination result corresponding to the feature vector matching the first search vector, and the determination result of the answer and / or situation description information is determined based on the first determination result. Thus, a targeted similarity comparison can be performed in the knowledge base in combination with the key information, thereby enhancing the influence of the key features during retrieval, improving the retrieval accuracy, and further improving the determination accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of a determination method provided in an embodiment of the present application;
[0035] Figure 2 is a schematic diagram of an intelligent determination system provided in a specific embodiment of the present application;
[0036] Figure 3 is a structural schematic diagram of a determination device provided in an embodiment of the present application;
[0037] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0039] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0040] The judgment task involved in this application means: for a question, a judgment result is given in combination with the answer result / situation description. That is, each case to be judged consists of a question, an answer / situation description, and a judgment result for the answer / situation description.
[0041] Optionally, the solution in this application can be applied to large model result evaluation systems, automatic grading systems in the education field, complaint handling systems in the customer service field, intelligent review systems in the communication field, etc.
[0042] In order to solve the problem of poor judgment accuracy of the current knowledge base-based judgment method, the retrieval link based on the knowledge base is optimized in the embodiment of the present application, thereby improving the efficiency and effect of the retrieval, and further improving the judgment accuracy.
[0043] The following, in conjunction with the accompanying drawings, describes in detail the determination method, device, electronic device, and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.
[0044] See also Figure 1 , Figure 1 is a flow chart of a determination method provided in an embodiment of the present application, the method is applied to electronic equipment, such as Figure 1 As shown, the method comprises the following steps:
[0045] Step 11: Obtain a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information;
[0046] Step 12: In response to the determination request, extract key information from the answer and / or situation description information, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector;
[0047] Step 13: Matching the first search vector with a feature vector stored in a knowledge base to obtain a first determination result corresponding to the feature vector matching the first search vector;
[0048] Step 14: Determine a determination result of the answer and / or situation description information based on the first determination result.
[0049] In an embodiment of the present application, the judgment task may be a judgment task of the type such as testing in the field of artificial intelligence, automatic grading in the field of education, complaint handling in the field of customer service, or intelligent review in the field of communication. The corresponding judgment result may be correct or incorrect (such as grading), or may be able to solve the problem (such as complaint handling), or may be clearly described or unclear (such as review), and so on.
[0050] The first question may be a question related to a determination task of the type of testing in the field of artificial intelligence, automatic grading in the field of education, complaint handling in the field of customer service, or intelligent review in the field of communication.
[0051] Based on the different types of the first question, different results can be obtained, such as obtaining an answer to the first question, or obtaining a description of the situation of the first question, or obtaining an answer and a description of the situation of the first question. For example, taking a question related to a determination task of automatic grading in the field of education as an example, an answer can be obtained for the question; or taking a question related to a determination task of complaint handling in the field of customer service as an example, a description of the situation can be obtained for the question, such as a description of the situation for the question; or taking a question related to a determination task of testing in the field of artificial intelligence as an example, an answer and a description of the situation can be obtained for the question at the same time.
[0052] The key information may specifically be sentences and / or words that can better characterize the corresponding answer and / or situation description information, or sentences and / or words that play a key role in the characterization / understanding of the corresponding answer and / or situation description information. Optionally, the key information may include at least one of the following: at least one key sentence, at least one characteristic word.
[0053] The knowledge base is specifically a vector database. In order to improve the accuracy and efficiency of retrieval, the knowledge base uses the judgment case as the granularity, and stores the feature vector constructed according to the key information (such as key sentences, feature words, etc.) in the question and the corresponding answer and / or situation description information, as well as the judgment result corresponding to the feature vector. The judgment result can be correct or incorrect (such as the situation of paper grading), can or cannot solve the problem (such as the situation of complaint handling), can also be clear or unclear description (such as the situation of audit), etc. In this way, multi-dimensional features can be integrated to generate an embedded vector stored in the knowledge base, thereby improving the retrieval accuracy.
[0054] Optionally, when extracting key information from the answer and / or situation description information, existing related methods may be used for extraction, which is not limited in this embodiment.
[0055] Optionally, when the first question and the key information are represented by feature vectors, the first question and the key information can be converted into embedding vectors respectively, and the obtained embedding vectors are concatenated to obtain the first search vector. However, this embodiment does not limit the specific vector representation method (such as the dimension of the vector representation, etc.).
[0056] Optionally, when obtaining the first search vector, the overall characteristics of the answer to the first question and / or the situation description information can be added, that is, the first question, the answer to the first question and / or the situation description information and the key information are represented by a feature vector to obtain the first search vector.
[0057] Optionally, when matching the first search vector with a feature vector stored in the knowledge base, the matching can be performed based on a similarity comparison. For example, if the similarity between the first search vector and a feature vector stored in the knowledge base exceeds a preset threshold, it can be determined that the feature vector matches the first search vector.
[0058] Optionally, after obtaining the first judgment result, if the first judgment result only includes one judgment result, the judgment result can be directly selected as the answer to the first question and / or the judgment result of the situation description information; if the first judgment result includes multiple judgment results, the multiple judgment results can be integrated to determine the answer to the first question and / or the judgment result of the situation description information, for example: if the multiple judgment results are close / similar judgment results, one of them can be selected as the answer to the first question and / or the judgment result of the situation description information; if there are opposite judgment results among the multiple judgment results, the judgment result of the answer to the first question and / or the situation description information can be determined based on the principle of minority obeys majority, or the answer to the first question and / or the situation description information can be judged using a pre-trained large language model.
[0059] Through the scheme of the embodiment of the present application, in response to a determination request, the key information in the answer to the first question in the determination request and / or the situation description information can be extracted, and the first question and the key information are represented by feature vectors to obtain a first search vector, and the first search vector is matched with the feature vector stored in the knowledge base to obtain a first determination result corresponding to the feature vector matching the first search vector, and the determination result of the answer and / or situation description information is determined based on the first determination result. Thus, a targeted similarity comparison can be performed in the knowledge base in combination with the key information, thereby enhancing the influence of the key features during retrieval, improving the retrieval accuracy, and further improving the determination accuracy.
[0060] Considering that when there is too much limited key information, it may not be possible to retrieve relevant judgment results in the knowledge base, in the embodiment of the present application, a supplementary search can be performed based on the search results.
[0061] Optionally, if, when matching the first search vector with the feature vectors stored in the knowledge base, no feature vector matching the first search vector is obtained, or the number of feature vectors matching the first search vector obtained is less than a preset number threshold, then after matching the first search vector with the feature vectors stored in the knowledge base, the determination method in this embodiment may further include:
[0062] Perform at least one of the following to obtain a second search vector: represent the key information by a feature vector; select at least one target key information from the key information, and represent the first question and the at least one target key information by a feature vector, or represent the at least one target key information by a feature vector; for example, if the key information includes multiple key sentences, at least one key sentence (less than the multiple key sentences) may be selected as the target key information; or if the key information includes multiple feature words, at least one feature word (less than the multiple feature words) may be selected as the target key information;
[0063] Matching the second search vector with the feature vector stored in the knowledge base to obtain a second determination result corresponding to the feature vector matching the second search vector; for example, matching may be performed based on a similarity comparison;
[0064] According to the second determination result, a determination result of the answer and / or situation description information is determined.
[0065] Optionally, after obtaining the second judgment result, if the second judgment result contains only one judgment result, the judgment result can be directly selected as the answer to the first question and / or the judgment result of the situation description information; and if the second judgment result contains multiple judgment results, the multiple judgment results can be integrated to determine the answer to the first question and / or the judgment result of the situation description information.
[0066] In this way, with the help of supplementary search, it can be ensured that relevant judgment results are retrieved based on the knowledge base, thereby improving the search efficiency.
[0067] Regarding the knowledge base, the relevant schemes are mainly proposed for general fields, which has the problem of poor retrieval effect and low retrieval efficiency. To address this problem, the knowledge base in this scheme is proposed for specialized fields, such as a knowledge base related to the determination task of test type in the field of artificial intelligence, or a knowledge base related to the determination task of automatic grading type in the field of education, or a knowledge base related to the determination task of complaint handling type in the field of customer service, or a knowledge base related to the determination task of intelligent review type in the field of communication, etc.
[0068] Considering that among the massive number of cases to be judged, there is a high proportion of cases with low decision difficulty but high frequency, and a certain proportion of cases with high decision difficulty but low frequency. If all of them are entered into the knowledge base without screening, a large amount of computing resources will be consumed during retrieval, and the difficulty of correct retrieval will be increased, affecting the retrieval effect. When constructing the knowledge base, the solution in this application can aim at the diversity of answer results, and include as many cases of language expression and problem-solving ideas as possible for the same question.
[0069] Optionally, the determination method in the embodiment of the present application may further include:
[0070] Obtaining multiple answers and / or situation description information for a second question and a determination result corresponding to each answer and / or situation description information; the second question is a question stored in the knowledge base, that is, a feature vector corresponding to the second question is stored in the knowledge base;
[0071] Calculate a diversity score for each answer and / or situation description information according to the knowledge base;
[0072] According to the calculated diversity score, the feature vector corresponding to the first answer and / or situation description information and the determination result corresponding to the first answer and / or situation description information are stored in the knowledge base; wherein the first answer and / or situation description information is the answer and / or situation description information whose corresponding diversity score exceeds a preset score threshold among the multiple answers and / or situation description information; for the feature vector corresponding to the first answer and / or situation description information, key information (such as key sentences, feature words, etc.) in the first answer and / or situation description information can be first extracted, and then the second question and the key information are represented by feature vectors to obtain the feature vector corresponding to the first answer and / or situation description information.
[0073] It should be noted that the diversity score is used to indicate the degree of difference between the corresponding information, and the higher the diversity score, the greater the degree of difference. By calculating the diversity score, new data that is not homogeneous with the cases in the knowledge base can be selected to be added to the knowledge base, so as to include as many rich judgment cases as possible into the knowledge base, keep the content in the knowledge base concise and representative, thereby avoiding the waste of retrieval resources, reducing the difficulty of correct retrieval, and improving the retrieval effect.
[0074] Optionally, calculating the diversity score of each answer and / or situation description information according to the knowledge base may include:
[0075] According to at least one of the word dimension, the sentence dimension and the semantic dimension, the diversity score of each answer and / or situation description information is calculated according to the knowledge base. In this way, the diversity score can be calculated from multiple dimensions, which is helpful to select non-homogeneous new data to be added to the knowledge base.
[0076] In an optional implementation, the diversity scores may be calculated according to the word dimension, sentence dimension, and semantic dimension respectively to obtain three diversity scores under the three dimensions, and then the three diversity scores are weighted averaged to obtain the final diversity score.
[0077] In an optional implementation, when calculating the diversity score according to the word dimension, the information quantity correlation theory can be used to obtain the diversity score by counting the information quantity of the characteristic words in each answer and / or situation description information in the knowledge base. For example, the following formulas (1) to (3) can be used to calculate the diversity score F under the word dimension:
[0078] P c =freq(c) / N (1)
[0079]
[0080] F=H(T) / unique N (3)
[0081] Where freq(c) represents the frequency of the feature word c in the corresponding answer and / or situation description information T, N represents the frequency of the feature word c in the knowledge base, and P c represents the probability of the feature word c appearing in the knowledge base; H(T) represents the information content of the feature word W, V represents the number of feature words W in the corresponding answer and / or situation description information T, W i represents the i-th feature word W, i = {1,…,V}; unique N Represents the total information content of all feature words in the knowledge base.
[0082] In an optional implementation, when calculating the diversity score according to the sentence dimension, Self-BLEU can be used to calculate the diversity score under the sentence dimension. For example, the diversity score can be determined by calculating the similarity between the key sentences in each answer and / or situation description information and the corresponding sentences in the knowledge base; the higher the similarity, the lower the diversity score.
[0083] In an optional implementation, when calculating the diversity score according to the semantic dimension, the cosine similarity index can be used to calculate the diversity score under the semantic dimension. For example, the diversity score can be determined by calculating the cosine similarity between the semantic feature vector of each answer and / or situation description information and the corresponding feature vector in the knowledge base; the higher the similarity, the lower the diversity score.
[0084] In the embodiment of the present application, in order to improve the accuracy of the obtained determination result, the large model technology can be further combined to perform result determination. Before determining the determination result of the answer and / or situation description information according to the first determination result, the determination method in this embodiment can also include:
[0085] The first question and the answer and / or the situation description information are input into a pre-trained large language model to obtain a third determination result of the answer and / or the situation description information. For example, the large language model (LLM) can be obtained by fine-tuning an open source LLM using sample data.
[0086] The above-mentioned determining the determination result of the answer and / or the situation description information according to the first determination result includes: determining the determination result of the answer and / or the situation description information according to the first determination result and the third determination result. In this way, the final determination result can be obtained by integrating the determination result obtained based on the large model and the determination result obtained based on the knowledge base, thereby achieving the goal of intelligent determination and improving the accuracy of determination.
[0087] Optionally, when determining the answer to the first question and / or the determination result of the situation description information based on the first determination result and the third determination result, if the first determination result and the third determination result are close / similar determination results, one of them can be selected as the answer to the first question and / or the determination result of the situation description information; if they are opposite determination results, the answer to the first question and / or the determination result of the situation description information can be determined based on the principle of minority obeys majority.
[0088] The present application is described below in conjunction with specific embodiments.
[0089] like Figure 2 As shown, a specific embodiment of the present application provides an intelligent determination system, which mainly includes a core processing module, a storage and retrieval module and a fine-tuning module, wherein the main functions of each module are as follows:
[0090] Core processing module: obtains the judgment request and processes the question, answer and / or situation description information contained in the judgment request, such as extracting key information and performing vector representation to obtain a retrieval vector, and initiates a retrieval request to the storage and retrieval module based on the retrieval vector; then, after obtaining the judgment result fed back by the storage and retrieval module and the judgment result obtained based on the pre-trained judgment LLM, all the obtained judgment results are integrated to obtain the final judgment result.
[0091] Storage and retrieval module: construct a knowledge base (i.e., vector database) based on the judgment case data (including questions, answers / situation description information, and judgment conclusions). The specific construction process can be found in the above embodiment and will not be repeated here. Initiate a search in the knowledge base based on the search vector, obtain the search result, i.e., the judgment result corresponding to the feature vector matching the search vector, and return it to the core processing module.
[0092] Fine-tuning module: Based on the judgment case data (including questions, answers / situation description information and judgment conclusions), the open source LLM is fine-tuned (such as LoRA fine-tuning) trained to perform adjustment parameters (such as Adapter parameters), weight merging, etc. to obtain the judgment LLM.
[0093] It should be noted that the determination method provided in the embodiment of the present application can be executed by a determination device or a control module in the determination device for executing the determination method. In the embodiment of the present application, the determination device provided in the embodiment of the present application is described by taking the determination device executing the determination method as an example.
[0094] See also Figure 3 , Figure 3 is a schematic diagram of a structure of a determination device provided in an embodiment of the present application, and the device is applied to electronic equipment, such as Figure 3 As shown, the determination device 30 includes:
[0095] A first acquisition module 31 is used to acquire a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information;
[0096] A processing module 32, configured to extract key information from the answer and / or situation description information in response to the determination request, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector;
[0097] A first matching module 33, used to match the first search vector with a feature vector stored in a knowledge base, and obtain a first determination result corresponding to the feature vector matching the first search vector;
[0098] The first determination module 34 is used to determine a determination result of the answer and / or situation description information according to the first determination result.
[0099] Optionally, the key information includes at least one of the following: at least one key sentence, at least one characteristic word.
[0100] Optionally, the determination device 30 further includes:
[0101] an execution module, configured to, when matching the first search vector with feature vectors stored in a knowledge base, perform at least one of the following to obtain a second search vector when no feature vector matching the first search vector is obtained or the number of feature vectors matching the first search vector obtained is less than a preset number threshold: perform feature vector representation on the key information; select at least one target key information from the key information and perform feature vector representation on the first question and the at least one target key information, or perform feature vector representation on the at least one target key information;
[0102] A second matching module, used for matching the second search vector with a feature vector stored in the knowledge base to obtain a second determination result corresponding to the feature vector matching the second search vector;
[0103] The second determination module is used to determine a determination result of the answer and / or situation description information according to the second determination result.
[0104] Optionally, the determination device 30 further includes:
[0105] A second acquisition module is used to obtain multiple answers and / or situation description information of a second question and a determination result corresponding to each answer and / or situation description information; the second question is a question stored in the knowledge base;
[0106] A calculation module, used for calculating the diversity score of each answer and / or situation description information according to the knowledge base;
[0107] A storage module is used to store a feature vector corresponding to a first answer and / or situation description information and a determination result corresponding to the first answer and / or situation description information in the knowledge base according to the calculated diversity score; the first answer and / or situation description information is an answer and / or situation description information whose corresponding diversity score among the multiple answers and / or situation description information exceeds a preset score threshold.
[0108] Optionally, the calculation module is specifically used to calculate the diversity score of each answer and / or situation description information according to the knowledge base according to at least one of the word dimension, sentence dimension and semantic dimension.
[0109] Optionally, the determination device 30 further includes:
[0110] an input module, configured to input the first question and the answer and / or the situation description information into a pre-trained large language model before determining a determination result of the answer and / or the situation description information according to the first determination result, so as to obtain a third determination result of the answer and / or the situation description information;
[0111] The first determination module 34 is specifically used to determine a determination result of the answer and / or situation description information according to the first determination result and the third determination result.
[0112] The determination device 30 of the embodiment of the present application can realize the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0113] Optional, such as Figure 4 As shown, an embodiment of the present application also provides an electronic device 40, including a processor 41, a memory 42, and a program or instruction stored in the memory 42 and executable on the processor 41. When the program or instruction is executed by the processor 41, each process of the above-mentioned determination method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0114] The present application also provides a computer program product, including computer instructions, which can achieve the above-mentioned Figure 1 The various processes of the determination method embodiment shown can achieve the same technical effect, and to avoid repetition, they will not be described again here.
[0115] The embodiment of the present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned determination method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0116] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0117] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0118] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0120] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A determination method, characterized in that: include: Obtaining a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information; In response to the determination request, extract key information from the answer and / or the situation description information, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector; Matching the first search vector with a feature vector stored in a knowledge base to obtain a first determination result corresponding to the feature vector matching the first search vector; A determination result on the answer and / or situation description information is determined based on the first determination result.
2. The method according to claim 1, characterized in that The key information includes at least one of the following: at least one key sentence and at least one characteristic word.
3. The method according to claim 1 or 2, characterized in that: If, when matching the first search vector with the feature vectors stored in the knowledge base, no feature vector matching the first search vector is obtained, or the number of feature vectors matching the first search vector obtained is less than a preset number threshold, then after matching the first search vector with the feature vectors stored in the knowledge base, the method further includes: Perform at least one of the following to obtain a second search vector: represent the key information by a feature vector; select at least one target key information from the key information, and represent the first question and the at least one target key information by a feature vector, or represent the at least one target key information by a feature vector; Matching the second search vector with a feature vector stored in the knowledge base to obtain a second determination result corresponding to the feature vector matching the second search vector; According to the second determination result, a determination result of the answer and / or situation description information is determined.
4. The method according to claim 1, characterized in that: The method further comprises: Obtaining multiple answers and / or situation description information for a second question and a determination result corresponding to each answer and / or situation description information; the second question is a question stored in the knowledge base; Calculate a diversity score for each answer and / or situation description information according to the knowledge base; According to the calculated diversity score, the feature vector corresponding to the first answer and / or situation description information and the determination result corresponding to the first answer and / or situation description information are stored in the knowledge base; wherein the first answer and / or situation description information is the answer and / or situation description information whose corresponding diversity score among the multiple answers and / or situation description information exceeds a preset score threshold.
5. The method according to claim 4, characterized in that Calculating the diversity score of each answer and / or situation description information according to the knowledge base includes: According to at least one of a word dimension, a sentence dimension and a semantic dimension, a diversity score of each answer and / or situation description information is calculated based on the knowledge base.
6. The method according to claim 1, characterized in that Before determining the determination result of the answer and / or the situation description information according to the first determination result, the method further includes: Inputting the first question and the answer and / or the situation description information into a pre-trained large language model to obtain a third determination result of the answer and / or the situation description information; Wherein, determining the determination result of the answer and / or situation description information according to the first determination result includes: A determination result on the answer and / or situation description information is determined based on the first determination result and the third determination result.
7. A determination device, characterized in that: include: A first acquisition module is used to acquire a determination request, wherein the determination request includes a first question and an answer to the first question and / or situation description information; a processing module, configured to extract key information from the answer and / or situation description information in response to the determination request, and perform feature vector representation on the first question and the key information to obtain a first retrieval vector; A first matching module, used to match the first search vector with a feature vector stored in a knowledge base, and obtain a first determination result corresponding to the feature vector matching the first search vector; The first determination module is used to determine a determination result of the answer and / or situation description information according to the first determination result.
8. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 6.
9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 6.
