Data quality inspection method and device, equipment, storage medium and program product

Abstract

The invention provides a data quality inspection method and device, equipment, a storage medium and a program product, and the method comprises the steps: generating a plurality of quality inspection replies corresponding to the to-be-inspected question and answer data based on the to-be-inspected question and answer data; wherein the quality inspection reply is used for indicating whether the question and answer data to be subjected to quality inspection is qualified; under the condition that the multiple quality inspection replies corresponding to the question and answer data to be subjected to quality inspection meet preset conditions, determining that the question and answer data to be subjected to quality inspection is qualified question and answer data; wherein the preset condition comprises that the consistency of the plurality of quality inspection replies indicates that the quality inspection of the question and answer data to be subjected to quality inspection is qualified, or the consistency of the plurality of quality inspection replies is inconsistent and the quality inspection of the question and answer data to be subjected to quality inspection is confirmed to be qualified after rechecking; the quality inspection qualified question-answer data is used for training the question-answer model. According to the method, high-quality question and answer data can be generated.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data quality inspection method, device, equipment, storage medium and program product. Background Art

[0002] With the popularization of the Internet and the advancement of informatization, online question-and-answer platforms have gradually become an important channel for the public to acquire knowledge and solve problems. However, the answers on the platform are usually generated by practitioners in professional fields, machines or other users, and it is difficult to ensure the accuracy and reliability of the answers generated on the platform. How to conduct quality inspection on question-and-answer data and then generate high-quality question-and-answer data is a problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a data quality inspection method, apparatus, device, storage medium and program product for generating high-quality question and answer data.

[0004] According to a first aspect of an embodiment of the present application, a data quality inspection method is provided, comprising:

[0005] Based on the question and answer data to be inspected, generate multiple quality inspection replies corresponding to the question and answer data to be inspected; wherein the quality inspection replies are used to indicate whether the question and answer data to be inspected has passed the quality inspection;

[0006] When the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, the question and answer data to be quality inspected is determined to be the question and answer data that has passed the quality inspection; wherein the preset conditions include the multiple quality inspection replies consistently indicating that the question and answer data to be quality inspected has passed the quality inspection, or the multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed the quality inspection; the question and answer data that has passed the quality inspection is used to train the question and answer model.

[0007] Optionally, when a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected meet a preset condition, determining that the question and answer data to be quality inspected is quality inspection qualified question and answer data includes:

[0008] In the case that the multiple quality inspection replies corresponding to the question and answer data to be inspected are inconsistent, generating reasons corresponding to the multiple quality inspection replies respectively; wherein the reasons include reasons for passing the quality inspection or reasons for failing the quality inspection;

[0009] When it is confirmed that the question and answer data to be inspected has passed the quality inspection based on the reasons corresponding to each of the plurality of quality inspection replies, the question and answer data to be inspected is determined to be the question and answer data that has passed the quality inspection.

[0010] Optionally, when the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, generating reasons corresponding to the multiple quality inspection replies respectively includes:

[0011] In the case that the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, obtaining, from the multiple quality inspection replies, a first number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and a second number of quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection;

[0012] Based on the question and answer data to be inspected, generating the first number of reasons for passing the quality inspection;

[0013] Based on the question and answer data to be inspected, generating the second number of reasons for quality inspection failure;

[0014] Based on the first number of reasons for passing the quality inspection and the second number of reasons for failing the quality inspection, reasons corresponding to each of the plurality of quality inspection replies are determined.

[0015] Optionally, in the case where the question and answer data to be inspected is confirmed to have passed the quality inspection based on the reasons corresponding to each of the plurality of quality inspection replies, determining that the question and answer data to be inspected is the question and answer data that has passed the quality inspection includes:

[0016] Based on the question and answer data to be inspected and the reasons corresponding to the plurality of quality inspection replies, a processing method for the question and answer data to be inspected is obtained through a large model;

[0017] In the case where the processing method includes manual review, obtaining a manual review result of the question and answer data to be inspected, and in the case where the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, determining that the question and answer data to be inspected is the question and answer data that has passed the quality inspection;

[0018] In a case where the processing method includes determining that the question and answer data to be inspected has passed quality inspection, the question and answer data to be inspected is determined to be question and answer data that has passed quality inspection.

[0019] Optionally, the method further comprises:

[0020] If the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, a preset value is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection;

[0021] In the case where the processing method includes determining that the question and answer data to be inspected has passed the quality inspection, based on the reasons corresponding to the question and answer data to be inspected and the plurality of quality inspection replies, the quality inspection confidence of the question and answer data that has passed the quality inspection is obtained through the large model, and the quality inspection confidence is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection;

[0022] Among them, the preset value is greater than or equal to the quality inspection confidence, and the quality weight corresponding to the quality inspection qualified question and answer data is used to train the question and answer model.

[0023] Optionally, generating a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected based on the question and answer data to be quality inspected includes:

[0024] Based on the question and answer data to be inspected, generating a first prompt word;

[0025] The first prompt word is input into the big model to obtain a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected, wherein the first prompt word is used to instruct the big model to generate a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected on the basis of ensuring diversity.

[0026] Optionally, generating a first prompt word based on the question and answer data to be inspected includes:

[0027] Based on the question and answer data to be inspected, a classification result corresponding to the question and answer data to be inspected is obtained through a template classification model; wherein the template classification model is used to predict the degree of fit between the question and answer data to be inspected and the prompt word template;

[0028] Based on the classification result corresponding to the question and answer data to be inspected, determining a target prompt word template corresponding to the question and answer data to be inspected from a plurality of prompt word templates;

[0029] The question and answer data to be inspected and the target prompt word template are spliced ​​to generate a first prompt word.

[0030] Optionally, generating the first number of quality inspection qualified reasons based on the question and answer data to be inspected includes:

[0031] Based on the question and answer data to be inspected, generating a second prompt word;

[0032] The second prompt word is input into the large model to obtain the first number of quality inspection qualified reasons, wherein the second prompt word is used to instruct the large model to generate the first number of quality inspection qualified reasons on the basis of ensuring diversity.

[0033] Optionally, generating the second number of quality inspection failure reasons based on the question and answer data to be quality inspected includes:

[0034] Based on the question and answer data to be inspected, generating a third prompt word;

[0035] The third prompt word is input into the large model to obtain the second number of quality inspection failure reasons, wherein the third prompt word is used to instruct the large model to generate the second number of quality inspection failure reasons on the basis of ensuring diversity.

[0036] According to a second aspect of an embodiment of the present application, a data quality inspection device is provided, including:

[0037] A generating unit, configured to generate a plurality of quality inspection replies corresponding to the question and answer data to be inspected based on the question and answer data to be inspected; wherein the quality inspection replies are used to indicate whether the question and answer data to be inspected has passed the quality inspection;

[0038] A processing unit is used to determine that the question and answer data to be quality inspected is a question and answer data that has passed quality inspection if a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected meet preset conditions; wherein the preset conditions include that a plurality of quality inspection replies consistently indicate that the question and answer data to be quality inspected has passed quality inspection, or that a plurality of quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed quality inspection; the question and answer data that has passed quality inspection is used to train a question and answer model.

[0039] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including a memory and a processor;

[0040] The memory is connected to the processor and is used to store programs;

[0041] The processor is used to implement the data quality inspection method as described in the first aspect by running the program in the memory.

[0042] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data quality inspection method as described in the first aspect is implemented.

[0043] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the data quality inspection method as described in the first aspect.

[0044] In the present application, based on the question and answer data to be quality inspected, multiple quality inspection replies corresponding to the question and answer data to be quality inspected are generated, wherein the quality inspection replies are used to indicate whether the question and answer data to be quality inspected has passed the quality inspection. When the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, the question and answer data to be quality inspected is determined to be the question and answer data that has passed the quality inspection, wherein the preset conditions include that the multiple quality inspection replies consistently indicate that the question and answer data to be quality inspected has passed the quality inspection, or that the multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed the quality inspection, and the question and answer data that has passed the quality inspection is used to train the question and answer model. One question and answer data to be quality inspected corresponds to multiple quality inspection replies, and multiple quality inspections are performed on one question and answer data to be quality inspected multiple times, and then the question and answer data that has passed the quality inspection is determined, which improves the accuracy of the question and answer data that has passed the quality inspection, and can generate high-quality question and answer data that has passed the quality inspection. Moreover, the question and answer data that has passed the quality inspection is used to train the question and answer model, which further improves the accuracy of the answers corresponding to the questions to be answered obtained by the question and answer model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0046] Figure 1 A schematic diagram of a data quality inspection method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a process of step 101 provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a process of step 201 provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a process of step 102 provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a process of step 401 provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of a process of step 502 provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of a process of step 503 provided in an embodiment of the present application;

[0053] Figure 8A schematic diagram of a process of step 402 provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram of the structure of a data quality inspection device provided in an embodiment of the present application;

[0055] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] With the popularization of the Internet and the advancement of informatization, online question-and-answer platforms have gradually become an important channel for the public to acquire knowledge and solve problems. However, the answers on the platform are usually generated by practitioners in professional fields, machines or other users, and it is difficult to ensure the accuracy and reliability of the answers generated on the platform. How to conduct quality inspection on question-and-answer data and then generate high-quality question-and-answer data is a problem that needs to be solved urgently.

[0057] Traditional data quality inspection methods mostly rely on manual review or rule matching technology, which has problems such as high cost, low efficiency, and poor scalability. First, manual review is inefficient, time-consuming, and costly, especially in highly professional fields such as medicine, where it is difficult to deploy reviewers on a large scale. Second, although simple automated rules can improve review efficiency to a certain extent, they lack flexibility and accuracy and are difficult to capture and process complex associations and implicit information in the data.

[0058] With the rapid development of large-scale pre-trained language models (such as GPT-4), natural language processing technologies based on these models have shown excellent performance in text understanding and generation capabilities. This provides new possibilities for efficient and accurate QA data quality inspection. At present, some studies have applied large models to professional fields such as medicine, but there are still few mature solutions for QA data quality inspection specifically for professional fields such as medicine.

[0059] In order to generate high-quality question-and-answer data, the present application provides a data quality inspection method, apparatus, device, storage medium, and program product.

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] Exemplary Implementation Environment

[0062] The data quality inspection method according to the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory.

[0063] Exemplary Methods

[0064] See also Figure 1 In an exemplary embodiment, a data quality inspection method is provided. Figure 1 As shown in the figure, the process of data quality inspection method mainly includes:

[0065] Step 101: Based on the question and answer data to be quality inspected, generate a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected.

[0066] Among them, the quality inspection reply is used to indicate whether the question and answer data to be inspected has passed the quality inspection.

[0067] In an exemplary embodiment, the question-and-answer data to be inspected may be question-and-answer data in various fields, such as the medical field, the electronic field, the mechanical field, etc., and this application does not limit this. This application explains that the question-and-answer data to be inspected is question-and-answer data in the medical field.

[0068] In an exemplary embodiment, the question-and-answer data to be inspected may include a single round of question-and-answer data, and one question-and-answer data to be inspected includes a question to be inspected and an answer to be inspected. For example, one question-and-answer data to be inspected is as follows: Question to be inspected Q: Can niacinamide really whiten the skin? Answer to be inspected A: Niacinamide generally has a certain whitening effect, but it cannot be completely relied on to whiten the skin using niacinamide. It can be whitened in other ways. Niacinamide is a water-soluble vitamin. After entering the skin, the drug can be used to inhibit the production of melanin and reduce the deposition of melanin on the skin, thereby achieving a whitening effect. However, the skin whitening effect is relatively slow and not suitable for everyone. Some people are allergic to the drug and it is generally not recommended. If you are not particularly satisfied with your skin, you can go to a regular hospital for laser whitening treatment. The principle of laser can act on the inside of the skin to decompose the melanin particles in the skin to achieve a whitening effect.

[0069] In an exemplary embodiment, the quality inspection reply may include whether the question and answer data to be quality inspected has passed the quality inspection and whether the question and answer data to be quality inspected has failed the quality inspection; the quality inspection reply may also include whether the question to be quality inspected and the answer to be quality inspected match, and whether the question to be quality inspected and the answer to be quality inspected do not match; the quality inspection reply may also include whether the answer to be quality inspected is the correct answer to the question to be quality inspected, and whether the answer to be quality inspected is the incorrect answer to the question to be quality inspected; as long as the quality inspection reply can indicate whether the question and answer data to be quality inspected has passed the quality inspection, this application does not limit the specific content of the quality inspection reply.

[0070] In some embodiments, step 101 includes: inputting the question and answer data to be quality inspected into multiple quality inspection response generation models, and obtaining quality inspection responses corresponding to each of the multiple quality inspection response generation models.

[0071] In other embodiments, Figure 2 As shown, step 101 includes:

[0072] Step 201: Generate a first prompt word based on the question and answer data to be inspected.

[0073] In some embodiments, step 201 includes: splicing the question and answer data to be inspected and a preset prompt word template to generate a first prompt word.

[0074] For example, the preset prompt word template is as follows:

[0075] "The following is a pair of knowledge questions and answers. Please judge whether the answer is correct based on the question and answer. If it contains errors, please reply "No". If it is completely correct, please reply "Yes".

[0076] Here are the questions: Q

[0077] Here is the answer: A".

[0078] In other embodiments, Figure 3 As shown, step 201 includes:

[0079] Step 301, based on the question and answer data to be inspected, a classification result corresponding to the question and answer data to be inspected is obtained through a template classification model.

[0080] Among them, the template classification model is used to predict the degree of fit between the question and answer data to be inspected and the prompt word template.

[0081] In an exemplary embodiment, the template classification model may be BERT (Bidirectional Encoder Representation from Transformers) or other models used for classification. In this application, BERT is used as an example for explanation of the template classification model.

[0082] In an exemplary embodiment, step 301 may include:

[0083] The question to be inspected Q and the answer to be inspected A in the question and answer data to be inspected are concatenated according to the following template to obtain the input M1 of the template classification model: [CLS]Q[SEP]A[SEP], where [CLS] represents the classification identifier and [SEP] represents the separation identifier;

[0084] Input M1 into the template classification model to obtain the adaptation scores corresponding to the question and answer data to be inspected and the multiple prompt word templates;

[0085] Based on the adaptation scores corresponding to the question and answer data to be inspected and the plurality of prompt word templates, a classification result corresponding to the question and answer data to be inspected is obtained.

[0086] In an exemplary embodiment, the template classification model is BERT, which is a pre-trained model after learning the cloze / context understanding task. The architecture of BERT is a multi-layer transformer-encoder network. The template classification model first converts the input M1 into a static semantic encoding vector of shape (L, E), where L is the sentence length of the input M1 and E is the vector length corresponding to each character; secondly, the sentence of the input M1 is positionally encoded, and the position encoding vector is added to the static semantic encoding vector to obtain the result of the addition; thirdly, the result of the addition is input into the self-attention module and the feedforward neural network module to obtain a static semantic representation vector of shape (L, E), and the first vector, i.e., the vector of length E at the corresponding position of [CLS] is taken as the representation vector; the representation vector is passed through a layer of fully connected neural network with an input dimension of E and an output dimension of N to obtain the adaptation scores corresponding to the question and answer data to be inspected and the multiple prompt word templates.

[0087] In an exemplary embodiment, the template classification model is BERT, BERT has a 6-layer structure, and E is 768.

[0088] In an exemplary embodiment, the template classification model is a GPT model, which is a large-scale pre-trained language model. The calculation method of the GPT model is similar to that of BERT, but the difference is that when the data in BERT is self-attention calculated, the data at each position can be calculated with the data at all positions, while when the data of the GPT model is self-attention calculated, the data at each position can only be self-attention calculated with the data before that position.

[0089] In an exemplary embodiment, the classification result corresponding to the question and answer data to be quality inspected may include the adaptation scores corresponding to the question and answer data to be quality inspected and the multiple prompt word templates respectively; the classification result corresponding to the question and answer data to be quality inspected may also include whether the question and answer data to be quality inspected and the multiple prompt word templates are adapted. If the adaptation score is greater than a preset adaptation threshold, the classification result is that the question and answer data to be quality inspected and the prompt word template are adapted; if the adaptation score is less than or equal to the preset adaptation threshold, the classification result is that the question and answer data to be quality inspected and the prompt word template are not adapted.

[0090] Step 302: Based on the classification result corresponding to the question and answer data to be inspected, a target prompt word template corresponding to the question and answer data to be inspected is determined from a plurality of prompt word templates.

[0091] In an exemplary embodiment, the plurality of prompt word templates refers to respective prompt word templates in a preset prompt word library.

[0092] In an exemplary embodiment, multiple prompt word templates can be prompt word templates for different fields, for example, prompt word templates for different fields such as cosmetics and medicines. The field keywords in different prompt word templates are different, but the purposes of different prompt word templates are the same, which is to determine whether the question and answer data to be inspected has passed the quality inspection.

[0093] In an exemplary embodiment, the prompt word template is as follows:

[0094] “The following is a pair of knowledge questions and answers mainly involving health products and cosmetics. Please judge whether the answer is correct based on the question and answer. If it contains errors, please reply "No". If it is completely correct, please reply "Yes".

[0095] Here are the questions: Q

[0096] Here is the answer: A".

[0097] In an exemplary embodiment, the training data of the template classification model includes data annotated by medical experts based on the domain adaptability of the prompt word template and the existing medical question and answer data. The loss function of the template classification model adopts the cross entropy loss function. Compared with the training data scale of the traditional large model, this part only requires a small amount of manually annotated data. The main purpose of this step is to automatically select the appropriate target prompt word template for the question and answer data to be inspected for subsequent data quality inspection based on the use of a small-scale model.

[0098] In an exemplary embodiment, when the classification result corresponding to the question and answer data to be quality inspected includes the adaptation scores corresponding to the question and answer data to be quality inspected and multiple prompt word templates, step 302 may include determining the prompt word template with the highest adaptation score as the target prompt word template corresponding to the question and answer data to be quality inspected; the classification result corresponding to the question and answer data to be quality inspected may also include whether the question and answer data to be quality inspected and multiple prompt word templates are compatible, and step 302 may include determining the prompt word template that is compatible with the question and answer data to be quality inspected as the target prompt word template corresponding to the question and answer data to be quality inspected.

[0099] Step 303: Concatenate the question and answer data to be inspected and the target prompt word template to generate a first prompt word.

[0100] Based on the question and answer data to be inspected, a classification result corresponding to the question and answer data to be inspected is obtained through a template classification model. Based on the classification result corresponding to the question and answer data to be inspected, a target prompt word template corresponding to the question and answer data to be inspected is determined from multiple prompt word templates. The question and answer data to be inspected and the target prompt word template are spliced ​​to generate a first prompt word. The system can automatically select a suitable target prompt word template for the question and answer data to be inspected to generate a first prompt word, and perform subsequent data quality inspection, thereby improving the accuracy of multiple quality inspection replies corresponding to the question and answer data to be inspected obtained according to the first prompt word.

[0101] Step 202: input the first prompt word into the large model to obtain multiple quality inspection replies corresponding to the question and answer data to be quality inspected.

[0102] Among them, the first prompt word is used to instruct the large model to generate multiple quality inspection replies corresponding to the question and answer data to be quality inspected on the basis of ensuring diversity.

[0103] In the exemplary embodiment, a large model (also called a foundation model) refers to a machine learning model with large-scale parameters and complex computing structure, which can process massive data and complete various complex tasks, such as natural language processing, computer vision, speech recognition, etc. Large models usually have billions or even hundreds of billions of parameters.

[0104] In an exemplary embodiment, the large model may specifically be a large language model (Large Language Models, LLMs for short), which refers to a deep learning model trained using a large amount of text data, and is intended to understand and generate natural language text.

[0105] In an exemplary embodiment, step 202 may be to input the first prompt word into the large model each time, generate a quality inspection reply corresponding to the question and answer data to be inspected, repeat the process of inputting the first prompt word into the large model K times, and obtain K quality inspection replies corresponding to the question and answer data to be inspected. By adjusting the parameters of the large model each time, the replies of the large model are made to generate K replies: {D1, D2, ..., DK} according to the same input, i.e., the first prompt word, while ensuring diversity as much as possible.

[0106] In an exemplary embodiment, K=10, and the large model is a large model that has been pre-trained with a large amount of medical texts, and the number of parameters generally reaches tens of billions.

[0107] Based on the question and answer data to be quality inspected, a first prompt word is generated, and the first prompt word is input into the big model to obtain multiple quality inspection replies corresponding to the question and answer data to be quality inspected, wherein the first prompt word is used to instruct the big model to generate multiple quality inspection replies corresponding to the question and answer data to be quality inspected while ensuring diversity. It can automatically generate multiple quality inspection replies corresponding to the question and answer data to be quality inspected, and improve the diversity of the multiple quality inspection replies corresponding to the question and answer data to be quality inspected.

[0108] Step 102, when the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, determine that the question and answer data to be quality inspected is the question and answer data that has passed the quality inspection.

[0109] Among them, the preset conditions include multiple quality inspection replies consistently indicating that the question and answer data to be inspected has passed the quality inspection, or multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be inspected has passed the quality inspection; the question and answer data that pass the quality inspection is used to train the question and answer model.

[0110] At present, the data quality inspection plan remains at a single evaluation stage, and only a single quality inspection is carried out on all data. That is, one quality inspection determines the quality evaluation of the data. There is no secondary or multiple quality inspection review process for some data that may be controversial, which has a certain impact on the quality of quality inspection and ignores the fact that data quality has gradient changes, and the quality of some data may need to undergo more detailed evaluation.

[0111] In the present application, one question and answer data to be quality inspected corresponds to multiple quality inspection replies, and multiple quality inspections are performed on one question and answer data to be quality inspected, forming an automated multiple quality inspection process. When multiple quality inspection replies consistently indicate that the question and answer data to be quality inspected has passed the quality inspection, the question and answer data to be quality inspected is directly determined to be the question and answer data that has passed the quality inspection. When multiple quality inspection replies are inconsistent, the question and answer data to be quality inspected is reviewed. After the review, if it is confirmed that the question and answer data to be quality inspected has passed the quality inspection, the question and answer data to be quality inspected is determined to be the question and answer data that has passed the quality inspection, thereby improving the accuracy of the question and answer data that has passed the quality inspection, and being able to generate high-quality question and answer data that has passed the quality inspection. Moreover, the question and answer data that has passed the quality inspection is used to train the question and answer model, so that the training of the question and answer model can cover a more comprehensive and higher-quality data set, further improving the accuracy of the answers corresponding to the questions to be answered obtained by the question and answer model.

[0112] In some embodiments, when multiple quality inspection replies corresponding to the question and answer data to be inspected unanimously indicate that the question and answer data to be inspected fails the quality inspection, it is determined that the question and answer data to be inspected fails the quality inspection and the question and answer data to be inspected is deleted.

[0113] In some embodiments, step 102 includes: if multiple quality inspection replies corresponding to the question and answer data to be quality inspected unanimously indicate that the question and answer data to be quality inspected has passed the quality inspection, determining that the question and answer data to be quality inspected is question and answer data that has passed the quality inspection.

[0114] When multiple quality inspection replies corresponding to the question and answer data to be inspected consistently indicate that the question and answer data to be inspected has passed the quality inspection, the question and answer data to be inspected is determined to be the question and answer data that has passed the quality inspection, which improves the accuracy of the question and answer data that has passed the quality inspection compared to a single quality inspection.

[0115] In other embodiments, Figure 4 As shown, step 102 includes:

[0116] Step 401 , when the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, generate reasons corresponding to the multiple quality inspection replies.

[0117] Among them, the reasons include reasons for passing the quality inspection or reasons for failing the quality inspection.

[0118] In some embodiments, step 401 includes: when multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, generating reasons corresponding to each of the multiple quality inspection replies based on the chain of thought.

[0119] In the exemplary embodiment, the Chain of Thought (COT) refers to a series of intermediate reasoning steps, which is a chain composed of multiple prompt words. By allowing the large model to gradually participate in the process of decomposing a complex problem into sub-problems step by step and solving them in sequence, the performance of the large model can be significantly improved. For example, by adding a preset instruction "Let's think step by step" to the first prompt word, the large model can output multiple quality inspection replies corresponding to the question and answer data to be inspected and the reasoning steps corresponding to each of the multiple quality inspection replies. The reasoning steps corresponding to each of the multiple quality inspection replies are the reasons corresponding to each of the multiple quality inspection replies.

[0120] In other embodiments, Figure 5 As shown, step 401 includes:

[0121] Step 501, when multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, obtain from the multiple quality inspection replies a first number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and a second number of quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection.

[0122] In an exemplary embodiment, the number of multiple quality inspection replies corresponding to the question and answer data to be quality inspected is K. Among the K quality inspection replies, R quality inspection replies indicate that the question and answer data to be quality inspected has passed the quality inspection, and W quality inspection replies indicate that the question and answer data to be quality inspected has failed the quality inspection. Then the first number is R and the second number is W.

[0123] Step 502: Generate a first number of quality inspection qualified reasons based on the question and answer data to be inspected.

[0124] In some embodiments, step 502 includes: inputting the question and answer data to be quality inspected into a first number of quality inspection qualified reason generation models, and obtaining the quality inspection qualified reasons corresponding to each of the first number of quality inspection qualified reason generation models.

[0125] In other embodiments, Figure 6 As shown, step 502 includes:

[0126] Step 601: Generate a second prompt word based on the question and answer data to be inspected.

[0127] In an exemplary embodiment, step 601 may include: splicing the question and answer data to be inspected and the quality inspection qualified reason prompt word template to generate a second prompt word.

[0128] In the exemplary embodiment, the quality inspection qualified reason prompt word template is as follows:

[0129] “The following is a pair of medical knowledge questions and answers. It is known that this pair of knowledge questions and answers is correct. Please write relevant reasons and justifications for the correctness of the answers while ensuring diversity as much as possible.

[0130] The following are the questions: …

[0131] Here is the answer:…”

[0132] Step 602: input the second prompt word into the large model to obtain a first number of quality inspection qualified reasons.

[0133] The second prompt word is used to instruct the large model to generate a first number of quality inspection qualified reasons on the basis of ensuring diversity.

[0134] In an exemplary embodiment, the first number is R, and step 602 may be to input the second prompt word into the large model each time, generate a quality inspection reason, repeat the process of inputting the second prompt word into the large model R times, and obtain R quality inspection reasons. By adjusting the parameters of the large model each time, the response of the large model is made to generate R quality inspection reasons {011, 012, ..., 01R} according to the same input, that is, the second prompt word, while ensuring diversity as much as possible.

[0135] Based on the question and answer data to be quality inspected, a second prompt word is generated, and the second prompt word is input into the big model to obtain a first number of reasons for passing the quality inspection, wherein the second prompt word is used to instruct the big model to generate a first number of reasons for passing the quality inspection on the basis of ensuring diversity, and can automatically generate the first number of reasons for passing the quality inspection corresponding to the question and answer data to be quality inspected, and improve the diversity of the reasons for passing the quality inspection corresponding to the question and answer data to be quality inspected.

[0136] Step 503: Generate a second number of quality inspection failure reasons based on the question and answer data to be quality inspected.

[0137] In some embodiments, step 503 includes: inputting the question and answer data to be quality inspected into a second number of quality inspection failure reason generation models, and obtaining quality inspection failure reasons corresponding to each of the second number of quality inspection failure reason generation models.

[0138] In other embodiments, Figure 7 As shown, step 503 includes:

[0139] Step 701: Generate a third prompt word based on the question and answer data to be inspected.

[0140] In an exemplary embodiment, step 701 may include: splicing the question and answer data to be quality inspected and the prompt word template of the reason for quality inspection failure to generate a third prompt word.

[0141] In an exemplary embodiment, the quality inspection failure reason prompt word template is as follows:

[0142] “The following is a pair of medical-related knowledge questions and answers. It is known that this pair of knowledge questions and answers is wrong. Please find out the errors in this pair of knowledge questions and answers while ensuring diversity as much as possible, and write relevant explanations and reasons for this error.

[0143] The following are the questions: …

[0144] Here is the answer:…”

[0145] Step 702: input the third prompt word into the large model to obtain a second number of quality inspection failure reasons.

[0146] The third prompt word is used to instruct the large model to generate a second number of quality inspection failure reasons on the basis of ensuring diversity.

[0147] In an exemplary embodiment, the second number is W, and step 702 may be to input the third prompt word into the large model each time, generate a quality inspection failure reason, repeat the process of inputting the third prompt word into the large model W times, and obtain W quality inspection failure reasons. By adjusting the parameters of the large model each time, the response of the large model is made to generate W quality inspection failure reasons: {021, 022, ..., 02W} according to the same input, that is, the third prompt word, while ensuring diversity as much as possible.

[0148] Based on the question and answer data to be quality inspected, a third prompt word is generated, and the third prompt word is input into the big model to obtain a second number of reasons for quality inspection failure, wherein the third prompt word is used to instruct the big model to generate a second number of reasons for quality inspection failure on the basis of ensuring diversity, and can automatically generate the second number of reasons for quality inspection failure corresponding to the question and answer data to be quality inspected, and improve the diversity of the reasons for quality inspection failure corresponding to the question and answer data to be quality inspected.

[0149] Step 504: Determine the reasons corresponding to each of the plurality of quality inspection replies based on the first number of reasons for passing the quality inspection and the second number of reasons for failing the quality inspection.

[0150] In an exemplary embodiment, step 504 may be to determine the first number of reasons for passing the quality inspection and the second number of reasons for failing the quality inspection as reasons corresponding to each of the multiple quality inspection replies.

[0151] When multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, a first number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and a second number of quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection are obtained from the multiple quality inspection replies, the number of reasons for passing the quality inspection is consistent with the number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and the number of reasons for failing the quality inspection is consistent with the quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection, and generating reasons corresponding to multiple quality inspection replies based on the question and answer data to be quality inspected and the corresponding quantities is simple, easy to operate, and relatively accurate.

[0152] Step 402, when it is confirmed that the question and answer data to be inspected has passed the quality inspection based on the reasons corresponding to the multiple quality inspection replies, the question and answer data to be inspected is determined to be the question and answer data that has passed the quality inspection.

[0153] Currently, data quality inspection solutions can usually only provide a basic qualified or unqualified result, lacking the explainability of the judgment process, and therefore have deficiencies in credibility and reliability.

[0154] In this application, when multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, reasons corresponding to multiple quality inspection replies are generated. By introducing an explanation mechanism, not only quality inspection replies are provided, but also the logical basis for the evaluation process, that is, the reasons corresponding to the quality inspection replies, is given, which significantly enhances the transparency and credibility of data quality inspection. When the quality inspection of the question and answer data to be quality inspected is confirmed to be qualified based on the reasons corresponding to multiple quality inspection replies, the question and answer data to be quality inspected is determined to be qualified question and answer data, thereby improving the accuracy of the qualified question and answer data.

[0155] In some embodiments, step 402 includes: obtaining a manual review result of the question and answer data to be quality inspected based on the reasons corresponding to each of the multiple quality inspection replies, and determining that the question and answer data to be quality inspected is qualified question and answer data when the manual review result of the question and answer data to be quality inspected indicates that the quality inspection of the question and answer data to be quality inspected is qualified.

[0156] In other embodiments, Figure 8 As shown, step 402 includes:

[0157] Step 801, based on the question and answer data to be quality inspected and the reasons corresponding to each of the multiple quality inspection replies, a processing method for the question and answer data to be quality inspected is obtained through a large model.

[0158] In an exemplary embodiment, step 801 may include: splicing the question and answer data to be quality inspected, the reasons corresponding to each of the multiple quality inspection replies, and a processing method prompt word template to generate a fourth prompt word, inputting the fourth prompt word into the big model, and obtaining the processing method of the question and answer data to be quality inspected, wherein the fourth prompt word is used to instruct the big model to generate a processing method for the question and answer data to be quality inspected based on the question and answer data to be quality inspected and the reasons corresponding to each of the multiple quality inspection replies.

[0159] In an exemplary embodiment, the processing prompt word template is as follows:

[0160] "The following is a pair of medical-related knowledge questions and answers. We have collected some correct and incorrect reasons for judging this pair of knowledge questions and answers. Please judge the quality of this pair of knowledge questions and answers and choose the appropriate follow-up steps for this pair of knowledge questions and answers.

[0161] The following are the questions: …

[0162] Here are the answers:…

[0163] Here are the reasons why this is correct:

[0164] xxxx

[0165] Here are the reasons why this quiz is wrong:

[0166] xxxx

[0167] Please display the output results in JSON format. If you think you are completely unable to judge the correctness based on these reasons, please output: {"Judgment result": "Manual review"};

[0168] If you think you need more reasons to judge the correctness, please output: {"judgment result": "change the prompt word template"};

[0169] If you think this is wrong for the knowledge quiz, please output: {"judgment result": "discard"};

[0170] If you think this pair of knowledge questions and answers is correct, please output your confidence in this pair of knowledge questions and answers from the range of 0-1 according to your confidence level in judging that this pair of knowledge questions and answers is correct, and output {"judgment result": "correct", "confidence level": xxx}".

[0171] Step 802, when the processing method includes manual review, obtain the manual review result of the question and answer data to be quality inspected, and when the manual review result of the question and answer data to be quality inspected indicates that the question and answer data to be quality inspected has passed the quality inspection, determine that the question and answer data to be quality inspected is the question and answer data that has passed the quality inspection.

[0172] In an exemplary embodiment, when the processing method includes manual review, that is, when the "judgment result" output by the large model is "manual review", the question and answer data to be quality inspected are submitted to a professional doctor for subsequent review, and the manual review result of the question and answer data to be quality inspected is obtained. When the manual review result of the question and answer data to be quality inspected indicates that the quality inspection of the question and answer data to be quality inspected is qualified, it is determined that the question and answer data to be quality inspected is qualified; when the manual review result of the question and answer data to be quality inspected indicates that the quality inspection of the question and answer data to be quality inspected is unqualified, it is determined that the quality inspection of the question and answer data to be quality inspected is unqualified, and the question and answer data to be quality inspected is deleted.

[0173] Step 803, when the processing method includes determining that the question and answer data to be inspected is qualified in quality inspection, determine that the question and answer data to be inspected is qualified in quality inspection.

[0174] In an exemplary embodiment, when the processing method includes determining that the question and answer data to be inspected has passed the quality inspection, that is, when the "judgment result" output by the large model is "correct" and the "confidence level": xxx, the question and answer data to be inspected is determined to be the question and answer data that has passed the quality inspection.

[0175] In an exemplary embodiment, when the processing method includes determining that the question and answer data to be inspected fails the quality inspection, that is, when the "judgment result" output by the large model is "discard", it is determined that the question and answer data to be inspected fails the quality inspection, and the question and answer data to be inspected is deleted.

[0176] In an exemplary embodiment, when the processing method includes requiring more reasons to determine whether the question and answer data to be inspected is qualified for quality inspection, that is, when the "judgment result" output by the large model is "change the prompt word template", the prompt word template with the highest adaptation score and that has not been selected is determined from multiple prompt word templates as the target prompt word template corresponding to the question and answer data to be inspected, the question and answer data to be inspected and the target prompt word template are spliced ​​to generate a first prompt word, and then subsequent steps are executed.

[0177] At present, there are only two operations for data processing after data quality inspection: retention and cleaning. It is impossible to conduct detailed quality assessment based on the data quality inspection results.

[0178] In this application, the big model intelligently schedules and selects the processing method, which is diversified and can realize automated quality inspection. Compared with traditional rules and single quality inspection, only individual data needs to be manually reviewed, which can save manpower and effectively improve the effect of data quality inspection. The big model writes reasons for the correctness and error of the data, and finally makes the judgment by the big model. It can effectively use the reasoning ability of the big model to effectively judge the correctness and make the judgment process more transparent, and can automatically generate the confidence level to facilitate subsequent training.

[0179] In some embodiments, the data quality inspection method further includes:

[0180] When the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, the preset value is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection;

[0181] In the case where the processing method includes determining that the question and answer data to be inspected has passed the quality inspection, based on the reasons corresponding to the question and answer data to be inspected and the multiple quality inspection replies, the quality inspection confidence of the question and answer data that has passed the quality inspection is obtained through the big model, and the quality inspection confidence is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection;

[0182] Among them, the preset value is greater than or equal to the quality inspection confidence level, and the quality weight corresponding to the question and answer data that passes the quality inspection is used to train the question and answer model.

[0183] In an exemplary embodiment, the preset value is 1.0, and the quality inspection confidence is a value in the range of 0-1.

[0184] In an exemplary embodiment, when the processing method includes determining that the question and answer data to be inspected has passed the quality inspection, that is, the "judgment result" output by the large model is "correct", and the "confidence level" is xxx, if the confidence level is a value in the range of 0-1, the confidence level is determined as the quality inspection confidence level of the question and answer data that has passed the quality inspection; if the confidence level is not in the range of 0-1, for example, the confidence level is 2, -1, etc., it means that the data is abnormal, and the quality inspection confidence level of the question and answer data that has passed the quality inspection can be directly set to 0.5.

[0185] Currently, data quality inspection solutions often only provide simple binary (pass / fail) results, failing to fully utilize the judgment potential of the model.

[0186] In this application, by implementing a fine-grained data quality scoring mechanism based on model quality inspection confidence, the quality level of the data can be divided more accurately, and richer quality inspection information can be provided, thereby providing more targeted support for the subsequent processing and application of the data. Weights are set for the quality of the data so that more samples of higher quality data can be collected in subsequent training, thereby improving the effect of the model. Moreover, the quality weight that is manually reviewed correctly is definitely greater than or equal to the quality weight that is confirmed correct by the model.

[0187] In some embodiments, the data quality inspection method further includes: when multiple quality inspection replies corresponding to the question and answer data to be inspected unanimously indicate that the question and answer data to be inspected has passed the quality inspection, determining a preset value as the quality weight corresponding to the question and answer data that has passed the quality inspection.

[0188] In an exemplary embodiment, the preset value is 1.0.

[0189] In summary, in the present application, based on the question and answer data to be quality inspected, multiple quality inspection replies corresponding to the question and answer data to be quality inspected are generated, wherein the quality inspection reply is used to indicate whether the question and answer data to be quality inspected has passed the quality inspection, and when the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, the question and answer data to be quality inspected is determined to be the question and answer data that has passed the quality inspection, wherein the preset conditions include that multiple quality inspection replies consistently indicate that the question and answer data to be quality inspected has passed the quality inspection, or that multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed the quality inspection, and the question and answer data that has passed the quality inspection is used to train the question and answer model. One question and answer data to be quality inspected corresponds to multiple quality inspection replies, and multiple quality inspections are performed on one question and answer data to be quality inspected multiple times, and then the question and answer data that has passed the quality inspection is determined, which improves the accuracy of the question and answer data that has passed the quality inspection, and can generate high-quality question and answer data that has passed the quality inspection. Moreover, the question and answer data that has passed the quality inspection is used to train the question and answer model, which further improves the accuracy of the answers corresponding to the questions to be answered obtained by the question and answer model.

[0190] Exemplary Devices

[0191] Accordingly, the present application also provides a data quality inspection device, such as Fig. 9 As shown, the data quality inspection device includes:

[0192] A generating unit 901 is used to generate a plurality of quality inspection replies corresponding to the question and answer data to be inspected based on the question and answer data to be inspected; wherein the quality inspection replies are used to indicate whether the question and answer data to be inspected is qualified for quality inspection;

[0193] Processing unit 902 is used to determine that the question and answer data to be quality inspected is a question and answer data that has passed quality inspection if multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet preset conditions; wherein the preset conditions include multiple quality inspection replies consistently indicating that the question and answer data to be quality inspected has passed quality inspection, or multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed quality inspection; the question and answer data that has passed quality inspection is used to train the question and answer model.

[0194] Optionally, the processing unit 902 includes:

[0195] A first generating subunit is used to generate a plurality of reasons corresponding to each of the quality inspection replies when the plurality of quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent; wherein the reasons include a reason for passing the quality inspection or a reason for failing the quality inspection;

[0196] The first processing sub-unit is used to determine that the question and answer data to be quality inspected is the question and answer data that has passed the quality inspection, if the question and answer data to be quality inspected is confirmed to have passed the quality inspection based on the reasons corresponding to each of the plurality of quality inspection replies.

[0197] Optionally, the first generating subunit is specifically used for:

[0198] In the case that the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, obtaining, from the multiple quality inspection replies, a first number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and a second number of quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection;

[0199] Based on the question and answer data to be inspected, generating the first number of reasons for passing the quality inspection;

[0200] Based on the question and answer data to be inspected, generating the second number of reasons for quality inspection failure;

[0201] Based on the first number of reasons for passing the quality inspection and the second number of reasons for failing the quality inspection, reasons corresponding to each of the plurality of quality inspection replies are determined.

[0202] Optionally, the first processing subunit is specifically configured to:

[0203] Based on the question and answer data to be inspected and the reasons corresponding to the plurality of quality inspection replies, a processing method for the question and answer data to be inspected is obtained through a large model;

[0204] In the case where the processing method includes manual review, obtaining a manual review result of the question and answer data to be inspected, and in the case where the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, determining that the question and answer data to be inspected is the question and answer data that has passed the quality inspection;

[0205] In a case where the processing method includes determining that the question and answer data to be inspected has passed quality inspection, the question and answer data to be inspected is determined to be question and answer data that has passed quality inspection.

[0206] Optionally, the data quality inspection device further includes a first weight determination unit and a second weight determination unit;

[0207] A first weight determination unit is used to determine a preset value as a quality weight corresponding to the question and answer data that has passed the quality inspection, if the manual review result of the question and answer data to be quality inspected indicates that the question and answer data to be quality inspected has passed the quality inspection;

[0208] A second weight determination unit is used for, when the processing method includes determining that the question and answer data to be quality inspected is qualified, obtaining the quality inspection confidence of the qualified question and answer data through the large model based on the reasons corresponding to the question and answer data to be quality inspected and the plurality of quality inspection replies, and determining the quality inspection confidence as the quality weight corresponding to the qualified question and answer data;

[0209] Among them, the preset value is greater than or equal to the quality inspection confidence, and the quality weight corresponding to the quality inspection qualified question and answer data is used to train the question and answer model.

[0210] Optionally, the generating unit 901 includes:

[0211] A second generating subunit is used to generate a first prompt word based on the question and answer data to be inspected;

[0212] The second processing sub-unit is used to input the first prompt word into the big model to obtain multiple quality inspection replies corresponding to the question and answer data to be quality inspected, wherein the first prompt word is used to instruct the big model to generate multiple quality inspection replies corresponding to the question and answer data to be quality inspected on the basis of ensuring diversity.

[0213] Optionally, the second generating subunit is specifically used for:

[0214] Based on the question and answer data to be inspected, a classification result corresponding to the question and answer data to be inspected is obtained through a template classification model; wherein the template classification model is used to predict the degree of fit between the question and answer data to be inspected and the prompt word template;

[0215] Based on the classification result corresponding to the question and answer data to be inspected, determining a target prompt word template corresponding to the question and answer data to be inspected from a plurality of prompt word templates;

[0216] The question and answer data to be inspected and the target prompt word template are spliced ​​to generate a first prompt word.

[0217] Optionally, the first generating subunit is specifically used for:

[0218] Based on the question and answer data to be inspected, generating a second prompt word;

[0219] The second prompt word is input into the large model to obtain the first number of quality inspection qualified reasons, wherein the second prompt word is used to instruct the large model to generate the first number of quality inspection qualified reasons on the basis of ensuring diversity.

[0220] Optionally, the first generating subunit is specifically used for:

[0221] Based on the question and answer data to be inspected, generating a third prompt word;

[0222] The third prompt word is input into the large model to obtain the second number of quality inspection failure reasons, wherein the third prompt word is used to instruct the large model to generate the second number of quality inspection failure reasons on the basis of ensuring diversity.

[0223] The data quality inspection device provided in this embodiment belongs to the same application concept as the data quality inspection method provided in the above embodiments of this application, and can execute the data quality inspection method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the data quality inspection method. For technical details not fully described in this embodiment, please refer to the specific processing content of the data quality inspection method provided in the above embodiments of this application, and will not be repeated here.

[0224] The functions implemented by the above generation unit 901 and the processing unit 902 may be implemented by the same or different processors, respectively, and are not limited in the embodiments of the present application.

[0225] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0226] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0227] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0228] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0229] Exemplary Electronic Devices

[0230] An embodiment of the present application provides an electronic device, see Fig.10 As shown, the device includes:

[0231] Memory 200 and processor 210;

[0232] The memory 200 is connected to the processor 210 and is used to store programs;

[0233] The processor 210 is used to implement the data quality inspection method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0234] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0235] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0236] A bus may include a pathway that transfers information between components of a computer system.

[0237] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0238] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0239] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0240] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0241] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0242] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0243] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any data quality inspection method provided in the above embodiments of the present application.

[0244] Exemplary computer program products and storage media

[0245] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the data quality inspection method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0246] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0247] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the data quality inspection method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0248] Step 101: Based on the question and answer data to be quality inspected, generate a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected.

[0249] Among them, the quality inspection reply is used to indicate whether the question and answer data to be inspected has passed the quality inspection.

[0250] Step 102, when the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, determine that the question and answer data to be quality inspected is the question and answer data that has passed the quality inspection.

[0251] Among them, the preset conditions include multiple quality inspection replies consistently indicating that the question and answer data to be inspected has passed the quality inspection, or multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be inspected has passed the quality inspection; the question and answer data that pass the quality inspection is used to train the question and answer model.

[0252] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0253] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0254] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0255] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0256] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0257] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0258] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0259] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0260] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0261] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0262] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data quality inspection method, characterized in that: include: Based on the question and answer data to be inspected, generate multiple quality inspection replies corresponding to the question and answer data to be inspected; wherein the quality inspection replies are used to indicate whether the question and answer data to be inspected has passed the quality inspection; When the multiple quality inspection replies corresponding to the question and answer data to be quality inspected meet the preset conditions, the question and answer data to be quality inspected is determined to be the question and answer data that has passed the quality inspection; wherein the preset conditions include the multiple quality inspection replies consistently indicating that the question and answer data to be quality inspected has passed the quality inspection, or the multiple quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed the quality inspection; the question and answer data that has passed the quality inspection is used to train the question and answer model.

2. The data quality inspection method according to claim 1, characterized in that: When the plurality of quality inspection replies corresponding to the question and answer data to be inspected meet a preset condition, determining that the question and answer data to be inspected is the question and answer data that has passed the quality inspection includes: In the case that the multiple quality inspection replies corresponding to the question and answer data to be inspected are inconsistent, generating reasons corresponding to the multiple quality inspection replies respectively; wherein the reasons include reasons for passing the quality inspection or reasons for failing the quality inspection; When it is confirmed that the question and answer data to be inspected has passed the quality inspection based on the reasons corresponding to each of the plurality of quality inspection replies, the question and answer data to be inspected is determined to be the question and answer data that has passed the quality inspection.

3. The data quality inspection method according to claim 2, characterized in that: When the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, generating reasons corresponding to the multiple quality inspection replies respectively includes: In the case that the multiple quality inspection replies corresponding to the question and answer data to be quality inspected are inconsistent, obtaining, from the multiple quality inspection replies, a first number of quality inspection replies indicating that the question and answer data to be quality inspected has passed the quality inspection, and a second number of quality inspection replies indicating that the question and answer data to be quality inspected has failed the quality inspection; Based on the question and answer data to be inspected, generating the first number of reasons for passing the quality inspection; Based on the question and answer data to be inspected, generating the second number of reasons for quality inspection failure; Based on the first number of reasons for passing the quality inspection and the second number of reasons for failing the quality inspection, reasons corresponding to each of the plurality of quality inspection replies are determined.

4. The data quality inspection method according to claim 2, characterized in that: In the case where the question and answer data to be inspected is confirmed to be qualified in quality inspection based on the reasons corresponding to the multiple quality inspection replies, determining that the question and answer data to be inspected is qualified question and answer data includes: Based on the question and answer data to be inspected and the reasons corresponding to the plurality of quality inspection replies, a processing method for the question and answer data to be inspected is obtained through a large model; In the case where the processing method includes manual review, obtaining a manual review result of the question and answer data to be inspected, and in the case where the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, determining that the question and answer data to be inspected is the question and answer data that has passed the quality inspection; In a case where the processing method includes determining that the question and answer data to be inspected has passed quality inspection, the question and answer data to be inspected is determined to be question and answer data that has passed quality inspection.

5. The data quality inspection method according to claim 4, characterized in that: The method further comprises: If the manual review result of the question and answer data to be inspected indicates that the question and answer data to be inspected has passed the quality inspection, a preset value is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection; In the case where the processing method includes determining that the question and answer data to be inspected has passed the quality inspection, based on the reasons corresponding to the question and answer data to be inspected and the plurality of quality inspection replies, the quality inspection confidence of the question and answer data that has passed the quality inspection is obtained through the large model, and the quality inspection confidence is determined as the quality weight corresponding to the question and answer data that has passed the quality inspection; Among them, the preset value is greater than or equal to the quality inspection confidence, and the quality weight corresponding to the quality inspection qualified question and answer data is used to train the question and answer model.

6. The data quality inspection method according to claim 1, characterized in that: The step of generating a plurality of quality inspection replies corresponding to the question and answer data to be inspected based on the question and answer data to be inspected includes: Based on the question and answer data to be inspected, generating a first prompt word; The first prompt word is input into the big model to obtain a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected, wherein the first prompt word is used to instruct the big model to generate a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected on the basis of ensuring diversity.

7. The data quality inspection method according to claim 6, characterized in that: The step of generating a first prompt word based on the question and answer data to be inspected includes: Based on the question and answer data to be inspected, a classification result corresponding to the question and answer data to be inspected is obtained through a template classification model; wherein the template classification model is used to predict the degree of fit between the question and answer data to be inspected and the prompt word template; Based on the classification result corresponding to the question and answer data to be inspected, determining a target prompt word template corresponding to the question and answer data to be inspected from a plurality of prompt word templates; The question and answer data to be inspected and the target prompt word template are spliced ​​to generate a first prompt word.

8. The data quality inspection method according to claim 3, characterized in that: The generating the first number of quality inspection qualified reasons based on the question and answer data to be inspected includes: Based on the question and answer data to be inspected, generating a second prompt word; The second prompt word is input into the large model to obtain the first number of quality inspection qualified reasons, wherein the second prompt word is used to instruct the large model to generate the first number of quality inspection qualified reasons on the basis of ensuring diversity.

9. The data quality inspection method according to claim 3, characterized in that: The generating the second number of quality inspection failure reasons based on the question and answer data to be inspected includes: Based on the question and answer data to be inspected, generating a third prompt word; The third prompt word is input into the large model to obtain the second number of quality inspection failure reasons, wherein the third prompt word is used to instruct the large model to generate the second number of quality inspection failure reasons on the basis of ensuring diversity.

10. A data quality inspection device, characterized in that: include: A generating unit, configured to generate a plurality of quality inspection replies corresponding to the question and answer data to be inspected based on the question and answer data to be inspected; wherein the quality inspection replies are used to indicate whether the question and answer data to be inspected has passed the quality inspection; A processing unit is used to determine that the question and answer data to be quality inspected is a question and answer data that has passed quality inspection if a plurality of quality inspection replies corresponding to the question and answer data to be quality inspected meet preset conditions; wherein the preset conditions include that a plurality of quality inspection replies consistently indicate that the question and answer data to be quality inspected has passed quality inspection, or that a plurality of quality inspection replies are inconsistent and after review, it is confirmed that the question and answer data to be quality inspected has passed quality inspection; the question and answer data that has passed quality inspection is used to train a question and answer model.

11. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the data quality inspection method according to any one of claims 1 to 9 by running the program in the memory.

12. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the data quality inspection method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to execute the data quality inspection method according to any one of claims 1 to 9.

Images