Model defect detection method, device and equipment and storage medium

Abstract

The invention discloses a model defect detection method and device, equipment and a storage medium. The method comprises the following steps: establishing an optimized training data sample set for a pre-established question and answer model; wherein the optimization training data sample set comprises multiple pieces of text data; performing word and sentence transformation on each piece of text data to obtain transformation data corresponding to each piece of text data; inputting each piece of text data and each piece of transformation data into a question and answer model, so as to determine each piece of target data from each piece of transformation data; and summarizing the target data to obtain a defect sample set corresponding to the question and answer model. According to the method, the target data is determined according to the output of the model for the text data and the transformation data, and the target data is summarized into the set to serve as the defect sample set for the question and answer model, so that defect detection of the question and answer model is completed, the defect sample set can be subsequently utilized to optimize the question and answer model to eliminate defects, and the detection efficiency is improved. Therefore, the comprehensiveness of the question and answer model can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of model defect detection, and in particular to a model defect detection method, device, equipment and storage medium. Background Art

[0002] In today's era of rapid digital development, various models play a vital role in many fields. By inputting data into the model, the model can quickly output accurate results. However, with the widespread application of the model, the defects of the model are becoming increasingly prominent.

[0003] The traditional model building process often focuses on improving the accuracy and performance of the model, but pays insufficient attention to potential defects. For example, after the model training is completed, no attention is paid to the subsequent application of the model. However, after the model is built and trained, the subsequent defect detection and optimization process is also crucial, otherwise it will be detrimental to the comprehensiveness of the model. Summary of the invention

[0004] In view of this, the present application provides a model defect detection method, device, equipment and storage medium for performing defect detection on the model, which can detect the defects of the model and help improve the comprehensiveness of the model.

[0005] To achieve the above objectives, the proposed solution is as follows:

[0006] In a first aspect, a model defect detection method comprises:

[0007] Establishing an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data;

[0008] Performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data;

[0009] Inputting each of the text data and each of the transformed data into the question-answering model respectively, so as to determine each of the target data from each of the transformed data;

[0010] The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

[0011] Preferably, the step of performing word or sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data comprises:

[0012] For each piece of text data, split the text data to obtain each initial character;

[0013] Determining the type of each of the initial characters;

[0014] Using each initial character of a preset Chinese character symbol type as each first character;

[0015] Using each initial character of a preset word type as each second character;

[0016] The text data is transformed based on each of the first characters and each of the second characters to obtain transformed data corresponding to the text data.

[0017] Preferably, the transforming the piece of text data based on each of the first characters and each of the second characters to obtain the transformed data corresponding to the piece of text data includes:

[0018] For each of the first characters, selecting each synonymous character corresponding to the first character from an open source database;

[0019] Calculating the similarity between each of the synonymous characters and the first character respectively;

[0020] The synonym with the smallest similarity is used as the target synonym corresponding to the first character;

[0021] For each of the second characters, selecting synonyms corresponding to the second character from an open source database;

[0022] Calculating the similarity between each of the synonyms and the second character respectively;

[0023] The synonym with the smallest similarity is used as the target synonym corresponding to the second character;

[0024] The text data is transformed based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain transformed data corresponding to the text data.

[0025] Preferably, the step of transforming the text data based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain transformed data corresponding to the text data includes:

[0026] Replacing each of the first characters in the text data with a corresponding target synonym;

[0027] replacing each of the second characters in the text data with a corresponding target synonym;

[0028] The replaced text data is regarded as the conversion data corresponding to the text data.

[0029] Preferably, the step of inputting each of the text data and each of the transformed data into the question-answering model to determine each of the target data from each of the transformed data comprises:

[0030] Selecting one piece of text data from each piece of text data as the first sentence;

[0031] Inputting the first sentence into the question-answering model, so that the question-answering model outputs first answer data according to the first sentence;

[0032] Inputting the transformed data corresponding to the first sentence into the question-answering model, so that the question-answering model outputs second question-answering data according to the transformed data corresponding to the first sentence;

[0033] If the second question-and-answer data is different from the first question-and-answer data, using the transformed data corresponding to the first sentence as the target data;

[0034] Returning to the step of selecting a piece of text data from each piece of the text data as the first sentence, until each piece of the text data has been selected, so as to obtain one or more pieces of target data;

[0035] The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

[0036] Preferably, before inputting each of the text data and each of the transformed data into the question-answering model, the method further includes: performing similarity expansion on each of the transformed data, including:

[0037] For each piece of the transformed data, split the piece of transformed data to obtain each third character;

[0038] determining one or more key characters from each of the third characters;

[0039] Determining a current application field of the question-answering model, and determining a noun type of each of the key characters according to the current application field;

[0040] For each of the key characters, a plurality of extended characters corresponding to the key character are set;

[0041] randomly selecting an extended character from the plurality of extended characters as a fourth character;

[0042] In the piece of transformed data, the key character is replaced with the fourth character to complete the similarity extension.

[0043] In a second aspect, a model defect detection device includes:

[0044] A sample set building module, used to build an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data;

[0045] A sentence transformation module, used for performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data;

[0046] A target data obtaining module, used for inputting each of the text data and each of the transformed data into the question-answering model, so as to determine each of the target data from each of the transformed data;

[0047] The summarizing module is used to summarize the target data to obtain a set of defect samples corresponding to the question-answering model.

[0048] Preferably, the sentence transformation module includes:

[0049] A splitting module, for splitting each piece of text data to obtain each initial character;

[0050] A type determination module, used to determine the type of each of the initial characters;

[0051] A first character determination module, used for taking each initial character belonging to a preset Chinese character symbol type as each first character;

[0052] A second character determination module, used for taking each initial character belonging to a preset word type as each second character;

[0053] The transformation module is used to transform the text data based on each of the first characters and each of the second characters to obtain transformation data corresponding to the text data.

[0054] In a third aspect, a model defect detection device includes a memory and a processor;

[0055] The memory is used to store programs;

[0056] The processor is used to execute the program to implement each step of the model defect detection method as described in any one of the first aspects.

[0057] In a fourth aspect, a storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the model defect detection method as described in any one of the first aspects are implemented.

[0058] It can be seen from the above technical scheme that the present application establishes an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set contains multiple text data; each of the text data is transformed into words and sentences respectively to obtain transformed data corresponding to each of the text data; each of the text data and each of the transformed data is input into the question-answering model respectively to determine each target data from each of the transformed data; each of the target data is summarized to obtain a defect sample set corresponding to the question-answering model. The present application performs defect detection on a pre-built question-and-answer model. First, an optimized training data sample set is established. This sample set includes multiple text data. The question-and-answer model can output a result based on the text data. The result is the answer to the text data. In order to detect defects in the model, the present application performs word and sentence transformation on each text data to obtain transformed data, thereby determining the output of the question-and-answer model for the transformed data, and determining each target data based on the model's output for the text data and the transformed data respectively. The target data are aggregated into a set as a defect sample set for the question-and-answer model, thereby completing defect detection for the question-and-answer model. The defect sample set can then be used to optimize the question-and-answer model to eliminate defects, thereby improving the comprehensiveness of the question-and-answer model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0060] Figure 1 An optional flow chart of a model defect detection method provided in an embodiment of the present application;

[0061] Figure 2 An optional flow chart of another model defect detection method provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the structure of a model defect detection device provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of the structure of a model defect detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] In today's era of rapid digital development, various models play a vital role in many fields. By inputting data into the model, the model can quickly output accurate results. However, with the widespread application of the model, the defects of the model are becoming increasingly prominent.

[0066] The traditional model building process often focuses on improving the accuracy and performance of the model, but pays insufficient attention to potential defects. For example, after the model training is completed, no attention is paid to the subsequent application of the model. However, after the model is built and trained, the subsequent defect detection and optimization process is also crucial, otherwise it will be detrimental to the comprehensiveness of the model.

[0067] The present invention can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or devices, etc.

[0068] The embodiment of the present invention provides a model defect detection method, which can be applied to various computer terminals or smart terminals, and its execution subject can be a processor or server of the computer terminal or smart terminal. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0069] S1: Establish an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes multiple text data.

[0070] The pre-built question-and-answer model in this application refers to a model into which users can input question-and-answer statements, text data and other information, and can output corresponding answers or responses. The input of the model is text, the output is also text, and the output text is the model's answer to the input text.

[0071] The optimized training data sample set also includes text data. The text data set in this application can be input into the question-answering model, and the question-answering model can output results based on the text data.

[0072] In one example, a question-and-answer model is a model that can identify user intent. User intent refers to the user's intention to ask a question. For example, the user wants to know the area of ​​a city, wants to know who the famous singers are, what the quality of this product is, etc. Then the model can identify the user's intent as other scenario questions and answers - geographical knowledge, other scenario questions and answers - brand information, and product knowledge questions and answers - evaluation.

[0073] S2: Perform word or sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data.

[0074] The word and sentence transformation of text data is not limited to replacing the text data with synonyms, replacing synonymous characters, changing the order of words, changing the order of subject, predicate and object, randomly disrupting the order of words without changing the meaning, etc., to ensure that the transformed data corresponds to the text data.

[0075] The purpose of the transformation is to detect whether the question-answering model can output the same answer when the input data is different but the meaning / meaning is the same, so as to determine whether the question-answering model has corresponding defects under this text data.

[0076] S3: Inputting each of the text data and each of the transformed data into the question-answering model respectively, so as to determine each of the target data from each of the transformed data.

[0077] After the transformation is completed, the original text data is input into the question-answering model, and the question-answering model outputs the answer. Then the transformed data is input into the question-answering model, and the question-answering model also outputs the answer. Specifically, it can be determined whether the two answers are the same. If they are the same, it means that the question-answering model can output the same answer for two different but identical data. If they are different, it means that the question-answering model cannot output the same answer for two different but identical data. In this case, it can be considered that the question-answering model has defects within the scope of this text data / transformed data, and the transformed data is determined as the target data.

[0078] S4: Aggregate the target data to obtain a defect sample set corresponding to the question-answering model.

[0079] These target data are sentences that the question-answering model cannot accurately recognize / cannot accurately answer. Therefore, these target data can be summarized into a data set as the defect sample set corresponding to the question-answering model. The defect sample set can then be used to optimize the training of the question-answering model to improve the accuracy of question answering.

[0080] It can be seen from the above technical scheme that the present application establishes an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set contains multiple text data; each of the text data is transformed into words and sentences respectively to obtain transformed data corresponding to each of the text data; each of the text data and each of the transformed data is input into the question-answering model respectively to determine each target data from each of the transformed data; each of the target data is summarized to obtain a defect sample set corresponding to the question-answering model. The present application performs defect detection on a pre-built question-and-answer model. First, an optimized training data sample set is established. This sample set includes multiple text data. The question-and-answer model can output a result based on the text data. The result is the answer to the text data. In order to detect defects in the model, the present application performs word and sentence transformation on each text data to obtain transformed data, thereby determining the output of the question-and-answer model for the transformed data, and determining each target data based on the model's output for the text data and the transformed data respectively. The target data are aggregated into a set as a defect sample set for the question-and-answer model, thereby completing defect detection for the question-and-answer model. The defect sample set can then be used to optimize the question-and-answer model to eliminate defects, thereby improving the comprehensiveness of the question-and-answer model.

[0081] In the method provided by the embodiment of the present invention, each of the text data is transformed into words and sentences to obtain the transformation data corresponding to each of the text data. Figure 2 The specific instructions are as follows:

[0082] S21: For each piece of text data, split the text data to obtain initial characters;

[0083] S22: Determine the type of each of the initial characters;

[0084] S23: taking each initial character belonging to a preset Chinese character symbol type as each first character;

[0085] S24: using each initial character belonging to a preset word type as each second character;

[0086] S25: transforming the piece of text data based on each of the first characters and each of the second characters to obtain transformed data corresponding to the piece of text data.

[0087] Specifically, this step mainly introduces the process of transforming text data into words and sentences, wherein each text data needs to be transformed into words and sentences, so as to obtain the transformation data corresponding to each text data. The following is a detailed description for a text data: the text data is split, and each initial character can be obtained after the split, and the type of each initial character needs to be determined. It can be considered that the types to which the initial characters belong include Chinese character symbol type, punctuation mark type, word type, and other character types (types of other characters that do not belong to Chinese characters, punctuation marks, and words). Generally, it is considered that punctuation mark types and other character types are not suitable for word and sentence transformation. Therefore, this step first takes the initial character belonging to the Chinese character symbol type as the first character, and the initial character belonging to the word type as the second character. Only the first character and the second character are transformed, and the transformed text data is the transformed data.

[0088] The process of transforming the piece of text data based on each of the first characters and each of the second characters to obtain transformed data corresponding to the piece of text data may specifically include:

[0089] For each of the first characters, selecting each synonymous character corresponding to the first character from an open source database;

[0090] Calculating the similarity between each of the synonymous characters and the first character respectively;

[0091] The synonym with the smallest similarity is used as the target synonym corresponding to the first character;

[0092] For each of the second characters, selecting synonyms corresponding to the second character from an open source database;

[0093] Calculating the similarity between each of the synonyms and the second character respectively;

[0094] The synonym with the smallest similarity is used as the target synonym corresponding to the second character;

[0095] The text data is transformed based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain transformed data corresponding to the text data.

[0096] Specifically, there are many characters stored in the open source database, such as Chinese characters, words, idioms, etc. For a first character, the synonyms corresponding to the first character can be screened from the open source database, because there are many synonyms close to the first character. The screening method can be to use the characters of the same type and meaning as the first character as synonyms, or to calculate the matching degree and use the characters with a matching degree greater than a preset matching threshold as synonyms; then the similarity of the determined synonyms and the first character is calculated respectively, and the synonym with the smallest similarity is used as the target synonym; the same is true for synonyms. The selection of synonyms and synonyms with the smallest similarity is to detect whether the question-answering model can identify and detect the most marginal and least similar synonyms or synonyms, and to detect whether the question-answering model can accurately identify and detect the most dissimilar sentences among multiple sentences similar to the text data, so that resources and time can be saved. If the question-answering model can accurately identify and detect, there is no need to identify and detect sentences composed of other synonyms or synonyms. If it cannot be accurately identified, it is necessary to monitor step by step.

[0097] Furthermore, the step of transforming the text data based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters in the above process to obtain transformed data corresponding to the text data is explained in detail below:

[0098] Replacing each of the first characters in the text data with a corresponding target synonym;

[0099] replacing each of the second characters in the text data with a corresponding target synonym;

[0100] The replaced text data is regarded as the conversion data corresponding to the text data.

[0101] Specifically, in the above process, the target synonymous characters and target synonymous words have been determined, and then the corresponding first characters and second characters in the text data are replaced to obtain the transformed data corresponding to the text data.

[0102] Next, the process of inputting each of the text data and each of the transformed data into the question-answering model to determine each of the target data from each of the transformed data in the present application is specifically described.

[0103] Selecting one piece of text data from each piece of text data as the first sentence;

[0104] Inputting the first sentence into the question-answering model, so that the question-answering model outputs first answer data according to the first sentence;

[0105] Inputting the transformed data corresponding to the first sentence into the question-answering model, so that the question-answering model outputs second question-answering data according to the transformed data corresponding to the first sentence;

[0106] If the second question-and-answer data is different from the first question-and-answer data, using the transformed data corresponding to the first sentence as the target data;

[0107] Returning to the step of selecting a piece of text data from each piece of the text data as the first sentence, until each piece of the text data has been selected, so as to obtain one or more pieces of target data;

[0108] The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

[0109] Specifically, it should be noted that in this step, when comparing the second question and answer data with the first question and answer data, it is necessary to determine whether the second question and answer data is exactly the same as the first question and answer data. If not, it is considered that the question and answer model cannot accurately identify the data. Therefore, the transformed data corresponding to the second question and answer data is used as the target data, and it is considered that the question and answer data has defects in this target data. Next, continue to judge other text data in the optimized training data sample set, and if the two answers are exactly the same, then judge other text data in the same way until all other text data in the optimized training data sample set have been selected, thereby preventing omissions and improving the accuracy of defect detection of the question and answer model.

[0110] Furthermore, before inputting each of the text data and each of the transformed data into the question-answering model, the present application further comprises the following steps: for each of the transformed data, splitting the transformed data to obtain each third character;

[0111] determining one or more key characters from each of the third characters;

[0112] Determining a current application field of the question-answering model, and determining a noun type of each of the key characters according to the current application field;

[0113] For each of the key characters, a plurality of extended characters corresponding to the key character are set;

[0114] randomly selecting an extended character from the plurality of extended characters as a fourth character;

[0115] In the piece of transformed data, the key character is replaced with the fourth character to complete the similarity extension.

[0116] Specifically, in order to improve the detection accuracy of the question-answering model, after the text data is transformed into the transformed data, the transformed data is converted again to a certain extent, so as to increase the question-answering difficulty of the question-answering model, so as to detect the more hidden defects of the question-answering model, and split it into each third character, from which the key characters are determined. The key characters can be understood as the characters in the transformed data that can represent the overall meaning of the transformed data. Starting from the current application field, the key characters are expanded, and then one is randomly selected from the expanded characters as the fourth character to increase randomness and ensure detection accuracy. Finally, the key character is replaced with the fourth character, thereby completing the similarity expansion and obtaining a second-transformed data, so that the question-answering model processes the text data and the second-transformed data, and outputs the answer, which is compared to obtain the final defect sample set.

[0117] and Figure 1 Corresponding to the method described above, the embodiment of the present invention also provides a model defect detection device for Figure 1 In the specific implementation of the method, the model defect detection device provided by the embodiment of the present invention can be used in a computer terminal or various mobile devices, combined with Figure 3 , the model defect detection device is introduced, such as Figure 3 As shown, the device may include:

[0118] The sample set establishment module 10 is used to establish an optimized training data sample set for the pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data;

[0119] A sentence transformation module 20, used to transform each piece of text data into words and sentences to obtain transformation data corresponding to each piece of text data;

[0120] A target data obtaining module 30, for inputting each of the text data and each of the transformed data into the question-answering model, so as to determine each of the target data from each of the transformed data;

[0121] The summarizing module 40 is used to summarize the target data to obtain a defect sample set corresponding to the question-answering model.

[0122] It can be seen from the above technical scheme that the present application establishes an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set contains multiple text data; each of the text data is transformed into words and sentences respectively to obtain transformed data corresponding to each of the text data; each of the text data and each of the transformed data is input into the question-answering model respectively to determine each target data from each of the transformed data; each of the target data is summarized to obtain a defect sample set corresponding to the question-answering model. The present application performs defect detection on a pre-built question-and-answer model. First, an optimized training data sample set is established. This sample set includes multiple text data. The question-and-answer model can output a result based on the text data. The result is the answer to the text data. In order to detect defects in the model, the present application performs word and sentence transformation on each text data to obtain transformed data, thereby determining the output of the question-and-answer model for the transformed data, and determining each target data based on the model's output for the text data and the transformed data respectively. The target data are aggregated into a set as a defect sample set for the question-and-answer model, thereby completing defect detection for the question-and-answer model. The defect sample set can then be used to optimize the question-and-answer model to eliminate defects, thereby improving the comprehensiveness of the question-and-answer model.

[0123] In one example, the sentence transformation module 20 may include:

[0124] A splitting module, for splitting each piece of text data to obtain each initial character;

[0125] A type determination module, used to determine the type of each of the initial characters;

[0126] A first character determination module, used for taking each initial character belonging to a preset Chinese character symbol type as each first character;

[0127] A second character determination module, used for taking each initial character belonging to a preset word type as each second character;

[0128] The transformation module is used to transform the text data based on each of the first characters and each of the second characters to obtain transformation data corresponding to the text data.

[0129] Furthermore, the embodiment of the present application provides a model defect detection device. Optionally, Figure 4 The hardware structure diagram of the model defect detection device is shown in FIG. Figure 4 The hardware structure of the model defect detection device may include: at least one processor 01, at least one communication interface 02, at least one memory 03 and at least one communication bus 04.

[0130] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0131] The processor 01 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0132] The memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory.

[0133] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to execute the following model defect detection method, including:

[0134] Establishing an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data;

[0135] Performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data;

[0136] Inputting each of the text data and each of the transformed data into the question-answering model respectively, so as to determine each of the target data from each of the transformed data;

[0137] The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

[0138] Optionally, the detailed functions and extended functions of the program may refer to the description of the model defect detection method in the method embodiment.

[0139] The embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, and when the program is executed, controls the device where the storage medium is located to execute the following model defect detection method, including:

[0140] Establishing an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data;

[0141] Performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data;

[0142] Inputting each of the text data and each of the transformed data into the question-answering model respectively, so as to determine each of the target data from each of the transformed data;

[0143] The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

[0144] Specifically, the storage medium may be a computer-readable storage medium, and the computer-readable storage medium may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk or a ROM.

[0145] Optionally, the detailed functions and extended functions of the program may refer to the description of the model defect detection method in the method embodiment.

[0146] In addition, the functional modules in the various embodiments of the present disclosure can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a live broadcast device, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present disclosure.

[0147] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0148] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0149] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model defect detection method, characterized in that: include: Establishing an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data; Performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data; Inputting each of the text data and each of the transformed data into the question-answering model respectively, so as to determine each of the target data from each of the transformed data; The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

2. The method according to claim 1, characterized in that The step of performing word or sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data includes: For each piece of text data, split the text data to obtain each initial character; Determining the type of each of the initial characters; Using each initial character of a preset Chinese character symbol type as each first character; Using each initial character of a preset word type as each second character; The text data is transformed based on each of the first characters and each of the second characters to obtain transformed data corresponding to the text data.

3. The method according to claim 2, characterized in that The step of transforming the text data based on each of the first characters and each of the second characters to obtain transformed data corresponding to the text data includes: For each of the first characters, selecting each synonymous character corresponding to the first character from an open source database; Calculating the similarity between each of the synonymous characters and the first character respectively; The synonym with the smallest similarity is used as the target synonym corresponding to the first character; For each of the second characters, selecting synonyms corresponding to the second character from an open source database; Calculating the similarity between each of the synonyms and the second character respectively; The synonym with the smallest similarity is used as the target synonym corresponding to the second character; The text data is transformed based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain transformed data corresponding to the text data.

4. The method according to claim 3, characterized in that The step of transforming the text data based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain transformed data corresponding to the text data includes: Replacing each of the first characters in the text data with a corresponding target synonym; replacing each of the second characters in the text data with a corresponding target synonym; The replaced text data is regarded as the conversion data corresponding to the text data.

5. The method according to claim 1, characterized in that The step of inputting each of the text data and each of the transformed data into the question-answering model to determine each of the target data from each of the transformed data comprises: Selecting one piece of text data from each piece of text data as the first sentence; Inputting the first sentence into the question-answering model, so that the question-answering model outputs first answer data according to the first sentence; Inputting the transformed data corresponding to the first sentence into the question-answering model, so that the question-answering model outputs second question-answering data according to the transformed data corresponding to the first sentence; If the second question-and-answer data is different from the first question-and-answer data, using the transformed data corresponding to the first sentence as the target data; Returning to the step of selecting a piece of text data from each piece of the text data as the first sentence, until each piece of the text data has been selected, so as to obtain one or more pieces of target data; The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

6. The method according to any one of claims 1 to 5, characterized in that Before inputting each of the text data and each of the transformed data into the question-answering model, the method further includes: performing similarity expansion on each of the transformed data, including: For each piece of the transformed data, split the piece of transformed data to obtain each third character; determining one or more key characters from each of the third characters; Determining a current application field of the question-answering model, and determining a noun type of each of the key characters according to the current application field; For each of the key characters, a plurality of extended characters corresponding to the key character are set; randomly selecting an extended character from the plurality of extended characters as a fourth character; In the piece of transformed data, the key character is replaced with the fourth character to complete the similarity extension.

7. A model defect detection device, characterized in that: include: A sample set building module, used to build an optimized training data sample set for a pre-built question-answering model; wherein the optimized training data sample set includes a plurality of text data; A sentence transformation module, used for performing word and sentence transformation on each piece of the text data to obtain transformation data corresponding to each piece of the text data; A target data obtaining module, used for inputting each of the text data and each of the transformed data into the question-answering model, so as to determine each of the target data from each of the transformed data; The summarizing module is used to summarize the target data to obtain a set of defect samples corresponding to the question-answering model.

8. The device according to claim 7, characterized in that The sentence transformation module comprises: A splitting module, for splitting each piece of text data to obtain each initial character; A type determination module, used to determine the type of each of the initial characters; A first character determination module, used for taking each initial character belonging to a preset Chinese character symbol type as each first character; A second character determination module, used for taking each initial character belonging to a preset word type as each second character; The transformation module is used to transform the text data based on each of the first characters and each of the second characters to obtain transformation data corresponding to the text data.

9. A model defect detection device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the model defect detection method according to any one of claims 1 to 6.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the model defect detection method according to any one of claims 1 to 6 is implemented.

Images