A model defect detection method, device, equipment and storage medium

Abstract

The application discloses a model defect detection method, device and equipment and a storage medium. An optimized training data sample set is established for a pre-constructed question and answer model. The optimized training data sample set contains multiple pieces of text data. Each piece of text data is subjected to word and sentence transformation to obtain transformed data corresponding to each piece of text data. Each piece of text data and each piece of transformed data is input into the question and answer model to determine target data from each piece of transformed data. The target data is summarized to obtain a defect sample set corresponding to the question and answer model. The application determines target data according to the output of the model for text data and transformed data, and summarizes the target data into a set as a defect sample set for the question and answer model, that is, the defect detection of the question and answer model is completed. The defect sample set can be used to optimize the question and answer model to eliminate defects, so that the comprehensiveness of the question and answer model can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model defect detection, in particular to a model defect detection method, device, equipment and storage medium. BACKGROUND

[0002] In today's era of rapid digital development, various models play a crucial role in many fields. Data is input into the model, and the model can quickly output accurate results. However, as models are widely used, their defects are increasingly prominent.

[0003] Traditional model building process often focuses on the accuracy and performance improvement of the model, but pays little attention to potential defects. For example, after the model training is completed, no attention is paid to the subsequent application of the model. However, after the model is built and trained, the detection and optimization process for defects in the later stage is also crucial, otherwise it will not be conducive to the comprehensiveness of the model. SUMMARY

[0004] Therefore, the present application provides a model defect detection method, device, equipment and storage medium for detecting defects in the model, which can detect the defects in the model and improve the comprehensiveness of the model.

[0005] To achieve the above purpose, the present scheme is as follows:

[0006] In a first aspect, a model defect detection method comprises:

[0007] An optimization training data sample set is established for a pre-built question and answer model; wherein the optimization training data sample set contains multiple text data;

[0008] Each piece of text data is respectively transformed into a transformed data corresponding to each piece of text data;

[0009] Each piece of text data and each piece of transformed data is input into the question and answer model to determine each piece of target data from each piece of transformed data;

[0010] Each piece of target data is summarized to obtain a defect sample set corresponding to the question and answer model.

[0011] Preferably, the transformation of each piece of text data into a transformed data corresponding to each piece of text data comprises:

[0012] For each piece of text data, the text data is split to obtain each initial character;

[0013] The type of each initial character is determined;

[0014] each initial character belonging to a preset Chinese character type as each first character;

[0015] each initial character belonging to a preset word type as each second character;

[0016] transforming the piece of text data based on each first character and each second character to obtain transformation data corresponding to the piece of text data.

[0017] Preferably, the transforming the piece of text data based on each first character and each second character to obtain transformation data corresponding to the piece of text data comprises:

[0018] for each first character, screening each synonym from an open source database corresponding to the first character;

[0019] respectively calculating the similarity between each synonym and the first character;

[0020] taking the synonym with the smallest similarity as a target synonym corresponding to the first character;

[0021] for each second character, screening each synonym from an open source database corresponding to the second character;

[0022] respectively calculating the similarity between each synonym and the second character;

[0023] taking the synonym with the smallest similarity as a target synonym corresponding to the second character;

[0024] transforming the piece of text data based on each target synonym corresponding to each first character and each target synonym corresponding to each second character to obtain transformation data corresponding to the piece of text data.

[0025] Preferably, the transforming the piece of text data based on each target synonym corresponding to each first character and each target synonym corresponding to each second character to obtain transformation data corresponding to the piece of text data comprises:

[0026] replacing each first character in the piece of text data with the target synonym corresponding thereto;

[0027] replacing each second character in the piece of text data with the target synonym corresponding thereto;

[0028] taking the text data after the replacement as the transformation data corresponding to the piece of text data.

[0029] Preferably, the inputting each piece of the text data and each piece of the transformed data into the question and answer model respectively to determine each piece of target data from each piece of the transformed data comprises:

[0030] selecting a piece of text data from the text data as a first sentence;

[0031] inputting the first sentence into the question and answer model, so that the question and answer model outputs first answer data according to the first sentence;

[0032] inputting the transformed data corresponding to the first sentence into the question and answer model, so that the question and answer model outputs second answer data according to the transformed data corresponding to the first sentence;

[0033] if the second answer data is different from the first answer data, taking the transformed data corresponding to the first sentence as target data;

[0034] returning to the step of selecting a piece of text data from the text data as a first sentence, until each piece of the text data is selected, to obtain one or more pieces of target data;

[0035] summarizing each piece of the target data to obtain a defect sample set corresponding to the question and answer model.

[0036] Preferably, before the inputting each piece of the text data and each piece of the transformed data into the question and answer model, the method further comprises: performing similarity expansion on each piece of the transformed data, comprising:

[0037] for each piece of the transformed data, splitting the piece of transformed data to obtain each third character;

[0038] determining one or more key characters from each third character;

[0039] determining a current application field of the question and answer model, and determining a noun type of each key character according to the current application field;

[0040] for each key character, setting a plurality of expansion characters corresponding to the key character;

[0041] randomly selecting one expansion character from the plurality of expansion characters as a fourth character;

[0042] replacing the key character with the fourth character in the piece of transformed data to complete the similarity expansion.

[0043] In a second aspect, a model defect detection device comprises:

[0044] The sample set establishing module is configured to establish an optimized training data sample set for a pre-constructed question and answer model, wherein the optimized training data sample set comprises a plurality of pieces of text data;

[0045] The sentence transformation module is configured to transform each piece of the text data into transformed data corresponding to the text data respectively.

[0046] The target data obtaining module is configured to input each piece of the text data and each piece of the transformed data into the question and answer model, so as to determine each piece of target data from each piece of the transformed data.

[0047] The summary module is configured to summarize each piece of the target data to obtain a defect sample set corresponding to the question and answer model.

[0048] Preferably, the sentence transformation module comprises:

[0049] The splitting module is configured to split each piece of the text data to obtain initial characters.

[0050] The type determining module is configured to determine the type of each initial character.

[0051] The first character determining module is configured to determine each initial character of a preset Chinese character type as a first character.

[0052] The second character determining module is configured to determine each initial character of a preset word type as a second character.

[0053] The transformation module is configured to transform each piece of the text data based on each first character and each second character to obtain transformed data corresponding to the text data.

[0054] In a third aspect, a model defect detection device comprises a memory and a processor.

[0055] The memory is configured to store a program.

[0056] The processor is configured to execute the program to implement each step of the model defect detection method according to any one of the first aspect.

[0057] In a fourth aspect, a storage medium stores a computer program, and the computer program is executed by a processor to implement each step of the model defect detection method according to any one of the first aspect.

[0058] From the above technical solution can be seen, the application is established for the pre-constructed question and answer model optimization training data sample set; wherein, the optimization training data sample set contains a plurality of text data; each text data is respectively transformed into a sentence, and the transformed data corresponding to each text data is obtained; each text data and each transformed data is input into the question and answer model, so as to determine each target data from each transformed data; each target data is summarized to obtain a defect sample set corresponding to the question and answer model. The application detects defects for the pre-constructed question and answer model. First, an optimized training data sample set is established, and the sample set contains a plurality of text data. The question and answer model can output results according to the text data. The results are answers to the text data. In order to detect defects of the model, each text data is transformed into a sentence to obtain transformed data, so as to determine the output of the question and answer model for the transformed data. According to the output of the model for the text data and the transformed data, each target data is determined. Each target data is summarized as a defect sample set for the question and answer model, that is, the defect detection of the question and answer model is completed. The defect sample set can be used to optimize the question and answer model to eliminate defects, so as to improve the comprehensiveness of the question and answer model. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0060] Figure 1 An optional flowchart of a model defect detection method provided by the embodiment of the present application;

[0061] Figure 2 An optional flowchart of another model defect detection method provided by the embodiment of the present application;

[0062] Figure 3 A structural schematic diagram of a model defect detection device provided by the embodiment of the present application;

[0063] Figure 4 A structural schematic diagram of a model defect detection device provided by the embodiment of the present application; DETAILED DESCRIPTION

[0064] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0065] In today's era of rapid digital development, various models play a crucial role in many fields. By inputting data into the model, the model can quickly output accurate results. However, with the widespread application of models, the defects of models are increasingly prominent.

[0066] Traditional model building process often focuses on the accuracy and performance improvement of the model, and pays little attention to potential defects. For example, after the model training is completed, no attention is paid to the subsequent application of the model. However, after the model is built and trained, the detection and optimization process for defects in the later stage is also crucial, otherwise it will be detrimental to the comprehensiveness of the model.

[0067] The present application can be used in many general or special computing device environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or devices, etc.

[0068] The embodiment of the present application provides a model defect detection method, which can be applied in various computer terminals or intelligent terminals, and the execution subject can be the processor or server of the computer terminal or intelligent terminal. The method flow chart of the method is as shown in Figure 1 The specific steps include:

[0069] S1: An optimization training data sample set is established for a pre-built question and answer model; wherein the optimization training data sample set contains multiple text data.

[0070] The pre-built question and answer model in the present application refers to a model that users can input question and answer statements, text data and other information into, and can output corresponding answers or responses. The input of the model is text, and the output is also text, and the output text is the answer of the model to the input text.

[0071] The optimization training data sample set also contains text data. The text data set in the present application can be input into the question and answer model, and the question and answer model can output results according to the text data.

[0072] In one example, the question and answer model is a model that can identify the user's intention, where the user's intention refers to the user's questioning intention, such as the user wanting to know the area of a city, wanting to know the famous singers, wanting to know the quality of a product, etc., and the model can identify that the user's intention is other scene question and answer-geographical knowledge, other scene question and answer-brand information, and product knowledge question and answer-evaluation, respectively.

[0073] S2: Each piece of text data is respectively subjected to word and sentence transformation to obtain transformation data corresponding to each piece of text data.

[0074] The word and sentence transformation of the text data is not limited to synonym replacement, synonym replacement, changing the order of words, changing the subject-predicate-object sequence, randomly rearranging the order of words under the premise of unchanged meaning, etc., to ensure that the transformation data obtained by transformation corresponds to the text data.

[0075] The purpose of transformation is to detect whether the question and answer model can output the same answer under different input data but the same meaning / meaning, so as to determine whether the question and answer model has corresponding defects under this piece of text data.

[0076] S3: Each piece of text data and each piece of transformation data is respectively input into the question and answer model to determine each piece of target data from each piece of transformation data.

[0077] After transformation, the original text data is input into the question and answer model, and the question and answer model outputs an answer, and then the transformation data is input into the question and answer model, and the question and answer model also outputs an answer. Specifically, it can be determined whether the two answers are the same. If they are the same, it means that the question and answer model can output the same answer for two different but identical data. If they are different, it means that the question and answer model cannot output the same answer for two different but identical data. It can be considered that the question and answer model has defects in this piece of text data / transformation data range, and the piece of transformation data is determined as the target data.

[0078] S4: The target data is summarized to obtain a defect sample set corresponding to the question and answer model.

[0079] These target data are sentences that the question and answer model cannot accurately identify / cannot accurately answer, so these target data can be summarized as a data set as a defect sample set corresponding to the question and answer model. Subsequently, the defect sample set can be used to optimize and train the question and answer model to improve the question and answer accuracy.

[0080] As can be seen from the above technical solution, this application establishes an optimized training data sample set for a pre-constructed question-answering model; wherein, the optimized training data sample set contains multiple text data; each text data is transformed into words and sentences to obtain transformed data corresponding to each text data; each text data and each transformed data are input into the question-answering model to determine each target data from each transformed data; and the target data are summarized to obtain a defect sample set corresponding to the question-answering model. This application performs defect detection on a pre-built question-answering model. First, an optimized training data sample set is established, containing multiple text data. The question-answering model can output results based on the text data, which are the answers given to the text data. To detect defects in the model, this application performs word and sentence transformations on each text data to obtain transformed data, thereby determining the output of the question-answering model for the transformed data. Based on the model's output for both the text data and the transformed data, each target data is determined. All target data are aggregated into a set as the defect sample set for the question-answering model, thus completing the defect detection of the question-answering model. Subsequently, this defect sample set can be used to optimize the question-answering model to eliminate defects, thereby improving the comprehensiveness of the question-answering model.

[0081] The method provided in this embodiment of the invention involves performing word and sentence transformations on each piece of text data to obtain transformed data corresponding to each piece of text data, as follows: Figure 2 As shown, the specific explanation is as follows:

[0082] S21: For each piece of text data, split the text data to obtain each initial character;

[0083] S22: Determine the type of each of the initial characters;

[0084] S23: Take each initial character of type 'Chinese character symbol' as the first character;

[0085] S24: Use each initial character belonging to the preset word type as the second character;

[0086] S25: Based on each of the first characters and each of the second characters, transform the text data to obtain transformed data corresponding to the text data.

[0087] Specifically, the step mainly introduces the process of word and sentence transformation of text data, wherein each piece of text data needs to be transformed, so as to obtain the respective transformation data corresponding to each piece of text data. The following will be described in detail with respect to one piece of text data: the text data is split to obtain each initial character, and the type of each initial character needs to be determined. It can be considered that the type to which the initial character belongs includes Chinese character symbol type, punctuation symbol type, word type and other character type (the type to which other characters not belonging to Chinese character symbol, punctuation symbol and word belong). Generally, it is considered that the punctuation symbol type and the other character type are not suitable for word and sentence transformation. Therefore, the step first takes the initial character belonging to the Chinese character symbol type as a first character, and takes the initial character belonging to the word type as a second character, and only transforms the first character and the second character. The transformed text data is the transformation data.

[0088] The process of transforming the text data based on each first character and each second character to obtain the transformation data corresponding to the text data can specifically include:

[0089] For each first character, each synonym corresponding to the first character is screened from an open source database.

[0090] The similarity of each synonym and the first character is calculated respectively.

[0091] The synonym with the smallest similarity is taken as the target synonym corresponding to the first character.

[0092] For each second character, each synonym corresponding to the second character is screened from an open source database.

[0093] The similarity of each synonym and the second character is calculated respectively.

[0094] The synonym with the smallest similarity is taken as the target synonym corresponding to the second character.

[0095] The text data is transformed based on the target synonym corresponding to each first character and the target synonym corresponding to each second character to obtain the transformation data corresponding to the text data.

[0096] Specifically, a lot of characters, such as Chinese characters, words, idioms, and the like, are stored in the open source database. For a first character, each synonym corresponding to the first character can be filtered from the open source database. Because there are many synonyms similar to the first character, the filtering manner can be that all characters of the same type and the same meaning as the first character are regarded as synonyms, or the matching degree is calculated, and the character with a matching degree greater than a preset matching threshold is regarded as a synonym. Then, the already determined synonym is respectively compared with the first character for similarity calculation, and the synonym with the smallest similarity is regarded as a target synonym. The same principle applies to the synonym. The synonym with the smallest similarity is selected in order to detect whether the question and answer model can identify and detect the most marginal and least similar synonym or synonym. It is detected whether the question and answer model can accurately identify and detect the least similar sentence in the multiple sentences similar to the text data. In this way, resources and time can be saved. If the question and answer model can accurately identify and detect, it is not necessary to identify and detect the sentence composed of other synonyms or synonyms. If it cannot accurately identify, it is necessary to further monitor and monitor.

[0097] Further, the step of transforming the text data according to the target synonym corresponding to each of the first characters and the target synonym corresponding to each of the second characters in the above process will be explained in detail as follows:

[0098] Each of the first characters in the text data is replaced by the corresponding target synonym.

[0099] Each of the second characters in the text data is replaced by the corresponding target synonym.

[0100] The replaced text data is used as the transformed data corresponding to the text data.

[0101] Specifically, in the above process, the target synonym and the target synonym have been determined, and then the corresponding first character and the second character in the text data are replaced, so that the transformed data corresponding to the text data is obtained.

[0102] Next, the process of inputting each of the text data and each of the transformed data into the question and answer model to determine each of the target data from each of the transformed data in the present application will be described in detail.

[0103] A text data is selected from the text data as a first sentence.

[0104] The first sentence is input into the question and answer model, so that the question and answer model outputs first answer data according to the first sentence.

[0105] input the transformation data corresponding to the first sentence into the question and answer model, so that the question and answer model outputs second question and answer data according to the transformation data corresponding to the first sentence;

[0106] If the second question and answer data is different from the first question and answer data, the transformation data corresponding to the first sentence is taken as target data;

[0107] Return to the step of selecting a piece of text data as a first sentence from each piece of text data until each piece of text data is selected to obtain one or more target data;

[0108] Summarize each piece of target data to obtain a defect sample set corresponding to the question and answer model.

[0109] Specifically, it should be noted that in the process of comparing the second question and answer data and the first question and answer data, it is necessary to determine whether the second question and answer data is completely the same as the first question and answer data. If not, it is considered that the question and answer model cannot accurately identify, and the transformation data corresponding to the second question and answer data is taken as the target data, and it is considered that the question and answer data is defective on this target data. Next, continue to judge other pieces of text data in the optimization training data sample set, and if the two answers are completely the same, other pieces of text data also need to be judged until all the other pieces of text data in the optimization training data sample set are selected, so as to prevent omission and improve the accuracy of question and answer model defect detection.

[0110] Further, before inputting each piece of text data and each piece of transformation data into the question and answer model, the application further includes the following steps: for each piece of transformation data, the transformation data is split to obtain each third character;

[0111] Determine one or more key characters from each third character;

[0112] Determine the current application field of the question and answer model, and determine the noun type of each key character according to the current application field;

[0113] For each key character, set a plurality of expansion characters corresponding to the key character;

[0114] Randomly select one expansion character as a fourth character from a plurality of expansion characters;

[0115] Replace the key character with the fourth character in the transformation data to complete the similarity expansion.

[0116] Specifically, in order to improve the detection accuracy of the question and answer model, after the text data is transformed into transformed data, the transformed data is again converted to a certain extent, so that the question and answer difficulty of the question and answer model can be increased to detect the hidden defects of the question and answer model, and the transformed data is split into each third character, the key character is determined from the third character, the key character can be understood as a character in the transformed data that can represent the overall meaning of the transformed data, and the key character is expanded from the current application field, and a fourth character is randomly selected from the expanded character to increase randomness and ensure detection accuracy. Finally, the key character is replaced by the fourth character, thereby completing the similarity expansion to obtain a twice-transformed data, so that the question and answer model processes the text data and the twice-transformed data and outputs an answer, and the answer is compared to obtain a final defect sample set.

[0117] With Figure 1 Corresponding to the method, the embodiment of the application also provides a model defect detection device for detecting Figure 1 The model defect detection device provided by the embodiment of the application can be used in a computer terminal or various mobile devices, and can be combined with Figure 3 The model defect detection device will be introduced as follows, as shown in Figure 3 The device can include:

[0118] The sample set establishing module 10 is configured to establish an optimized training data sample set for a pre-constructed question and answer model, wherein the optimized training data sample set includes a plurality of text data.

[0119] The sentence transformation module 20 is configured to perform word and sentence transformation on each piece of text data to obtain transformed data corresponding to each piece of text data.

[0120] The target data obtaining module 30 is configured to input each piece of text data and each piece of transformed data into the question and answer model to determine each piece of target data from each piece of transformed data.

[0121] The summary module 40 is configured to summarize each piece of target data to obtain a defect sample set corresponding to the question and answer model.

[0122] As can be seen from the above technical solution, this application establishes an optimized training data sample set for a pre-constructed question-answering model; wherein, the optimized training data sample set contains multiple text data; each text data is transformed into words and sentences to obtain transformed data corresponding to each text data; each text data and each transformed data are input into the question-answering model to determine each target data from each transformed data; and the target data are summarized to obtain a defect sample set corresponding to the question-answering model. This application performs defect detection on a pre-built question-answering model. First, an optimized training data sample set is established, containing multiple text data. The question-answering model can output results based on the text data, which are the answers given to the text data. To detect defects in the model, this application performs word and sentence transformations on each text data to obtain transformed data, thereby determining the output of the question-answering model for the transformed data. Based on the model's output for both the text data and the transformed data, each target data is determined. All target data are aggregated into a set as the defect sample set for the question-answering model, thus completing the defect detection of the question-answering model. Subsequently, this defect sample set can be used to optimize the question-answering model to eliminate defects, thereby improving the comprehensiveness of the question-answering model.

[0123] In one example, the statement transformation module 20 may include:

[0124] The splitting module is used to split each piece of text data to obtain individual initial characters.

[0125] A type determination module is used to determine the type of each of the initial characters;

[0126] The first character determination module is used to determine each initial character of a preset Chinese character type as the first character.

[0127] The second character determination module is used to determine each initial character belonging to a preset word type as a second character.

[0128] The transformation module is used to transform the text data based on each of the first characters and each of the second characters to obtain transformed data corresponding to the text data.

[0129] Furthermore, embodiments of this application provide a model defect detection device. Optionally, Figure 4 The hardware structure block diagram of the model defect detection device is shown. (Refer to...) Figure 4 The hardware structure of the model defect detection device may include: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.

[0130] In the embodiments of the present application, the number of the processor 01, the communication interface 02, the memory 03 and the communication bus 04 is at least one, and the processor 01, the communication interface 02 and the memory 03 complete the communication with each other through the communication bus 04.

[0131] The processor 01 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement the embodiments of the present application, etc.

[0132] The memory 03 can include a high-speed RAM memory, and can also include a non-volatile memory, etc., such as at least one disk memory.

[0133] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to execute the following model defect detection method, which comprises:

[0134] An optimized training data sample set is established for a pre-constructed question and answer model, wherein the optimized training data sample set includes a plurality of text data;

[0135] Each piece of text data is respectively subjected to word and sentence transformation to obtain transformation data corresponding to each piece of text data;

[0136] Each piece of text data and each piece of transformation data is respectively input into the question and answer model to determine target data from each piece of transformation data;

[0137] The target data is summarized to obtain a defect sample set corresponding to the question and answer model.

[0138] Optionally, the refinement function and the extension function of the program can refer to the description of the model defect detection method in the method embodiments.

[0139] The embodiments of the present application also provide a storage medium which can store a program suitable for a processor to execute, and when the program runs, controls a device where the storage medium is located to execute the following model defect detection method, which comprises:

[0140] An optimized training data sample set is established for a pre-constructed question and answer model, wherein the optimized training data sample set includes a plurality of text data;

[0141] Each piece of text data is respectively subjected to word and sentence transformation to obtain transformation data corresponding to each piece of text data;

[0142] Input each piece of the text data and each piece of the transformation data into the question and answer model to determine each piece of target data from each piece of the transformation data;

[0143] Collect each piece of the target data to obtain a defect sample set corresponding to the question and answer model.

[0144] Specifically, the storage medium can be a computer-readable storage medium, which can be an electronic storage such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM.

[0145] Optionally, the refinement function and the expansion function of the program can refer to the description of the model defect detection method in the method embodiment.

[0146] In addition, each functional module in each embodiment of the present disclosure can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is realized in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present disclosure essentially or say the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a live device, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present disclosure.

[0147] Finally, it should be noted that in this document, relationship terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0148] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between various embodiments can be referred to each other.

[0149] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting defects in a model, characterized in that, include: An optimized training data sample set is established for a pre-built question-answering model; wherein, the optimized training data sample set contains multiple text data; Each piece of text data is transformed into words and phrases to obtain transformed data corresponding to each piece of text data; this includes: splitting each piece of text data into initial characters; determining the type of each initial character; taking each initial character of type belonging to a preset Chinese character type as each first character; taking each initial character of type belonging to a preset word type as each second character; and transforming the text data based on each first character and each second character to obtain transformed data corresponding to that piece of text data. Each piece of text data and each piece of transformed data are input into the question-answering model to determine each piece of target data from the transformed data; this includes: selecting one piece of text data as a first statement from each piece of text data; inputting the first statement into the question-answering model so that the question-answering model outputs first question-answer data based on the first statement; inputting the transformed data corresponding to the first statement into the question-answering model so that the question-answering model outputs second question-answer data based on the transformed data corresponding to the first statement; if the second question-answer data is different from the first question-answer data, then the transformed data corresponding to the first statement is taken as the target data; returning to the step of selecting one piece of text data as the first statement from each piece of text data until each piece of text data has been selected to obtain one or more pieces of target data; summarizing each piece of target data to obtain a defect sample set corresponding to the question-answering model; The target data are aggregated to obtain a defect sample set corresponding to the question-answering model.

2. The method according to claim 1, characterized in that, The process of transforming the text data based on each of the first characters and each of the second characters to obtain transformed data corresponding to the text data includes: For each of the first characters, select the synonyms corresponding to the first character from the open-source database; Calculate the similarity between each of the aforementioned synonyms and the first character; The closest similarity is selected as the target and the target synonym corresponding to the first character; For each of the second characters, select the synonyms corresponding to that second character from the open-source database; Calculate the similarity between each of the aforementioned synonyms and the second character; The synonym with the lowest similarity is selected as the target synonym corresponding to the second character; The text data is transformed based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain the transformed data corresponding to the text data.

3. The method according to claim 2, characterized in that, The transformation of the text data based on the target synonyms corresponding to each of the first characters and the target synonyms corresponding to each of the second characters to obtain the transformed data corresponding to the text data includes: Replace each of the first characters in the text data with its corresponding target synonym; Replace each of the second characters in the text data with its corresponding target synonym; The replaced text data is used as the transformation data corresponding to that text data.

4. The method according to any one of claims 1 to 3, characterized in that, Before inputting each piece of text data and each piece of transformed data into the question-answering model, the method further includes: performing similarity expansion on each piece of transformed data, including: For each piece of transformed data, the transformed data is split to obtain individual third characters; Determine one or more key characters from each of the aforementioned third characters; Determine the current application domain of the question-answering model, and determine the noun type of each key character according to the current application domain; For each of the aforementioned key characters, set multiple extended characters corresponding to that key character; Randomly select one of the extended characters from the plurality of extended characters as the fourth character; In this transformed data, the key character is replaced with the fourth character to complete the similarity expansion.

5. A model defect detection device, characterized in that, include: The sample set establishment module is used to establish an optimized training data sample set for a pre-built question-answering model; wherein, the optimized training data sample set contains multiple text data; The sentence transformation module is used to transform each piece of text data into words and sentences to obtain transformed data corresponding to each piece of text data. It includes: a splitting module for splitting each piece of text data to obtain initial characters; a type determination module for determining the type of each initial character; a first character determination module for identifying initial characters of a preset Chinese character type as first characters; a second character determination module for identifying initial characters of a preset word type as second characters; and a transformation module for transforming the text data based on the first and second characters to obtain transformed data corresponding to the text data. The target data acquisition module is used to input each piece of text data and each piece of transformed data into the question-answering model to determine each piece of target data from the transformed data. This includes: selecting one piece of text data as a first statement from each piece of text data; inputting the first statement into the question-answering model so that the model outputs first question-answer data based on the first statement; inputting the transformed data corresponding to the first statement into the question-answering model so that the model outputs second question-answer data based on the transformed data corresponding to the first statement; if the second question-answer data is different from the first question-answer data, then the transformed data corresponding to the first statement is taken as the target data; returning to the step of selecting one piece of text data as the first statement until each piece of text data has been selected to obtain one or more pieces of target data; and summarizing all the target data to obtain a defect sample set corresponding to the question-answering model. The aggregation module is used to aggregate the target data to obtain a set of defect samples corresponding to the question-answering model.

6. A model defect detection device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the model defect detection method as described in any one of claims 1-4.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the model defect detection method as described in any one of claims 1-4.

Images