Method, system and equipment for testing correctness of sensitive data checking model for strengthening effective sample priority and medium

Abstract

The invention discloses a sensitive data checking model correctness testing method, system and device for strengthening effective sample priority and a medium, and belongs to the technical field of computer and information science, and the method comprises the steps: carrying out the structure modification of a to-be-tested reference model according to a plurality of preset model structure level mutation rules, and generating a plurality of corresponding mutation models; respectively training a plurality of mutation models by using the training sample set, and determining a corresponding differential weight for each mutation model according to whether the training process of each mutation model is successful or not; obtaining the prediction results of the reference model and the successfully trained mutation model for the unlabeled samples, and calculating the difference degree between the prediction results in combination with the differential weight to generate the priority score of the unlabeled samples; and sorting the plurality of unlabeled samples based on the priority scores to obtain a sample priority list. The method for enhancing the priority of the effective samples is provided, and the model correctness testing efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer and information science, and particularly relates to a sensitive data verification model correctness testing method, system, device and medium for reinforcing effective sample priority. BACKGROUND

[0002] Data, as a new type of production factor, shows an indispensable strategic value, and the importance of data security in the transaction process is increasingly prominent. Building an efficient and high-precision sensitive data attribute verification model can help alleviate the problems of accurate identification of sensitive data attributes and mining of potential correlations between attributes in the application scenarios of business system traffic data. However, the reliability of artificial intelligence model construction itself is difficult to guarantee in specific application functions. If the sensitive data verification model makes a decision error, it may lead to more serious data security risks.

[0003] Therefore, for the sensitive data verification model that has been built, it is necessary to introduce artificial intelligence model testing technology to fully verify it, which can help to promote its safety and stability in terms of function and performance. Especially in the data transaction scenario, the available labeled sensitive data is lacking, and it is necessary to ensure that the test samples have high quality and can quickly expose model defects in a targeted manner, which has important theoretical significance and practical value in improving test efficiency and reducing test cost.

[0004] The core problem that the correctness testing method needs to solve is how to find samples with higher testing value. The existing advanced methods mainly include the following three categories:

[0005] 1. Using sample distribution method

[0006] The sample distribution method determines the priority of effective samples by calculating the distribution difference between the sample and the training sample set. In the existing method, LSA calculates the distribution density function of a given sample on the training sample set by kernel density estimation, and estimates the possibility of the sample in the training sample set distribution as the sample ranking standard according to the distribution density function. DSA calculates the distance relationship between the given sample and the training sample set as the sample ranking standard. However, this method needs to traverse the complete training sample set for each test sample, which makes the method unable to be applied to large-scale data transaction scenarios and high-complexity verification models in actual testing.

[0007] 2. Using model uncertainty method

[0008] The method using model uncertainty is used to measure the confidence degree of the model to the sample decision. Taking a binary classification model as an example, compared with the sample with an output of [0.6, 0.4], the model has higher confidence to the sample with an output of [0.9, 0.1], and the former is more likely to have a risk of incorrect decision. DeepGini is a typical method using model uncertainty, which determines the label priority of the sample by calculating the Gini value of the output of the target model for each test sample. However, its calculation process depends on the output probability matrix of the model and cannot be applied to the application scenarios of target models such as regression tasks.

[0009] 3. Method for using mutation test

[0010] The method for using mutation test measures the effectiveness of the sample by detecting the ability of the sample to find mutations. For example, the PRIMA method proposes multiple mutation rules for deep neural networks, image samples and text samples, etc. to generate mutation models or mutation samples. Six difference features of normal output and mutation output are extracted, and a ranking model is constructed to determine the label priority of the sample. This kind of method can find more defects of the target model under the same number of sample labels. However, this method directly applies mutation rules to the neuron weights of the target model, which is easily affected by mutation randomness and inappropriate mutation rate in complex model structures, and cannot be applied to test scenarios with insufficient labeled samples.

[0011] In summary, the existing deep learning test sample priority ranking methods mainly have the following problems: (1) lack of applicability in complex model structures, and as the complexity of the model structure increases, the existing methods are prone to problems such as explosive increase of iteration amount, uncontrollable random strategy or unsuitable calculation format, which hinders the identification of effective test samples; (2) high dependence on labeled samples, and the existing mutation test method confirms the priority by training a ranking model, which may fail when there are insufficient labeled samples. SUMMARY

[0012] In view of the above problems, the present application is proposed.

[0013] Therefore, the purpose of the application is to solve the problems of insufficient applicability of traditional test methods due to complex model structure and lack of sufficient known annotations of sensitive data in the correctness test of existing sensitive data verification model.

[0014] To solve the above technical problems, the application provides the following technical solutions: a sensitive data verification model correctness test method for reinforcing effective sample priority, which comprises the following steps: modifying the structure of a benchmark model to be tested according to a plurality of preset model structure level mutation rules to generate a plurality of corresponding mutant models; training the plurality of mutant models using a training sample set, and determining a corresponding differential weight for each of the plurality of mutant models according to whether the training process of each of the plurality of mutant models is successful; obtaining the prediction results of the benchmark model and the successfully trained mutant models for unlabeled samples respectively, and combining the differential weight to calculate the difference between the prediction results to generate a priority score of the unlabeled samples; and sorting a plurality of the unlabeled samples based on the priority score to obtain a sample priority list.

[0015] As a preferred scheme of the sensitive data verification model correctness test method for reinforcing effective sample priority, the model structure level mutation rules comprise: performing an increase or deletion operation on at least one of a hidden layer, a convolution layer or a pooling layer in the benchmark model, or an adjustment operation on the number of components in the layer, or a modification operation on the parameters of the components in the layer. The beneficial effects of the preferred technical scheme are: by directly acting on the structure level of the hidden layer, the convolution layer or the pooling layer of the model to perform mutation, instead of the traditional neuron weight mutation, the problem of random mutation sites and difficult to balance mutation rate in the existing mutation test method is solved. This structure level modification method is more strongly associated with the decision logic of the model, so that the mutant model can more stably and effectively identify valuable test samples from the logical level.

[0016] As a preferred scheme of the sensitive data verification model correctness test method for reinforcing effective sample priority, the adjustment operation on the number of components in the layer comprises: adjusting the number of neurons in the hidden layer or adjusting the number of convolution kernels in the convolution layer; and the modification operation on the parameters of the components in the layer comprises: modifying the size or moving step of the convolution kernel in the convolution layer, or modifying the size or moving step of the pooling window in the pooling layer.

[0017] As a preferred scheme of the method for testing the correctness of the sensitive data checking model for reinforcing the priority of the effective sample, wherein: the corresponding differential weight is determined for each mutation model, including: if the training process of the mutation model is successfully completed, the corresponding differential weight is set as a first preset value; if the training process of the mutation model fails or is interrupted, the corresponding differential weight is set as a second preset value. The beneficial effect of the preferred technical scheme is that the differential weight mechanism is introduced, and the weight is allocated according to whether the mutation model can be successfully trained after training, so that the invalid mutation model caused by excessive structural modification and leading to training failure or collapse can be automatically filtered out. This avoids the mutation rule invalid to the structure of the target model being included in the subsequent priority calculation system, ensures the effectiveness and stability of the model set finally used for evaluating the priority of the sample, and thus improves the robustness of the test method.

[0018] As a preferred scheme of the method for testing the correctness of the sensitive data checking model for reinforcing the priority of the effective sample, wherein: the difference between the prediction results is calculated to generate the priority score of the unlabeled sample, including: the distance between the prediction results of the baseline model and each successfully trained mutation model for the unlabeled sample is calculated respectively; the distance is combined with the corresponding differential weight to obtain the priority score of the unlabeled sample. The beneficial effect of the preferred technical scheme is that the influence degree of multiple different structural mutations on the prediction results of the model is comprehensively considered, and the prediction differences of multiple model pairs are combined by weighting, so that the effectiveness of the sample for exposing the functional and performance defects of the model is comprehensively evaluated from different angles. The multi-dimensional cooperative evaluation mode reinforces the performance of the effective sample in the final priority score, ensures that the samples with high priority can more probably and more quickly expose the deep-seated defects of the model, and thus improves the test efficiency and reduces the test cost.

[0019] As a preferred scheme of the method for testing the correctness of the sensitive data checking model for reinforcing the priority of the effective sample, wherein: the prediction result is a prediction vector, and the distance is the Euclidean distance between the prediction vector output by the baseline model and the prediction vector output by the mutation model.

[0020] As a preferred scheme of the method for testing the correctness of the sensitive data checking model for reinforcing the priority of the effective sample, wherein: further comprising: selecting the unlabeled sample with high priority from the sample priority list for manual labeling, and using the manually labeled sample for defect detection of the baseline model.

[0021] The application provides a system for testing the correctness of a sensitive data checking model for reinforcing the priority of an effective sample.

[0022] To solve the above technical problems, the application further provides the following technical solutions: a sensitive data verification model correctness test system for strengthening effective sample priority, comprising: a generation module configured to modify the structure of a benchmark model to be tested according to a plurality of preset model structure level mutation rules to generate a plurality of corresponding mutant models; a training weight module configured to train the plurality of mutant models using a training sample set, and determine a corresponding differential weight for each of the mutant models according to whether the training process of each of the mutant models is successful; a scoring module configured to obtain the prediction results of the benchmark model and the successfully trained mutant models for unannotated samples respectively, and calculate the difference between the prediction results in combination with the differential weight to generate a priority score of the unannotated samples; and a sorting module configured to sort a plurality of the unannotated samples based on the priority score to obtain a sample priority list.

[0023] The application provides a computer device, comprising a memory and a processor, and the memory stores a computer program, characterized in that the processor implements the steps of the sensitive data verification model correctness test method for strengthening effective sample priority when executing the computer program.

[0024] The application provides a computer readable storage medium, which stores a computer program, characterized in that the computer program implements the steps of the sensitive data verification model correctness test method for strengthening effective sample priority when executed by a processor.

[0025] The application has the following beneficial effects: compared with the existing sensitive data verification model test method, the application can use a model structure level mutation strategy to alleviate the problems of random mutation sites and difficult mutation rate balance in mutation test methods, so that it is easier to identify effective samples at the model decision logic level; in combination with the synergistic effect of multiple types of mutations, the scoring results of effective samples in the priority aspect are strengthened to ensure that effective samples can be labeled and used as soon as possible; the method overall promotes the test effectiveness of the sensitive data verification model test, and improves the protection capability of the sensitive data verification model. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 The overall flowchart of the sensitive data verification model correctness test method for strengthening effective sample priority provided by an embodiment of the application is shown in the figure.

[0028] Figure 2 A method principle diagram of sample priority ranking of the sensitive data verification model correctness test method for reinforcing effective sample priority provided by an embodiment of the present application;

[0029] Figure 3 A principle diagram comparison of model level mutation and model structure level mutation of the sensitive data verification model correctness test method for reinforcing effective sample priority provided by an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the above objectives, characteristics and advantages of the present application more apparent, obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0031] Embodiment 1, reference Figure 1 For the first embodiment of the present application, the embodiment provides a sensitive data verification model correctness test method for reinforcing effective sample priority, comprising:

[0032] S100: According to a plurality of preset model structure level mutation rules, modifying the structure of the to-be-tested reference model to generate a plurality of corresponding mutant models;

[0033] S200: Training the plurality of mutant models using the training sample set, and determining the corresponding difference weight for each mutant model according to whether the training process of each mutant model is successful;

[0034] S300: Obtaining the prediction results of the reference model and the successfully trained mutant model respectively for the unlabeled samples, and combining the difference weight to calculate the difference degree between the prediction results to generate the priority score of the unlabeled samples;

[0035] S400: Ranking the plurality of unlabeled samples based on the priority score to obtain a sample priority list.

[0036] It should be noted that in the context of data as a new production factor, it is crucial to build an efficient and high-precision sensitive data verification model, but the reliability of such data-driven AI model in function is difficult to guarantee, and once the decision is wrong, it may cause serious data security risks. Therefore, sufficient correctness testing of the constructed model is a necessary link to ensure its safety and stability. However, the core problem of testing is how to find the most valuable samples from the vast amount of data. In the data transaction and other scenarios, there is a serious lack of labeled sensitive data available, resulting in high testing cost and low efficiency. The existing methods have many problems: the sample distribution-based method has a huge amount of calculation and is difficult to apply to large-scale scenarios; the model uncertainty-based method has limited application scenarios; and the traditional mutation test method is easily affected by mutation randomness and inappropriate mutation rate, and has high dependence on labeled samples, which is not effective in model structure.

[0037] Therefore, in view of the above problems of difficult test sample screening and high dependence on labeled samples, through the steps of S100 to S400, the method achieves the following effects: S100 introduces model structure level mutation, generates mutants from the model decision logic level rather than the random neuron weight level, and alleviates the problem of random mutation sites and difficult mutation rate balance; the differential weight mechanism of S200 effectively filters invalid mutant models that cannot be successfully trained due to structural modification failure, ensuring the effectiveness of subsequent evaluation; S300 combines the synergistic effect of multiple types of mutations to comprehensively score unlabeled samples, strengthens the priority of effective samples, and solves the problem of over-reliance on labeled samples; and finally, the priority list output by S400 can guide the test personnel to preferentially invest limited manual annotation resources into samples most likely to expose model defects, thereby improving the efficiency and accuracy of sensitive data verification model correctness testing.

[0038] Embodiment 2, refer to Figures 1-3 The second embodiment of the present application provides a sensitive data verification model correctness testing method for strengthening the priority of effective samples.

[0039] S100: According to a plurality of preset model structure level mutation rules, a structure of a to-be-tested reference model is modified to generate a plurality of corresponding mutant models.

[0040] In the embodiment of the present application, this step specifically parses and extracts the original topology structure of the to-be-tested reference model from the code level or model definition file (such as model structure definition in Keras, TensorFlow), then traverses a preset mutation rule list, and applies each model structure level mutation rule in the list to the original model structure to generate a plurality of mutant models corresponding to the rules.

[0041] In the embodiments of the present application, the preset model structure level mutation rule includes:

[0042] For at least one of the hidden layer, the convolutional layer or the pooling layer in the benchmark model, a random increase or deletion operation of the layer, an adjustment operation of the number of components in the layer or a modification operation of the parameters of the components in the layer is performed, and the validity of the model after the operation is ensured.

[0043] The adjustment operation of the number of components in the layer includes adjusting the number of neurons in the hidden layer or adjusting the number of convolutional kernels in the convolutional layer; the modification operation of the parameters of the components in the layer includes modifying the size or moving step of the convolutional kernel in the convolutional layer, or modifying the size or moving step of the pooling window in the pooling layer.

[0044] It should be noted that the mutation mode directly acting on the model structure has the core advantage that its influence is macroscopic and has logical explainability. For example, deleting a convolutional layer directly weakens the deep feature extraction ability of the model; expanding the number of neurons in the hidden layer enhances the fitting and expression ability of the model. These changes directly touch the core learning and decision mechanism of the model, and the effect is significant and predictable, so as to ensure that the generated mutant set can effectively disturb the benchmark model from multiple key logical dimensions. This completely avoids the problem of a large number of invalid or micro-effective mutations caused by the randomness of the mutation site and the difficulty in setting the mutation rate in the traditional weight mutation method.

[0045] In an optional implementation, multiple models with different behaviors can also be generated by mutating the hyperparameters of the model. In this way, the structure of the benchmark model is kept unchanged, and key hyperparameters in the training process are modified, such as replacing the optimizer from Adam to SGD, or adjusting the learning rate, batch size, etc., to generate multiple models with different behavior characteristics. Although these models have the same structure, the convergence path and optimization dynamics in the training process are different, and the final decision boundary is also different, which can also be used for subsequent difference evaluation.

[0046] In another optional implementation, it can also be realized by training data subsampling. In this way, the model structure and hyperparameters are kept unchanged, and multiple different training subsets are obtained by randomly sampling the complete training sample set (such as using the Bagging idea) with or without replacement. The benchmark model is trained independently using these different training subsets, and since the input data distribution is different, the decision logic of the multiple model instances obtained finally will also be different.

[0047] S200: training multiple mutant models using the training sample set, and determining the corresponding difference weight for each mutant model according to whether the training process of each mutant model is successful.

[0048] In this embodiment of the invention, determining the corresponding difference weights for each mutation model includes:

[0049] If the number of training rounds of the mutation model reaches the maximum number of iterations n, that is, the model training is completed, then the corresponding differential weights are set to the first preset value.

[0050] If the training process of the mutation model fails or is interrupted, its corresponding differential weights are set to the second preset value.

[0051] It should be noted that this step is a crucial filtering mechanism to ensure the validity of the test. "Successful completion of training" means that the model can load data and complete all preset training epochs without error, without program crashes or interruptions caused by structural incompatibility (such as tensor dimension conflicts) or numerical computation instability (such as gradient explosion or vanishing). The "first preset value" is set to 1 in this invention, and the "second preset value" is set to 0. The fundamental purpose of introducing this differential weighting mechanism is to ensure that the final priority score is contributed only by logically valid and workable mutation models. Any "invalid" mutation model that becomes untrainable due to improper structural modifications has a weight of 0, and its difference from the baseline model will not be included in the total score, thus eliminating noise interference from invalid mutations and greatly enhancing the robustness of the entire testing process and the reliability of the final score.

[0052] In an alternative implementation, weights can be determined using performance-based continuous weights. Instead of using binary weights of 0 or 1, this approach rapidly evaluates all successfully trained mutation models on a small, independent validation set and assigns a continuous value between 0 and 1 as a differential weight based on their performance metrics (such as accuracy, F1 score, etc.). For example, a higher validation accuracy for a mutation model corresponds to a higher weight. This approach more precisely reflects the importance of different effective mutation models.

[0053] S300: Obtain the prediction results of the baseline model and the successfully trained mutation model for unlabeled samples, and calculate the difference between the prediction results by combining the differential weights to generate a priority score for the unlabeled samples.

[0054] In this embodiment of the invention, the difference between prediction results is calculated to generate priority scores for unlabeled samples, including steps A1 and A2:

[0055] Step A1: Calculate the distance between the baseline model and the prediction results of each successfully trained mutation model for unlabeled samples;

[0056] Step A2: combine the distance and the corresponding differential weight to get the priority score of the unlabeled sample. Let p j be the sample x j the predicted vector under the base model; be the sample x j the predicted vector under the ith successfully trained mutant model; w i be the differential weight of the mutant model; N is the total number of mutant models. Then the priority score S j of the sample x i is:

[0057]

[0058] where the distance function dist is the Euclidean distance by default:

[0059]

[0060] It should be noted that the prediction result is the predicted vector, and the distance is the Euclidean distance between the predicted vector output by the base model and the predicted vector output by the mutant model. Specifically, for the jth unlabeled sample, its priority score score_j is the weighted sum of the prediction vector differences on the 20-group model pair consisting of the base model and 20 mutant models. In essence, this score quantifies the "consensus" of the "committee" consisting of a series of logically related model variants for the same input sample. A high priority score means that there is a great "disagreement" in the judgment of the model committee on the sample, which strongly suggests that the sample is in the ambiguous or unstable region of the model decision, and is most likely a potential decision error point, and therefore has the highest test priority.

[0061] In an optional implementation, in step A1, in addition to the Euclidean distance, other distances or similarity measures can also be selected according to the task type to measure the distance between the prediction results. For example, Manhattan distance (L1 norm) can be used to measure the vector difference; or when processing the probability output vector of the classification task, the KL divergence (Kullback-Leibler divergence) or JS divergence (Jensen-Shannon divergence) in information theory can be used. The KL divergence can measure the information loss when one probability distribution is used to approximate another, and is particularly suitable for measuring the difference in model prediction probability.

[0062] S400: Sort the multiple unlabeled samples based on the priority score to get a sample priority list.

[0063] In the embodiment of the present application, this step ranks all unlabeled samples in descending order according to the priority scores calculated in S300, and the samples with higher scores are ranked in front, thereby forming a final sample priority list that can be directly used by testers.

[0064] In addition, in this embodiment, it also includes: selecting the top-K samples with high priority rankings from the sample priority list, K is the proportion of the total number of samples with high priority rankings, manually labeling the unlabeled samples, and using the manually labeled samples for defect detection of the benchmark model.

[0065] Effect analysis: In the actual application scenario of sensitive data verification, enterprises or institutions often face millions of unlabeled data streams, and manual labeling is costly and time-consuming. Through the process of S100-S400, the present application can automatically and efficiently select a high-value test subset from the massive unlabeled data.

[0066] For example, the present application is applied to test a sensitive image content verification model trained on the CIFAR-10 image dataset. There are 10,000 unlabeled test images in this test task. First, a priority score is calculated for each of the 10,000 images through S100-S300. Then, the samples are sorted according to S400. In the traditional test process, testers may need to randomly sample or label all samples to find enough model defects. After applying the present application, testers can only select the top 10% (i.e. 1000) samples with the highest priority scores for manual labeling. The experimental results in the technical disclosure show that, compared with existing methods, the number of model defects detected by the 10% of samples selected by the present application (i.e. defect detection rate DDR) is much higher than that of samples selected by random sampling or other methods. At the same time, the area under the curve (RAUC) value of the present application is also the highest, which mathematically proves that the present application can rank the most valuable samples at the front of the list the fastest. This effect directly translates into huge commercial value: it enables enterprises to find more than 90% of model defects with only 10% of the labeling cost, greatly shortening the model testing and iteration cycle, and reducing the data security risks caused by model decision errors.

[0067] In summary, the present application can efficiently and accurately identify the most valuable samples for testing model defects in the absence of large-scale labeled data by introducing model structure-level mutations, combining differential weight mechanisms and multi-model difference metrics. The present application does not depend on a specific model type, has wide applicability and strong robustness, and can improve the efficiency of correctness testing of sensitive data verification models and other AI systems, thereby better ensuring their safety and reliability.

[0068] Embodiment 3 is a third embodiment of the present application, which provides a method for testing the correctness of the sensitive data checking model for enhancing the priority of the effective sample. Based on the first two embodiments, the embodiment provides a specific implementation.

[0069] Step 1: Apply each model structure level mutation rule to the tested model to generate the corresponding mutant model.

[0070] Step 1.1: Extract the original model structure design of the tested model from the code level.

[0071] Step 1.2: Traverse the mutation rules in the mutation rule list.

[0072] Step 1.3: Apply the mutation rules in step 1.2 to the original model structure in step 1.1.

[0073] Step 1.4: Repeat steps 1.2 and 1.3 to obtain 20 mutant models corresponding to 20 rules.

[0074] Step 2: Train the mutant models generated in step 1 with the training sample set and assign differential weights.

[0075] Step 2.1: Train each mutant model using the training sample set.

[0076] Step 2.2: If the model completes training normally, set its differential weight to 1, if the model fails to train or crashes during training due to structural modification, set its differential weight to 0.

[0077] Step 2.3: Repeat steps 2.1 and 2.2 until all training is completed to obtain trained mutant models and corresponding differential weights.

[0078] Step 3: Input the unlabeled samples into the tested model and the 20 mutant models obtained in step 2 to obtain the corresponding prediction vectors.

[0079] Step 3.1: Input the unlabeled samples into the tested model to obtain the original prediction vector.

[0080] Step 3.2: Input the unlabeled samples into the mutant model to obtain the mutated prediction vector.

[0081] Step 3.3: Repeat step 3.2 until all mutant models are covered.

[0082] Step 3.4: Repeat steps 3.1 to 3.3 until all unlabeled samples are traversed.

[0083] Step 4: Combine the differential weights obtained in step 2 to analyze the differences in the prediction vectors obtained in step 3 to obtain the priority score corresponding to each sample.

[0084] Step 4.1: Calculate the Euclidean distance between the predicted vector of the sample on each mutant model and the original predicted vector on the measured model.

[0085] Step 4.2: Multiply the distance of step 4.1 by the weight of step 2.3, sum up on 20 groups of model pairs (mutant model and measured model) to get the priority score.

[0086] Step 4.3: Repeat steps 4.1 and 4.2 until all unlabeled samples are traversed.

[0087] Step 5: Sort the priority scores obtained in step 4 from large to small.

[0088] Embodiment 4, as the fourth embodiment of the present application, provides a sensitive data verification model correctness test system for strengthening the priority of effective samples, comprising:

[0089] A generating module is configured to modify the structure of a to-be-tested reference model according to a plurality of preset model structure level mutation rules, and generate a plurality of corresponding mutant models;

[0090] A training weight module is configured to train the plurality of mutant models using a training sample set, and determine a corresponding differential weight for each mutant model according to whether the training process of each mutant model is successful;

[0091] A scoring module is configured to obtain the prediction results of the reference model and the successfully trained mutant models for the unlabeled samples, and calculate the difference between the prediction results in combination with the differential weight, to generate a priority score of the unlabeled samples;

[0092] A sorting module is configured to sort the plurality of unlabeled samples based on the priority score to obtain a sample priority list.

[0093] Embodiment 5, which is the fifth embodiment of the present application, is different from the previous four embodiments in that: the function, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0094] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a processor-based system, or other system that can fetch the instructions from a instruction execution system, apparatus, or device and execute the instructions, or in conjunction with such an instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device, or in conjunction with such an instruction execution system, apparatus, or device.

[0095] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic editing, interpretation, or necessary processing, and then stored in a computer memory if necessary. Other suitable media can also be used.

[0096] It should be understood that various portions of the application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies known in the art or their combinations can be used: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with appropriate combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0097] Example 6, reference Figures 1-3 As a sixth embodiment of the present application, a method for testing the correctness of the sensitive data verification model for strengthening the priority of the effective sample is provided. In order to verify the beneficial effects of the present application, scientific demonstration is carried out through experiments.

[0098] The experimental data is verified based on five image data sets of open source data sets MNIST, Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and their information is as follows:

[0099] (1) MNIST (1998): MNIST is an image data set of handwritten digits, containing 60000 training samples and 10000 test samples, with pixel size of 28*28. The data set is sponsored by the National Institute of Standards and Technology, including handwritten number pictures of 250 different people, 50% from high school students and 50% from census staff.

[0100] (2) Fashion-MNIST (2017): Fashion-MNIST is an image data set constructed with reference to MNIST, provided by the research department of Zalando. It covers 70000 front images of different articles of 10 categories (T-shirt, jeans, pullover, skirt, coat, sandal, shirt, sneaker, bag and short boots). The size, format and training test set division of Fashion MNIST are the same as MNIST.

[0101] (3) SVHN (2011): SVHN is a street view home digital data set, which comes from Google street view house number. The training set contains a total of 73257 training image samples, the test set contains a total of 26032 test image samples, and the whole data set contains 10 categories, i.e. numbers 0-9.

[0102] (4) CIFAR-10 (2009): CIFAR-10 is a small image dataset compiled by Alex et al. for recognizing common objects, including RGB images of airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks. The images are 32*32 pixels, and the dataset contains a total of 50,000 training samples and 10,000 test samples.

[0103] (5) CIFAR-100 (2009): CIFAR-100 is similar to CIFAR-10, containing RGB images for 20 main categories and 100 subcategories. The images are 32*32 pixels, and the dataset contains a total of 50,000 training samples and 10,000 test samples.

[0104] The experiment used the following three evaluation methods to assess the results:

[0105] (1) Defect Detection Rate (DDR): DDR measures the ability of a method to detect defects during testing. For a set of unlabeled test samples that require manual labeling, a good method can detect more model defects with a limited number of labels. Let N be the number of model defects detected in a set of unlabeled test samples when fully labeled. all For a correct test of the model, manually labeling the top n test samples can identify N model defects. n Then the method's DDR n Represented as N n and N all The ratio is shown in formula (1).

[0106] A higher DDR indicates that the method can detect more model defects in the target model with a limited number of labeled samples. We set n to 10%, 25%, and 50% of the total sample size to evaluate the method's ability to detect model defects with different numbers of labeled samples.

[0107] (2) Rank Area Under Curve (RAUC): RAUC is an evaluation metric for testing methods in existing research. The x-axis represents the number of test samples, and the y-axis represents the number of model defects detected. Let A be the area under the curve for the first m manually labeled test samples. m Ideal case (first N) all The sample just detected N all The area under the curve (of the model defect) is A. ide RAUC of the method m Represented as A m and A ide The ratio is shown in formula (2).

[0108] The larger the RAUC, the stronger the discrimination ability of the test method for the valid samples. In the experiments of this chapter, m was set to 100, 500, 1000 and all respectively to evaluate the discrimination ability of the test method for the valid samples in different priority intervals.

[0109] (3) Last Valid Sample Ratio (LVS): LVS is also used to evaluate the discrimination ability of the test method for the valid samples. RAUC is more inclined to the discrimination ability of the method for the valid samples in the high priority interval to evaluate the upper limit of the method. While LVS is designed to evaluate the overall discrimination ability of the method for the valid samples. Let the total number of samples be Num, and the ranking sequence of the valid samples with a ranking ratio of l in all valid samples in all samples be R l , then the LVS of the test method is l , which is the ratio of R l and Num, as shown in equation (3).

[0110] The smaller the LVS, the stronger the discrimination ability of the method for the overall valid samples. In the experiments, l was set to 0.5 and 0.9 respectively to evaluate the discrimination ability of the test method for the valid samples in different priority intervals.

[0111] The experiments were carried out on a standalone computer with the following configuration: 64-bit Windows 10 operating system, Intel(R) Core(TM) i7-9750H@2.60GHz CPU, 16GB RAM, NVIDIA GeForce RTX 2070 Max-Q version GPU. The versions of the deep learning frameworks used are TensorFlow v1.11.0 and Keras v2.2.4.

[0112] The specific process is as follows:

[0113] Step 1, apply each model structure mutation rule to the model under test to generate the corresponding mutant model.

[0114] Step 1.1, extract the original model structure design of the model under test from the code level

[0115] Step 1.2, traverse the mutation rule list and select the ith mutation rule R i , where the mutation rule name and description information is shown in Table 1, which contains 20 groups of proposed model structure mutation rules.

[0116] Table 1 Model structure mutation rule name and description

[0117]

[0118]

[0119] Step 1.3, the mutation rule R selected in step 1.2 i Apply to the original model structure obtained in step 1.1 Mutation model M was obtained i .

[0120] Step 1.4: Repeat steps 1.2 and 1.3 until all mutation rules have been used, finally obtaining the mutation model M1, ..., M 20 .

[0121] Step 2: Train the mutation models generated in Step 1 using the training sample set, and assign differential weights to each mutation model.

[0122] Step 2.1: Train the i-th mutation model M using the training sample set. i Obtain the trained mutation model M i '.

[0123] Step 2.2: If the model completes training normally, then set its differential weights w. i If the model fails to train or crashes midway due to structural modifications, its differential weights w are set to 1. i Set to 0.

[0124] Step 2.3: Repeat steps 2.1 and 2.2 until all training is complete, obtaining the trained mutation models M1', ..., M 20 ', and the corresponding difference weights w1, ..., w 20 .

[0125] Step 3: Input the unlabeled samples into the tested model and the mutation models obtained in Step 2 to obtain the corresponding prediction vectors.

[0126] Step 3.1: Input the j-th unlabeled sample into the model under test to obtain the prediction vector O. 0,j .

[0127] Step 3.2: Input the j-th unlabeled sample into the i-th mutation model M. i 'Obtain the prediction vector O i,j .

[0128] Step 3.3, repeat step 3.2, until all mutation models are input, and the prediction vector O is obtained. 1,j , ..., O 20,j .

[0129] Step 3.4: Repeat steps 3.1 to 3.3 until all unlabeled samples have been traversed.

[0130] Step 4, combine the differential weight obtained in step 2 with the prediction vector obtained in step 3 to perform difference analysis to obtain a priority score corresponding to each sample.

[0131] Step 4.1, calculate the prediction vector O i,j of the jth sample in the ith mutation model. 0,j The Euclidean distance is defined as shown in formula (4), where N represents the dimension of the vector.

[0132] Step 4.2, calculate the priority score score j of the jth sample, which is calculated as shown in formula (5).

[0133] Step 4.3, repeat steps 4.1 and 4.2 until all unlabeled samples are traversed to obtain priority scores score1, …, score n , where n represents the total number of unlabeled samples.

[0134] Step 5, sort the priority scores obtained in step 4 from high to low, manually label the samples with higher priority, and use them for defect detection of the sensitive data verification model.

[0135] The present application is a method for testing the correctness of a sensitive data verification model by strengthening the priority of effective samples. Based on five open-source image data sets, the DDR of the present application is the highest, the RAUC value is the largest, and the LVS is the smallest compared to several existing most advanced methods. This indicates that the present application has higher testing efficiency and lower testing cost in testing sensitive data verification models, which can better ensure the safety and reliability of sensitive data verification models and promote the use of artificial intelligence models to verify sensitive data to ensure data security.

[0136] It should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A method for testing the correctness of a sensitive data audit model that prioritizes samples for fortification, the method comprising: The method comprises the steps of: ​ According to a plurality of preset model structure level mutation rules, modifying the structure of the benchmark model to generate a plurality of corresponding mutant models; Using a training sample set to train a plurality of mutant models, and determining a corresponding differential weight for each mutant model according to whether the training process of each mutant model is successful; Obtaining the prediction results of the benchmark model and the successfully trained mutant models for the unlabeled samples, and combining the differential weights to calculate the difference between the prediction results to generate the priority score of the unlabeled samples; Based on the priority score, a plurality of unlabeled samples are sorted to obtain a sample priority list.

2. The method of claim 1, wherein the method of testing the correctness of the model of sensitive data verification of the strengthened valid sample priority is characterized in that: The model structure level mutation rules comprise: At least one of the hidden layer, convolutional layer or pooling layer in the benchmark model is subjected to an increase or deletion operation, an adjustment operation of the number of components within the layer, or a modification operation of the parameters of the components within the layer.

3. The method of claim 2, wherein the method of testing the correctness of the model of sensitive data verification of the strengthened valid sample priority is characterized in that: The adjustment operation of the number of components within the layer comprises adjusting the number of neurons in the hidden layer or adjusting the number of convolutional kernels in the convolutional layer; the modification operation of the parameters of the components within the layer comprises modifying the size or moving step of the convolutional kernel in the convolutional layer, or modifying the size or moving step of the pooling window in the pooling layer.

4. The method for testing the correctness of a sensitive data verification model that prioritizes effective samples as described in claim 3, characterized in that: The method further comprises the steps of: If the training process of the mutant model is successfully completed, the corresponding differential weight is set to a first preset value; If the training process of the mutant model fails or is interrupted, the corresponding differential weight is set to a second preset value.

5. The method for testing the correctness of a sensitive data verification model that prioritizes effective samples as described in claim 4, characterized in that: The method further comprises the steps of: Respectively calculating the distance between the prediction results of the benchmark model and each successfully trained mutant model for the unlabeled samples; Combining the distance and the corresponding differential weight by weighting to obtain the priority score of the unlabeled samples.

6. The method of claim 5, wherein the method of testing the correctness of the model of sensitive data verification of the strengthened valid sample priority is characterized in that: The prediction result is a prediction vector, and the distance is the Euclidean distance between the prediction vector output by the benchmark model and the prediction vector output by the mutant model.

7. The method of testing the correctness of a sensitive data audit model with reinforced valid sample priority as claimed in claim 6, wherein: The method further comprises the steps of:

8. A system for testing the correctness of a sensitive data checking model with enhanced sample priority, applying the method for testing the correctness of a sensitive data checking model with enhanced sample priority according to any one of claims 1 to 7, characterized in that, Selecting the unlabeled samples with high priority from the sample priority list for manual labeling, and using the manually labeled samples for defect detection of the benchmark model. The method comprises the steps of: A generating module is configured to modify the structure of the benchmark model according to a plurality of preset model structure level mutation rules to generate a plurality of corresponding mutant models; A training weight module is configured to use a training sample set to train a plurality of mutant models, and determine a corresponding differential weight for each mutant model according to whether the training process of each mutant model is successful; A scoring module is configured to obtain the prediction results of the benchmark model and the successfully trained mutant models for the unlabeled samples, and combine the differential weights to calculate the difference between the prediction results to generate the priority score of the unlabeled samples; A sorting module is configured to sort a plurality of unlabeled samples based on the priority score to obtain a sample priority list. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program is executed by the processor to implement the steps of the sensitive data checking model correctness testing method of reinforcing the effective sample priority according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the sensitive data checking model correctness testing method of reinforcing the effective sample priority according to any one of claims 1 to 7.

Images