Black-box Model Classification Method Based on Confidence Analysis

Abstract

The present invention discloses a black-box model type classification method based on confidence analysis. The implementation steps are as follows: classify the type of the black-box model to be classified by using the characteristics of confidence. Determine the type of the model to be detected with discrete confidence as a decision tree or a forest, determine the type of the model to be detected with symmetric confidence as a Gaussian kernel SVM, and determine the type of the model to be detected with linear solvability confidence as linear. The present invention detects the model by observing the different characteristics exhibited by the confidence output by the black-box model to be detected when the input data changes, and classifies the model, having the advantages of few accesses to the model to be detected and high efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and further relates to a black-box model classification method based on confidence analysis in the field of machine learning security technology. The present invention can obtain the type of the model to be detected by detecting the black-box model. Background Art

[0002] With the wide application of machine learning, some large enterprises provide machine learning as a service to users. As the core resource of these services, the information leakage of machine learning models will not only cause huge economic losses to service providers, but also cause the leakage of sensitive information of training data providers. Therefore, service providers only allow customers to access machine learning models through interfaces, and only feedback the results to customers, without providing additional information about the models. In this environment, the models of machine learning service platforms are black-box models for users. The detection of black-box models usually generates a surrogate model similar to the function of the black-box model. When the surrogate model is of the same type as the black-box model, the performance of the detection method is the best. However, there is currently no method for classifying black-box models.

[0003] General Motors Global Technology Operations LLC proposed a method and system for generating adversarial attacks on a black-box object detection algorithm for sensors in its patent document "Adversarial Attacks on Black-Box Object Detection Algorithms" (patent application number CN202010194023.6, publication number CN111753868A). The method first accesses the black-box model to obtain an initial training dataset, selects an architecture for the surrogate model, trains the surrogate model with the initial training set, and accesses the black-box model multiple times to update the training dataset and train the surrogate model, so that the output from the surrogate model can replicate the output of the black-box model, completing the detection of the black-box model. The disadvantage of this method is that it needs to access the black-box model multiple times to update the training dataset and train the surrogate model, resulting in a large number of accesses to the black-box model to be detected during the detection process and high overhead. Summary of the Invention

[0004] The purpose of the present invention is to propose a black-box model type detection method based on confidence analysis for the deficiencies of the above-mentioned existing technologies, so as to solve the problems of a large number of accesses to the black-box model to be detected and high overhead in the existing technologies.

[0005] The technical idea for achieving the object of the present invention is as follows: When constructing the data set, the present invention uses the confidence array output by the black box model to be detected, repeatedly utilizes the output of the black box model to be detected for the same data, avoids multiple accesses caused by the same data, and reduces the number of accesses to the black box model to be detected; the present invention detects the model type by observing the different characteristics shown by the confidence output by the black box model to be detected when the input data changes. The characteristics of different types of models are obtained by analyzing the relationship between the input and output of a large number of white box models. Only a single-digit number of input data is required to discover the characteristics of the confidence, significantly reducing the number of accesses to the black box model to be detected; based on the above idea, the present invention solves the problems of a large number of accesses and high overhead to the black box model to be detected.

[0006] The specific steps of the present invention are as follows:

[0007] Step 1, detecting the black box model to be detected by using the discreteness of the confidence:

[0008] Step 1.1, randomly generate at least five data matrices, and input all the generated data matrices into the black box model to be detected, output the confidence corresponding to each data matrix, and form a data set with all the data matrices and their corresponding confidences;

[0009] Step 1.2, randomly select at least three data matrices from the data set, and at least one of the data matrices has a confidence that is not equal to the confidences of the other data matrices;

[0010] Step 1.3, arbitrarily select at least one unselected element from each selected data matrix, select a random number for each selected element, and update the corresponding element by using the sum of the selected element and the selected random number respectively, where the set where the random number is located is {a|a∈(-0.2, -0.1)∪(0.1, 0.2)}, and the sum of the squares of the differences between the elements of the selected data matrix and the data matrix after updating its corresponding elements is less than ε, ε≤0.05;

[0011] Step 1.4, input all the data matrices after updating the elements into the black box model to be detected in sequence, and output the confidence corresponding to each data matrix;

[0012] Step 1.5, count the number of data matrices with equal confidences before and after updating for each selected data matrix;

[0013] Step 2, determine whether the proportion of the number of data matrices with equal confidences in the total number of selected data matrices is greater than 80%. If so, execute Step 3; otherwise, execute Step 4;

[0014] Step 3, determine whether there is at least one confidence level in the confidence levels of the data matrix after all updated elements that is not equal to any other confidence level. If so, determine that the confidence level of the black box model to be detected has discreteness and then execute Step 14; otherwise, execute Step 4;

[0015] Step 4, detect the black box model to be detected by using the symmetry of the confidence level:

[0016] Step 4.1, add all the data matrices after the updated elements in Step 1.3 and their corresponding confidence levels to the data set to obtain the updated data set;

[0017] Step 4.2, establish an empty confidence level list;

[0018] Step 4.3, arbitrarily select at least 3 unselected data matrices from the updated data set, and arbitrarily select at least 1 element from the selected data matrices;

[0019] Step 4.4, update the values of all the selected elements to b, input all the data matrices after the element values are updated into the black box model to be detected, output the confidence levels corresponding to the data matrices after the element values are updated, and save all the confidence levels to the confidence level list, where the initial value of b, b0 ∈ {10000k|k ∈ Z *};

[0020] Step 5, calculate the distance between every two confidence levels in the confidence level list;

[0021] Step 6, determine whether the distance between every two confidence levels in the confidence level list is less than δ. If so, execute Step 7; otherwise, execute Step 9, where δ ∈ (0, 10 -5 ;

[0022] Step 7, determine whether the absolute value of each element value in each confidence level saved in the confidence level list is less than λ. If so, execute Step 8; otherwise, execute Step 9, where λ ∈ [0.99, 1);

[0023] Step 8, determine whether the absolute value of each element value in each confidence level saved in the confidence level list is greater than β. If so, determine that the confidence level of the black box model to be detected has symmetry and then execute Step 13; otherwise, execute Step 9, where β ∈ (0, 0.001];

[0024] Step 9, detect the black box model to be detected by using linear solvability:

[0025] Step 9.1, construct the linear matrix equation y j = x j W + D, where y j is the confidence level corresponding to the jth data matrix, xj The j-th data matrix participating in solving the linear matrix equation, W is a matrix composed of g rows and k columns, the value of g is equal to the total number of elements in the data matrix, D is a matrix composed of 1 row and k columns. When the black box model to be detected is a binary classification model, the value of k is 1. When the black box model to be detected is a multi-classification model, the value of k is equal to the number of categories of the black box model to be detected;

[0026] Step 9.2, determine whether the total number of data matrices in the dataset updated in step 4.1 is less than e. If so, execute step 9.3; otherwise, execute step 9.4, where the value of e is equal to the sum of the total number of elements in matrix W and the total number of elements in matrix D;

[0027] Step 9.3, randomly generate t data matrices, input all the generated data matrices into the black box model to be detected, output the confidence level corresponding to each data matrix, and add all the data matrices and their corresponding confidence levels to the dataset updated in step 4.1 to obtain the currently updated dataset, where t is equal to the difference between e and the total number of data matrices contained in the dataset updated in step 4.1;

[0028] Step 9.4, convert all the data matrices in the current dataset into data matrices with only one row in turn, and use all the converted data matrices, the confidence level corresponding to each data matrix, and the iterative solution method of the linear equation system to solve the linear matrix equation y j = x j W + D, to obtain two parameter matrices W and D, and use W and D to form a new linear matrix equation Y = XW + D, where Y is the function value corresponding to the input matrix, and X is the input data matrix;

[0029] Step 9.5, input each data matrix in the dataset into the linear matrix equation Y = XW + D in turn to calculate its corresponding function value, and count the number of data matrices whose indices corresponding to the maximum value in the function values corresponding to each data matrix in the dataset are equal to the indices corresponding to the maximum value in the confidence levels. Among them, when the black box model to be detected is a binary classification model, the 2 elements of the function value corresponding to the data matrix are the calculation result of the linear matrix equation and the difference between 1 and the calculation result of the linear matrix equation. When the black box model to be detected is a multi-classification model, the function value corresponding to the data matrix is a one-dimensional matrix containing k elements;

[0030] Step 9.6, use the formula h = A / B to calculate the similarity between the linear matrix equation and the black box model to be detected, where A represents the number of data matrices whose maximum values in the function values corresponding to each data matrix in the dataset are equal to the maximum values in the confidence levels, and B represents the number of data matrices contained in the dataset;

[0031] Step 10: Determine whether the similarity between the linear matrix equation and the black box model to be detected is greater than 0.8. If so, determine that the confidence level of the black box model to be detected has linear solvability and then execute Step 11; otherwise, execute Step 12;

[0032] Step 11: Determine that the type of the black box model to be detected is a linear type;

[0033] Step 12: Determine that the type of the black box model to be detected is an unknown type;

[0034] Step 13: Determine that the type of the black box model to be detected is a Gaussian kernel support vector machine;

[0035] Step 14: Determine that the type of the black box model to be detected is a decision type.

[0036] The present invention has the following advantages compared with the prior art:

[0037] First, since the present invention detects the model type by observing the different characteristics shown by the confidence level output by the black box model to be detected when the input data changes, only a small amount of input data is required to discover the characteristics of the confidence level, which can significantly reduce the number of accesses to the black box model to be detected; overcoming the problem of the large number of accesses to the black box model to be detected existing in the prior art, making the present invention have the advantage of fewer accesses to the black box model to be detected.

[0038] Second, since the present invention uses a data matrix and its corresponding confidence level array to form a data set, the output of the black box model to be detected for the same data can be reused, thereby avoiding the problem of large overhead caused by repeated accesses to the same data, making the present invention have small overhead and improving the efficiency of black box model classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] Refer to Figure 1 and the embodiments for a further detailed description of the steps implemented by the present invention.

[0041] Step 1: Detect the black box model to be detected by using the discreteness of the confidence level.

[0042] Step 1.1, An embodiment of the present invention uses a logistic regression model trained with the breast cancer dataset as the black-box model to be detected, and its expression is Y = wX + p, where w, X, and p represent the weight matrix, the input data matrix, and the bias matrix respectively. According to the requirement that the input data format of the black-box model is 1 row and 30 columns, 5 data matrices are randomly selected from the breast cancer dataset, and all data matrices and their corresponding confidence levels are combined into a dataset. In the technical field, the data type of the confidence level is usually in matrix form.

[0043] Step 1.2, Randomly select three data matrices x1, x2, and x3 from the dataset, and the confidence levels corresponding to each selected data matrix are not equal.

[0044] Step 1.3, Arbitrarily select 2 elements from each selected data matrix, and the 6 elements of the three data matrices are respectively and Use and to update the corresponding element values respectively. The sum of the squares of the differences between the elements of the selected data matrix and the data matrix after updating its corresponding elements is less than ε. The larger the value of ε, the greater the difference between the selected data matrix and the data matrix after updating its corresponding elements, and the greater the possibility that the black-box model to be detected classifies the above two data matrices into different categories. Therefore, the possibility of judging the confidence levels of the above two data as discrete is smaller, which will reduce the correct rate of classifying the type of the black-box model. The experimental results of the present invention show that there is no obvious difference in the correct rate of classifying the type of the black-box model when ε is in the range (0, 0.05]. Therefore, in the embodiment of the present invention, ε = 0.05.

[0045] Step 1.4, Input the three data matrices with updated element values into the black-box model to be detected in sequence, and output the confidence level corresponding to each data matrix.

[0046] Step 1.5, Count the number of equal confidence levels before and after the update for each selected data matrix.

[0047] Step 2, Judge whether the number of equal confidence levels is equal to 3. If so, execute Step 3; otherwise, execute Step 4.

[0048] Step 3, Judge whether the confidence levels of all data matrices after updating the elements are not equal. If so, determine that the confidence level of the black-box model to be detected is discrete and then execute Step 14; otherwise, execute Step 4.

[0049] Step 4, Detect the black-box model to be detected by using the symmetry of the confidence level.

[0050] Step 4.1: Add all the data matrices after updating the elements in Step 1.3 and their corresponding confidence levels to the dataset to obtain an updated dataset.

[0051] Step 4.2: Establish an empty confidence level list.

[0052] Step 4.3: Arbitrarily select at least 3 unselected data matrices from the updated dataset, and arbitrarily select 1 element from the selected data matrices.

[0053] Step 4.4: Update the value of the selected element to 100000, input all the data matrices after updating the element values into the black box model to be detected, output the confidence levels corresponding to the data matrices after updating the element values, and save all the confidence levels to the confidence level list.

[0054] Step 5: Select any one of the two formulas and to calculate the distance between every two confidence levels in the confidence level list.

[0055] In the embodiment of the present invention, select the formula to calculate the distance between every two confidence levels in the confidence level list. C3 and C4 represent two confidence levels. When C3 and C4 are equal, the distance between them is 0. n represents the total number of elements contained in C3 and C4. |·| represents the absolute value operation, and ∑ represents the summation operation. In the embodiment of the present invention, n = 2.

[0056] Step 6: Determine whether the distance between every two confidence levels in the confidence level list is less than δ. If so, execute Step 7; otherwise, execute Step 9. Wherein, δ ∈ (0, 10 -5 . The distance between two confidence levels being small enough is a prerequisite for the symmetry of the confidence levels. The larger the value of δ, the greater the possibility of judging the confidence levels as having symmetry, which will reduce the correct rate of classifying the type of the black box model. The results of multiple experiments of the present invention show that there is no obvious difference in the correct rate of classifying the type of the black box model when δ is in the range (0, 10 -5 . In the embodiment of the present invention, δ = 10 -5 .

[0057] The distance between the two confidence levels is calculated by arbitrarily selecting one formula from the two formulas and . Wherein, L represents the distance between two confidence levels C1 and C2. When C1 and C2 are equal, the distance between them is 0. n represents the total number of elements contained in C1 and C2. |·| represents the absolute value operation, and ∑ represents the summation operation.

[0058] In the embodiment of the present invention, select Formula for calculating the distance between every two confidences in the confidence list. C3 and C4 represent two confidences. When C3 and C4 are equal, the distance between them is 0. n represents the total number of elements in C3 and C4. |·| represents the absolute value operation, and ∑ represents the summation operation. In the embodiments of the present invention, n = 2.

[0059] Step 7: Determine whether the absolute value of each element value in each confidence saved in the confidence list is less than λ. If so, execute Step 8; otherwise, execute Step 9. Here, λ ∈ [0.99, 1), and λ is used to determine whether there is an element value in the confidence whose absolute value tends to 1. The smaller the λ value, the more likely the absolute values of all element values in the confidence tend to 1. If the confidence includes an element whose absolute value of the numerical value tends to 1, it will be determined that it has no symmetry with all the other confidences in the confidence list, thereby reducing the correct rate of black box model type classification. The experimental results of the present invention show that when λ is in the range [0.99, 1), there is no obvious difference in the correct rate of black box model type classification of the present invention. Therefore, in the embodiments of the present invention, λ = 0.999.

[0060] Step 8: Determine whether the absolute value of each element value in each confidence saved in the confidence list is greater than β. If so, after determining that the confidence of the black box model to be detected has symmetry, execute Step 13; otherwise, execute Step 9. Here, β ∈ (0, 0.001], and β is used to determine whether there is an element value in the confidence whose absolute value tends to 0. The larger the β value, the more likely the absolute values of all element values in the confidence tend to 0. If the confidence contains an element whose absolute value of the numerical value tends to 0, it will be determined that it has no symmetry with all the other confidences in the confidence list, thus reducing the correct rate of black box model type classification. The experimental results of the present invention show that when β is in the range (0, 0.001], there is no obvious difference in the correct rate of black box model type classification of the present invention. Therefore, in the embodiments of the present invention, β = 0.0001.

[0061] Step 9: Detect the black box model to be detected using linear solvability.

[0062] Step 9.1: Construct a linear matrix equation y j = x j W + D, where y j is the confidence corresponding to the jth data matrix, and x jThe j-th data matrix participating in solving the linear matrix equation. All data matrices participating in solving the linear matrix equation have the same total number of elements. W is a matrix composed of g rows and k columns, where the value of g is equal to the total number of elements in the data matrix. D is a matrix composed of 1 row and k columns. When the black box model to be detected is a binary classification model, the value of k is 1. When the black box model to be detected is a multi-classification model, the value of k is equal to the number of categories of the black box model to be detected. In the embodiments of the present invention, the black box model to be detected is a binary classification model, k = 1, and g = 30.

[0063] Step 9.2, determine whether the total number of data matrices in the dataset updated in step 4.1 is less than e. If so, execute step 9.3; otherwise, execute step 9.4. Here, the value of e is equal to the sum of the number of elements in matrix W and the number of elements in matrix D. In the embodiments of the present invention, e = 31.

[0064] Step 9.3, randomly select t data matrices from the breast cancer dataset according to the requirements of the black box model to be detected for the input data matrix. Sequentially input the selected data matrices into the black box model to be detected, and output the confidence level corresponding to each data matrix. Add all data matrices and their corresponding confidence levels to the dataset updated in step 4.1 to obtain the currently updated dataset. Here, t is equal to the difference between e and the total number of data matrices contained in the dataset updated in step 4.1. In the embodiments of the present invention, e = 31, there are 8 data matrices in the dataset, and t = 23.

[0065] Step 9.4, sequentially convert all data matrices in the current dataset into data matrices with only one row. Use all the converted data matrices, the confidence level corresponding to each data matrix, and the iterative solution method of the linear equations to solve the linear matrix equation y j = x j W + D to obtain two parameter matrices W and D. The linear matrix equation can be converted into a system of linear equations, so the method of solving the system of linear equations can be used to solve the linear matrix equation. And use W and D to form a new linear matrix equation Y = XW + D, where Y is the function value corresponding to the input matrix, and X is the input data matrix. In the embodiments of the present invention, the iterative solution method of the linear equations used is the Newton iterative solution method.

[0066] Step 9.5, input each data matrix in the dataset into the linear matrix equation Y = XW + D in sequence to calculate its corresponding function value, and count the number of data matrices in the dataset where the index corresponding to the maximum value in the function values corresponding to each data matrix is equal to the index corresponding to the maximum value in the confidence levels. Among them, when the black-box model to be detected is a binary classification model, the two elements of the function value corresponding to the data matrix are respectively the calculation result of the linear matrix equation and the difference between 1 and the calculation result of the linear matrix equation. When the black-box model to be detected is a multi-classification model, the function value corresponding to the data matrix is an array containing k elements.

[0067] Step 9.6, use the formula h = A / B to calculate the similarity between the linear matrix equation and the black-box model to be detected, where A represents the number of data matrices in the dataset where the maximum value in the function values corresponding to each data matrix is equal to the maximum value in the confidence levels, and B represents the number of data matrices contained in the dataset.

[0068] Step 10, determine whether the similarity between the linear matrix equation and the black-box model to be detected is greater than 0.8. If so, determine that the confidence level of the black-box model to be detected has linear solvability and then execute Step 11. Otherwise, execute Step 12.

[0069] Step 11, determine the type of the black-box model to be detected as a linear type. The described linear type includes types such as a linear kernel support vector machine model and a logistic regression model.

[0070] Step 12, determine the type of the black-box model to be detected as an unknown type.

[0071] Step 13, determine the type of the black-box model to be detected as a Gaussian kernel support vector machine.

[0072] Step 14, determine the type of the black-box model to be detected as a decision type. The described decision type is a type composed of an ensemble model integrated by using a boosting aggregation algorithm or a boosting algorithm with multiple decision trees and a decision tree model.

Claims

1. A method for classifying black-box model types based on confidence analysis, characterized in that Classify the type of the black-box model to be classified by using the characteristics of confidence. Determine the type of the model to be detected with discrete confidence as a decision tree or forest, determine the type of the model to be detected with symmetric confidence as Gaussian kernel SVM, and determine the type of the model to be detected with linear solvability confidence as linear. The specific steps of this method are as follows: Step 1, detect the black-box model to be detected by using the discreteness of confidence: Step 1.1, randomly generate at least five data matrices, input all the generated data matrices into the black-box model to be detected, output the confidence corresponding to each data matrix, and form a data set with all the data matrices and their corresponding confidences; Step 1.2, randomly select at least three data matrices from the data set, and at least one of the confidences corresponding to the data matrices is not equal to the confidences corresponding to the other data matrices; Step 1.3, randomly select at least one unselected element from each selected data matrix, select a random number for each selected element, and update the corresponding element with the sum of the selected element and the selected random number respectively, where the set where the random number is located is {a|a∈(-0.2, -0.1)∪(0.1, 0.2)}, and the sum of the squares of the differences between the elements of the selected data matrix and the data matrix after updating its elements is less than ε, ε≤0.05; Step 1.4, input all the data matrices after updating the elements into the black-box model to be detected in sequence, and output the confidence corresponding to each data matrix; Step 1.5, count the number of data matrices with equal confidence before and after updating for each selected data matrix; Step 2, judge whether the proportion of the number of data matrices with equal confidence in the total number of selected data matrices is greater than 80%. If so, execute Step 3; otherwise, execute Step 4; Step 3, judge whether there is at least one confidence in the confidences of all the data matrices after updating the elements that is not equal to the other confidences. If so, determine that the confidence of the black-box model to be detected has discreteness and then execute Step 14; otherwise, execute Step 4; Step 4, detect the black-box model to be detected by using the symmetry of confidence: Step 4.1, add all the data matrices after updating the elements in Step 1.3 and their corresponding confidences to the data set to obtain the updated data set; Step 4.2, establish an empty confidence list; Step 4.3, randomly select at least 3 unselected data matrices from the updated data set, and randomly select at least 1 element from the selected data matrices; Step 4.4, update the values of all selected elements to b, input all the data matrices after the element value update into the black box model to be detected, output the confidence corresponding to the data matrix after the element value update, and save all the confidences into the confidence list, where the initial value b0 of b ∈ {10000k|k ∈ Z *}; Step 5, calculate the distance between every two confidences in the confidence list; Step 6, determine whether the distance between every two confidences in the confidence list is less than δ. If so, execute Step 7; otherwise, execute Step 9, where δ ∈ (0, 10 -5 ; Step 7, judge whether the absolute value of each element value in each confidence saved in the confidence list is less than λ. If so, execute Step 8; otherwise, execute Step 9, where λ∈[0.99, 1); Step 8, judge whether the absolute value of each element value in each confidence saved in the confidence list is greater than β. If so, determine that the confidence of the black-box model to be detected has symmetry and then execute Step 13; otherwise, execute Step 9, where β∈(0, 0.001]; Step 9. Detect the black-box model to be detected using linear solvability: Step 9.1, construct the linear matrix equation y j = x j W + D, where y j is the confidence corresponding to the j-th data matrix, x j is the j-th data matrix participating in solving the linear matrix equation, W is a matrix composed of g rows and k columns, the value of g is equal to the total number of elements in the data matrix, D is a matrix composed of 1 row and k columns, when the black box model to be detected is a binary classification model, the value of k is 1, and when the black box model to be detected is a multi-classification model, the value of k is equal to the number of types of the black box model to be detected; Step 9.

2. Determine whether the total number of data matrices in the dataset updated in Step 4.1 is less than e. If so, execute Step 9.3; otherwise, execute Step 9.4, where the value of e is equal to the sum of the total number of elements in matrix W and the total number of elements in matrix D; Step 9.

3. Randomly generate t data matrices, input all the generated data matrices into the black-box model to be detected, output the confidence level corresponding to each data matrix, and add all the data matrices and their corresponding confidence levels to the dataset updated in Step 4.1 to obtain the currently updated dataset, where t is equal to the difference between e and the total number of data matrices contained in the dataset updated in Step 4.1; Step 9.4, convert all data matrices in the current dataset into data matrices with only one row in sequence, and use all the converted data matrices, the confidence corresponding to each data matrix, and the iterative solution method of the linear equation system to solve the linear matrix equation y j = x j W + D to obtain two parameter matrices W and D, and use W and D to form a new linear matrix equation Y = XW + D, where Y is the function value corresponding to the input matrix, and X is the input data matrix; Step 9.

5. Input each data matrix in the dataset into the linear matrix equation Y = XW + D in turn to calculate its corresponding function value, and count the number of data matrices whose indices corresponding to the maximum value in the function values corresponding to each data matrix in the dataset are equal to the indices corresponding to the maximum value in the confidence levels. When the black-box model to be detected is a binary classification model, the two elements of the function value corresponding to the data matrix are the calculation result of the linear matrix equation and the difference between 1 and the calculation result of the linear matrix equation. When the black-box model to be detected is a multi-classification model, the function value corresponding to the data matrix is a one-dimensional matrix containing k elements; Step 9.

6. Use the formula h = A / B to calculate the similarity between the linear matrix equation and the black-box model to be detected, where A represents the number of data matrices whose function values corresponding to each data matrix in the dataset and the maximum values in the confidence levels are equal, and B represents the number of data matrices contained in the dataset; Step 10. Determine whether the similarity between the linear matrix equation and the black-box model to be detected is greater than 0.

8. If so, determine that the confidence level of the black-box model to be detected has linear solvability and then execute Step 11; otherwise, execute Step 12; Step 11. Determine the type of the black-box model to be detected as a linear type; Step 12. Determine the type of the black-box model to be detected as an unknown type; Step 13. Determine the type of the black-box model to be detected as a Gaussian kernel support vector machine; Step 14. Determine the type of the black-box model to be detected as a decision type.

2. The method for detecting the black box model type based on confidence analysis according to claim 1, wherein The data matrices generated in Step 1.1 and Step 9.3 refer to data matrices that meet the format requirements of the black-box model to be detected for input data, and the value of each element in the matrix is within the value range required by the black-box model to be detected.

3. The method for detecting the black box model type based on confidence analysis according to claim 1, characterized in that The distance between the two confidence levels in Step 5 and Step 6 is the distance calculated by selecting any one of the following two formulas: where L represents the distance between the two confidence levels C1 and C2. When C1 and C2 are equal, the distance between them is 0, n represents the total number of elements in C1 and C2, |·| represents the absolute value operation, and ∑ represents the summation operation.

4. The method for detecting the black box model type based on confidence analysis according to claim 1, wherein The linear type described in Step 11 includes types such as linear kernel support vector machine models and logistic regression models.

5. The method for detecting the black box model type based on confidence analysis according to claim 1, characterized in that, The decision type described in step 14 refers to the type composed of an ensemble model formed by integrating multiple decision trees using the guided aggregation algorithm or the boosting algorithm and a decision tree model.

Images