A method for evaluating the performance of artificial intelligence models based on privacy computing

Abstract

The present invention provides a method for evaluating the performance of artificial intelligence models based on privacy computing, comprising the following steps: S1: randomly generating a manufacturer queue; S2: comparing the performance of two artificial intelligence models; S3: sorting the performance of n artificial intelligence models; S4: calculating the mean of the performance of n artificial intelligence models; S5: calculating the variance of the performance of n artificial intelligence models; The method for evaluating the performance of artificial intelligence models based on privacy computing proposed by the present invention uses a privacy computing method to compare all artificial intelligence model performance data in pairs, and obtains the performance ranking of all models through a sorting algorithm; uses a privacy computing method to compare and count the performance of artificial intelligence models, and realizes the evaluation and statistics of artificial intelligence model performance under the condition that humans cannot disclose their own performance data or leak any information, thereby protecting the privacy of each manufacturer's artificial intelligence model performance data.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for evaluating the performance of an artificial intelligence model based on privacy computing. Background Art

[0002] Privacy computing is an information technology that aims to analyze and calculate data without leaking the original data. Its core is to ensure the security and privacy of data while allowing data users to perform necessary data analysis and processing. As users' demand for data privacy protection becomes increasingly strong, privacy computing technology is becoming more and more important.

[0003] In recent years, with the emergence of models such as ChatGPT and Sora, artificial intelligence models have increasingly come into the public eye. Artificial intelligence models are a combination of mathematical and statistical methods used to solve specific artificial intelligence tasks; these models are usually trained with large amounts of data so that they can discover patterns and regularities in the data to make predictions or perform specific tasks; artificial intelligence models can learn regularities and patterns from large amounts of data and make decisions or predictions based on what they have learned.

[0004] The ability of an AI model is usually verified by performing a specific task, such as classification, prediction, generation, etc. The ability of a model to perform a task is called the performance of the model. The performance of the model is measured by how well the task is performed by using the same set of data for testing. Different AI models are tested using the same data, and the test results are compared with the correct answer. If they are consistent with the correct answer, they are correct, and if they are inconsistent, they are wrong. The accuracy or recall rate of the model under this set of test data is thus obtained, which is the performance of the model. The industry has compiled many similar data sets, such as ImageNet, CoCo, etc. Generally, a higher accuracy rate tested on a certain data set is used to measure whether the performance of a new AI model has been improved compared to the previous one.

[0005] The accuracy of an artificial intelligence model in performing a task on a certain data set is an important attribute for measuring the model's capabilities. To some extent, this accuracy is an important data indicator of the artificial intelligence model. In the early stages of the current development of artificial intelligence, scientific research institutions or companies generally take the initiative to publish such data in order to highlight the performance of the model; however, in the later stages when competition is more intense, more and more companies may regard the true performance of the model as a company secret and prohibit it from being disclosed to the public; however, in the process of model capability evaluation or bidding, the organizer or Party A needs to know the quality of the model capabilities participating in the selection in order to give a ranking or evaluation result, and needs to understand the mean and variance of the performance of all models in order to understand the performance distribution of the model; how to complete the performance evaluation between models without disclosing the specific performance parameters of each model and calculate indicators such as the mean and variance of the artificial intelligence model is an urgent problem to be solved. Summary of the invention

[0006] The purpose of this invention is to solve the shortcomings existing in the prior art and to propose a method for evaluating the performance of artificial intelligence models based on privacy computing.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for evaluating the performance of an artificial intelligence model based on privacy computing, comprising the following steps:

[0009] S1: Randomly generate manufacturer queues;

[0010] Assume that there are n AI models participating in the evaluation, each AI model corresponds to a manufacturer, and the manufacturers of the n models are randomly arranged into a queue M, where M={M1, M2, ..., M i ,......,M n};

[0011] S2: Compare the performance of two AI models;

[0012] Randomly select two manufacturers from the queue M, denoted as Manufacturer A and Manufacturer B. The two artificial intelligence models held by Manufacturer A and Manufacturer B have completed the test of the same data in the evaluation and obtained the performance data of their own artificial intelligence models; the performance data of Manufacturer A is denoted as i, and the performance data of Manufacturer B is denoted as j;

[0013] The i and j are positive integers in the range [0-1000]. The performance data of each manufacturer is private and not disclosed to the public.

[0014] The following sub-steps are included:

[0015] S21: Generate public and private keys of the performance data of each artificial intelligence model through encryption algorithms. Manufacturer A obtains ciphertext based on the public key of Manufacturer B, converts the ciphertext into data w and transmits it to Manufacturer B;

[0016] The two parties jointly determine an asymmetric encryption algorithm, and use the asymmetric encryption algorithm to generate public and private keys for the performance of their respective artificial intelligence models, where the private key is kept by the manufacturer itself and the public key is disclosed to the other party;

[0017] Manufacturer A randomly selects data x and y within the range of values ​​allowed by the formula, and obtains a positive integer z based on x and y. The formula is as follows:

[0018] ;

[0019] Manufacturer A encrypts data z according to encryption function Encrypt and manufacturer B's public key to obtain ciphertext v; v=Encrpt(z) 乙 , where B refers to the public key of manufacturer B;

[0020] The ciphertext v is converted into a positive integer. Manufacturer A adds the positive integer v to the performance data of the AI ​​model corresponding to Manufacturer A to obtain w=v+i, and transmits the value of w to Manufacturer B;

[0021] S22: Manufacturer B receives data w, constructs a sequence pair based on the received data, and decrypts the sequence;

[0022] Manufacturer B receives data w, but cannot obtain valid information from it. It takes the following steps:

[0023] Manufacturer B uses the received w to construct a sequence W containing 10,000 elements;

[0024] W={w-1, w-2, w-3,……, wi, …,w-10000}, i is a positive integer;

[0025] Manufacturer B decrypts all elements in the W sequence using the decryption function Decrypt and Manufacturer B's private key, and obtains Decrypt(W) 乙 sequence:

[0026] Decrypt(W) 乙 = { Decrypt(w-1) 乙 ,Decrypt(w-2) 乙 ,Decrypt(w-3) 乙 ,……,Decrypt(wi) 乙 ,……,Decrypt(w-10000) 乙};

[0027] And because w=v+i;

[0028] but ;

[0029] Since z is obtained by Manufacturer A randomly selecting x and y according to the formula, Manufacturer B is unaware of it, so Manufacturer B cannot identify which element z is in the sequence.

[0030] S23: Taking the modulus of the decrypted sequence to obtain a remainder sequence E;

[0031] Manufacturer B randomly selects a prime number p and Decrypt(W) 乙 All elements in the sequence are modulo, that is, each element in the sequence is divided by p. Since the divisor p is a prime number, the remainder of the element divided by p is a positive integer greater than 0, and the remainder sequence E={e1, e2, e3,……, e i , ……, e 10000};

[0032] S24: obtain sequence G;

[0033] In the interval (1, ) randomly select a positive integer q;

[0034] The model performance data of manufacturer B is j. The following operations are performed on the elements before and including the jth position in sequence E:

[0035] If the value of the element after adding q is greater than or equal to p, then subtract the positive integer q from the element; conversely, if the value of the element after adding q is less than p, then add or subtract the positive integer q from the element;

[0036] We get the sequence G={e1±q, e2±q, e3±q, …, e j ±q, e j+1 , e j+2 ……, e 10000};

[0037] If i is less than or equal to j, the i-th element e in sequence E i It will become e i ±q, if i is greater than j, then the i-th element e in the sequence E i remain unchanged;

[0038] S25: Encrypt all elements in the G sequence to obtain the H sequence;

[0039] Manufacturer B encrypts all elements of the G sequence using the encryption function Encrypt and the public key of Manufacturer A, and obtains the sequence H=Encrpt(G) 甲 = { Encrpt (e1±q)甲 , Encrypt (e2±q) 甲 , Encrypt (e3±q) 甲 ,……, Encrypt (e j ±q) 甲 , Encrypt (e j+1 ) 甲 , Encrypt (e j+2 ) 甲 …… Encrypt (e 10000 ) 甲};

[0040] Transmit sequence H and prime number p to manufacturer A;

[0041] S26: Complete the comparison of the performance of the AI models of manufacturers A and B;

[0042] Manufacturer A receives sequence H and prime number p, decrypts the elements in sequence H through the decryption function Decrypt and the private key of manufacturer A, and obtains sequence G = {e1±q, e2±q, e3±q, ……, e j ±q, e j+1 , e j+2 ……,e 10000};

[0043] Take the modulus of the positive integer z with the prime number p to obtain the remainder s, compare the i-th element in sequence G with the remainder s. If the two are equal, it means that the i-th element has not been added or subtracted by q, that is, the i-th bit is after the j-th bit, i>j, and the performance data of manufacturer A's model is greater than that of manufacturer B's model, and the performance of manufacturer A is higher than that of manufacturer B;

[0044] If the two are not equal, it means that the i-th element has been added or subtracted by q by manufacturer B, that is, the i-th bit is before the j-th bit, i<j, and the performance data of manufacturer A's model is less than that of manufacturer B's model, and the performance of manufacturer A is lower than that of manufacturer B;

[0045] Realize the comparison of the performance of the AI models of manufacturers A and B.

[0046] S3: Sort the performance of n AI models;

[0047] It includes the following sub-steps:

[0048] S31: Compare the performance of the AI models of adjacent manufacturers in queue M in turn, obtain the model with the best performance in queue M, remove this model from queue M and put it into queue C;

[0049] Gradually compare the performance of two adjacent models in queue M according to the method for comparing the performance of two artificial intelligence models in step S2;

[0050] Specifically, the model performances of the first manufacturer and the second manufacturer in queue M are compared. If the model performance of the first manufacturer is lower than that of the second manufacturer, their positions remain unchanged; if the model performance of the first manufacturer is higher than that of the second manufacturer, their positions in queue M are swapped.

[0051] Then compare the model performances of the second and third manufacturers in queue M. If the model performance of the second manufacturer is lower than that of the third manufacturer, the positions of the two manufacturers remain unchanged; if the model performance of the second manufacturer is higher than that of the third manufacturer, the positions of the two manufacturers are swapped.

[0052] According to the above method, the model performances of two adjacent manufacturers in queue M are compared in turn until the model performances of the n-1th and nth manufacturers are compared. If the model performance of the n-1th manufacturer is greater than that of the nth manufacturer, the model of the n-1th manufacturer is the best model in queue M; otherwise, the model of the nth manufacturer is the best model in queue M. The best model in queue M is removed from queue M and enters the empty queue C. At this time, the length of queue M is reduced by 1 and the length of queue C is increased by 1.

[0053] S32: complete the performance ranking of n artificial intelligence models;

[0054] Step S31 is repeated until all elements in queue M are removed to queue C in sequence, that is, the length of queue M is 0 and the length of queue C is n; queue C={C1, C2, C3, ... C n};

[0055] At this point, the order of the elements in queue C is the order of the performance of the n vendors’ artificial intelligence models from high to low, and the performance ranking of the n artificial intelligence models is completed;

[0056] S4: Calculate the mean of the performance of n artificial intelligence models;

[0057] Set the queue C={C1, C2, C3, … C n Each manufacturer in the data multiplies its corresponding AI model performance data by 10,000 and randomly divides it into n-1 parts, and randomly transmits these n-1 parts of split data to other n-1 manufacturers. Then each manufacturer will get the split data from the other n-1 manufacturers, and because they only get the split data, each manufacturer cannot get any valid information from these n-1 parts of data.

[0058] Each manufacturer will add up the n-1 pieces of data it receives and make the result public, obtaining n pieces of public data. The n pieces of public data are added together to obtain the sum of all model performance data. The sum of all model performance data is divided by n, and then divided by 10,000 to obtain the mean performance of these n artificial intelligence models.

[0059] S5: Calculate the variance of the performance of n AI models;

[0060] Compute queue C = {C1, C2, C3, ... C n} and the square value of the difference between the artificial intelligence model performance data of each manufacturer in step S4, each manufacturer randomly divides the corresponding square value into n-1 parts, and randomly transmits the n-1 parts of the split data to the other n-1 manufacturers, then each manufacturer will obtain the split data from the other n-1 manufacturers, and because the obtained data is the split data, each manufacturer cannot obtain any valid information from the n-1 parts of the data;

[0061] Each manufacturer will add up the n-1 pieces of data it receives and make the result public, obtaining n pieces of public data. The n pieces of public data will be added together to get the total, and the total will be divided by n, and then divided by 10,000 to obtain the variance of the performance data of these n artificial intelligence models.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The method for evaluating the performance of artificial intelligence models based on privacy computing proposed in the present invention compares the performance data of all artificial intelligence models in pairs through the privacy computing method, and obtains the performance ranking of all models through the sorting algorithm; then obtains the measurement of all models through the privacy computing method, and calculates the mean and variance of all models;

[0064] Use privacy computing methods to compare and count the performance of artificial intelligence models. Under the condition that humans only keep their own performance data confidential and do not leak any information, it is possible to evaluate and count the performance of artificial intelligence models and protect the privacy of each manufacturer's artificial intelligence model performance data. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flowchart of the steps of a method for evaluating the performance of artificial intelligence models based on privacy computing in the present invention. DETAILED DESCRIPTION

[0066] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.

[0067] like Figure 1 As shown, a method for evaluating the performance of artificial intelligence models based on privacy computing includes the following steps:

[0068] S1: Randomly generate manufacturer queues;

[0069] Assume that there are n AI models participating in the evaluation, each AI model corresponds to a manufacturer, and the manufacturers of the n models are randomly arranged into a queue M, where M={M1, M2, ..., M i ,......,M n};

[0070] S2: Compare the performance of two AI models;

[0071] Randomly select two manufacturers from the queue M, denoted as Manufacturer A and Manufacturer B. The two artificial intelligence models held by Manufacturer A and Manufacturer B have completed the test of the same data in the evaluation and obtained the performance data of their own artificial intelligence models; the performance data of Manufacturer A is denoted as i, and the performance data of Manufacturer B is denoted as j;

[0072] The i and j are positive integers in the range [0-1000]. The performance data of each manufacturer is private and not disclosed to the public.

[0073] The following sub-steps are included:

[0074] S21: Generate public and private keys of the performance data of each artificial intelligence model through encryption algorithms. Manufacturer A obtains ciphertext based on the public key of Manufacturer B, converts the ciphertext into data w and transmits it to Manufacturer B;

[0075] The two parties jointly determine an asymmetric encryption algorithm, and use the asymmetric encryption algorithm to generate public and private keys for the performance of their respective artificial intelligence models, where the private key is kept by the manufacturer itself and the public key is disclosed to the other party;

[0076] Manufacturer A randomly selects data x and y within the range of values ​​allowed by the formula, and obtains a positive integer z based on x and y. The formula is as follows:

[0077] ;

[0078] Manufacturer A encrypts data z according to encryption function Encrypt and manufacturer B's public key to obtain ciphertext v; v=Encrpt(z) 乙 , where B refers to the public key of manufacturer B;

[0079] The ciphertext v is converted into a positive integer. Manufacturer A adds the positive integer v to the performance data of the AI ​​model corresponding to Manufacturer A to obtain w=v+i, and transmits the value of w to Manufacturer B;

[0080] S22: Manufacturer B receives data w, constructs a sequence pair based on the received data, and decrypts the sequence;

[0081] Manufacturer B receives data w, but cannot obtain valid information from it. It takes the following steps:

[0082] Manufacturer B uses the received w to construct a sequence W containing 10,000 elements;

[0083] W={w-1, w-2, w-3,……, wi, …,w-10000}, i is a positive integer;

[0084] Manufacturer B decrypts all elements in the W sequence using the decryption function Decrypt and Manufacturer B's private key, and obtains Decrypt(W) 乙 sequence:

[0085] Decrypt(W) 乙 = { Decrypt(w-1) 乙 ,Decrypt(w-2) 乙 ,Decrypt(w-3) 乙 ,……,Decrypt(wi) 乙 ,……,Decrypt(w-10000) 乙};

[0086] And because w=v+i;

[0087] but ;

[0088] Since z is obtained by Manufacturer A randomly selecting x and y according to the formula, Manufacturer B is unaware of it, so Manufacturer B cannot identify which element z is in the sequence.

[0089] S23: Taking the modulus of the decrypted sequence to obtain a remainder sequence E;

[0090] Manufacturer B randomly selects a prime number p and Decrypt(W) 乙 All elements in the sequence are modulo, that is, each element in the sequence is divided by p. Since the divisor p is a prime number, the remainder of the element divided by p is a positive integer greater than 0, and the remainder sequence E={e1, e2, e3,……, e i , ……, e 10000};

[0091] S24: obtain sequence G;

[0092] In the interval (1, ) randomly select a positive integer q;

[0093] The model performance data of manufacturer B is j. The following operations are performed on the elements before and including the jth position in sequence E:

[0094] If the value of the element after adding q is greater than or equal to p, then subtract the positive integer q from the element; conversely, if the value of the element after adding q is less than p, then add or subtract the positive integer q from the element;

[0095] We get the sequence G={e1±q, e2±q, e3±q, …, e j ±q, e j+1 , e j+2 ……, e 10000};

[0096] If i is less than or equal to j, the i-th element e in sequence E i It will become e i ±q, if i is greater than j, then the i-th element e in the sequence E i remain unchanged;

[0097] S25: Encrypt all elements in the G sequence to obtain the H sequence;

[0098] Manufacturer B encrypts all elements of the G sequence using the encryption function Encrypt and the public key of Manufacturer A, and obtains the sequence H=Encrpt(G) 甲 = { Encrpt (e1±q) 甲 , Encrpt (e2±q) 甲 , Encrpt (e3±q) 甲 ,……, Encrpt (e j ±q) 甲 , Encrpt (e j+1 ) 甲 , Encrpt (e j+2 ) 甲 …… Encrpt (e 10000 ) 甲};

[0099] Transmit sequence H and prime number p to manufacturer A;

[0100] S26: Complete the comparison of the performance of the artificial intelligence models of Manufacturer A and Manufacturer B;

[0101] Manufacturer A receives the sequence H and prime number p, and decrypts the elements in sequence H using the decryption function Decrypt and Manufacturer A's private key to obtain the sequence G={ e1±q, e2±q, e3±q, …, e j ±q, e j+1 , e j+2 ……,e 10000};

[0102] The remainder s is obtained by taking the positive integer z modulo the prime number p. The element at the i-th position in the sequence G is compared with the remainder s. If the two are equal, it means that the element at the i-th position has not been added or subtracted by q, that is, the i-th position is after the j-th position, i > j, and the performance data of the model of Manufacturer A is greater than that of the model of Manufacturer B, so the performance of Manufacturer A is higher than that of Manufacturer B;

[0103] If the two are not equal, it means that the element at the i-th position has been added or subtracted by q by Manufacturer B, that is, the i-th position is before the j-th position, i < j, and the performance data of the model of Manufacturer A is less than that of the model of Manufacturer B, so the performance of Manufacturer A is lower than that of Manufacturer B;

[0104] Compare the performance of the artificial intelligence models of Manufacturer A and Manufacturer B.

[0105] S3: Sort the performance of n artificial intelligence models;

[0106] It includes the following sub-steps:

[0107] S31: Compare the performance of the artificial intelligence models of two adjacent manufacturers in the queue M in turn, and obtain the model with the best performance in the queue M. Remove this model from the queue M and move it into the queue C;

[0108] Compare the performance of two adjacent models in the queue M step by step according to the method of comparing the performance of two artificial intelligence models in step S2;

[0109] Specifically, compare the model performance of the first manufacturer and the second manufacturer in the queue M. If the model performance of the first manufacturer is less than that of the second manufacturer, their positions remain unchanged; if the model performance of the first manufacturer is greater than that of the second manufacturer, exchange their positions in the queue M;

[0110] Then compare the model performance of the second manufacturer and the third manufacturer in the queue M. If the model performance of the second manufacturer is less than that of the third manufacturer, their positions remain unchanged; if the model performance of the second manufacturer is greater than that of the third manufacturer, exchange their positions;

[0111] Compare the model performance of two adjacent manufacturers in the queue M in turn according to the above method until comparing the model performance of the (n - 1)-th and the n-th manufacturers. If the model performance of the (n - 1)-th manufacturer is greater than that of the n-th manufacturer, the model of the (n - 1)-th manufacturer is the model with the best performance in the queue M; otherwise, the model of the n-th manufacturer is the model with the best performance in the queue M; Remove the model with the best performance in the queue M from the queue M and move it into the empty queue C. At this time, the length of the queue M is reduced by 1 and the length of the queue C is increased by 1;

[0112] S32: Complete the sorting of the performance of n artificial intelligence models;

[0113] Step S31 is repeated until all elements in queue M are removed to queue C in sequence, that is, the length of queue M is 0 and the length of queue C is n; queue C={C1, C2, C3, ... C n};

[0114] At this point, the order of the elements in queue C is the order of the performance of the n vendors’ artificial intelligence models from high to low, and the performance ranking of the n artificial intelligence models is completed;

[0115] The present invention generates two kinds of keys, a public key and a private key, by using an asymmetric encryption algorithm. The public key is made public and the private key is kept by oneself. The data encrypted by the other party using the public key can be restored by oneself using the private key, thus ensuring that no secrets are leaked during the entire transmission process.

[0116] S4: Calculate the mean of the performance of n artificial intelligence models;

[0117] Set the queue C={C1, C2, C3, … C n Each manufacturer in the data multiplies its corresponding AI model performance data by 10,000 and randomly divides it into n-1 parts, and randomly transmits these n-1 parts of split data to other n-1 manufacturers. Then each manufacturer will get the split data from the other n-1 manufacturers, and because they only get the split data, each manufacturer cannot get any valid information from these n-1 parts of data.

[0118] Each manufacturer will add up the n-1 pieces of data it receives and make the result public, obtaining n pieces of public data. The n pieces of public data are added together to obtain the sum of all model performance data. The sum of all model performance data is divided by n, and then divided by 10,000 to obtain the mean performance of these n artificial intelligence models.

[0119] S5: Calculate the variance of the performance of n AI models;

[0120] Compute queue C = {C1, C2, C3, ... C n} and the square value of the difference between the artificial intelligence model performance data of each manufacturer in step S4, each manufacturer randomly divides the corresponding square value into n-1 parts, and randomly transmits the n-1 parts of the split data to the other n-1 manufacturers, then each manufacturer will obtain the split data from the other n-1 manufacturers, and because the obtained data is the split data, each manufacturer cannot obtain any valid information from the n-1 parts of the data;

[0121] Each manufacturer will add up the n-1 pieces of data it receives and make the result public, obtaining n pieces of public data. The n pieces of public data will be added together to get the total, and the total will be divided by n, and then divided by 10,000 to obtain the variance of the performance data of these n artificial intelligence models.

[0122] After completing the performance ranking of n artificial intelligence models, the present invention calculates the mean and variance of the performance of the n artificial intelligence models, and understands the average performance level of all evaluated artificial intelligence models by calculating the mean; the discrete degree, that is, the fluctuation range, of the performance of all evaluated artificial intelligence models is measured by calculating the variance. A smaller variance indicates that the performance of the model is more consistent, and a larger variance indicates that the performance of the model is more different; statistical data such as the mean and variance can be used as a performance benchmark and a reference for future model improvements; help formulate improvement strategies and optimize the model development and evaluation process.

[0123] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.

Claims

1. A method for evaluating the performance of artificial intelligence models based on privacy computing, characterized in that: It includes the following steps: S1: Randomly generate a manufacturer queue; Assume that there are n AI models participating in the evaluation, each AI model corresponds to a manufacturer, and the manufacturers of the n models are randomly arranged into a queue M, where M = {M1, M2, ..., M i ,......,M n }; S2: Compare the performance of two artificial intelligence models; Randomly select two manufacturers from queue M, denoted as Manufacturer A and Manufacturer B. The two artificial intelligence models held by Manufacturer A and Manufacturer B have completed tests on the same data in the evaluation, and the performance data of their own artificial intelligence models have been obtained. Denote the performance data of Manufacturer A as i and the performance data of Manufacturer B as j; The i and j are positive integers within the interval [0 - 1000], and the performance data of each manufacturer is private and not publicly disclosed; It includes the following sub-steps: S21: Generate public and private keys for the performance data of their respective artificial intelligence models through an encryption algorithm. Manufacturer A obtains the ciphertext according to the public key of Manufacturer B, converts the ciphertext into data w and transmits it to Manufacturer B; S22: Manufacturer B receives data w, constructs a sequence pair based on the received data and decrypts the sequence; S23: Take the modulus of the decrypted sequence to obtain the remainder sequence E; Manufacturer B randomly selects a prime number p and Decrypt(W) 乙 All elements in the sequence are modulo, that is, each element in the sequence is divided by p. Since the divisor p is a prime number, the remainder of the element divided by p is a positive integer greater than 0, and the remainder sequence E = {e1, e2, e3, ..., e i ,……,e 10000 }; Among them, Decrypt(W) 乙 The W in the sequence is a sequence, specifically a sequence W containing 10,000 elements constructed by manufacturer B using the received data w; S24: Obtain sequence G; In the interval Randomly select a positive integer q from the The model performance data of Manufacturer B is j. Perform the following operations on the j-th element and the elements before the j-th element in sequence E: If the value after adding q to the element is greater than or equal to p, then subtract the positive integer q from the element; conversely, if the value after adding q to the element is less than p, then add or subtract the positive integer q to the element; The sequence G = {e1±q,e2±q,e3±q,……,e j ±q,e j+1 ,e j+2 ……,e 10000 }; If i is less than or equal to j, the i-th element e in sequence E i It will become e i ±q, if i is greater than j, then the i-th element e in the sequence E i remain unchanged; S25: Encrypt all elements in sequence G to obtain sequence H; Manufacturer B encrypts all elements in the G sequence using the encryption function Encrypt and the public key of Manufacturer A, and obtains the sequence H = Encrpt (G) 甲 = {Encrpt(e1±q) 甲 ,Encrpt(e2±q) 甲 ,Encrpt(e3±q) 甲 ,……,Encrpt(e j ±q) 甲 ,Encrpt(e j+1 ) 甲 ,Encrpt(e j+2 ) 甲 ……Encrpt(e 10000 ) 甲 }; Transmit sequence H and prime number p to Manufacturer A; S26: Complete the comparison of the performance of the artificial intelligence models of Manufacturer A and Manufacturer B; Manufacturer A receives the sequence H and prime number p, and decrypts the elements in sequence H using the decryption function Decrypt and the private key of Manufacturer A to obtain the sequence G = {e1±q,e2±q,e3±q,……,e j ±q,e j+1 ,e j+2 ……,e 10000 }; Take the modulus of the positive integer z with prime number p to obtain the remainder s. Compare the i-th element in sequence G with the remainder s. If the two are equal, it means that the i-th element has not been added or subtracted by q, that is, the i-th bit is after the j-th bit, i > j, and the performance data of Manufacturer A's model is greater than that of Manufacturer B's model, and the performance of Manufacturer A is higher than that of Manufacturer B; If the two are not equal, it means that the i-th element has been added or subtracted by q by Manufacturer B, that is, the i-th bit is before the j-th bit, i < j, and the performance data of Manufacturer A's model is less than that of Manufacturer B's model, and the performance of Manufacturer A is lower than that of Manufacturer B; Realize the comparison of the performance of the artificial intelligence models of Manufacturer A and Manufacturer B; S3: Sort the performance of n artificial intelligence models; It includes the following sub-steps: S31: Compare the performance of the artificial intelligence models of adjacent manufacturers in queue M in turn, obtain the model with the best performance in queue M, remove this model from queue M and enter it into queue C; S32: Complete the sorting of the performance of n artificial intelligence models; Step S31 is repeated until all elements in queue M are removed to queue C in sequence, that is, the length of queue M is 0 and the length of queue C is n; queue C = {C1, C2, C3, ... C n }; At this time, the order of the elements in queue C is the order of the performance of the artificial intelligence models of n manufacturers from high to low, and the sorting of the performance of n artificial intelligence models is completed; S4: Calculate the mean value of the performance of n artificial intelligence models; Set queue C = {C1, C2, C3, ... C n Each manufacturer in the data multiplies its corresponding AI model performance data by 10,000 and randomly divides it into n-1 parts, and randomly transmits these n-1 parts of split data to other n-1 manufacturers. Then each manufacturer will get the split data from the other n-1 manufacturers, and because they only get the split data, each manufacturer cannot get any valid information from these n-1 parts of data. Each manufacturer adds the n - 1 pieces of data received and makes the result public, then n pieces of public data are obtained. Add the n pieces of public data to get the sum of all model performance data. Divide the sum of the performance data of all models by n, and then divide by 10000 to obtain the mean value of the performance of these n artificial intelligence models; S5: Calculate the variance of the performance of n artificial intelligence models; Calculation queue C = {C1, C2, C3, ... C n } and the square value of the difference between the artificial intelligence model performance data of each manufacturer in step S4, each manufacturer randomly divides the corresponding square value into n-1 parts, and randomly transmits the n-1 parts of the split data to the other n-1 manufacturers, then each manufacturer will obtain the split data from the other n-1 manufacturers, and because the obtained data is the split data, each manufacturer cannot obtain any valid information from the n-1 parts of the data; Each manufacturer will add up the n-1 pieces of data it receives and make the result public, obtaining n pieces of public data. The n pieces of public data will be added together to get the total, and the total will be divided by n, and then divided by 10,000 to obtain the variance of the performance data of these n artificial intelligence models.

2. The method for evaluating the performance of artificial intelligence models based on privacy computing as claimed in claim 1, characterized in that: The specific contents of step S21 are as follows: Manufacturer A and Manufacturer B jointly determine an asymmetric encryption algorithm, and use the asymmetric encryption algorithm to generate public and private keys for the performance of their respective artificial intelligence models, where the private key is kept by the manufacturer itself and the public key is disclosed to the other party; Manufacturer A randomly selects data x and y within the range of values ​​allowed by the formula, and obtains a positive integer z based on x and y. The formula is as follows: z=2 x +y(x∈{x|100≤x≤200},y∈{y|-2 x-1 ≤y≤2 x-1 }); Manufacturer A encrypts data z according to encryption function Encrypt and manufacturer B's public key to obtain ciphertext v; v = Encrpt(z) 乙 , where B refers to the public key of manufacturer B; Convert the ciphertext v into a positive integer. Manufacturer A adds the positive integer v to the performance data of the artificial intelligence model corresponding to Manufacturer A to obtain w=v+i, and passes the value of w to Manufacturer B.

3. The method for evaluating the performance of artificial intelligence models based on privacy computing as claimed in claim 1, characterized in that: The specific contents of step S22 are as follows: Manufacturer B receives data w, but cannot obtain valid information from it. It takes the following steps: Manufacturer B uses the received w to construct a sequence W containing 10,000 elements; W = {w-1, w-2, w-3, ..., wi, ..., w-10000}, i is a positive integer; Manufacturer B decrypts all elements in the W sequence using the decryption function Decrypt and Manufacturer B's private key, and obtains Decrypt(W) 乙 sequence: Decrypt(W) 乙 ={Decrypt(w-1) 乙 ,Decrypt(w-2) 乙 ,Decrypt(w-3) 乙 ,……, Decrypt(w-i) 乙 ,……,Decrypt(w-10000) 乙 }; And because w=v+i; Then Decrypt(w - i) 乙 = Decrypt(v + i - i) 乙 = Decrypt(v) 乙 = z; Since z is obtained by Manufacturer A randomly selecting x and y according to the formula, Manufacturer B is unaware of it, so Manufacturer B cannot identify which element z is in the sequence.

4. The method for evaluating the performance of artificial intelligence models based on privacy computing as claimed in claim 1, characterized in that: The specific contents of step S31 are as follows: Gradually compare the performance of two adjacent models in queue M according to the method for comparing the performance of two artificial intelligence models in step S2; Specifically, the model performances of the first manufacturer and the second manufacturer in queue M are compared. If the model performance of the first manufacturer is lower than that of the second manufacturer, the positions of the two manufacturers remain unchanged. If the model performance of the first manufacturer is greater than that of the second manufacturer, the positions of the two in queue M are swapped; Then compare the model performance of the second and third manufacturers in queue M. If the model performance of the second manufacturer is lower than that of the third manufacturer, the positions of the two manufacturers remain unchanged. If the model performance of the second manufacturer is greater than that of the third manufacturer, the positions of the two manufacturers are swapped; According to the above method, the model performances of two adjacent manufacturers in queue M are compared in turn until the model performances of the n-1th and nth manufacturers are compared. If the model performance of the n-1th manufacturer is greater than that of the nth manufacturer, the model of the n-1th manufacturer is the best performing model in queue M; otherwise, the model of the nth manufacturer is the best performing model in queue M; the best performing model in queue M is moved out of queue M and enters the empty queue C. At this time, the length of queue M is reduced by 1 and the length of queue C is increased by 1.

Images