A privacy computing program evaluation method, device, equipment and medium

Abstract

The embodiments of the present specification disclose an evaluation method, device, equipment and medium of a privacy computing program. The scheme comprises: obtaining running data generated in the process of executing a privacy computing program; obtaining a protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules to be executed by the privacy computing program; based on the running data and the protocol description text, generating a prompt word containing at least the running data and the protocol description text; the prompt word is used to indicate that a large model evaluates the matching degree of the execution process of the privacy computing program and the protocol description text; the prompt word is provided to the large model, and an evaluation result representing the matching degree output by the large model is obtained.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for evaluating privacy computing programs. Background Technology

[0002] With the development of privacy-preserving computation technology, its applications are increasing, and more and more service providers are participating in it. However, because privacy-preserving computation has high requirements for computing power and security, some service providers may sacrifice security to improve computing performance during execution. The lack of transparency in the privacy-preserving computation process often leads to unfair competition and may also compromise the security of the executed privacy-preserving computation services. Therefore, determining whether the executed computation program faithfully executes the privacy-preserving computation protocol is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This specification provides an evaluation method, apparatus, device, and medium for privacy computing programs to address the problem of difficulty in determining whether an executed computing program faithfully performs a privacy computing protocol.

[0004] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0005] This specification provides an example of an evaluation method for a privacy computing program, comprising:

[0006] Obtain runtime data generated during the execution of privacy-preserving computation programs;

[0007] Obtain the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute;

[0008] Based on the runtime data and the protocol description text, a prompt word is generated that contains at least the runtime data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text;

[0009] The prompt words are provided to the large model, and the large model outputs an evaluation result representing the degree of matching.

[0010] An evaluation apparatus for a privacy computing program provided in the embodiments of this specification includes:

[0011] The first data acquisition module is used to acquire runtime data generated during the execution of the privacy computing program.

[0012] The second data acquisition module is used to acquire the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute;

[0013] The prompt word generation module is used to generate prompt words that contain at least the running data and the protocol description text based on the running data and the protocol description text; the prompt words are used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text;

[0014] The result determination module is used to provide the prompt words to the large model and obtain the evaluation result representing the degree of matching output by the large model.

[0015] This specification provides an embodiment of an evaluation device for a privacy computing program, comprising:

[0016] At least one processor; and,

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0019] Obtain runtime data generated during the execution of privacy-preserving computation programs;

[0020] Obtain the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute;

[0021] Based on the runtime data and the protocol description text, a prompt word is generated that contains at least the runtime data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text;

[0022] The prompt words are provided to the large model, and the large model outputs an evaluation result representing the degree of matching.

[0023] This specification provides an embodiment of a computer-readable medium storing computer-readable instructions that can be executed by a processor to implement an evaluation method for a privacy computing program.

[0024] One embodiment of this specification achieves the following beneficial effects:

[0025] The embodiments in this specification utilize the inductive and organizational capabilities of large models to compare the acquired runtime data with existing protocol description texts, thereby determining the extent to which the privacy computing program executes according to the protocol text. This effectively assesses whether the privacy computing program used to process privacy computing tasks faithfully executes the secure computing protocol required for the privacy computing task, and also helps ensure the processing quality of privacy computing tasks. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This diagram illustrates an application scenario of an evaluation method for a privacy computing program provided in the embodiments of this specification.

[0028] Figure 2 A flowchart illustrating an evaluation method for a privacy computing program provided in an embodiment of this specification;

[0029] Figure 3 Swimlane diagram of an evaluation method for a privacy computing program provided in the embodiments of this specification;

[0030] Figure 4 A schematic diagram of the structure of an evaluation device for a privacy computing program provided in the embodiments of this specification;

[0031] Figure 5 This is a schematic diagram of the structure of an evaluation device for a privacy computing program provided in an embodiment of this specification. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0033] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0034] To more clearly illustrate the methods provided in the embodiments of this specification, some technical concepts are explained first:

[0035] Large-scale models refer to machine learning models with a large number of parameters and complex computational structures. They are language models with strong analytical capabilities achieved by increasing model parameters, typically implemented through natural language interfaces. Usually constructed from deep neural networks, they can handle complex tasks and data, and have a wide range of applications, including natural language processing, computer vision, speech recognition, and recommendation systems. Large-scale models learn complex patterns and features by training on massive amounts of data, and are deep learning models trained on massive amounts of text data. Large-scale language models can not only generate natural language text, but also deeply understand the meaning of text and handle various natural language tasks, such as text summarization, question answering, and translation.

[0036] Privacy-preserving computation is a collective term for a series of technical approaches to achieve data analysis and computation while protecting the data itself from external disclosure. It aims to achieve the goal of "data usable but not visible," realizing the transformation and release of data value while fully protecting data and privacy security. Privacy-preserving computation is not a single technology, but rather a collective term encompassing various privacy protection and enhancement technologies, involving multiple disciplines such as cryptography, security hardware, information theory, and distributed computing.

[0037] From a technical perspective, privacy computing can be an interdisciplinary technology encompassing numerous disciplines. It can include cryptography-based privacy computing techniques such as multi-party secure computation, technologies derived from the integration of artificial intelligence and privacy protection, such as federated learning, and privacy computing techniques based on trusted hardware, such as trusted execution environments. Different technologies are often used in combination to complete computational and analytical tasks while ensuring the security and privacy of the original data.

[0038] The call stack can be understood as the call path obtained through memory snapshots during program runtime.

[0039] Vector matching can be understood as finding the most matching segment from massive amounts of data or code through logic such as text similarity judgment.

[0040] To address the shortcomings of existing technologies, this solution provides the following embodiments:

[0041] Figure 1 This diagram illustrates an application scenario of an evaluation method for a privacy computing program provided in an embodiment of this specification. Figure 1As shown, this scheme generates a prompt for the large model based on the runtime data 1 of the privacy-computing program executed by the participants and the corresponding protocol description text 2. This prompt instructs the large model to evaluate the execution level of the privacy program. Then, leveraging the inductive reasoning capabilities of the large model 3, it determines whether the privacy-computing program executed by the participants complies with the secure computing protocol defined according to the privacy task to be processed. This determines the fidelity of the privacy-computing program to the computing protocol and reflects the capability and trustworthiness of the service provider offering the privacy-computing program. Specifically, the large model 3 can summarize information representing the execution level of the privacy-computing program according to the protocol description text based on the prompt, which can also be understood as the matching degree 4. This can be a score from 0 to 100, with higher scores indicating a higher matching degree and a greater degree of privacy-computing execution, thus better meeting the protocol requirements. In practical applications, the privacy-computing program can be provided by the participants in the privacy-computing process or by other service providers capable of compiling privacy-computing programs.

[0042] In practical applications, if a privacy-preserving computation program faithfully executes the computation protocol, it indicates a high degree of reliability in the computation results provided by that program. In scenarios requiring privacy-preserving computation, a suitable service provider can be selected based on the determined level of execution.

[0043] Next, an evaluation method for a privacy computing program provided in the embodiments of the specification will be described in detail with reference to the accompanying drawings:

[0044] Figure 2 This is a flowchart illustrating an evaluation method for a privacy computing program provided in an embodiment of this specification. From a program perspective, the entity executing the process can be a program or application client hosted on an application server. Specifically, it can be an evaluation platform or server with evaluation capabilities.

[0045] like Figure 2 As shown, the process may include the following steps:

[0046] Step 202: Obtain runtime data generated during the execution of the privacy computing program.

[0047] In practical applications, privacy-preserving computation may be necessary to ensure data security in certain business processes. For example, in the healthcare field, privacy-preserving computation technology can protect the privacy of individual health data. Multiple medical institutions can collaboratively analyze disease transmission trends through secure multi-party computation without sharing specific patient information. In the financial risk assessment field, privacy-preserving computation technology can protect the privacy of individual financial data. For instance, financial institutions can use homomorphic encryption to encrypt customer account information, enabling data analysis and risk assessment without exposing specific user account information. In the internet sector, privacy-preserving computation technology can protect users' privacy data. For example, advertisers can obtain users' advertising preferences through secure multi-party computation without accessing specific personal information. In the field of artificial intelligence, privacy-preserving computation technology can protect the privacy of training data. For instance, different institutions can collaboratively train models through secure multi-party computation without sharing training data, thus protecting data privacy.

[0048] In the embodiments of this specification, the privacy computation program can be a program used by the participating parties to perform privacy computation tasks; the runtime data can be runtime data generated during the execution of the privacy computation program. In practical applications, runtime data can be collected through some monitoring tools, provided by the participating parties, or obtained from the device that hosts and runs the privacy computation program; the specific method is not specifically limited here.

[0049] Step 204: Obtain the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute.

[0050] The protocol description text can represent the program protocol text corresponding to the privacy-preserving computation program. Specifically, it can be a secure computation protocol for privacy-preserving computation. This protocol can include information about the privacy-preserving computation steps to be performed, the security level information of the algorithms to be used, and other execution rules. For example, the protocol may specify the use of high-strength encryption algorithms, the transmission of all encrypted data, or the use of differential privacy, etc. In practical applications, during the processing of privacy-preserving computation services, a secure computation protocol can be pre-defined according to actual business needs. For example, the vendor providing the privacy-preserving computation program can provide a secure computation protocol based on business requirements. Alternatively, the protocol description content can be determined based on publicly available and recognized information in the field, such as interoperability algorithm descriptions.

[0051] The protocol description text can be a secure computing protocol determined based on business requirements, or it can be descriptive text generated based on a secure computing protocol. It can include source code snippets demonstrating the privacy-preserving computation level, or it can include natural language process descriptions used to describe the privacy-preserving computation process. No specific restrictions are placed on the source or specific format of the protocol description text.

[0052] Step 206: Based on the running data and the protocol description text, generate a prompt word that contains at least the running data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text.

[0053] Step 208: Provide the prompt words to the large model to obtain the evaluation result representing the degree of matching output by the large model.

[0054] A prompt is a form of input that instructs an AI model what action to take or what output to produce when performing a specific task. A prompt is a natural language input, similar to a command or instruction, letting the AI ​​model know what it needs to do.

[0055] In the embodiments described in this specification, a prompt word used by the large model can be generated based on the acquired runtime data and protocol description text. This prompt instructs the large model to evaluate the privacy computing program. The inductive and summarizing capabilities of the large model can be used to determine the degree to which the privacy computing program executes according to the protocol description text. The degree of execution can be understood as the matching degree between the privacy computing program and the protocol description text. A higher matching degree indicates that the privacy computing program is more consistent with the protocol description text, and that the settings of the privacy computing program are more in line with actual business needs.

[0056] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0057] Figure 2 The method described in the paper, through the inductive and organizational capabilities of the large model, can compare the acquired runtime data with existing protocol description texts to determine the extent to which the privacy computing program executes according to the protocol text. This can effectively determine whether the privacy computing program used to process privacy computing tasks faithfully executes the secure computing protocol that the privacy computing task needs to satisfy, and also helps to ensure the processing quality of privacy computing tasks.

[0058] On the other hand, it can reduce human intervention, eliminating the need for people to judge the faithful execution of privacy computing programs based on their own experience, thereby improving evaluation efficiency and ensuring the uniformity of evaluation standards.

[0059] In practical applications, a secure computing protocol that meets the requirements of the privacy-preserving computation task can be determined based on the task to be processed. This can be understood as the protocol description text in the embodiments of this specification being a secure computing protocol that meets the task requirements, and can be a manually reviewed and correct protocol text. In practical applications, some service providers offering privacy-preserving computation programs may deviate from the protocol for the sake of computational efficiency. For example, the protocol may specify the use of a high-strength encryption algorithm, but this may not actually be used; the protocol may state that full ciphertext is used, but partial plaintext is actually sent; the protocol may state that differential privacy is used, but the relevant functions are not actually called, etc. The method in the embodiments of this specification can effectively evaluate whether the privacy-preserving computation program deviates from the security protocol. When multiple service providers compete, it can also ensure that everyone competes at the same security level, and can also be used as a factor in evaluating service providers.

[0060] based on Figure 2 In addition to the method described herein, this specification also provides some specific implementation schemes of the method, which will be described below.

[0061] To improve the accuracy of the evaluation, the program source code can be used as part of the prompts, allowing the large model to combine multi-dimensional information to arrive at a more accurate evaluation result. Optionally, the method in the embodiments of this specification may also include:

[0062] Obtain the source code of the privacy computing program;

[0063] The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes:

[0064] Based on the source code of the privacy computing program, the runtime data, and the protocol description text, generate a prompt word that includes at least the source code, the runtime data, and the protocol description text.

[0065] The source code can be provided by the service provider offering the privacy computation program. If multiple parties participate in the privacy computation, the source code can include the source code of the privacy computation program executed by each participant. To facilitate analysis of large models, the source code can be converted into an embedding vector database in the embodiments of this specification.

[0066] The embodiments in this specification can also obtain a list of software packages that the source code depends on, which can be used to build an embedding vector database together with the source code. Existing methods can be used for construction, which will not be detailed here.

[0067] In practical applications, prompts can include instructions, scenarios, input data, output metrics, and so on. Instructions can represent a task or instruction that the model is expected to complete. For example, in this embodiment, we hope the model can provide the degree of execution of the privacy computing program according to the protocol description text, and the degree of matching between the execution process of the privacy computing program and the protocol description text. Scenarios can represent external information or additional contextual information that can guide the model to generate better responses, such as in this embodiment, which focuses on the privacy computing domain. Input data can represent the input or question we are interested in, and the response we hope to obtain, such as the running data, protocol text, and problem information that needs to be solved by the large model in this embodiment. Output metrics can indicate the type or format of the output, such as the output format of the execution degree in this embodiment, whether it is a score or a rating, etc.

[0068] In practical applications, the obtained runtime data may not be exactly the runtime data generated by the privacy computing program being executed once. For example, it might be runtime data generated from executing a portion of the program, or it might be runtime data generated from executing multiple times. To provide accuracy in the evaluation, the source code corresponding to the runtime data to be evaluated can be determined through stack information, and the evaluation can be performed in conjunction with the source code. Optionally, the runtime data in the embodiments of this specification may include stack information. The aforementioned acquisition of the source code of the secure computing privacy computing program may specifically include:

[0069] Based on the stack in the runtime data, the source code corresponding to the stack is determined.

[0070] The stack trace information can be collected using stack tracking tools such as SystemTap or OProfile. In the embodiments described in this specification, prompts can be generated based on the source code determined from the stack trace, along with protocol description text and runtime data.

[0071] In the embodiments described in this specification, the stack information and all source code can also be provided to the large model, and instructions to find the source code through the stack can be set in the prompt, so that the large model can perform the above steps.

[0072] To facilitate data collection and avoid impacting the business processes of parties involved in executing the privacy-preserving computation program, the runtime data obtained in the embodiments of this specification can be obtained by executing the privacy-preserving computation program in a virtual machine. Optionally, the runtime data generated during the execution of the privacy-preserving computation program can specifically include:

[0073] Obtain runtime data generated in a virtual machine on which the privacy computing program is deployed; the virtual machine has the same configuration as the execution device used by the participants in the privacy computing to execute the privacy computing program.

[0074] The configuration information of the virtual machine, such as its operating system and version, number of processors, memory, and disk size, can be determined based on the configuration of the execution device. In the embodiments of this specification, multiple virtual machines can be used. If there are multiple parties involved in privacy computation, such as in multi-party security computation requiring multiple participants, each participant can have at least one virtual machine. Of course, if the virtual machine has high performance, a single virtual machine can also be used. The number of virtual machines and their specific deployment are not limited here, as long as the actual needs are met.

[0075] To facilitate data collection, relevant data collection or monitoring programs can be installed in the virtual machine. These programs may include stack tracking tools, tools for collecting information on CPU, memory, network, and hard disk resources, and log collection tools. In the embodiments described in this specification, the runtime data may include at least one of the following: log data, stack data, and resource usage data; wherein, resources include at least one of CPU, memory, network, and hard disk. If multiple parties participate in the secure computation, the obtained runtime data may include stack information, log information, and resource information such as CPU for each party.

[0076] It is understood that if the operational data used in the embodiments of this specification includes data involving privacy, such data is collected and used after obtaining authorization.

[0077] Given the complexity of privacy-preserving computation, the protocol descriptions for privacy computation can be quite large. When dealing with large datasets, even large models may not provide accurate results, or the computation time may be excessive. As an alternative implementation, the methods in the embodiments of this specification may further include:

[0078] The protocol description text is split into several sub-texts;

[0079] The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes:

[0080] For any one of the plurality of sub-texts, based on the any one sub-text and the running data, generate a prompt word that contains at least the any one sub-text and the running data.

[0081] In this embodiment, at least one prompt word can be generated for each subtext, and evaluation results can be obtained for each subtext. Specifically, assuming the protocol description text is divided into M subtexts, M prompt words can be generated. For each subtext, a large model can be used for at least one matching, meaning the large model can perform at least M operations, and evaluation results can be obtained for the M subtexts.

[0082] In practical applications, the matching results of each subtext can be integrated using large models, other small models, or manual sorting to obtain information that reflects the degree of matching of the privacy computing program with respect to the entire protocol description text.

[0083] As one implementation method, optionally, the evaluation result representing the degree of matching obtained from the large model output in the embodiments of this specification may specifically include:

[0084] For any subtext within each subtext, obtain the sub-evaluation result generated by the large model for the prompt words of that subtext; the sub-evaluation result is used to represent the degree of matching between the privacy calculation program and the subtext.

[0085] Based on the sub-evaluation results corresponding to each sub-text, the overall evaluation result of the privacy procedure and the protocol description text is obtained.

[0086] In the embodiments of this specification, after obtaining the matching information between the privacy computation program and each sub-text using a large model, a small model with weighted summation function can also be used to integrate the obtained information on the matching information between the privacy computation program and each sub-text to obtain information on the degree of matching between the privacy computation program and the entire protocol description text. The specific weight values ​​can be positively correlated with the importance of each sub-file, or with the number of characters contained in each sub-file, or can be set based on expert experience; no specific limitations are made here.

[0087] In practical applications, the need to generate an overall evaluation result can also be included in the prompt words. This leverages the inductive and analytical capabilities of the large model to obtain the overall evaluation result of the protocol description text. For example, the generated prompt words can instruct the large model to determine the matching procedure between the privacy computation procedure and each sub-text, or to determine the degree of matching between the privacy computation procedure and the entire protocol description text based on the matching procedure between the privacy computation procedure and each sub-text. Alternatively, the prompt words can instruct the large model to output information about the matching procedure between the privacy computation procedure and each sub-text, or to output information about the degree of matching between the privacy computation procedure and the entire protocol description text.

[0088] In practical applications, the protocol description text can be generated according to the needs of the privacy computing task, and may include steps or requirements that the participating parties need to perform in order to complete the task. As one implementation, the protocol description text in the embodiments of this specification may include descriptions of multiple execution steps; the above-mentioned division of the protocol description text into several sub-texts may specifically include:

[0089] According to the steps contained in the protocol description text, the protocol description text is split into several subtexts; wherein each subtext contains at least one description of an execution step.

[0090] The execution steps can represent the computational steps required to complete the privacy task. For example, it might involve first retrieving a certain amount of data from some data according to a first rule, then processing the retrieved data according to the first rule, and finally sending the processed data to a node, and so on. The steps in the protocol description file can be set according to the actual privacy task to be processed; no specific limitations are made here.

[0091] In practical applications, the steps in the protocol description text can be steps of the same level or steps of different levels. If the protocol description text contains steps of different levels, the division can be based on the first level of steps or on steps of other levels; no specific limitation is made here.

[0092] Alternatively, the text can be divided based on the size of the text contained in each step. For example, if a step contains three sub-steps, and the entire large step is treated as a sub-text, it is still quite large. Therefore, the large step can be further divided into three sub-texts, or into two sub-texts. One sub-text contains the description of one sub-step, and the other sub-text contains the description of the two sub-steps.

[0093] In practical applications, text can also be divided solely based on data volume, such as dividing the protocol description text into multiple word texts of a predetermined data volume. The specific division method is not limited here, as long as it meets the actual needs.

[0094] To improve the accuracy of the assessment, the operational data obtained in the embodiments of this specification can be operational data from multiple time periods, providing more data for reference to the largest model, which is beneficial to improving the accuracy of the assessment. Optionally, the operational data mentioned above in the embodiments of this specification may include operational data from multiple time periods during the operation of the privacy computing program;

[0095] The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text may specifically include:

[0096] Based on the running data of any one of the multiple time periods and the protocol description text, generate a prompt word that contains at least the running data of any one of the time periods and the protocol description text.

[0097] Assuming N time periods of operational data are collected, at least one prompt word can be generated based on the data within each time period. A large model can perform an evaluation or matching process once based on a prompt word. In this way, the large model can perform at least N evaluation or matching processes and output at least N evaluation results. Alternatively, the various evaluation results can be integrated into a single overall evaluation result. This integration can be achieved manually, using a small model, or by using a large model, similar to the method described above for integrating the entire evaluation result; details will not be elaborated further here.

[0098] In the embodiments of this specification, relevant data for each time period can be organized into a text file. The duration of the aforementioned time period can be greater than or equal to the duration of one execution of the secure computation privacy computation program. This ensures that each file contains as much runtime data as possible for the entire privacy computation execution process. If multiple parties participate in the privacy computation, the file can contain runtime information for each party. In practical applications, large models have inductive and organizational capabilities, and the results given by large models are closely related to the prompt words provided to them. Providing high-quality prompt words helps large models provide accurate results. In the embodiments of this specification, prompt word templates can also be pre-defined, and prompt words can be obtained based on these templates. Optionally, the method in the embodiments of this specification may further include:

[0099] Obtain a preset prompt word template; the prompt word template contains areas for filling in runtime data and protocol description text;

[0100] Based on the aforementioned operational data and the protocol description text, a prompt word is generated that includes at least the operational data and the protocol description text. Specifically, this may include:

[0101] The running data and the protocol description text are added to the prompt word template to generate the prompt word.

[0102] The prompt word template can be obtained based on expert experience or by using machine models to learn and analyze historical prompt words and their corresponding historical results.

[0103] In the embodiments of this specification, the program source code can also be provided to the largest model as a reference. Optionally, the prompt word template can include a region for filling the source code, and the source code can also be used as part of the prompt word.

[0104] To make the large model clearer about the tasks it needs to perform, roles can be defined for the large model, instructions on the problems it needs to handle, output formats, etc. Optionally, the prompt word template in the embodiments of this specification may also include at least one of the following: information indicating the problems the large model needs to solve, role definition information for the large model, output format information, and information for filling in relevant source code.

[0105] As an example, a large model prompt word could be used as follows:

[0106] You are an expert in privacy-preserving computation security protocols and adept at determining whether program execution conforms to the steps defined in the standards. You will evaluate the following program:

[0107] First, the protocol text definition for this program in the standard is: ${protocol text definition};

[0108] The relevant source code for the program is: ${Program source code information found through the stack trace};

[0109] By running the program, the following information was collected:

[0110] The runtime stack information is: ${A's runtime stack}, ${B's runtime stack};

[0111] Related log output: ${A's log}, ${B's log};

[0112] Resource usage during this operation: ${Resource usage of A}, ${Resource usage of B};

[0113] Now, based on the information above, determine whether the current program is faithfully executing the procedure defined in the protocol text. Give a score from 0 to 100, where a higher score indicates a higher degree of matching. Do not output any other information. If you cannot determine this, please output "Cannot determine".

[0114] The above can be an example of prompts for providing a large-scale model. The protocol text definition can be the protocol description text described above, which can be a description of standard protocol text or protocol text specifically designed for privacy computing services. The prompts can contain instructions for the large-scale model, such as the program source code information found through the stack trace in the example above. This instructs the large-scale model to find the corresponding source code according to the stack trace. In practical applications, the server can also find the program source code based on the stack trace and write the found source code into the prompts. For example, in the example above, the part {program source code information found through the stack trace} can be filled with the corresponding program source code.

[0115] In practical applications, multiple parties may participate in privacy-preserving computations. For example, multi-party secure computation requires at least two parties. In the embodiments of this specification, the prompts may include operational data such as logs, stack traces, and resource usage of each participant. In the example above, A can represent one participant in the privacy-preserving computation, and B can represent another participant. If there is only one participant, the prompts may not indicate the owner of the operational data.

[0116] The prompts in this example may also include instructions on the problem the large model needs to solve, such as "Now, based on the information above, determine whether the current program is faithfully executing the process defined in the protocol text." The prompts may also include descriptive information regarding the output format, such as "Give a score from 0 to 100; a higher score indicates a higher match. Do not output any other information. If you cannot determine, output 'Cannot determine'," which can instruct the large model on the output format.

[0117] It can be understood that the above example of prompt words is only an example given to more clearly illustrate the method provided in the embodiments of this specification. The specific description of the prompt words can be set according to actual needs, and no specific limitation is made here.

[0118] To more clearly illustrate the evaluation method of the privacy computing program provided in the embodiments of this specification, Figure 3 Swimlane diagram of an evaluation method for a privacy computing program provided in the embodiments of this specification. For example... Figure 3 As shown, the solution can include a preparation phase, a data acquisition phase, and an analysis results phase, which can specifically include:

[0119] Step 302: The server obtains the source code and the list of dependent software packages provided by the service provider of the privacy computing program, and establishes an embedding vector library.

[0120] This step can be done offline. The above data can be obtained and the vector library can be built in advance before the program runs, thus avoiding the consumption of the program's resources.

[0121] Step 304: The server can obtain the runtime data generated by executing the privacy computing program.

[0122] This can involve preparing one or more virtual machines, on which the vendor or other service provider can install privacy computing programs. These virtual machines can also contain monitoring programs, such as log collection tools, to obtain operational data.

[0123] In practical applications, the duration of the privacy computing program can be set, such as 1 minute or 2 minutes, and the server can save the execution data. If there is a large amount of execution data, it can be split. For example, the relevant data for each time period can be organized into a text file, and the data can be split into 10-second intervals. The split files can also be sorted according to the chronological order in which the data was collected, so that they can be traversed by the larger model later.

[0124] In practical applications, the duration required for a privacy computing program to be executed once can be determined based on experience or historical execution data. The duration of the privacy computing program can be greater than the duration required for the privacy computing program to be executed once, usually a preset multiple of that duration, such as 5 times, 10 times, etc.

[0125] Step 306: The server obtains the protocol description text of the privacy computing program.

[0126] The protocol description text can be a natural language process description of the computation protocol of the privacy computing program. In embodiments of this specification, each independent process in the protocol description text can also be broken down, such as by steps.

[0127] Step 308: Based on the constructed embedding vector library, the obtained runtime data, and the protocol description text, the server generates a prompt word and submits it to the large model.

[0128] Step 310: The large model can organize and summarize the obtained prompts and analyze the matching between the privacy computation program and the protocol description text.

[0129] The large model can search or traverse the collected time-series files to find matches between the time-series files and the protocol description files, or vice versa, and determine the degree of match between the privacy computation program and the protocol description files. The time-series files can be understood as the files containing the runtime data mentioned above, and can be multiple text files. The degree of match can also be understood as similarity, representing the extent to which the privacy computation program executes according to the protocol description text.

[0130] Step 312: The server obtains the matching information given by the large model and gets the evaluation result.

[0131] In practical applications, the large model can be a large language model, which can provide a text description that is relatively in line with human language habits, including the matching degree. In this way, the server can also display the results given by the large model as the evaluation result.

[0132] In practical applications, if the protocol description text is split into M sub-texts, and the obtained runtime data consists of N time periods (which can be simply understood as N text files containing runtime data), then M×N prompt words can be generated. The large model can obtain a result for each prompt word, thus yielding M×N evaluation information representing the degree of matching. In the embodiments of this specification, prompt words summarizing the evaluation information can also be generated based on this evaluation information, leveraging the inductive and organizational capabilities of the large model to obtain the overall evaluation result of the degree of matching. Alternatively, a pre-set small model can be used to organize the obtained evaluation information. The evaluation information can also be organized and summarized manually.

[0133] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 4 This is a schematic diagram of the structure of an evaluation device for a privacy computing program provided in an embodiment of this specification. Figure 4 As shown, the device may include:

[0134] The first data acquisition module 402 is used to acquire runtime data generated during the execution of the privacy computing program.

[0135] The second data acquisition module 404 is used to acquire the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute;

[0136] The prompt word generation module 406 is used to generate prompt words that contain at least the running data and the protocol description text based on the running data and the protocol description text; the prompt words are used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text;

[0137] The result determination module 408 is used to provide the prompt words to the large model and obtain the evaluation result representing the degree of matching output by the large model.

[0138] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0139] Figure 5 This is a schematic diagram of the structure of an evaluation device for a privacy computing program provided as an embodiment of this specification. Figure 5 As shown, device 500 may include:

[0140] At least one processor 510; and,

[0141] Memory 530 communicatively connected to the at least one processor; wherein,

[0142] The memory 530 stores instructions 520 that can be executed by the at least one processor 510, the instructions being executed by the at least one processor 510 to enable the at least one processor 510 to:

[0143] Obtain runtime data generated during the execution of privacy-preserving computation programs;

[0144] Obtain the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute;

[0145] Based on the runtime data and the protocol description text, a prompt word is generated that contains at least the runtime data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text;

[0146] The prompt words are provided to the large model, and the large model outputs an evaluation result representing the degree of matching.

[0147] Based on the same approach, embodiments of this specification also provide a computer-readable medium corresponding to the above-described method. The computer-readable medium stores computer-readable instructions that can be executed by a processor to implement the above-described method for evaluating privacy-preserving computation programs.

[0148] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 5 As the device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0149] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0150] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0151] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0152] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0153] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0159] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0163] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for evaluating privacy-preserving computation programs, comprising: Obtain runtime data generated during the execution of privacy-preserving computation programs; Obtain the protocol description text for the privacy computing program; The protocol description text is used to describe the execution rules that the privacy computing program needs to follow; Based on the runtime data and the protocol description text, a prompt word is generated that contains at least the runtime data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text; The prompt words are provided to the large model, and the large model outputs an evaluation result representing the degree of matching.

2. The method according to claim 1, further comprising: Obtain the source code of the privacy computing program; The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes: Based on the source code of the privacy computing program, the runtime data, and the protocol description text, generate a prompt word that includes at least the source code, the runtime data, and the protocol description text.

3. The method according to claim 1, wherein obtaining the runtime data generated during the execution of the privacy computing program specifically includes: Obtain runtime data generated in a virtual machine on which the privacy computing program is deployed; the virtual machine has the same configuration as the execution device used by the participants in the privacy computing to execute the privacy computing program.

4. The method according to claim 1, further comprising: The protocol description text is split into several sub-texts; The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes: For any one of the plurality of subtexts, based on the any one subtext and the running data, generate a prompt word that contains at least the any one subtext and the running data.

5. The method according to claim 4, wherein the protocol description text includes descriptions of multiple execution steps; The step of splitting the protocol description text into several sub-texts specifically includes: According to the execution steps contained in the protocol description text, the protocol description text is split into several subtexts; wherein each subtext contains at least one description of an execution step.

6. The method according to claim 4, wherein obtaining the evaluation result representing the degree of matching output by the large model specifically includes: For any subtext in each subtext, obtain the sub-evaluation result generated by the prompt word of the large model for that subtext; The sub-evaluation result is used to indicate the degree of matching between the privacy calculation program and any of the sub-texts; Based on the sub-evaluation results corresponding to each sub-text, the overall evaluation result of the privacy computing program and the protocol description text is obtained.

7. The method according to claim 1, wherein the runtime data includes runtime data from multiple time periods during the execution of the privacy computing program; The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes: Based on the running data of any one of the multiple time periods and the protocol description text, generate a prompt word that contains at least the running data of any one of the time periods and the protocol description text.

8. The method according to claim 7, wherein the duration of the time period is greater than or equal to the duration of one execution of the privacy computing program.

9. The method according to claim 1, further comprising: Get the preset prompt word template; The prompt word template includes areas for filling in runtime data and protocol description text; The step of generating prompt words that contain at least the operational data and the protocol description text based on the operational data and the protocol description text specifically includes: The running data and the protocol description text are added to the prompt word template to generate the prompt word.

10. The method according to claim 9, wherein the prompt word template further comprises: Information indicating the problem that the large model needs to solve, role definition information for the large model, output format information, and information used to populate the relevant source code are included.

11. The method according to any one of claims 1 to 10, wherein the operating data includes: The data includes at least one of log data, stack data, and resource usage data; the resources include at least one of CPU, memory, network, and hard disk.

12. An evaluation apparatus for a privacy computing program, comprising: The first data acquisition module is used to acquire runtime data generated during the execution of the privacy computing program. The second data acquisition module is used to acquire the protocol description text for the privacy computing program; The protocol description text is used to describe the execution rules that the privacy computing program needs to follow; The prompt word generation module is used to generate prompt words that contain at least the running data and the protocol description text based on the running data and the protocol description text; the prompt words are used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text; The result determination module is used to provide the prompt words to the large model and obtain the evaluation result representing the degree of matching output by the large model.

13. An evaluation device for a privacy computing program, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Obtain runtime data generated during the execution of privacy-preserving computation programs; Obtain the protocol description text for the privacy computing program; the protocol description text is used to describe the execution rules that the privacy computing program needs to execute; Based on the runtime data and the protocol description text, a prompt word is generated that contains at least the runtime data and the protocol description text; the prompt word is used to instruct the large model to evaluate the degree of matching between the execution process of the privacy computing program and the protocol description text; The prompt words are provided to the large model, and the large model outputs an evaluation result representing the degree of matching.

14. A computer-readable medium having stored thereon computer-readable instructions that can be executed by a processor to implement the evaluation method of the privacy computing program according to any one of claims 1 to 11.

Images