Secure two-party fast convolution calculation method and device based on im2col

Abstract

The invention discloses a secure two-party fast convolution calculation method and device based on im2col, and relates to the technical field of privacy calculation, in the method, a first participant node and a second participant node carry out preprocessing based on the im2col method on private data matrixes of the first participant node and the second participant node respectively, and the private data matrixes of the first participant node and the second participant node are obtained; executing a safe two-square matrix hybrid multiplication protocol based on the preprocessed data matrix to obtain respective hybrid multiplication results; and the first participant node and the second participant node perform vector reconstruction on respective hybrid multiplication results to obtain respective reconstruction matrixes, and send the respective reconstruction matrixes to the calculation requester for aggregation to obtain a convolution calculation result. In the secure two-party fast convolution calculation method, a matrix transformation preprocessing means based on an im2col method and a secure two-party matrix hybrid multiplication protocol are introduced, the efficiency and precision of secure two-party fast convolution calculation can be improved, and the reliability of a secure two-party fast convolution calculation result is ensured.

Description

Technical Field

[0001] This application relates to the field of privacy computing technology, and in particular to a secure two-way fast convolution computation method and apparatus based on im2col. Background Technology

[0002] In the era of data-driven decision-making, machine learning models are widely used in fields such as intelligent traffic management and public health monitoring, playing a crucial role in promoting the application of artificial intelligence in society. Faced with the massive amounts of data generated by multi-source data acquisition devices and sensor networks, ensuring privacy and efficiency when processing sensitive information is a critical issue. However, as reliance on data deepens, data privacy and security issues become increasingly severe. The frequent occurrence of data breaches and unauthorized access incidents highlights the urgency of effectively addressing privacy protection issues. To address these challenges, privacy-preserving computation (a series of information technologies that analyze and compute data while ensuring that data providers do not disclose the original data, guaranteeing that data is "usable but not visible" during circulation and fusion) is applied in various scenarios such as signal processing, big data joint mining, and machine learning, providing effective solutions for protecting sensitive data. These related works involve computational processes involving mixed convolution operations of multi-source data and the challenge of accelerating feature extraction. For example, in common two-party secure convolutional neural networks, it is necessary to achieve efficient convolution calculations between the convolution kernel and the data without leaking massive sample feature information during training or classification. This requires a secure and fast two-party convolution calculation protocol to improve the calculation speed and ensure that data privacy is not leaked.

[0003] Existing solutions to the problem of secure two-party fast convolution computation often employ a stack of cryptographic techniques such as homomorphic encryption, unintentional transmission, and secret sharing to serially perform the inner product calculation of the matrix corresponding to the decomposed convolution operation. This leads to a significant increase in the computational complexity of the ciphertext space, a doubling of communication overhead, and a reduction in computational efficiency. Furthermore, the use of large prime number encryption in existing solutions to the problem of secure two-party fast convolution computation increases the number of bits in the ciphertext space. At the same time, the fixed length of the ciphertext bit directly causes a loss of numerical computational precision, affecting the reliability of the computational results. Summary of the Invention

[0004] The purpose of this application is to provide a secure two-sided fast convolution calculation method and apparatus based on im2col, which can improve the efficiency and accuracy of secure two-sided fast convolution calculation and ensure the reliability of the secure two-sided fast convolution calculation results.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a secure two-party fast convolution calculation method based on im2col, characterized by involving two participating nodes. A first participating node holds a first initial input matrix and a first initial convolution kernel matrix, and a second participating node holds a second initial input matrix and a second initial convolution kernel matrix. The secure two-party fast convolution calculation method based on im2col includes: The first participating node preprocesses the first initial input matrix and the first initial convolution kernel matrix to obtain the first preprocessed input matrix and the first convolution kernel vector. The preprocessing refers to expanding the window matrix generated by sliding the first initial convolution kernel matrix on the first initial input matrix into a vector and then recombining it based on the im2col method, as well as vectorizing the first initial convolution kernel matrix. The second participating node preprocesses the second initial input matrix and the second initial convolution kernel matrix to obtain the second preprocessed input matrix and the second convolution kernel vector. The first participating node uses the first preprocessed input matrix and the first convolution kernel vector as one of the inputs to the secure two-way matrix hybrid multiplication protocol. The second participating node uses the second preprocessed input matrix and the second convolution kernel vector as one of the inputs to the secure two-way matrix hybrid multiplication protocol. The first and second participating nodes obtain the first hybrid multiplication result and the second hybrid multiplication result, respectively. In the secure two-way matrix hybrid multiplication protocol, the secure two-way matrix multiplication calculation protocol is applied to the first preprocessed input matrix and the second convolution kernel vector to perform secure two-way matrix multiplication calculation, and the secure two-way matrix multiplication calculation protocol is applied to the second preprocessed input matrix and the first convolution kernel vector to perform secure two-way matrix multiplication calculation. The first participating node performs vector reconstruction on the first mixed multiplication result to obtain the first reconstruction matrix; the second participating node performs vector reconstruction on the second mixed multiplication result to obtain the second reconstruction matrix. The first participating node and the second participating node send the first reconstruction matrix and the second reconstruction matrix to the computation requester for aggregation to obtain the convolution calculation result.

[0006] Secondly, this application provides a secure two-party fast convolutional computing device based on im2col, characterized in that the secure two-party fast convolutional computing device based on im2col is deployed on each participating node in the network, and the secure two-party fast convolutional computing device based on im2col includes: a task acquisition module, a secure computing module, a rule generation module, a consensus computing module, and a data sending module. The task acquisition module is used to receive and parse fast convolution calculation requests from the calculation requester; The secure computing module is used to automatically match the corresponding secure computing protocol based on the parsed fast convolution computing request; the secure computing protocol includes a secure two-way matrix multiplication protocol and a secure two-way matrix hybrid multiplication protocol; The rule generation module is used to split the fast convolution calculation request according to the asynchronous instruction set of the matched secure computing protocol. The two participating nodes in the fast convolution calculation request perform collaborative computing according to the secure two-party fast convolution calculation method based on im2col. The consensus computation module is used to ensure the synchronization and consistency of results between the two participating nodes in the fast convolution computation request during collaborative computation. The data sending module is used to send the first convolution calculation result obtained by the first participating node or the second convolution calculation result obtained by the second participating node to the calculation requester for aggregation to obtain a fast convolution calculation result.

[0007] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described secure two-way fast convolution calculation method based on im2col.

[0008] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described secure two-way fast convolution calculation method based on im2col.

[0009] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described secure two-way fast convolution calculation method based on im2col.

[0010] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a secure two-way fast convolution calculation method and apparatus based on im2col. A first participating node and a second participating node each preprocess their private data matrices using the im2col method. Then, a secure two-way matrix hybrid multiplication protocol is executed based on the preprocessed data matrices to obtain their respective hybrid multiplication results. The first and second participating nodes then reconstruct their respective hybrid multiplication results into vectors, obtaining their reconstructed matrices, which are sent to the computation requester for aggregation to obtain the convolution calculation result. This application introduces a matrix transformation preprocessing method based on the im2col method and a secure two-way matrix hybrid multiplication protocol into the secure two-way fast convolution calculation method, which can improve the efficiency and accuracy of secure two-way fast convolution calculation and ensure the reliability of the secure two-way fast convolution calculation results. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating a secure two-way fast convolution computation problem in one embodiment of this application; Figure 2 A flowchart illustrating a secure two-way fast convolution calculation method based on im2col provided in an embodiment of this application; Figure 3 A flowchart of a secure two-way fast convolution computation protocol based on im2col provided in one embodiment of this application; Figure 4 A flowchart illustrating vector reconstruction provided in one embodiment of this application; Figure 5 This is a flowchart of the preprocessing process in a secure two-way fast convolution calculation method based on im2col provided in an embodiment of this application; Figure 6 A schematic diagram illustrating a secure two-way matrix multiplication computation problem provided in an embodiment of this application; Figure 7 A flowchart of a secure two-way matrix hybrid multiplication computation protocol based on S2PM is provided in one embodiment of this application; Figure 8 A schematic diagram of a data obfuscation technique provided in an embodiment of this application; Figure 9 This is a schematic diagram illustrating a secure two-way matrix multiplication computation problem provided in an embodiment of this application; Figure 10 A flowchart of a secure two-party matrix multiplication computation protocol provided in an embodiment of this application; Figure 11 A functional module diagram of a secure two-way fast convolution calculation device based on im2col provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] This application focuses on the problem of fast convolution computation of secure two-sided matrices, aiming to achieve an efficient, secure, reliable, and highly coupled fundamental privacy computing method and apparatus. The definition of the problem of fast convolution computation of secure two-sided matrices is as follows: Figure 1 As shown, two computational participant nodes are known. and They are independent and distrustful of each other. It possesses a set of private matrices stored only on its own computing node. and private convolution kernel matrix , Holding a private matrix stored only on its own computing node and private convolution kernel matrix The two participating nodes jointly execute a two-way matrix fast convolution computation protocol. Default private matrix and Fill value The stride of the convolution kernel matrix Ultimately, each participating node in the computation obtains its corresponding output matrix. The results are then sent to the requesting party to aggregate and obtain the desired two-way convolution calculation results. During the calculation process, each participating node can only obtain the input and output information involved in its own calculation process, and cannot obtain the intermediate settlement results or private data information held by other participants. The following is a summary and description of the relevant technologies at home and abroad: Existing technology 1: Xu et al. decomposed convolution computation into the convolution kernel in the matrix The inner product between the sliding window matrix generated by the upward sliding and the convolution kernel matrix is ​​constructed using function encryption technology, thereby achieving secure convolution computation. Xu et al.'s function encryption scheme effectively improves computational security, but because it requires performing an inner product calculation for each sliding window and convolution kernel matrix, the computational overhead increases significantly.

[0016] Existing technology 2: Kim et al., based on the CKKS fully homomorphic encryption scheme, provided a concise representation of convolution operations in plaintext space, packing each input into polynomial coefficients of a ring, thus enabling convolution calculation through a single multiplication without rotation operations. While Kim et al.'s method of packing inputs to achieve single-multiplication convolution calculations improves the speed of convolution operations, it may lead to a loss of computational precision due to the involvement of finite ring domains. Furthermore, the complexity introduced by homomorphic encryption significantly reduces computational efficiency.

[0017] Existing technology 3: Wang et al. constructed a publicly verifiable convolutional neural network model based on inadvertent transmission and secret sharing techniques. They achieved secure convolution computation by calling a privacy multiplication protocol to calculate the inner product between each sliding window matrix and the convolution kernel matrix. However, Wang et al.'s inadvertent transmission scheme requires a large number of rounds of communication, and each sliding window and convolution kernel matrix needs to have its inner product calculated separately. This results in high communication costs and low computational efficiency in large-scale mixed inner product operations.

[0018] Existing technology 4: Ding et al. constructed a trusted privacy computing model in a cloud service environment based on homomorphic encryption technology, realizing decentralized ciphertext convolution computing; Ding et al.'s homomorphic encryption scheme effectively protects the privacy of input data and provides verifiability of computing results, but due to the high computational complexity of homomorphic encryption technology, it will lead to a significant decrease in overall computing efficiency, and relying on third-party cloud service providers may pose a risk of collusion attacks.

[0019] The above solutions are all based on cryptographic technology stacks and ensure security in a semi-honest environment. However, for the problem of secure two-way convolution computation, none of them can solve the problem of privacy risks to the original data that may be caused by the leakage of the output result.

[0020] Example 1 To address the aforementioned issues, this embodiment proposes a secure two-party fast convolution computation method based on im2col, specifically a secure two-party fast convolution computation method for semi-honest scenarios. The semi-honest model (Semi-Honest Adversaries Security) is a specific protocol that assumes all participating parties will honestly participate in privacy computation and strictly adhere to each step of the process. However, there is a risk of corrupted parties attempting to infer the privacy of other parties through intermediate or final results of the protocol execution. Secure two-party fast convolution computation has important applications in multiple fields, particularly in privacy-preserving feature extraction in information retrieval, privacy-preserving filtering and edge detection in medical images, and privacy-preserving deep neural networks, demonstrating broad research value. In tasks such as scene understanding, capturing a wide range of contextual information is crucial for the recognition of the overall scene or large objects. Large convolutional kernels can capture larger regions at once, thereby quickly extracting global features and reducing the need to stack multiple small convolutional kernels to obtain similar receptive fields. This approach can extract larger features in fewer layers, effectively reducing network depth and improving computational efficiency. Therefore, this embodiment proposes a secure two-party fast convolution computation protocol (S2PFC) based on im2col for scenarios requiring a large receptive field to capture global image context information. For generality, this is illustrated using two participating nodes as an example. The first participating node (participating node...) (Node) holds the first initial input matrix and the first initial convolution kernel matrix Second participating node (participant) (Node) holds the second initial input matrix Second initial convolution kernel matrix ,like Figure 2 and Figure 3 As shown, an efficient, parallel, secure, and fast two-way convolution computation method is presented, which includes the following steps.

[0021] Step 101: The first participating node preprocesses the first initial input matrix and the first initial convolution kernel matrix to obtain the first preprocessed input matrix and the first convolution kernel vector. The preprocessing refers to expanding the window matrix generated by sliding the first initial convolution kernel matrix on the first initial input matrix into a vector and then recombining it based on the im2col method, as well as vectorizing the first initial convolution kernel matrix.

[0022] Step 102: The second participating node preprocesses the second initial input matrix and the second initial convolution kernel matrix to obtain the second preprocessed input matrix and the second convolution kernel vector.

[0023] Step 103: The first participating node uses the first preprocessed input matrix and the first convolution kernel vector as one of the inputs to the Secure Two-Party Matrix Hybrid Multiplication Protocol (S2PHM). The second participating node uses the second preprocessed input matrix and the second convolution kernel vector as one of the inputs to the Secure Two-Party Matrix Hybrid Multiplication Protocol. The first and second participating nodes obtain the first hybrid multiplication result and the second hybrid multiplication result, respectively. In the Secure Two-Party Matrix Hybrid Multiplication Protocol, the Secure Two-Party Matrix Multiplication Calculation Protocol is applied to the first preprocessed input matrix and the second convolution kernel vector to perform Secure Two-Party Matrix Multiplication Calculation, and the Secure Two-Party Matrix Multiplication Calculation Protocol is applied to the second preprocessed input matrix and the first convolution kernel vector to perform Secure Two-Party Matrix Multiplication Calculation.

[0024] Step 104: The first participating node performs vector reconstruction on the first mixed multiplication result to obtain the first reconstruction matrix; the second participating node performs vector reconstruction on the second mixed multiplication result to obtain the second reconstruction matrix.

[0025] Step 105: The first participating node and the second participating node send the first reconstruction matrix and the second reconstruction matrix to the computation requester for aggregation to obtain the convolution calculation result.

[0026] Each element of the mixed multiplication result vector corresponds to the weighted average value in each sliding window of the traditional convolution operation. Since a shape transformation was performed during preprocessing, the result vectors in step 103 (the first mixed multiplication result and the second mixed multiplication result) need to be reconstructed to restore them to the state of the convolution operation result itself. An example of vector reconstruction is shown below. Figure 4 As shown.

[0027] Participants Nodes and Participants Each node will store its own private column vector. and Reconstructed into a matrix They then store the matrix in their respective local node's private storage space. Figure 3 middle, and These represent the results of the first and second mixed multiplications, respectively. Let represent the first reconstruction matrix and the second reconstruction matrix, respectively.

[0028] Participants Nodes and Participants Each node will generate its own private matrix after summarizing the final randomized intermediate results. and The result is sent to the party requesting the secure two-party fast convolution computation, which then aggregates the results to obtain the final computation outcome. Obviously, it is easy to verify that: .

[0029] This embodiment is based on the secure two-way matrix hybrid multiplication protocol. This paper presents an end-to-end parallel fast convolution computation scheme in both semi-honest and result-leaking environments, addressing the problems of high computational overhead and low efficiency in existing technologies caused by homomorphic encryption and unintended transmission techniques, which result in large computational and communication overhead in the ciphertext space. This embodiment employs a secure two-way matrix hybrid multiplication protocol that supports computation with a maximum float64-bit precision and has result reliability verification. This invention addresses the problem in existing technologies where floating-point calculations suffer from precision loss due to the limitation of fixed bit length in ciphertext computation. Furthermore, considering that existing solutions for secure two-party fast convolution computation often rely on outsourced cloud computing systems, and that low trustworthiness of third-party cloud computing nodes or attacks by malicious nodes could lead to key leakage and further compromise the privacy of the original data, this embodiment employs a secure two-party matrix hybrid multiplication protocol. This invention implements a parallel, secure, reliable, and secure two-way convolution computation scheme that does not require the introduction of third-party cloud services.

[0030] As an optional implementation, traditional convolution operations perform weighted summation by shifting the convolution kernel matrix. This invention, based on the Im2col method, expands the overlapping submatrices at each shift into row vectors to reduce the complexity of the convolution operation. An example of the preprocessing is shown below. Figure 5 As shown. The preprocessing process in step 101 is as follows: Participating parties The nodes sequentially convert the first initial convolution kernel matrix First initial input matrix The window matrix generated by the upward swipe is expanded into row vectors, resulting in a group of row vectors. They are then combined and reconstructed sequentially into the first preprocessing input matrix. Simultaneously, the first initial convolution kernel matrix is ​​expanded into column vectors to obtain the first convolution kernel vector. .

[0031] Similarly, the preprocessing procedure in step 102 is as follows: Participants The nodes sequentially convert the second initial convolution kernel matrix Second initial input matrix The window matrix generated by the upward swipe is expanded into row vectors, resulting in a group of row vectors. They are then combined and reconstructed sequentially into the second preprocessing input matrix. Simultaneously, the second initial convolution kernel matrix is ​​expanded into column vectors to obtain the second convolution kernel vector. .

[0032] As an optional implementation, in step 103, the problem definition of the secure two-party matrix hybrid computation protocol is as follows: Figure 6 As shown, two computational participants are known. and They are independent and distrustful of each other. It possesses a set of private data matrices stored only on its own computing nodes. and , It possesses a set of private data matrices stored only on its own computing nodes. and The two parties jointly execute a two-party matrix multiplication computation protocol. Ultimately, each participating node in the computation obtains its corresponding output matrix. The results are then sent to the requesting party to aggregate and obtain the desired two-party matrix multiplication result. During the calculation, each participating node can only obtain the input and output information involved in its own calculation process, and cannot obtain the intermediate settlement results or private data information held by other participants. (Regarding the secure two-party matrix multiplication protocol...) Multiplication calculation, participating parties Nodes and The node is based on a secure two-way matrix hybrid multiplication protocol. Input their respective preprocessed private matrices. , and private convolution kernel vector , ,exist After the protocol is executed, it returns a column vector of random intermediate computation results for each node based on random obfuscation techniques. , And these two private vectors , Satisfying the relation: .

[0033] The problem of secure two-party matrix multiplication often occurs in the intermediate computation process of multi-party modeling problems, with two-party multiplication being particularly frequent. For example, in regression and clustering problems, the intermediate computation results from two parties are often combined, superimposed, and then multiplied. To avoid loss of generality, let the participants in this protocol... Alice The initial input matrix of the node is , Participants Bob The initial input matrix of the node is , .

[0034] Based on this, such as Figure 7 As shown, in step 103, the first participating node uses the first preprocessed input matrix and the first convolution kernel vector as one of the inputs to the secure two-party matrix hybrid multiplication protocol, and the second participating node uses the second preprocessed input matrix and the second convolution kernel vector as one of the inputs to the secure two-party matrix hybrid multiplication protocol. The first participating node and the second participating node obtain the first hybrid multiplication result and the second hybrid multiplication result, respectively, which specifically includes the following steps.

[0035] (1) The first participating node takes the first preprocessed input matrix as one of the inputs of the secure two-party matrix multiplication calculation protocol, and the second participating node takes the second convolution kernel vector as one of the inputs of the secure two-party matrix multiplication calculation protocol. The secure two-party matrix multiplication calculation is performed, and the first participating node and the second participating node obtain the first multiplication result and the second multiplication result respectively.

[0036] like Figure 7 As shown, the participating nodes Alice and Bob input their respective private matrices based on the Secure Two-Party Matrix Multiplication Protocol (S2PM). and private matrix The first round of secure two-way matrix multiplication is performed. After the S2PM protocol calculation is completed, the intermediate results of this round of calculation will be randomly split into matrices using random obfuscation techniques. And send them to the participating Alice node and the participating Bob node respectively, and these two private output matrices satisfy the relation. .

[0037] Figure 7 middle, This represents the first preprocessing input matrix, corresponding to Figure 3 In ; Represents the first convolution kernel vector, corresponding to Figure 3 In ; This represents the second preprocessing input matrix, corresponding to Figure 3 In ; Represents the second convolution kernel vector, corresponding to Figure 3 In ; These represent the results of the first and second multiplications, respectively.

[0038] Data obfuscation techniques for secure multi-party computation are a data protection method used to protect intermediate results of secure multi-party computations. They achieve a one-time pad protection effect by randomly splitting the computation results into linear combinations. For most multi-party computations, the process of achieving secure computation typically involves multiple interactive steps. An unavoidable problem arises: how to ensure the security of intermediate results? For example, multiplying two matrices... As an intermediate result of the calculation, whether the participating Alice node or Bod node obtains the final matrix The results can potentially lead to the deduction of data from the other party. Therefore, privacy-preserving computation must ensure not only the security of the original input data but also the security of intermediate values. To address this issue, this application proposes a data obfuscation encryption technique, which involves decomposing any multi-ary operation into a new multi-ary addition to obfuscate the results of calculating intermediate values. To better illustrate its principle, this paper uses a two-way operation type as an example, and the principle is as follows: Figure 8 As shown. Assume ,in It is the objective calculation function. It is private data belonging to Alice, a participating node. This is private data belonging to Bob, one of the participating nodes. Intermediate results are processed at each step of the multi-party secure computation protocol. All will strictly adhere to the following constraints: Participating node Alice will only know the results of its own computation. Bob, the participating node, only knows ,and . formula This illustrates the process of passing intermediate values. Throughout this process, participant nodes Alice and Bod are not allowed to exchange data with each other, including the data after splitting the intermediate results. and As long as the intermediate value is split into two random data items at each step of the calculation, it can be guaranteed that no one can deduce the original data item from this obfuscated and encrypted data, thus making the entire privacy computing process highly secure.

[0039] (2) The first participating node generates the first matrix based on the first preprocessing input matrix and the first convolution kernel vector.

[0040] Specifically, the participating Alice node is based on the first preprocessed input matrix. and the first convolution kernel vector Execute private matrix multiplication calculations locally in parallel. After the calculation is completed, the result is stored in the private storage space inside the Alice local node. This represents the first matrix.

[0041] (3) The first participating node takes the first convolution kernel vector as one of the inputs of the secure two-party matrix multiplication calculation protocol, and the second participating node takes the second preprocessed input matrix as one of the inputs of the secure two-party matrix multiplication calculation protocol. The secure two-party matrix multiplication calculation is performed, and the first participating node and the second participating node obtain the third multiplication result and the fourth multiplication result, respectively.

[0042] like Figure 7 As shown, the participating nodes Alice and Bob input their respective private matrices based on the Secure Two-Party Matrix Multiplication Protocol (S2PM). and private matrix The second round of secure two-way matrix multiplication is performed in parallel. After the S2PM protocol computation is completed, the intermediate results of this round of computation will be randomly split into matrices using a random obfuscation technique. And send them to the participating Alice node and the participating Bob node respectively, and these two private output matrices satisfy the relation. . Figure 7 middle, These represent the results of the third and fourth multiplications, respectively.

[0043] (4) The second participating node generates the second matrix based on the second preprocessing input matrix and the second convolution kernel vector.

[0044] Specifically, the participating node Bob uses the second preprocessed input matrix. Second convolution kernel vector Execute private matrix multiplication calculations locally in parallel. After the calculation is completed, the result is stored in the private storage space inside Bob's local node. This represents the second matrix.

[0045] (5) The first participating node generates the first mixed multiplication result based on the first matrix, the first multiplication result and the third multiplication result.

[0046] Specifically, the participating Alice node will use the intermediate random splitting matrix generated by the first two rounds of S2PM protocol execution. Perform secret aggregation and execute private computations locally. ,in . Figure 7 middle, This represents the result of the first mixed multiplication, corresponding to Figure 3 In .

[0047] (6) The second participating node generates the second mixed multiplication result from the second matrix, the second multiplication result and the fourth multiplication result.

[0048] The participating node Bob will use the intermediate random splitting matrix generated by the first two rounds of S2PM protocol execution. Execute secret aggregation in parallel and perform private computation locally. ,in . Figure 7 middle, This represents the result of the second mixed multiplication, corresponding to Figure 3 In .

[0049] Participating nodes Alice and Bob each submit their private matrices summarizing the final randomly confused intermediate results. The result is sent to the party requesting the secure two-party mixed multiplication calculation, which then aggregates the results to obtain the final calculation result. Obviously, it is easy to verify that: .

[0050] As an alternative implementation method, two computational methods are known to exist. , They are independent and distrustful of each other. Holding a dimension as Private data matrix stored only on its own computing node , Holding a dimension as Private data matrix The two parties hope to achieve this by jointly implementing a secure matrix multiplication protocol. Ultimately, each participating node in the computation receives its corresponding dimension. The output matrix The results are then sent to the requesting party to aggregate and obtain the desired two-way matrix multiplication result. During the calculation process, each participating node can only know its own input and output information and cannot obtain the intermediate settlement results or held data information of other participants. A formal description of the secure two-way matrix multiplication problem can be found in [link to relevant documentation]. Figure 9 .

[0051] Therefore, as Figure 10 As shown, in the aforementioned step (1), the first participating node uses the first preprocessed input matrix as one of the inputs to the secure two-party matrix multiplication calculation protocol, and the second participating node uses the second convolution kernel vector as one of the inputs to the secure two-party matrix multiplication calculation protocol. The secure two-party matrix multiplication calculation is performed, and the first and second participating nodes obtain the first multiplication result and the second multiplication result, respectively, specifically including: (a) The first participating node and the second participating node receive a first random matrix pair and a second random matrix pair sent by the auxiliary computing node; the first random matrix pair includes a first random matrix and a second random matrix; the second random matrix pair includes a third random matrix and a fourth random matrix.

[0052] like Figure 10 As shown, the auxiliary computing nodes are also called commodity service nodes. Generate two pairs of random matrices, specifically in the form of a matrix with dimension 1. random matrix Dimensions random matrix and dimensions Two random matrices These random matrices must strictly satisfy the following constraints. Then by The auxiliary node will pair the first random matrix. Send to the participants Node, pair the second random matrix Send to the participants Nodes. Among them, Let the first random matrix be represented. Represents the second random matrix; Represents the third random matrix; Let represent the fourth random matrix.

[0053] (b) The first participating node derives the first intermediate matrix based on the first preprocessed input matrix and the first random matrix, and sends the first intermediate matrix to the second participating node.

[0054] like Figure 10 As shown, the participating parties Upon receiving the corresponding first random matrix pair Then, its internal calculation and send it to the participating nodes. . Figure 10 middle, This represents the first intermediate matrix; This represents the first preprocessing input matrix; Let represent the first random matrix.

[0055] (c) The second participating node derives the second intermediate matrix based on the second convolution kernel vector and the third random matrix, and sends the second intermediate matrix to the first participating node.

[0056] like Figure 10 As shown, the participating parties Upon receiving the corresponding second random matrix pair Then, its internal calculation and send it to the participating nodes. . Figure 10 middle, Indicates the second intermediate matrix; Represents the second convolution kernel vector; This represents the third random matrix.

[0057] (d) The second participating node randomly generates a matrix, denoted as the fifth random matrix, and derives the third intermediate matrix based on the second convolution kernel vector, the first intermediate matrix, the fourth random matrix, and the fifth random matrix, and sends it to the first participating node.

[0058] like Figure 10 As shown, the participating parties The node received from The first intermediate matrix sent by the node Then, a fifth random matrix is ​​secretly generated internally. And secretly calculate the third intermediate matrix locally. Send to node. Figure 10 In the fifth random matrix The result of the second multiplication is the corresponding Figure 7 In .

[0059] (e) The first participating node derives the fourth intermediate matrix based on the second intermediate matrix, the third intermediate matrix, the first random matrix, and the second random matrix; the fourth intermediate matrix The result of the first multiplication is the corresponding Figure 7 In .

[0060] like Figure 10 As shown, the participating parties The node receives the third intermediate matrix Subsequently, the fourth intermediate matrix is ​​obtained through local secret computation. . Figure 10 In the middle, the fourth intermediate matrix The result of the first multiplication is the corresponding Figure 7 In .

[0061] In the process of a secure two-party matrix multiplication computation protocol, the participating parties node, Each node represents its corresponding final obfuscation and splitting result. , The matrix multiplication is sent to the two parties requesting the matrix multiplication calculation, who then sums the results to obtain the final product. It is easy to verify that: .

[0062] The process of the secure two-party matrix multiplication calculation protocol in step (3) above is the same as the above steps ( a The process from step (3) to step (e) is the same, except that the input of the secure two-way matrix multiplication calculation protocol in step (3) is the first convolution kernel vector and the second preprocessed input matrix.

[0063] The solution proposed in this embodiment has the following advantages: 1) This embodiment adopts a secure two-way matrix hybrid multiplication basic protocol. This paper proposes a hybrid convolutional computation scheme that supports end-to-end parallel computing in a semi-honest environment. It effectively solves the problems of high ciphertext computation and communication overhead and low computational efficiency caused by the introduction of homomorphic encryption and unintentional transmission technology in existing technologies. It provides a parallel, secure and reliable convolutional computation scheme that does not rely on third-party cloud services, and solves the problems of high communication overhead, low computational efficiency, reliance on third-party cloud service computing nodes and data privacy risks caused by cryptographic technology stack implementation in the aforementioned existing technologies 1 to 4.

[0064] 2) This embodiment proposes a universally secure two-way fast convolution computation technique based on the im2col method. By transforming the traditional sliding window convolution operation into a single matrix privacy multiplication computation, it overcomes the shortcomings of existing technologies in large-scale convolution computation and parallel processing capabilities. It achieves fast convolution computation that supports large-scale data processing while protecting privacy. By using matrix privacy multiplication instead of the traditional sliding window summation method, the efficiency of convolution operations is significantly improved. This technique effectively solves the problems of low efficiency and insufficient parallel processing capabilities of existing methods when capturing global image context information and dealing with large-scale convolution computations. It also addresses the issues of high computational overhead, lack of computational accuracy, and low computational efficiency in the aforementioned existing technologies 1 and 2.

[0065] 3) This embodiment employs a secure two-way matrix multiplication protocol that supports calculations with a maximum float64-bit precision and has result reliability verification. This invention solves the problem of loss of numerical precision in floating-point calculations caused by the bottleneck of fixed bit length in ciphertext calculation in existing technologies. It realizes a secure two-way fast convolution calculation protocol that can support high-precision floating-point matrix calculations. It also solves the problems of high communication overhead, low calculation efficiency, reliance on third-party cloud service computing nodes, and data privacy risks caused by cryptographic technology stack implementation in the aforementioned existing technologies 1 to 4.

[0066] 4) This embodiment combines the idea of ​​random obfuscation encryption with a high-precision secure two-party matrix multiplication protocol. A high-precision, secure convolution computation protocol is proposed. Because... The protocol can support up to The bit-precision capability supports not only integer matrix calculations but also floating-point matrix calculations. Compared to convolution schemes based on obfuscated circuits and homomorphic encryption, which are limited to handling fixed-length decimals and integers, this embodiment expands the applicable scenarios for secure and fast convolution calculations, and the calculation results have the same order of magnitude numerical calculation precision as centralized calculation methods.

[0067] Example 2 This embodiment provides a secure two-party fast convolution computation device based on im2col. It requires deploying a corresponding distributed computing framework on the computational participant nodes involved in the two-party secure fast convolution computation task. That is, the secure two-party fast convolution computation device based on im2col is deployed on each participant node in the network. Figure 11 As shown, the secure two-party fast convolution calculation device based on im2col includes: a task acquisition module, a secure calculation module, a rule generation module, a consensus calculation module, and a data sending module.

[0068] The task acquisition module is used to receive and parse fast convolution calculation requests from the calculation requester.

[0069] The secure computing module is used to automatically match the corresponding secure computing protocol based on the parsed fast convolution computing request; the secure computing protocol includes a secure two-way matrix multiplication protocol and a secure two-way matrix hybrid multiplication protocol.

[0070] The rule generation module is used to split the fast convolution calculation request according to the asynchronous instruction set of the matched secure computing protocol. The two participating nodes in the fast convolution calculation request perform collaborative computing according to the secure two-party fast convolution calculation method based on im2col described in Embodiment 1.

[0071] The consensus computation module is used to ensure the synchronization and consistency of results between the two participating nodes in the fast convolution computation request during collaborative computation. After receiving the assigned sub-rules, the consensus computation module ensures the synchronization and consistency of results through the consensus protocol.

[0072] The data sending module is used to send the first convolution calculation result obtained by the first participating node or the second convolution calculation result obtained by the second participating node to the calculation requester for aggregation to obtain a fast convolution calculation result.

[0073] The specific implementation process is as follows: An external client sends a request for two-party fast matrix convolution calculation to the network terminal with distributed computing services via HTTP or gRPC communication protocol. When the task acquisition module of the network node receives the request for convolution calculation, it parses it and starts the secure computing service process of the corresponding computing participant node 1 and node 2. After the task acquisition module finishes parsing the corresponding computing requirements, it passes them to the secure computing module, which performs a joint query through its internal interface. After matching the corresponding secure computing protocol, it synchronizes it to the rule generation module in the two participating nodes. The rule generation module formulates different asynchronous parallel execution processes according to the different sub-tasks undertaken by the two different participating nodes, and maintains communication with the consensus computing module at each step of the execution. While the two participating nodes execute each step of the calculation instructions, the consensus computing module broadcasts and maintains the consistency of the results of the distributed computing nodes on the chain and controls the stability of the execution process. After the final calculation protocol is executed, the two participating nodes obtain the sub-results calculated by each other, and send the result sub-matrix after the two parties' confusion and splitting to the computing requester through the data sending module to obtain the correct calculation result.

[0074] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 12 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores intermediate and final results during the secure two-sided fast convolution calculation process. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a secure two-sided fast convolution calculation method based on im2col or a secure two-sided fast convolution calculation method based on the winograd convolution concept.

[0075] Those skilled in the art will understand that Figure 12The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0076] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0077] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A secure two-way fast convolution calculation method based on im2col, characterized in that, The method involves two participating nodes: a first participating node holds a first initial input matrix and a first initial convolution kernel matrix, and a second participating node holds a second initial input matrix and a second initial convolution kernel matrix. The secure two-party fast convolution calculation method based on im2col includes: The first participating node preprocesses the first initial input matrix and the first initial convolution kernel matrix to obtain the first preprocessed input matrix and the first convolution kernel vector. The preprocessing refers to expanding the window matrix generated by sliding the first initial convolution kernel matrix on the first initial input matrix into a vector and then recombining it based on the im2col method, as well as vectorizing the first initial convolution kernel matrix. The second participating node preprocesses the second initial input matrix and the second initial convolution kernel matrix to obtain the second preprocessed input matrix and the second convolution kernel vector. The first participating node uses the first preprocessed input matrix and the first convolution kernel vector as one of the inputs to the secure two-way matrix hybrid multiplication protocol. The second participating node uses the second preprocessed input matrix and the second convolution kernel vector as one of the inputs to the secure two-way matrix hybrid multiplication protocol. The first and second participating nodes obtain the first hybrid multiplication result and the second hybrid multiplication result, respectively. In the secure two-way matrix hybrid multiplication protocol, the secure two-way matrix multiplication calculation protocol is applied to the first preprocessed input matrix and the second convolution kernel vector to perform secure two-way matrix multiplication calculation, and the secure two-way matrix multiplication calculation protocol is applied to the second preprocessed input matrix and the first convolution kernel vector to perform secure two-way matrix multiplication calculation. The first participating node performs vector reconstruction on the first mixed multiplication result to obtain the first reconstruction matrix; the second participating node performs vector reconstruction on the second mixed multiplication result to obtain the second reconstruction matrix. The first participating node and the second participating node send the first reconstruction matrix and the second reconstruction matrix to the computation requester for aggregation to obtain the convolution calculation result.

2. The secure two-way fast convolution calculation method based on im2col according to claim 1, characterized in that, The first participating node preprocesses the first initial input matrix and the first initial convolution kernel matrix to obtain the first preprocessed input matrix and the first convolution kernel vector, specifically including: The first participating node expands the window matrix generated by sliding the first initial convolution kernel matrix on the first initial input matrix into row vectors, thus obtaining a group of row vectors. The row vector groups are combined and reconstructed into the first preprocessing input matrix in sequence; The first participating node expands the first initial convolution kernel matrix into column vectors to obtain the first convolution kernel vector.

3. The secure two-way fast convolution calculation method based on im2col according to claim 1, characterized in that, The first participating node uses the first preprocessed input matrix and the first convolution kernel vector as one of the inputs to the secure two-party matrix hybrid multiplication protocol. The second participating node uses the second preprocessed input matrix and the second convolution kernel vector as one of the inputs to the secure two-party matrix hybrid multiplication protocol. The first and second participating nodes obtain the first hybrid multiplication result and the second hybrid multiplication result, respectively, including: The first participating node uses the first preprocessed input matrix as one of the inputs to the secure two-party matrix multiplication computation protocol, and the second participating node uses the second convolution kernel vector as one of the inputs to the secure two-party matrix multiplication computation protocol. They then perform secure two-party matrix multiplication computation, and the first and second participating nodes obtain the first multiplication result and the second multiplication result, respectively. The first participating node generates the first matrix based on the first preprocessed input matrix and the first convolution kernel vector; The first participating node takes the first convolution kernel vector as one of the inputs to the secure two-party matrix multiplication computation protocol, and the second participating node takes the second preprocessed input matrix as one of the inputs to the secure two-party matrix multiplication computation protocol. They then perform secure two-party matrix multiplication computation, and the first and second participating nodes obtain the third and fourth multiplication results, respectively. The second participating node generates a second matrix based on the second preprocessed input matrix and the second convolution kernel vector; The first participating node generates the first mixed multiplication result based on the first matrix, the first multiplication result, and the third multiplication result; The second participating node generates a second mixed multiplication result from the second matrix, the second multiplication result, and the fourth multiplication result.

4. The secure two-way fast convolution calculation method based on im2col according to claim 3, characterized in that, The first participating node uses the first preprocessed input matrix as one of the inputs to the secure two-party matrix multiplication computation protocol, and the second participating node uses the second convolution kernel vector as one of the inputs to the secure two-party matrix multiplication computation protocol. They then perform the secure two-party matrix multiplication computation, and the first and second participating nodes obtain the first multiplication result and the second multiplication result, respectively, including: The first participating node and the second participating node receive a first random matrix pair and a second random matrix pair sent by the auxiliary computing node; the first random matrix pair includes a first random matrix and a second random matrix; the second random matrix pair includes a third random matrix and a fourth random matrix; The first participating node derives the first intermediate matrix based on the first preprocessed input matrix and the first random matrix, and then sends the first intermediate matrix to the second participating node; The second participating node derives the second intermediate matrix based on the second convolution kernel vector and the third random matrix, and sends the second intermediate matrix to the first participating node; The second participating node randomly generates a matrix, denoted as the fifth random matrix, and derives the third intermediate matrix based on the second convolution kernel vector, the first intermediate matrix, the fourth random matrix, and the fifth random matrix, and sends it to the first participating node; the fifth random matrix is ​​the result of the second multiplication. The first participating node derives the fourth intermediate matrix based on the second intermediate matrix, the third intermediate matrix, the first random matrix, and the second random matrix; the fourth intermediate matrix is ​​the result of the first multiplication.

5. The secure two-way fast convolution calculation method based on im2col according to claim 3, characterized in that, The first participating node generates a first matrix based on the first preprocessed input matrix and the first convolution kernel vector, specifically including: The first participating node performs a multiplication calculation on the first preprocessed input matrix and the first convolution kernel vector to obtain the first matrix.

6. The secure two-way fast convolution calculation method based on im2col according to claim 3, characterized in that, The first participating node generates the first mixed multiplication result based on the first matrix, the first multiplication result, and the third multiplication result, specifically including: The first participating node performs an addition operation on the first matrix, the first multiplication result, and the third multiplication result to obtain the first mixed multiplication result.

7. A secure two-way fast convolution calculation device based on im2col, characterized in that, The secure two-party fast convolutional computing device based on im2col is deployed on each participating node in the network. The secure two-party fast convolutional computing device based on im2col includes: a task acquisition module, a secure computing module, a rule generation module, a consensus computing module, and a data sending module. The task acquisition module is used to receive and parse fast convolution calculation requests from the calculation requester; The secure computing module is used to automatically match the corresponding secure computing protocol based on the parsed fast convolution computing request; the secure computing protocol includes a secure two-way matrix multiplication protocol and a secure two-way matrix hybrid multiplication protocol; The rule generation module is used to split the fast convolution calculation request according to the asynchronous instruction set of the matched secure computing protocol. The two participating nodes in the fast convolution calculation request perform collaborative computing according to the secure two-party fast convolution calculation method based on im2col as described in any one of claims 1-6. The consensus computation module is used to ensure the synchronization and consistency of results between the two participating nodes in the fast convolution computation request during collaborative computation. The data sending module is used to send the first convolution calculation result obtained by the first participating node or the second convolution calculation result obtained by the second participating node to the calculation requester for aggregation to obtain a fast convolution calculation result.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the secure two-way fast convolution computation method based on im2col as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the secure two-way fast convolution computation method based on im2col as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the secure two-way fast convolution computation method based on im2col as described in any one of claims 1-6.

Images