A privacy-protected distributed image feature extraction method and system

Abstract

The application relates to a privacy-protected distributed image feature extraction method and system, which comprises the following steps: performing feature extraction on collected image data on a distributed terminal, extracting a feature vector of each image in the image data; based on the feature vectors of the images, calculating a feature correlation matrix between the images on the distributed terminal and sending the feature correlation matrix to a server; the server receives the feature correlation matrices of the multiple distributed terminals, and calculates a complete feature correlation matrix between the image data of the multiple distributed terminals through a matrix completion algorithm; based on the complete feature correlation matrix, a Laplacian matrix is calculated on the server using a hypergraph construction method; and based on the Laplacian matrix, eigenvalue decomposition is performed on the server to obtain final image features. The application guarantees the privacy of distributed image data.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a privacy-preserving distributed image feature extraction method and system. Background Technology

[0002] With the continuous development of image recognition technology, deep learning-based methods have become mainstream. However, these methods often require large amounts of training data. Traditional training methods typically employ a centralized framework, collecting data and placing it on powerful servers for training. This process presents two main problems:

[0003] First, storing raw data directly can easily lead to privacy data leaks.

[0004] Secondly, unified collection and storage of data is not conducive to the rapid expansion and updating of data.

[0005] Therefore, the key to solving the above problems lies in how to collect data on different terminals while ensuring data privacy. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a privacy-preserving distributed image feature extraction method and system.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] On the one hand, a privacy-preserving distributed image feature extraction method includes:

[0009] Step 1: On the distributed terminal, feature extraction is performed on the collected image data, extracting a feature vector for each image in the image data;

[0010] Step 2: Based on the feature vectors extracted from each image, calculate the feature correlation matrix between each image on the distributed terminal and send it to the server;

[0011] Step 3: The server receives the feature association matrices of multiple distributed terminals and calculates the complete feature association matrix between the image data of multiple distributed terminals using a matrix completion algorithm.

[0012] Step 4: Based on the complete feature correlation matrix, calculate the Laplacian matrix on the server using the hypergraph construction method;

[0013] Step 5: Perform eigenvalue decomposition on the server based on the Laplacian matrix to obtain the final image features.

[0014] Preferably, the feature correlation matrix is ​​obtained by sequentially calculating the Euclidean distance between the feature vectors of the two images, and is expressed as:

[0015]

[0016] Among them, E i Let M represent the feature correlation matrix of the i-th distributed terminal; i = {1, 2, ..., n} represents the number of the distributed terminal, and n represents the number of distributed terminals; i This represents the number of images acquired on the i-th distributed terminal; Let j represent the Euclidean distance between the j-th image and the k-th image on the i-th distributed terminal, where j = {1, 2, ..., M}. i}, k={1,2,…,M i}

[0017] Preferably, a matrix completion algorithm is used to calculate the complete feature correlation matrix between image data from multiple distributed terminals, specifically including:

[0018] Step 3.1, combine the feature association matrix E on different distributed terminals. i The incomplete feature correlation matrix is ​​concatenated as follows:

[0019]

[0020] Step 3.2, calculate the correlation matrix C between images, using the following formula:

[0021]

[0022] Step 3.3, Calculate the mixture matrix The formula is:

[0023]

[0024] Where I is an identity matrix with the same width and height as C;

[0025] Step 3.4, randomly initialize matrices A and B, where the submatrix of the i-th distributed terminal is represented as A. i [0] and B i [0];

[0026] Step 3.5: The matrix to be completed on the i-th distributed terminal... Initialize to in, E represents i The submatrix on the i-th distributed terminal; during initialization, It is also equivalent to

[0027] Step 3.6, initialize m = 1, calculate A i [1], as follows:

[0028]

[0029] Among them, C il Let C represent the association between the i-th distributed terminal and the l-th distributed terminal, and be a submatrix of C;

[0030] Step 3.7, update A as follows:

[0031]

[0032] Among them, among them, This represents the association between the i-th distributed terminal and the l-th distributed terminal. The submatrix; α is a parameter controlling the update rate, with a value between 0 and 1;

[0033] Step 3.8, update B as follows:

[0034]

[0035] Among them, A′ i [m+1] is A i The Moore-Penrose generalized inverse matrix of [m+1];

[0036] Step 3.9, Update The formula is:

[0037]

[0038] in:

[0039]

[0040] in,

[0041] Step 3.10, calculate E diff ,as follows:

[0042]

[0043] Wherein, AVG represents the average value of the elements in the matrix;

[0044] Step 3.11, if E diff If the value is less than β, the loop ends; otherwise, increment m by 1 and return to step 3.7; the value is between 0.5 and 1.

[0045] Step 3.12, will Set all negative values ​​in the value set to 0;

[0046] Step 3.13, calculate the final completed matrix E:

[0047]

[0048] in, This represents the result after convergence in step 3.11.

[0049] Preferably, based on the complete feature correlation matrix, the Laplacian matrix is ​​calculated using the hypergraph construction method, specifically including:

[0050] Step 4.1, calculate the hypergraph incidence matrix H, as follows:

[0051]

[0052] Step 4.2, calculate the degree matrix of the hyperedges in the hypergraph, as follows:

[0053]

[0054] Among them, H k H represents jk The sum of all data in the k-th row;

[0055] Step 4.3, calculate the degree matrix η of the vertices in the hypergraph, as follows:

[0056]

[0057] Step 4.4, construct the hypergraph Laplacian matrix, as shown in the following formula:

[0058]

[0059] Where e represents an edge in the hypergraph, which is composed of a vertex v in the hypergraph and its K nearest neighbor; Represents matrix η v Square root; H′ denotes the transpose of H. Indicates η e The inverse matrix, η e Let represent the element in the e-th row of η, and let ∈′ represent the transpose of ∈.

[0060] Preferably, a privacy-preserving distributed image feature extraction system includes several distributed terminals and a server; the distributed terminals include a feature extraction module and an association matrix calculation module; the server includes a matrix completion module, a graph construction module, and an eigenvalue decomposition module.

[0061] The feature extraction module is used to extract features from the acquired image data, extracting a feature vector for each image in the image data;

[0062] The correlation matrix calculation module calculates the feature correlation matrix between each image based on the feature vectors extracted from each image, and sends it to the matrix completion module.

[0063] The matrix completion module is used to calculate the complete feature correlation matrix between image data from multiple distributed terminals using a matrix completion algorithm.

[0064] The graph construction module is used to calculate the Laplacian matrix using the hypergraph construction method based on the complete feature correlation matrix.

[0065] The eigenvalue decomposition module is used to perform eigenvalue decomposition based on the Laplacian matrix to obtain the final image features.

[0066] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0067] The original image data collected by this invention is stored on each distributed terminal, without needing to be uploaded to a server, thus preventing the leakage of privacy data. Specifically, each distributed terminal extracts features from its collected image data and calculates the feature correlation matrix between each image, then sends the feature correlation matrix to the server. The server calculates the complete feature correlation matrix between the image data of each distributed terminal using a matrix completion algorithm, and calculates the Laplacian matrix using a hypergraph construction method. Based on the Laplacian matrix, eigenvalue decomposition is performed to obtain the final image features, which are used for subsequent image recognition work.

[0068] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the privacy-preserving distributed image feature extraction method and system of the present invention are not limited to the embodiments. Attached Figure Description

[0069] Figure 1 This is a flowchart of the method of the present invention;

[0070] Figure 2 This is a structural block diagram of the system of the present invention. Detailed Implementation

[0071] The technical solutions in the embodiments of the present invention will be described and discussed in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0072] See Figure 1 As shown, the present invention provides a privacy-preserving distributed image feature extraction method, comprising the following steps:

[0073] On the one hand, a privacy-preserving distributed image feature extraction method includes:

[0074] Step 1: On the distributed terminal, feature extraction is performed on the collected image data, extracting a feature vector for each image in the image data;

[0075] Step 2: Based on the feature vectors extracted from each image, calculate the feature correlation matrix between each image on the distributed terminal and send it to the server;

[0076] Step 3: The server receives the feature association matrices of multiple distributed terminals and calculates the complete feature association matrix between the image data of multiple distributed terminals using a matrix completion algorithm.

[0077] Step 4: Based on the complete feature correlation matrix, calculate the Laplacian matrix on the server using the hypergraph construction method;

[0078] Step 5: Perform eigenvalue decomposition on the server based on the Laplacian matrix to obtain the final image features.

[0079] Specifically, in step 1, each image is represented by certain image features according to the characteristics and needs of the application. These features can be color, shape, and texture, etc. Therefore, the extracted features can also be color, shape, and texture, etc.

[0080] In step 2, the feature correlation matrix is ​​obtained by sequentially calculating the Euclidean distance between the feature vectors of the two images, and is expressed as:

[0081]

[0082] Among them, E i Let M represent the feature correlation matrix of the i-th distributed terminal; i = {1, 2, ..., n} represents the number of the distributed terminal, and n represents the number of distributed terminals; i This represents the number of images acquired on the i-th distributed terminal; Let j represent the Euclidean distance between the j-th image and the k-th image on the i-th distributed terminal, where j = {1, 2, ..., M}. i}, k={1,2,…,M i}

[0083] In step 3, the server calculates the complete feature correlation matrix between image data from multiple distributed terminals using a matrix completion algorithm, specifically including:

[0084] Step 3.1, combine the feature association matrix E on different distributed terminals. i The incomplete feature correlation matrix is ​​concatenated as follows:

[0085]

[0086] Step 3.2, calculate the correlation matrix C between images, using the following formula:

[0087]

[0088] Step 3.3, Calculate the mixture matrix The formula is:

[0089]

[0090] Where I is an identity matrix with the same width and height as C; the mixture matrix Indicates the proportion in which the preceding and following steps of the completion process are mixed;

[0091] Step 3.4, randomly initialize matrices A and B, where the submatrix of the i-th distributed terminal is represented as A. i [0] and B i [0]; A and B are the results of low-rank decomposition, through... get, This represents the Euclidean distance in step 2.

[0092] Step 3.5: The matrix to be completed on the i-th distributed terminal... Initialize to in, E represents i The submatrix on the i-th distributed terminal; during initialization, It is also equivalent to

[0093] Step 3.6, initialize m = 1, calculate A i [1], as follows:

[0094]

[0095] Among them, C il Let C represent the association between the i-th distributed terminal and the l-th distributed terminal, and be a submatrix of C;

[0096] Step 3.7, update A as follows:

[0097]

[0098] Among them, among them, This represents the association between the i-th distributed terminal and the l-th distributed terminal. The submatrix; α is a parameter controlling the update rate, with a value between 0 and 1;

[0099] Step 3.8, update B as follows:

[0100]

[0101] Among them, A′ i [m+1] is A iThe Moore-Penrose generalized inverse matrix of [m+1];

[0102] Step 3.9, Update The formula is:

[0103]

[0104] in:

[0105]

[0106] in,

[0107] Step 3.10, calculate E diff ,as follows:

[0108]

[0109] Wherein, AVG represents the average value of the elements in the matrix;

[0110] Step 3.11, if E diff If the value is less than β, the loop ends; otherwise, increment m by 1 and return to step 3.7; the value is between 0.5 and 1.

[0111] Step 3.12, will Set all negative values ​​in the value set to 0;

[0112] Step 3.13, calculate the final completed matrix E:

[0113]

[0114] in, This represents the result after convergence in step 3.11.

[0115] In step 4, based on the complete feature correlation matrix, the Laplacian matrix is ​​calculated using the hypergraph construction method, specifically including:

[0116] Step 4.1, calculate the hypergraph incidence matrix H, as follows:

[0117]

[0118] Step 4.2, calculate the degree matrix of the hyperedges in the hypergraph, as follows:

[0119]

[0120] Among them, H k H represents jk The sum of all data in the k-th row;

[0121] Step 4.3, calculate the degree matrix η of the vertices in the hypergraph, as follows:

[0122]

[0123] Step 4.4, construct the hypergraph Laplacian matrix, as shown in the following formula:

[0124]

[0125] Where e represents an edge in the hypergraph, which is composed of a vertex v in the hypergraph and its K nearest neighbor; Represents matrix η v Square root; H′ denotes the transpose of H. Indicates η e The inverse matrix, η e Let represent the element in the e-th row of η, and let ∈′ represent the transpose of ∈.

[0126] See Figure 2 As shown, this embodiment also includes a privacy-preserving distributed image feature extraction system, comprising a plurality of distributed terminals 10 and a server 20; the distributed terminal 10 includes a feature extraction module 101 and an association matrix calculation module 102; the server 20 includes a matrix completion module 201, a graph construction module 202 and an eigenvalue decomposition module 203;

[0127] The feature extraction module 101 is used to extract features from the acquired image data, extracting a feature vector for each image in the image data;

[0128] The correlation matrix calculation module 102 calculates the feature correlation matrix between each image based on the feature vectors extracted from each image, and sends it to the matrix completion module 201.

[0129] The matrix completion module 201 is used to calculate the complete feature correlation matrix between image data of multiple distributed terminals 10 through a matrix completion algorithm;

[0130] The graph construction module 202 is used to calculate the Laplacian matrix using the hypergraph construction method based on the complete feature correlation matrix.

[0131] The eigenvalue decomposition module 203 is used to perform eigenvalue decomposition based on the Laplacian matrix to obtain the final image features.

[0132] The specific implementation of a privacy-preserving distributed image feature extraction system is the same as that of a privacy-preserving distributed image feature extraction method, and will not be described again in this embodiment.

[0133] The above is merely a preferred embodiment of the present invention. However, the present invention is not limited to the above embodiment. Any equivalent changes and modifications made according to the present invention, provided that the resulting functional effects do not exceed the scope of the present invention, shall fall within the protection scope of the present invention.

Claims

1. A privacy-preserving distributed image feature extraction method, characterized in that, The method comprises the following steps: Step 1, feature extraction is performed on the collected image data on the distributed terminal, and a feature vector is extracted for each image in the image data; Step 2, based on the feature vectors extracted from each image, a feature correlation matrix between each image is calculated on the distributed terminal and sent to the server; Step 3, the server receives the feature correlation matrices of multiple distributed terminals, and calculates a complete feature correlation matrix between the image data of multiple distributed terminals by using a matrix completion algorithm; Step 4, based on the complete feature correlation matrix, a Laplacian matrix is calculated on the server by using a hypergraph construction method; Step 5, based on the Laplacian matrix, eigenvalue decomposition is performed on the server to obtain the final image features; The complete feature correlation matrix between the image data of multiple distributed terminals is calculated by using a matrix completion algorithm, and specifically comprises the following steps: Step 3.

1. Correlating feature association matrices on different distributed terminals into an incomplete feature association matrix, represented as follows: ； Step 3.2, computing the matrix of correlation between images The formula is: ； Step 3.3, Compute mixing matrix , the formula is: ； wherein is a unit matrix of the same width and height as is a unit matrix of the same width and height as Step 3.4, randomly initialize matrix and where the sub-matrix of the jthdistributed terminal is denoted as and and ; Step 3.5, the matrix to be completed on the first distributed terminal is initialized to ; where represents the sub-matrix on the first distributed terminal; at initialization, is also equal to ; Step 3.6, initialization , compute As follows: ； wherein, represents the association of the first distributed terminal and the second distributed terminal, and is a submatrix of . Step 3.7, Update As follows: ； wherein, wherein, represents the association of the first distributed terminal and the second distributed terminal, is a submatrix of ; and is a control The update rate parameter is a value between 0 and 1; Step 3.8, Update As follows: ； wherein is the Moore-Penrose generalized inverse matrix of Step 3.9, update , the formula is: ； Wherein: ； wherein ; Step 3.10, calculation As follows: ； wherein denotes the average value of the elements in the matrix; Step 3.11, if then end the loop, otherwise, return to step 3.7; is a number between 0.5 and 1. Step 3.12, set all negative values in the middle to 0; all negative values in the middle to 0; Step 3.13, calculating the final completion matrix : ； wherein represents the result of step 3.11 after convergence. 2.The privacy-preserving distributed image feature extraction method of claim 1, wherein, The feature correlation matrix is obtained by sequentially calculating the Euclidean distance between two image feature vectors, and is expressed as: ； wherein, represents a feature correlation matrix of the th distributed terminal; represents a number of the distributed terminal, represents a number of the distributed terminals; represents a number of images acquired on the th distributed terminal; represents a Euclidean distance between the th image and the th image on the th distributed terminal, , .

3. The method for privacy-preserving distributed image feature extraction of claim 2, wherein, Based on the complete feature correlation matrix, a Laplacian matrix is calculated by using a hypergraph construction method, and specifically comprises the following steps: Step 4.1, Calculating the hypergraph incidence matrix As follows: ； Step 4.2, computing the degree matrix of the hyperedges in the hypergraph As follows: ； wherein represents sum of all data in the kth row; Step 4.3, computing the degree matrix of vertices in the hypergraph As follows: ； Step 4.4, a hypergraph Laplacian matrix is constructed, and the formula is as follows: ； wherein, denotes an edge in a hypergraph, by vertices in the hypergraph and its K nearest neighbors; denotes a matrix square root; denotes transpose matrix, denotes inverse matrix of the element in the row of denotes transpose matrix of 4. A privacy-protected distributed image feature extraction system comprising a plurality of distributed terminals and a server; characterized in that, Based on the privacy protection distributed image feature extraction method according to any one of claims 1-3, the distributed terminal comprises a feature extraction module and a correlation matrix calculation module; the server comprises a matrix completion module, a hypergraph construction module and an eigenvalue decomposition module; The feature extraction module is configured to perform feature extraction on the collected image data, and extract a feature vector for each image in the image data; The correlation matrix calculation module is configured to calculate a feature correlation matrix between each image based on the feature vectors extracted from each image, and send the feature correlation matrix to the matrix completion module; The matrix completion module is configured to calculate a complete feature correlation matrix between the image data of multiple distributed terminals by using a matrix completion algorithm; The hypergraph construction module is configured to calculate a Laplacian matrix by using a hypergraph construction method based on the complete feature correlation matrix; The eigenvalue decomposition module is configured to perform eigenvalue decomposition based on the Laplacian matrix to obtain the final image features.

Images