Neural network-based continuous message private information retrieval coding method under multi-access channel

Abstract

The invention discloses a continuous message private information retrieval coding method under a multi-access channel based on a neural network, and belongs to the crossing field of machine learning and information theory. The method comprises the following steps: 1) constructing a KL divergence neural estimation (KLNE) module, estimating queried KL divergence through a multi-layer perceptron, and quantifying the KL divergence into privacy constraints; 2) constructing a query neural distribution converter (Q-NDT) and coding neural network (E-NN) module, and generating a coding response according to a database message; 3) constructing a message estimation neural network (ME-NN) module, and reconstructing an original message from the mixed response of the multi-access channel; and 4) training the modules by adopting an alternating optimization strategy, wherein the Q-NDT, the E-NN and the ME-NN are fixed in the first stage to train the KLNE so as to improve the KL divergence estimation precision, and the Q-NDT, the E-NN and the ME-NN are fixed in the second stage to train the Q-NDT, the E-NN and the ME-NN so as to minimize the message distortion degree. According to the invention, under the condition of any database and message number, private information retrieval coding and channel coding can be realized, and the degree of distortion between the estimated message and the original message is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine learning and information theory, and particularly relates to a continuous message private information retrieval coding method based on a neural network in a multi-access channel. BACKGROUND

[0002] With the development of digital transformation, information retrieval systems have evolved into intelligent infrastructure supporting medical, financial, and intelligent Internet of Things fields. How to improve retrieval efficiency while ensuring privacy security has become an important research topic. Private information retrieval is a retrieval technology that can effectively protect user privacy. The coding structure aims to completely hide the query information of users when accessing database messages and maximize retrieval speed. However, existing coding schemes cannot adapt to complex channel environments and specific traffic constraints. In recent years, the development of neural networks in coding design and mutual information estimation provides a new idea for private information retrieval coding under various constraints. SUMMARY

[0003] Technical problem: The technical problem to be solved by the application is to provide a continuous message private information retrieval coding method based on a neural network in a multi-access channel for any number of databases and messages. This method requires estimating the KL (Kullback-Leibler) divergence to strictly meet the privacy condition of the coding, and learning the optimal encoder and estimator to minimize the distortion between the estimated message and the original message.

[0004] Technical solution: The technical solution adopted by the application is as follows:

[0005] A continuous message private information retrieval coding method based on a neural network in a multi-access channel includes the following steps:

[0006] Step 1: Construct a KL divergence neural estimation module to estimate the KL divergence of the database response through a neural network and quantify it as a privacy constraint;

[0007] Step 2: Construct a query neural distribution transformer and an encoding neural network module to generate an encoded response according to the messages stored in the database;

[0008] Step 3: Construct a message estimation neural network module to estimate the original message according to the mixed response after the encoded response generated in step 2 is transmitted through the multi-access channel;

[0009] Step 4: Train the modules in steps 1-3 using an alternating optimization strategy:

[0010] First stage: fix the query neural distribution transformer and the encoding neural network module in step 2 and the message estimation neural network module in step 3, train the KL divergence neural estimation module in step 1;

[0011] Second stage: fix the KL divergence neural estimation module in step 1, train the query neural distribution transformer and the encoding neural network module in step 2 and the message estimation neural network module in step 3.

[0012] Repeat the alternating training until convergence to obtain the encoding scheme that minimizes the message distortion under privacy constraints.

[0013] Further, the KL divergence neural estimation module in step 1 is constructed based on the f-divergence representation of KL divergence, through a neural network Train the collected database message samples and corresponding encoding samples to obtain the estimated value of the KL divergence of different responses of the database; where n represents the database number, 1,j represents the message number of interest to the user, and the neural network parameters are A multilayer perceptron model is used, including 1 input layer, 3 hidden layers and 1 output layer, each hidden layer contains 128 neurons, and an ELU activation function is used for nonlinear transformation; for query request The subscript n represents the database number, and the superscript i represents the message of interest to the user; for m samples Let b = 1, 2, …, m, the training of the KL divergence neural estimation needs to maximize the following objective function:

[0014]

[0015] Further, the query neural distribution transformer and the encoding neural network module in step 2 first map a random vector subject to a uniform distribution into a database query, and then map the database stored message and the database query into continuous channel input samples to generate a database response; the query neural distribution transformer is defined as a neural network h φ , h φ A multilayer perceptron MLP model is used, including 1 input layer, 3 hidden layers and 1 output layer, the input layer maps the input data to the hidden layer, each hidden layer contains 64 neurons, and an ELU activation function is used for nonlinear transformation; the database query is calculated by , where N represents the number of databases, K represents the number of database stored messages, and U 2NK is a random variable subject to a uniform continuous distribution, i.e. The superscript 2NK represents the dimension d uIt is twice the product of the number of databases and the number of messages. U is a random variable that obeys uniform continuous distribution, that is, U~Unif([0,1]d u ); the encoding neural network is defined as The multi-layer perceptron MLP model is used, which includes 1 input layer, 3 hidden layers and 1 output layer. The input layer maps the input data to the hidden layer. Each hidden layer contains 128 neurons and uses the ELU activation function for nonlinear transformation. For any k=1,…,K, the database response number n is pass Calculate, where W 1:K is a uniformly distributed continuous random variable stored in the database; for the message sample w k , and its estimated sample is The training process of query neural distribution transformer and encoding neural network needs to minimize the following objective function: in, is a preset hyperparameter, m represents the number of samples, φ is the query neural network distribution transformer parameter, υ is the encoding neural network parameter, and ψ is the message estimation neural network parameter.

[0016] Furthermore, the message estimation neural network module in step 3 generates an estimated message by mapping the mixed response transmitted through the channel into an estimated message sample; the message estimation neural network is defined as g ψ , g ψ The multi-layer perceptron MLP model is used, which includes 1 input layer, 3 hidden layers and 1 output layer. The input layer maps the input data to the hidden layer. Each hidden layer contains 64 neurons and uses the ELU activation function for nonlinear transformation. pass Calculate, where is the mixed response of the kth message, The query request generated for the k-th message; the training of the message estimation neural network needs to minimize the following objective function: in, is a preset hyperparameter, m represents the number of samples, φ is the query neural network distribution transformer parameter, υ is the encoding neural network parameter, and ψ is the message estimation neural network parameter.

[0017] Furthermore, the alternating optimization strategy of step 4 specifically includes:

[0018] Step 4.1. Given the number of databases N and the number of messages K, initialize the KL divergence neural estimation parameter θ, the query neural distribution transformer parameter φ, the encoding neural network parameter υ, the message estimation neural network parameter ψ, the learning rate γ, the Lagrange multiplier λ, and the training batch size m.

[0019] Step 4.2, sampling auxiliary variables representing U 2NKm the distribution of which is subject to;

[0020] Step 4.3, generating database queries using the query neural distribution transformer module

[0021] Step 4.4, sampling message samples representing the distribution of which is subject to;

[0022] Step 4.5, generating database responses using the encoding neural network and computing the mixed responses according to the channel transition probability distribution

[0023] Step 4.6, generating estimated messages using the message estimation neural network module

[0024] Step 4.7, if it is the first stage, computing the KL divergence estimate updating the KL divergence neural estimation module parameters:

[0025]

[0026] Step 4.8, if it is the second stage, updating the query neural distribution transformer parameters: updating the encoding neural network parameters: updating the message estimation neural network module parameters:

[0027]

[0028] Step 4.9, if the algorithm has not converged, returning to Step 4.2 to repeat the above steps;

[0029] Step 4.10, obtaining the private information retrieval distortion estimation result under the current multi-access channel scenario

[0030] Beneficial effects: 1. Step 1 designs KL divergence neural estimation to realize the estimation of KL divergence, and quantifies the privacy condition in the private information retrieval problem as KL divergence, and describes the privacy constraint as a constraint optimization problem through neural estimation and Lagrange multiplier method; 2. Step 2 uses the query neural distribution transformer and the encoding neural network to realize the automatic encoding scheme of private information retrieval, without deriving the explicit encoding structure; 3. Step 3 uses the message estimation neural network to realize the neural estimation of the multi-access channel hybrid response; 4. Step 4 combines the above modules to build a joint encoding optimization framework, and uses the alternating optimization method for training, and solves the private information retrieval encoding and channel coding problem through gradient optimization. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The overall encoding optimization framework of the application. DETAILED DESCRIPTION

[0032] The technical scheme of the application will be described in detail below, but the protection scope of the application is not limited to the described embodiments.

[0033] As Figure 1 shown, the application discloses a continuous message private information retrieval encoding method based on neural network under multi-access channel, which can realize private information retrieval encoding and channel coding under the condition of any database and message number, and reduce the distortion degree between the estimated message and the original message. Figure 1 The random variable U 2NK Generate the query sequence through the query neural distribution transformer (Q-NDT) The KL divergence neural estimation calculates the KL divergence between the queries and combines it into the training target function, and then the encoding neural network (E-NN) generates the response sequence according to the sampled message W 1:K and the query in turn generates the response sequence Then the hybrid response sequence is obtained through channel transmission The message estimation neural network (ME-NN) estimates the original message sequence according to the received hybrid response The training of the above modules is carried out in the alternating optimization mode.

[0034] An embodiment is given below:

[0035] For the case of two databases and two messages, the multi-access channel is an additive multi-access channel, that is, the channel output Y=X1+X2+Z, where the noise Z obeys the uniform distribution on (-r, r), and X1, X2 represent the outputs of the two databases. The specific encoding optimization steps are as follows:

[0036] Step 1, initialize KL divergence neural estimation parameters θ, query neural distribution transformer parameters φ, encoding neural network parameters υ, message estimation neural network parameters ψ, learning rate γ = 0.0005, Lagrange multiplier λ = 1, training batch size m = 4000;

[0037] Step 2, sample auxiliary variables

[0038] Step 3, generate database queries using the query neural distribution transformer module

[0039] Step 4, sample message samples

[0040] Step 5, generate database responses using the encoding neural network and compute the mixed responses according to the channel transition probability distribution

[0041] Step 6, generate estimated messages using the message estimation neural network module

[0042] Step 7, if it is the first stage, compute the KL divergence estimate Update the KL divergence neural estimation module parameters:

[0043]

[0044] Step 8, if it is the second stage, update the query neural distribution transformer parameters: Update the encoding neural network parameters: Update the message estimation neural network module parameters:

[0045]

[0046] Step 9, if the algorithm has not converged, return to Step 2 and repeat the above steps;

[0047] Step 10, obtain the privacy information retrieval distortion estimation result under the additive multiple access channel with 2 databases and 2 messages

Claims

1. A neural network-based continuous message private information retrieval coding method under multi-access channels, applicable to the continuous message type private information retrieval problem under multi-access channels, characterized by: The following steps are involved: Step 1: Construct a KL divergence neural estimation module to estimate the KL divergence of the database response through a neural network and quantify it as a privacy constraint; Step 2: Construct a query neural network distribution transformer and an encoding neural network module to generate an encoded response based on the message stored in the database; Step 3: Construct a message estimation neural network module to estimate the original message based on the mixed response of the coded response generated in step 2 after transmission through the multi-access channel; Step 4: Use alternating optimization strategy to train the modules of steps 1-3: Phase 1: Fix the query neural distribution transformer and encoding neural network module of step 2 and the message estimation neural network module of step 3, and train the KL divergence neural estimation module of step 1; Phase 2: Fix the KL divergence neural estimation module in step 1, train the query neural distribution transformer and encoding neural network module in step 2 and the message estimation neural network module in step 3; Repeat the alternating training until convergence to obtain a coding scheme that minimizes message distortion under privacy constraints.

2. The method for retrieving and encoding private information of continuous messages in multiple access channels based on a neural network as claimed in claim 1, wherein: The KL divergence neural estimation module in step 1 is constructed based on the f-divergence representation of KL divergence, and is constructed through a neural network. Training the collected database message samples and the corresponding coded samples to obtain the estimated value of the KL divergence of different responses in the database; Where n represents the database number, 1, j represents the message number that the user is interested in, and the neural network parameters are A multilayer perceptron model is used, including 1 input layer, 3 hidden layers and 1 output layer. Each hidden layer contains 128 neurons, and the ELU activation function is used for nonlinear transformation. m samples of The subscript n represents the database number, and the superscript i represents the message that the user is interested in. Let b = 1, 2, ..., m. The training of the KL divergence neural estimator needs to maximize the following objective function:

3. The method for retrieving and encoding private information of continuous messages in multiple access channels based on a neural network as claimed in claim 2, wherein: The query neural distribution converter and encoding neural network module in step 2 first maps a random vector that follows a uniform distribution into a database query, and then maps the database stored messages and database queries into continuous channel input samples to generate a database response; The query neural distribution transformer is defined as a neural network h φ , h φ The multi-layer perceptron MLP model is used, which includes 1 input layer, 3 hidden layers and 1 output layer. The input layer maps the input data to the hidden layer. Each hidden layer contains 64 neurons and uses the ELU activation function for nonlinear transformation. pass Calculate, where N is the number of databases, K is the number of messages stored in the database, and U 2NK is a random variable that obeys a uniform continuous distribution, that is, ), where the superscript 2NK represents the dimension d of the random variable U u is twice the product of the number of databases and the number of messages; the encoding neural network is defined as The multi-layer perceptron MLP model is used, which includes 1 input layer, 3 hidden layers and 1 output layer. The input layer maps the input data to the hidden layer. Each hidden layer contains 128 neurons and uses the ELU activation function for nonlinear transformation. For any k=1,…,K, the database response number n is pass Calculate, where W 1:K is a uniformly distributed continuous random variable stored in the database; for the message sample w k , and its estimated sample is The training process of query neural distribution transformer and encoding neural network needs to minimize the following objective function: in, is a preset hyperparameter, m represents the number of samples, φ is the query neural network distribution transformer parameter, υ is the encoding neural network parameter, and ψ is the message estimation neural network parameter.

4. The method for retrieving and encoding private information of continuous messages in multiple access channels based on a neural network as claimed in claim 3, wherein: The message estimation neural network module in step 3 generates an estimated message by mapping the mixed response transmitted through the channel into an estimated message sample; The message estimation neural network is defined as g ψ , g ψ The multi-layer perceptron MLP model is used, which includes 1 input layer, 3 hidden layers and 1 output layer. The input layer maps the input data to the hidden layer. Each hidden layer contains 64 neurons and uses the ELU activation function for nonlinear transformation. pass Calculate, where is the mixed response of the kth message, The query request generated for the k-th message; the training of the message estimation neural network needs to minimize the following objective function: in, is a preset hyperparameter, m represents the number of samples, φ is the query neural network distribution transformer parameter, υ is the encoding neural network parameter, and ψ is the message estimation neural network parameter.

5. The method for retrieving and encoding private information of continuous messages in multiple access channels based on a neural network as claimed in claim 1, wherein: The alternating optimization strategy of step 4 specifically includes: Step 4.

1. Given the number of databases N and the number of messages K, initialize the KL divergence neural estimation parameter θ, the query neural distribution transformer parameter φ, the encoding neural network parameter υ, the message estimation neural network parameter ψ, the learning rate γ, the Lagrange multiplier λ, and the training batch size m. Step 4.2: Sampling auxiliary variables Indicates U 2NKm obey the distribution; Step 4.3: Generate database queries using the query neural distribution transformer module Step 4.4: Sample message express obey the distribution; Step 4.5: Generate database responses using the encoding neural network and calculate the mixed responses based on the channel transition probability distribution. Step 4.6: Generate estimated messages using the message estimation neural network module Step 4.7: If it is the first stage, calculate the KL divergence estimate Update the KL divergence neural estimation module parameters: Step 4.8: If it is the second stage, update the query neural distribution converter parameters: Update the encoding neural network parameters: Update message to estimate neural network module parameters: Step 4.9: If the algorithm does not converge, return to step 4.2 and repeat the above steps; Step 4.10: Obtain the distortion estimation result of private information retrieval under the current multi-access channel scenario 6. The method for retrieving and encoding private information of continuous messages in multiple access channels based on a neural network according to any one of claims 1 to 5, characterized in that: The method is applicable to multi-access channel scenarios with any number of databases and messages.

Images