Quantum-like multilayer sensing method for mining strong correlation between features

Abstract

The invention discloses a quasi-quantum multi-layer sensing method for mining strong correlation between features, which comprises the following steps: constructing a nonlinear neural network layer capable of mining strong correlation between features based on a multi-mass subsystem, and further constructing a quasi-quantum multi-layer perceptron based on the constructed neural network layer, so that the number of model parameters is reduced by hundreds of times. The principle and basis of the strong correlation between the QiMLP mining features are theoretically analyzed, the strong correlation between the features can be learned, the number of model parameters can be remarkably reduced, and the effectiveness of the constructed multi-layer perceptron is verified through experiments.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a quantum-like multi-layer perception method for mining strong correlations between features. Background Art

[0002] Multilayer Perceptron (MLP) models nonlinear relationships through nonlinear mapping of activation functions, and uses the combination of multiple neuron layers to learn more abstract and complex feature representations, thereby capturing deeper structures and patterns in the data. Although the multilayer perceptron is a simple application practice of traditional neuron layers, it has superior performance and outstanding performance in specific tasks, and is the cornerstone of machine learning, deep learning, and the construction of large language models.

[0003] However, each layer of neurons in the neural network model uses a weighted summation method to fuse the features output by the previous layer of neurons, which makes it difficult to learn the strong correlation between features and cannot capture the inherent strong connection between features. This method only assigns a certain weight value to the importance of the features and adds the features involved. In the real world, there are generally strong intrinsic connections between the features of things, such as causal relationships, strong statistical correlations, etc. Therefore, constructing a neuron layer that can mine the strong correlation between features to capture the real strong relationship or causal relationship between the features of things is more conducive to the model to implement effective decision-making. In addition, machine learning models built on traditional neuron layers, such as deep learning models, large language models, etc., require a large number of parameters to learn and require a large amount of computing power resources during the training process, which has become a bottleneck in the development of artificial intelligence (AI).

[0004] Chinese patent document CN115293254A discloses a "classification method and related equipment based on quantum multilayer perceptron". The method includes: obtaining the data to be classified, and encoding the data to be classified into a coded quantum state; inputting the coded quantum state into the first quantum computing layer to obtain an intermediate quantum state, the first quantum computing layer includes a parameter-containing quantum entanglement module; inputting the intermediate quantum state into the second quantum computing layer to obtain a result quantum state, the second quantum computing layer includes a cascaded quantum nonlinear transformation module and the parameter-containing quantum entanglement module; obtaining a classification result based on the result quantum state. The above technical solution has high requirements on the number of parameters, and it is difficult to simultaneously meet the effectiveness, strong correlation mining capabilities and parameter compression capabilities. Summary of the invention

[0005] The present invention mainly solves the technical problem that the original technical solution has high requirements on the number of parameters and is difficult to simultaneously meet the technical problems of effectiveness, strong correlation mining capability and parameter compression capability, and provides a quasi-quantum multilayer perception method for mining strong correlations between features. A nonlinear neural network layer that can mine strong correlations between features is constructed based on a multi-body quantum system, and then a quasi-quantum multilayer perceptron is constructed based on the constructed neural network layer. The principle and basis of QiMLP mining strong correlations between features are theoretically analyzed, which can not only learn the strong correlations between features, but also significantly reduce the number of model parameters.

[0006] The above technical problem of the present invention is mainly solved by the following technical solution: The present invention comprises the following steps: S1. obtaining a weight matrix by Kronecker product operation; S2. Obtain the bias term by Kronecker product operation; S3. Apply a weight matrix to the input vector of the quantum-like neuron layer and a bias term to construct the quantum-like neuron layer; S4. Applying an activation function to the output vector of the quantum-like neuron layer to obtain the final output vector of the quantum-like neuron layer; S5. Adjust the quantum-like neuron layer according to specific task requirements.

[0007] Preferably, the S1 specifically includes combining one or more small weight matrices using one or more Kronecker product operations to obtain a final weight matrix of the neuron layer. The reconstructed weight matrix can be expressed as in To construct the small weight matrix used in the final weight matrix, the operator Refers to the Kronecker product operation.

[0008] Preferably, the input of the neuron layer is N M dimensional vector, that is, the number of neurons in the previous layer is N M , reconstruct the weight matrix, and the number of Kronecker products used to construct the final weight matrix is ​​determined by the tasks that the model is applicable to and the number of parameters required for the tasks. The final bias term after reconstruction can be expressed as in A small bias term used to construct the final bias term.

[0009] Preferably, the S2 specifically includes combining one or more small bias terms using one or more Kronecker product operations to obtain a final bias term of the neuron layer.

[0010] As a preference, the final bias term is N M dimensional vector, representing the final bias term after reconstruction. The number of Kronecker products used to construct the bias term corresponds to the number of Kronecker products used to construct the weight matrix.

[0011] Preferably, the S3 specifically includes multiplying the quantum-like neuron layer input vector by the final weight matrix and adding the final bias term to construct the quantum-like neuron layer.

[0012] Preferably, the S4 specifically includes applying an activation function to the input vector multiplied by the final weight matrix and added with the final bias term to obtain the output result of the neuron layer, and selecting a suitable activation function according to the needs of the construction task. The output Y of the neuron layer, in is the output of the neuron layer, the operator * represents the matrix multiplication operation, and the subscript k represents the number of copies of the reconstructed weight matrix.

[0013] Preferably, a subscript k is set for the final constructed weight matrix to indicate the number of copies of the reconstructed weight matrix, thereby alleviating the problem of over-compression of model parameters.

[0014] Preferably, the S5 specifically includes constructing a quasi-quantum multilayer perceptron based on the constructed quasi-quantum neuron layer, and according to the vector of the input quasi-quantum multilayer perceptron being Set the input layer of the quantum-like multilayer perceptron. The input layer of the quantum-like multilayer perceptron is in is the output of the input layer. The activation function f(·) can be the commonly used RELU(·) or other. and B i is the weight matrix and bias term of the input layer.

[0015] As a preference, the vector of the input quantum multilayer perceptron is Set up the hidden layer and output layer of the quantum multilayer perceptron, using and B i represents the weight matrix and bias term of the input layer, and the hidden layer is a number of layers. The hidden layer is in is the output of the input layer. The activation function f(·) can be the commonly used RELU(·) or other, and B iis the weight matrix and bias term of the input layer, and the hidden layer can be multi-layer; The output layer is in is the output of the input layer. The activation function f(·) can be the commonly used Softmax(·) or other, and B i is the weight matrix and bias term of the input layer; based on the above steps, a quantum-like multilayer perceptron of any number of layers and any size can be constructed.

[0016] The beneficial effects of the present invention are: 1. Inspired by quantum many-body quantum systems, the present invention constructs a quantum-like neuron layer that can mine strong correlations between features, and based on the constructed neuron layer, constructs a quantum-like multilayer perceptron with practical application value. The multilayer perceptron has the ability to mine strong correlations between features and the parameters used can be compressed by more than hundreds of times compared to the classical multilayer perceptron.

[0017] 2. The quantum-like neuron layer proposed in this invention has important application value for reshaping machine learning, deep learning and large language models.

[0018] 3. The principle of the constructed quantum-like neuron layer to mine strong correlations is analyzed theoretically, and the effectiveness, strong correlation mining capability and parameter compression capability of the constructed quantum-like multilayer perceptron are verified through experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the present invention.

[0020] Figure 2 This is a graph of experimental results of a classical multilayer perceptron (Classic MLP) and a quasi-quantum multilayer perceptron under an MNIST data set of the present invention.

[0021] Figure 3 A diagram of the number of parameters used by a classical multilayer perceptron (Classic MLP) and a quasi-quantum multilayer perceptron under an MNIST data set of the present invention.

[0022] Figure 4 This is a graph of experimental results of quantum-like multilayer perceptrons of two-body (Two-body QiMLP), three-body (Three-bodyQiMLP), and four-body (Four-body QiMLP) quantum systems under the MNIST data set of the present invention at different k values.

[0023] Figure 5This is a diagram of the number of parameters used by the quantum-like multilayer perceptrons of two-body (Two-body QiMLP), three-body (Three-bodyQiMLP), and four-body (Four-body QiMLP) quantum systems under the MNIST data set of the present invention at different k values.

[0024] Figure 6 This is a graph of experimental results of a classical multi-layer perceptron and a quantum-like multi-layer perceptron under a CIFAR-10 data set of the present invention.

[0025] Figure 7 This is a diagram of the number of parameters used by a classical multilayer perceptron and a quasi-quantum multilayer perceptron under a CIFAR-10 data set of the present invention.

[0026] Figure 8 This is a graph of experimental results of a quantum-like multilayer perceptron for two-body (Two-body QiMLP), three-body (Three-bodyQiMLP), and four-body (Four-body QiMLP) quantum systems under different k values ​​in the CIFAR-10 dataset of the present invention.

[0027] Fig. 9 This is a diagram of the number of parameters used by the quantum-like multilayer perceptrons of two-body (Two-body QiMLP), three-body (Three-bodyQiMLP), and four-body (Four-body QiMLP) quantum systems under different k values ​​in the CIFAR-10 dataset of the present invention.

[0028] Fig.10 This is a graph showing the proportion of strongly correlated features mined from the input layer of a quantum-like multilayer perceptron under an MNIST data set of the present invention.

[0029] Fig.11 This is a graph showing the proportion of strongly correlated features mined from the hidden layer of a quantum-like multilayer perceptron under an MNIST data set of the present invention.

[0030] Fig.12 This is a graph showing the proportion of strongly correlated features mined from the input layer of a quantum-like multilayer perceptron under a CIFAR-10 dataset of the present invention.

[0031] Fig.13 This is a graph showing the proportion of strongly correlated features mined from the first hidden layer of a quantum-like multilayer perceptron under a CIFAR-10 dataset of the present invention.

[0032] Fig.14 This is a graph showing the proportion of strongly correlated features mined from the second hidden layer of a quantum-like multilayer perceptron under a CIFAR-10 dataset of the present invention.

[0033] Fig.15 This is a graph showing the proportion of strongly correlated features mined from the third hidden layer of a quantum-like multilayer perceptron under a CIFAR-10 dataset of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the technical solutions of the present application are further described in detail below through embodiments and in combination with the accompanying drawings. It should be understood that the specific implementation method described here is only an optimal embodiment of the present application, which is only used to explain the present application and does not limit the protection scope of the present application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] As for mining strong correlations between features, researchers have not yet found a suitable method to mine strong correlations, so they try to learn a disentangled representation method that is independent between features, thereby reducing the interference of strong correlations between features on downstream tasks (in fact, at this stage, people narrowly believe that it is harmful because they cannot effectively control this favorable resource). The most representative work is the disentangled representation method based on variational autoencoder (VAE), such as beta-VAE and FactorVAE. In order to reduce the number of parameters of neural network models, in recent years, researchers have proposed methods including model sparsification (also known as model pruning), model quantization, knowledge distillation, tensor decomposition, etc. to achieve model compression to reduce the number of parameters that the model needs to learn.

[0036] In quantum theory, multi-body quantum systems can characterize the strong correlation between system features, which includes not only statistical correlations in the classical world, but also non-classical correlations in the quantum world. In other words, the formal framework of quantum mechanics can describe the strong correlations between the features of things. It should be emphasized again here that the strong correlation in the present invention does not specifically refer to non-classical or quasi-quantum correlations between the features of things, such as quantum nonlocality, but refers to the strong statistical correlations between features, such as causality.

[0037] The present invention borrows the data framework of quantum mechanics, and reshapes the basic components of the neural network, namely the neuron layer, with the help of the theory of multi-body quantum systems, and further constructs a multi-layer perceptron model with practical application value, which is called a quantum-inspired MLP (QiMLP). Specifically, a weight matrix obtained by Kronecker product operation is applied to the input vector of the neuron layer, and a bias term obtained by Kronecker product operation is added, and finally an activation function is applied to achieve the reshaping of the classical neuron layer. The purpose of the Kronecker product operation is to obtain a matrix that can operate a multi-body quantum system. It is also possible to use multiple Kronecker product operations to obtain a matrix that acts on the input vector. This method can reduce the number of parameters in the matrix in an exponential manner, but this method easily leads to the problem of too few parameters. Finally, the present invention theoretically analyzes the principle of QiMLP mining strong correlations between features, and experimentally verifies it on multiple classic deep learning data sets. The experimental results show that QiMLP can not only learn the strong correlations between features, but also reduce the number of parameters by hundreds of times.

[0038] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the drawings; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0039] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0040] Embodiment: This embodiment is a quantum-like multi-layer perception method for mining strong correlations between features, such as Figure 1 As shown, the following steps are included: S1. Obtaining a weight matrix by Kronecker product operation, specifically including combining one or more small weight matrices using one or more Kronecker product operations to obtain a final weight matrix of the neuron layer.

[0041] S2. Obtaining a bias term by a Kronecker product operation, specifically comprising combining one or more small bias terms using one or more Kronecker product operations to obtain a final bias term of the neuron layer.

[0042] S3. Apply a weight matrix to the input vector of the quantum-like neuron layer and a bias term to construct the quantum-like neuron layer, specifically including multiplying the input vector of the quantum-like neuron layer by the final weight matrix and adding the final bias term to construct the quantum-like neuron layer.

[0043] S4. Apply an activation function to the output vector of the quantum-like neuron layer to obtain the final output vector of the quantum-like neuron layer. Apply an activation function to the input vector multiplied by the final weight matrix and added with the final bias term to obtain the output result of the neuron layer. Select a suitable activation function according to the needs of the construction task.

[0044] S5. Adjust the quantum-like neuron layer according to the specific task needs. Based on the constructed quantum-like neuron layer, a quantum-like multilayer perceptron is constructed. According to the vector of the input quantum-like multilayer perceptron, Set up the input layer, hidden layer, and output layer of the quantum-like multilayer perceptron. Based on the above steps, you can build a quantum-like multilayer perceptron of any number of layers and any size.

[0045] Specifically: (1) Use one or more Kronecker product operations to combine one or more small weight matrices to obtain the final weight matrix of the neuron layer: Assume that the input of the neuron layer is N M Dimensional vector That is, the number of neurons in the previous layer is N M , then the reconstructed weight matrix can be expressed as in To construct the small weight matrix used in the final weight matrix, the operator Refers to the Kronecker product operation; the number of Kronecker products used to construct the final weight matrix is ​​determined by the task for which the model is applicable and the number of parameters required for the task.

[0046] (2) Use one or more Kronecker product operations to combine one or more small bias terms to obtain the final bias term of the neuron layer: Assume that the final bias term is N M Dimensional vector Then the final bias term after reconstruction can be expressed as in A small bias term used to construct the final bias term; the number of Kronecker products used to construct the bias term corresponds to the number of Kronecker products used to construct the weight matrix.

[0047] (3) Apply an activation function to the input vector multiplied by the final weight matrix and the final bias term to obtain the output result of the neuron layer: Multiply the input vector X by the constructed final weight matrix W, add the constructed final bias term B, and apply an activation function f(·) to obtain the output Y of the neuron layer. in is the output of the neuron layer, the operator * represents the matrix multiplication operation, the subscript k represents the number of copies of the reconstructed weight matrix, which is used to alleviate the problem of over-compression of model parameters, and the activation function f(·) can select the appropriate activation function according to the needs of the construction task.

[0048] (4) Based on the quantum-like neuron layer constructed in the above three steps, a quantum-like multilayer perceptron is constructed: Assume that the input vector of the quantum-like multilayer perceptron is The input layer of the quantum-like multilayer perceptron is in is the output of the input layer. The activation function f(·) can be the commonly used RELU(·) or other. and B i is the weight matrix and bias term of the input layer; The hidden layer is in is the output of the input layer. The activation function f(·) can be the commonly used RELU(·) or other. and B i is the weight matrix and bias term of the input layer, and the hidden layer can be multi-layer; The output layer is in is the output of the input layer. The activation function f(·) can be the commonly used Softmax(·) or other, and B i is the weight matrix and bias term of the input layer; based on the above steps, a quantum-like multilayer perceptron of any number of layers and any size can be constructed.

[0049] This type of quantum multi-layer perceptron model uses the cross entropy loss function to train the model. During the model training process, the mainstream learning rate adaptive optimization algorithm Adam is used as the optimizer.

[0050] This section will organize experiments to test the performance of QiMLP in terms of model prediction accuracy, parameter compression, and strong correlation mining, specifically answering the following three questions: 1) Under similar structural design, does QiMLP outperform the baseline model in terms of prediction accuracy? 2)To what extent can QiMLP compress the model parameters? 3) Can QiMLP exploit the strong correlation between features to improve the prediction accuracy of the model? The following content will give in detail the construction details of the quantum-like neuron layer and the quantum-like multi-layer perceptron model based on the quantum-like neuron layer, as well as the analysis experiments implemented under the MNIST data set and the CIFAR-10 data set. It should be understood that the specific examples described here are only used to explain the present invention and are not intended to limit the present invention. The experiments were implemented under the MNIST data set and the CIFAR-10 data set, and the specific steps are as follows: Dataset: MINST and CIFAR-10 datasets are datasets widely used to test new models in the early days of deep learning. MINST is a handwritten digit image dataset, containing a training set of 60,000 images and a test set of 10,000 images. Each sample is a 28*28 pixel gray matter image from 0 to 9. CIFAR-10 is a real object image dataset, containing a training set of 50,000 images and a test set of 10,000 images. Each sample is a 32*32 pixel color image, divided into 10 categories, with 6,000 images per category.

[0051] The training set of the above dataset is divided into training set and development set in a ratio of 8:2, and its test set is directly used as the test set of this experiment. The measurement indicators of the experimental results are all evaluated by accuracy.

[0052] Model setup: In the MNIST dataset, the model structure of the control group is a classic multilayer perceptron MLP consisting of three layers of neurons, with 784 neurons in each layer. The activation function GELU is added to the first two layers, and the function Dropout is added to the third layer. The model structure of the experimental group only modifies the weight matrix and bias term of the control group model into the form of two-body (Two-body QiMLP), three-body (Three-body QiMLP), and four-body (Four-body QiMLP) quantum systems, and ensures that the dimensions of the weight matrix and bias term of the experimental group model are the same as those of the control group model. In the two-body quantum system, the weight matrix is ​​composed of 28*28 and 28*28 dimensional matrices; in the three-body quantum system, the weight matrix is ​​composed of 7*7, 7*7, and 16*16 dimensional matrices; in the four-body quantum system, the weight matrix is ​​composed of 4*4, 4*4, 7*7, and 7*7 dimensional matrices. When a composite matrix is ​​composed of multiple sub-matrices, the principle of selecting each sub-matrix is ​​to maximize all sub-matrices as much as possible. The construction method of the bias term of the two-body, three-body, and four-body quantum systems is similar to the construction method of the weight matrix. In addition, the experimental group model did not add the Dropout function in the third layer.

[0053] In the CIFAR-10 dataset, the model structure of the control group is a classic multi-layer perceptron MLP consisting of five neuron layers. The model structure of the experimental group is also designed with reference to the model structure of the control group, and the construction method is the same as the construction method of the experimental group model in the MNIST dataset. The activation function of the model in the CIFAR-10 dataset is consistent with that in the MNIST dataset.

[0054] Experimental accuracy and parameter compression results and analysis: In the MNIST dataset, the experimental results of the experimental group model (QiMLP for two-body, three-body, and four-body quantum systems) and the control group model (Classic MLP) are shown in the “Attached Figures”. Figure 1 , Figure 2 shown. Figure 2 is the accuracy of the model prediction results, Figure 3 is the number of model parameters.

[0055] In terms of the accuracy of the prediction results, the QiMLP based on the two-body quantum system is far superior to the control group model; the QiMLP based on the three-body quantum system is similar to the control group model; the QiMLP based on the four-body quantum system is not as good as the control group model. In terms of the number of model parameters, the QiMLP based on various quantum system configurations has far fewer parameters than the control group model. Combined with the number of model parameters, the accuracy of the QiMLP prediction results constructed for the three-body and four-body quantum systems was analyzed. The results show that the main reason for its low accuracy is that the number of model parameters is too small to learn better performance. From this conclusion, it can be inferred that QiMLP has a strong parameter compression capability, but in specific application scenarios, the information capacity of the model needs to be considered and the parameters cannot be over-compressed.

[0056] The subscript k in formula (3) represents the number of repetitions of the reconstruction weight matrix, which is used to alleviate the over-compression of the model parameters. The experimental results of different k values ​​are shown in the "Attachment to the Specification" Figure 4 , Figure 5 As shown in the figure. Overall, when k takes a larger value, the model parameters increase and the prediction results of the model also improve. However, the performance of QiMLP based on the four-body quantum system is average. The main reason is that the number of parameters is insufficient and the model parameters are over-compressed.

[0057] In the CIFAR-10 dataset, the experimental results of the experimental group model and the control group model are shown in the "Attached Figures" Figure 6 , Figure 7 The experimental results are similar to those of the MNIST dataset, indicating that QiMLP's parameter compression capability also performs well on larger datasets and the most complex tasks. The subscript k in formula (3) refers to the number of repetitions of the reconstructed weight matrix, which is used to alleviate the over-compression of the model parameters. The experimental results of different k values ​​are shown in the "Attached Figures in the Instruction Manual". Figure 8 , Fig. 9 As shown in the figure, when the k value is larger, the model parameters increase, but the prediction results of the model do not improve accordingly. The main reason is that the number of parameters is still insufficient, resulting in excessive compression of the model parameters.

[0058] Analysis of the model's strong correlation mining capabilities: This section will statistically analyze whether QiMLP can mine strong correlations between features during implementation. Since it is difficult to calculate mutual information, that is, the total correlation of a multi-body quantum system, this section only analyzes QiMLP based on a two-body quantum system. Train a neural network to obtain the output vector ρ of each neuron layer of QiMLP AB Kronecker product form let As much as possible with ρ AB Among them, ρ A, B and ρ AB In calculation, they are all represented by vector matrices. After obtaining the subsystems based on each neuron layer of the two-body quantum system QiMLP, the strong correlation can be calculated with the help of the strong correlation measurement formula C(ρ AB )=I(ρ AB )-S(ρ (A+B) )=S(ρ A )+S(ρ B )-S(ρ AB )-S(ρ (A+B) ) Where I(·) is the von Neumann mutual information calculation formula, and S(·) is the von Neumann entropy calculation formula.

[0059] The strong correlation statistical results of each neural layer of QiMLP on the MNIST test set are shown in the "Instruction Figure" Fig.10 , Fig.11 The statistical results of the strong correlation between each neural layer of QiMLP and the CIFAR-10 test set are shown in the "Attached Figures of the Instruction Manual" Fig.12 , Fig.13 , Fig.14 , Fig.15 As shown in the figure, since the last layer is the output layer, the dimension is determined by the prediction task and is not suitable for statistically analyzing its strong correlation.

[0060] contrast Fig.10 The input layer neurons and Fig.11 From the statistical results of strong correlations among neurons in the hidden layer, it can be seen that the hidden layer mines more strong correlations than the input layer. This shows that QiMLP can mine strong correlations between features, and as the number of hidden layers in the model increases, the strong correlations mined tend to increase gradually. This phenomenon also appears in QiMLP under the CIFAR-10 dataset. The above experimental results show that the core reason why QiMLP can still achieve good prediction results when the model parameters are greatly compressed is that it mines strong correlations between features, and the mined strong correlations can directly determine the accuracy of the model prediction results.

[0061] Inspired by quantum multi-body systems, the present invention constructs a nonlinear neural network layer based on multi-body quantum systems that can mine strong correlations between features, and then constructs a quantum-like multi-layer perceptron (Quantum-inspired MLP, QiMLP) based on the constructed neural network layer. QiMLP will have important application value and inspirational significance for reshaping machine learning, deep learning and large language models. Finally, the present invention theoretically analyzes the principle and basis of QiMLP mining strong correlations between features, and experimentally verifies it on multiple classic deep learning data sets. The experimental results show that QiMLP can not only learn the strong correlations between features, but also significantly reduce the number of model parameters. Specifically, the method for constructing a quantum-like neural network layer is as follows: 1) using one or more Kronecker product operations to obtain a weight matrix acting on the input vector of the neuron layer; 2) using one or more Kronecker product operations to obtain a bias term acting on the input vector of the neuron layer; 3) using an activation function to act on the result of multiplying the neuron layer input vector by the weight matrix and adding the bias term to obtain the output of the quantum-like neural network layer. Based on the reshaped quantum-like neural network layer, a corresponding quantum-like multi-layer perceptron can be constructed according to the needs of specific tasks.

[0062] The contributions of the present invention can be summarized as follows: 1) Based on a multi-body quantum system, a neuron layer is constructed that can mine strong correlations between features; 2) Based on the constructed neuron layer, a quantum-like multilayer perceptron with practical application value is constructed, which reduces the number of model parameters by hundreds of times; 3) Finally, the principle of the constructed neuron layer for mining strong correlations is analyzed theoretically, and the effectiveness of the constructed multilayer perceptron is verified through experiments.

[0063] The specific embodiments described herein are merely examples of the spirit of the present invention. The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that a technician in the technical field to which the present application belongs may make various modifications or supplements to the described specific embodiments or replace them in a similar manner, but will not deviate from the spirit of the present application or exceed the scope defined in the attached claims. For a person of ordinary skill in the art, multiple variations and improvements may also be made without departing from the concept of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A quantum-like multi-layer perception method for mining strong correlations between features, characterized in that: The following steps are involved: S1. The weight matrix acting on the input vector of the quantum-like neuron layer is obtained by Kronecker product operation; S2. The bias term acting on the input vector of the quantum-like neuron layer is obtained by Kronecker product operation; S3. Apply a weight matrix to the input vector of the quantum-like neuron layer and a bias term to construct the quantum-like neuron layer; S4. Applying an activation function to the output vector of the quantum-like neuron layer to obtain the final output vector of the quantum-like neuron layer; S5. Adjust the quantum-like neuron layer according to specific task requirements.

2. According to claim 1, a quantum-like multi-layer perception method for mining strong correlations between features is characterized in that: The S1 specifically includes combining one or more small weight matrices using one or more Kronecker product operations to obtain a final weight matrix of the neuron layer.

3. According to claim 2, a quantum-like multi-layer perception method for mining strong correlations between features is characterized in that: The input of the neuron layer is N M dimensional vector, that is, the number of neurons in the previous layer is N M , reconstruct the weight matrix, and the number of Kronecker products used to construct the final weight matrix is ​​determined by the tasks to which the model is applicable and the number of parameters required for the tasks.

4. According to claim 1, a quantum-like multi-layer perception method for mining strong correlations between features is characterized in that: The S2 specifically includes combining one or more small bias terms using one or more Kronecker product operations to obtain a final bias term of the neuron layer.

5. A quantum-like multi-layer sensing method for mining strong correlations between features according to claim 4, characterized in that: According to the final bias term is N M dimensional vector, representing the final bias term after reconstruction. The number of Kronecker products used to construct the bias term corresponds to the number of Kronecker products used to construct the weight matrix.

6. A quantum-like multi-layer sensing method for mining strong correlations between features according to claim 1, 3 or 5, characterized in that: The S3 specifically includes multiplying the quantum-like neuron layer input vector by the final weight matrix and adding the final bias term to construct the quantum-like neuron layer.

7. A quantum-like multi-layer sensing method for mining strong correlations between features according to claim 1, 3 or 5, characterized in that: The S4 specifically includes applying an activation function to the input vector multiplied by the final weight matrix and added with the final bias term to obtain the output result of the neuron layer, and selecting a suitable activation function according to the needs of the construction task.

8. A quantum-like multi-layer perception method for mining strong correlations between features according to claim 7, characterized in that: The subscript k is set for the final constructed weight matrix to indicate the number of copies of the reconstructed weight matrix, alleviating the problem of over-compression of model parameters.

9. A quantum-like multi-layer sensing method for mining strong correlations between features according to claim 1, characterized in that: The S5 specifically includes constructing a quasi-quantum multilayer perceptron based on the constructed quasi-quantum neuron layer, and according to the vector of the input quasi-quantum multilayer perceptron being Set up the input layer of the quantum-like multilayer perceptron.

10. A quantum-like multi-layer sensing method for mining strong correlations between features according to claim 9, characterized in that: According to the input vector of quantum multilayer perceptron, Set up the hidden layer and output layer of the quantum multilayer perceptron, using and B i represents the weight matrix and bias term of the input layer, and the hidden layer is a number of layers.

Images