Neural network system combining feedforward acceleration and feedback thinking, inference method and storage medium
By introducing feedforward acceleration and feedback thinking mechanisms into neural networks, the multi-layer network layers of the hidden layer are transmitted and calculated in sequence to realize jump and feedback transmission, the problem of redundant calculation of simple inputs and low accuracy of complex inputs in existing neural networks is solved, and the calculation accuracy and application effect of neural networks are improved.
Patent Information
- Application Number
- CN202510098938.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing neural networks have redundant calculation problems for simple input matrices, and the accuracy of inference results for complex input matrices is low.
A neural network system combining feedforward acceleration and feedback thinking is proposed. It passes backwards in sequence through the N-layer network layer of the hidden layer, and calculates the difference value between the input conversion vector and the output conversion vector to realize jump and feedback transmission, so as to omit the intermediate calculation process and improve the information conversion effect.
It effectively reduces the redundant calculation of simple input matrix, improves the processing ability of complex input matrix, and significantly improves the accuracy and application effect of the inference results of neural network systems.
Smart Images
Figure CN119538978A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network technology, and in particular to a neural network system, a reasoning method and a storage medium combining feedforward acceleration and feedback thinking. Background Art
[0002] In the field of deep learning, neural networks have attracted widespread attention due to their powerful data processing capabilities. These neural networks are usually designed with a serial structure. The existing serial structure neural networks have an input layer, a hidden layer, and an output layer (decoding layer). Information is gradually extracted and converted through the input layer and multiple hidden layers, and finally the inference (prediction) results are output through the output layer (decoding layer).
[0003] However, the existing serial structure of the neural network has certain limitations. All input matrices will undergo completely fixed logical calculations. For simple input matrices, the neural network may have a better "understanding" and use the front part of the hidden layer to perform more accurate information conversion. For complex input matrices, the fixed calculation process may not be able to fully convert information, resulting in a low accuracy rate of the output reasoning (prediction) results. Summary of the invention
[0004] The present invention proposes a neural network system, reasoning method and storage medium that combine feedforward acceleration and feedback thinking to solve the technical problems of redundant calculation of simple input matrices in existing neural networks and low accuracy of reasoning (prediction) results for complex input matrices.
[0005] One aspect of the present invention is to provide a neural network system combining feedforward acceleration and feedback thinking, the neural network system comprising at least: an input layer, a hidden layer and an output layer; The input layer is used to input the original matrix and process the original matrix into the input matrix of the hidden layer; The hidden layer includes N network layers, which are used to sequentially pass the input matrix of the hidden layer backward through the N network layers, where N≥5; When the input matrix is transmitted to the k-th network layer through the hidden layer, the input matrix of the k-th network layer is compressed and converted into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed and converted into the output conversion vector of the k-th network layer, N-2≥k>2; Calculate the difference between the input conversion vector and the output conversion vector of the k-th network layer; When the difference between the input conversion vector and the output conversion vector of the k-th network layer is less than the first threshold T1, the output matrix of the k-th network layer jumps to the last network layer and is used as the input matrix of the last network layer and input into the last network layer; otherwise, the output matrix of the k-th network layer is sequentially passed to the next network layer; The input matrix of the last network layer is compressed and converted into the input conversion vector of the last network layer, and the output matrix of the last network layer is compressed and converted into the output conversion vector of the last network layer; Calculate the difference between the input transformation vector and the output transformation vector of the last network layer; When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold T2; The output matrix of the last network layer is then triggered to be forwarded to the NLth network layer, and then it is further forwarded, and the above process is repeated, N>L≥1; The output layer is used to input the output matrix of the last network layer, and output the output matrix of the last network layer as an inference matrix.
[0006] In a preferred embodiment, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer by a compression averaging method; And, the input matrix of the last network layer is compressed into the input transformation vector of the last network layer, and the output matrix of the last network layer is compressed into the output transformation vector of the last network layer.
[0007] In a preferred embodiment, the difference between the input conversion vector and the output conversion vector of the kth network layer is calculated by a cosine similarity algorithm; And, the difference between the input transformation vector and the output transformation vector of the last network layer.
[0008] In a preferred embodiment, when the number of times the output matrix of the last network layer is forwarded to the NLth network layer reaches the triggering upper limit; The difference between the input conversion vector and the output conversion vector of the last network layer is no longer calculated.
[0009] Another aspect of the present invention is to provide a neural network reasoning method, the reasoning method comprising the following method steps: S1, the input layer inputs the original matrix, processes the original matrix into the input matrix of the hidden layer, and transmits it to the hidden layer; S2, the hidden layer transmits the input matrix of the hidden layer backward through the N layers of the network layer of the hidden layer in sequence, where N≥5; S21, when the input matrix is transmitted to the k-th network layer through the hidden layer, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer, N-2≥k>2; S22, calculating the difference between the input conversion vector and the output conversion vector of the k-th network layer; When the difference between the input conversion vector and the output conversion vector of the k-th network layer is less than the first threshold T1, the output matrix of the k-th network layer jumps to the last network layer and is used as the input matrix of the last network layer and input into the last network layer; otherwise, the output matrix of the k-th network layer is sequentially passed to the next network layer; S23, compressing the input matrix of the last network layer into the input conversion vector of the last network layer, and compressing the output matrix of the last network layer into the output conversion vector of the last network layer; S24, calculating the difference between the input conversion vector and the output conversion vector of the last network layer; When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold T2; The output matrix of the last network layer is then triggered to be forwarded to the NLth network layer, and then continued to be backwards, repeating steps S21 to S24, N>L≥1; S3. The output matrix of the last network layer is input into the output layer, and the output matrix of the last network layer is output as the inference matrix.
[0010] In a preferred embodiment, in step S21, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer by a compression averaging method; In step S23, the input matrix of the last network layer is compressed into the input conversion vector of the last network layer, and the output matrix of the last network layer is compressed into the output conversion vector of the last network layer by the compression averaging method.
[0011] In a preferred embodiment, in step S22, Calculate the difference between the input conversion vector and the output conversion vector of the kth network layer by using the cosine similarity algorithm; In step S24, the difference between the input conversion vector and the output conversion vector of the last network layer is calculated by using the cosine similarity algorithm.
[0012] In a preferred embodiment, when the number of times the output matrix of the last network layer is forwarded to the NLth network layer reaches the triggering upper limit; The difference between the input conversion vector and the output conversion vector of the last network layer is no longer calculated.
[0013] Another aspect of the present invention is to provide a computer storage medium, wherein the computer storage medium is used to store computer execution instructions; the computer execution instructions are used to execute a neural network reasoning method provided by the present invention.
[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a neural network system, reasoning method and storage medium that combines feedforward acceleration and feedback thinking. The input matrix is input into the hidden layer. In the process of being transmitted backward through the N layers of the hidden layer, by calculating whether the difference between the input conversion vector and the output conversion vector of the kth network layer is less than the first threshold, the output matrix of the kth network layer jumps to the last network layer, and the feedforward acceleration is transmitted backward, omitting the calculation process of multiple intermediate network layers, saving calculation time, and solving the problem of redundant calculation of simple input in the existing neural network. By calculating whether the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold, the information conversion effect of the last network layer of the hidden layer is evaluated. When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold, the output matrix of the last network layer is triggered to be transmitted forward to the NLth network layer, and the output matrix of the last network layer is input into the NLth network layer and continues to be transmitted backward, thereby realizing a more sufficient data processing capability of the hidden layer, improving the processing capability of the neural network system for complex data, and significantly improving the accuracy and application effect of the reasoning (prediction) results of the neural network system without increasing the number of parameters.
[0015] The present invention proposes a neural network system, reasoning method and storage medium that combines feedforward acceleration and feedback thinking, which can simulate the feedback thinking mechanism of human brain nerves. Without adding new network parameters, it performs more sufficient logical calculations on the input data, thereby simulating the effect of "deliberate thinking" of the human brain, thereby improving the calculation accuracy and application effect of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the following briefly introduces the drawings required for use in the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0017] Figure 1 It is a structural schematic diagram of a neural network system combining feedforward acceleration and feedback thinking according to the present invention. DETAILED DESCRIPTION
[0018] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.
[0019] Combination Figure 1 According to an embodiment of the present invention, a neural network system combining feedforward acceleration and feedback thinking is provided, including: an input layer 100, a hidden layer 200 and an output layer 300.
[0020] The hidden layer 200 includes N network layers, and each network layer is configured with a network layer weight. In this embodiment, the N network layers are respectively the first network layer 201, the second network layer 202, ..., the kth network layer 20k, ..., the 20N-2th network layer 20N-2, the 20N-1th network layer 20N-1, and the Nth network layer 20N (the last network layer).
[0021] The first network layer 201 is configured with a first network layer weight W1, the second network layer 202 is configured with a second network layer weight W2, ..., the kth network layer 20k is configured with a kth network layer weight W k , ..., 20N-2 network layer configuration 20N-2 network layer weight W N-2 、Configure the 20N-1th network layer and the 20N-1th network layer weight W N-1 , the Nth network layer 20N configures the Nth network layer 20N weight W N .
[0022] In the prior art, during the neural network training and reasoning (prediction) process, after the original matrix X is input to the input layer 100, the input matrix of the hidden layer 200 is output. The input matrix of the hidden layer 200 is sequentially passed backward through the N layers of the hidden layer 200, and finally the reasoning matrix Y is output by the output layer 300 as the reasoning (prediction) result.
[0023] During the neural network training and inference (prediction) process, since the input matrix of the network layer (e.g., the first network layer 201) located at the front end of the hidden layer 200 is closer to the original matrix X, and the network layer weight (e.g., the first network layer weight W1) configured by the network layer (e.g., the first network layer 201) located at the front end of the hidden layer 200 is larger, the difference between the input matrix and the output matrix of the network layer (e.g., the first network layer 201) located at the front end of the hidden layer 200 is relatively large.
[0024] During the neural network training and reasoning (prediction) process, since the input matrix of the network layer located at the back end of the hidden layer 200 (e.g., the Nth network layer 20N, the last network layer) has been transformed by multiple network layers (e.g., through the 1st network layer 201 to the N-1th network layer), and the network layer weights (e.g., the Nth network layer 20N weight W) configured by the network layer located at the back end of the hidden layer 200 (e.g., the Nth network layer 20N, the last network layer) are N ) is small, so the difference between the input matrix and the output matrix of the network layer located at the back of the hidden layer 200 (for example, the Nth network layer 20N, the last network layer) is small.
[0025] After the neural network is fully trained, the network weights configured in each network layer of the hidden layer 200 are usually stable, so that the fluctuation of the hidden layer 200 is stable. When the simple original matrix X is inferred (predicted), the neural network may have a better "understanding", and the inference matrix Y output by the output layer 300 has a higher accuracy. However, for the complex original matrix X, the hidden layer 200 may not be able to fully convert it, and the difference between the input matrix and the output matrix of the network layer at the back end of the hidden layer 200 (for example, the Nth network layer 20N, the last network layer) is still large, resulting in the situation that the inference matrix Y output by the output layer 300 has a lower accuracy.
[0026] In order to solve the above problems, Figure 1 As shown, the present invention provides a neural network system combining feedforward acceleration and feedback thinking. The input layer 100 is used to input the original matrix X and process the original matrix X into the input matrix X' of the hidden layer 200.
[0027] The hidden layer 200 includes N network layers, each of which is configured with a network layer weight. In this embodiment, the N network layers are respectively the first network layer 201, the second network layer 202, ..., the Nth network layer 20N (the last network layer).
[0028] The first network layer 201 is configured with a first network layer weight W1, the second network layer 202 is configured with a second network layer weight W2, ..., the Nth network layer 20N is configured with an Nth network layer 20N weight WN .
[0029] The hidden layer 200 is used to sequentially transmit the input matrix X' of the hidden layer 200 backward through the N network layers (the first network layer 201, the second network layer 202, ..., the Nth network layer 20N) of the hidden layer 200, where N≥5.
[0030] Specifically, the input matrix X' of the hidden layer 200 is used as the input matrix of the first network layer 201, and is input into the first network layer 201. After being converted and processed by the first network layer 201, the output matrix X1=W1×X' of the first network layer 201 is output.
[0031] The output matrix X1 of the first network layer 201 is used as the input matrix of the second network layer 202, and is input into the second network layer 202. After being converted and processed by the second network layer 202, the output matrix X2=W2×X1 of the second network layer 202 is output;…
[0032] By analogy, the output matrix X of the N-1th network layer is N-1 As the input matrix of the last network layer 20N, the Nth network layer 20N is input, and after the conversion processing of the Nth network layer 20N, the output matrix X of the Nth network layer 20N is output. N =W N ×X N-1 . …….
[0033] In the process of the input matrix X' of the hidden layer 200 of the present invention being sequentially transmitted backward through the N network layers (the first network layer 201, the second network layer 202, ..., the Nth network layer 20N) of the hidden layer 200, when the input matrix X' of the hidden layer 200 is transmitted to the kth network layer 20k through the hidden layer 200, the input matrix X' of the kth network layer 20k is k-1 Compressed to the input transformation vector H of the k-th network layer k-1 m , the output matrix X of the k-th network layer k Compressed to the output transformation vector H of the k-th network layer k m , where N-2≥k>2, that is, k=3, 4, 5,…, N-2.
[0034] The superscript m represents the output matrix X of the last network layer (the Nth network layer 20N) N The number of times the information is passed forward to the NLth network layer is m=0, 1, 2, 3, ..., which will be described in detail below.
[0035] The present invention uses a compression average method to convert the input matrix X of the kth network layer 20k intok-1 Compressed to the input transformation vector H of the k-th network layer k-1 m , the output matrix X of the k-th network layer k Compressed to the output transformation vector H of the k-th network layer k m .
[0036] For example, in this embodiment, the input matrix X of the kth network layer 20k is k-1 For X k-1 =[[1, 2], [3, 4]], the two-dimensional matrix is compressed into a one-dimensional vector [1.5, 3.5] by the compression average method. The compressed one-dimensional vector [1.5, 3.5] is used as the input transformation vector H of the k-th network layer k-1 m =[1.5, 3.5].
[0037] Calculate the input transformation vector H of the kth network layer k-1 m With the output conversion vector H k m The difference between .
[0038] The present invention calculates the input conversion vector H of the kth network layer by using the cosine similarity algorithm. k-1 m With the output conversion vector H k m The difference between .
[0039] Specifically, the input conversion vector H of the k-th network layer of the present invention is k-1 m With the output conversion vector H k m The difference between them is calculated as follows: score k = cosine(H k m , H k-1 m ); Among them, score k represents the difference between the input conversion vector and the output conversion vector of the kth network layer, H k-1 m represents the input transformation vector of the kth network layer, H k m Represents the output transformation vector of the k-th network layer.
[0040] When the input transformation vector H of the kth network layer k-1 mWith the output conversion vector H k m The difference between the score k When it is greater than or equal to the first threshold T1, the output matrix X of the kth network layer k It is passed sequentially to the next network layer until the last network layer (the Nth network layer 20N).
[0041] When the input transformation vector H of the kth network layer k-1 m With the output conversion vector H k m The difference between the score k When it is less than the first threshold T1, the output matrix X of the kth network layer k Jump to the last network layer (Nth network layer 20N), as the input matrix X of the last network layer (Nth network layer 20N) N-1 , input the last network layer (the Nth network layer 20N), thereby realizing feedforward acceleration and backward transmission. Otherwise, the output matrix X of the kth network layer k Passed to the last network layer in sequence.
[0042] The parameter value range of the first threshold value T1 of the present invention is: 0.4≥T1>0, and those skilled in the art can set it according to actual conditions.
[0043] Preferably, in this embodiment, the input conversion vector H of the kth network layer is k-1 m With the output conversion vector H k m The difference between the score k The first threshold T1 is set to 0.1.
[0044] When the input transformation vector H of the kth network layer k-1 m With the output conversion vector H k m The difference between the score k is less than 0.1, then the output matrix X of the k-th network layer k Jump to the last network layer (network layer N 20N).
[0045] It should be noted that the output matrix X of the kth network layer k In the process of being transmitted backward through Nk network layers in sequence to the last network layer (Nth network layer 20N), the above process is repeated to determine whether the difference between the input conversion vector and the output conversion vector of the current network layer is less than the first threshold T1.
[0046] The input matrix X of the last network layer (the Nth network layer 20N) N-1 Compressed into the input transformation vector H of the last network layer N-1 m , the output matrix X of the last network layer (the Nth network layer 20N) N Compressed to the output transformation vector H of the last network layer N m .
[0047] The present invention uses a compression averaging method to convert the input matrix X of the last network layer into N-1 Compressed into the input transformation vector H of the last network layer N-1 m , compress the output matrix of the last network layer into X N The output transformation vector H of the last network layer is reduced to N m .
[0048] Calculate the input transformation vector H of the last network layer N-1 m With the output conversion vector H N m The difference between .
[0049] The present invention calculates the input conversion vector H of the last network layer by using the cosine similarity algorithm. N-1 m With the output conversion vector H N m The difference between .
[0050] Specifically, the input transformation vector H of the last network layer (the Nth network layer 20N) is N-1 m With the output conversion vector H N m The difference between them is calculated as follows: score N = cosine(H N m , H N-1 m ); Among them, score N Represents the input transformation vector H of the last network layer (layer N 20N) N-1 m With the output conversion vector H N m The difference between N-1 m Represents the input transformation vector of the last network layer (layer N 20N), HN m represents the output conversion vector of the last network layer, and m represents the output matrix X of the last network layer (the Nth network layer 20N) N The number of times the data is passed forward to the NLth network layer, m=0, 1, 2, 3, …
[0051] When the input transformation vector H of the last network layer (the Nth network layer 20N) N-1 m With the output conversion vector H N m The difference between the score N When it is greater than the second threshold value T2; Then the output matrix X of the last network layer (the Nth network layer 20N) is triggered N Pass it forward to the NLth network layer and convert the output matrix X of the last network layer (Nth network layer 20N) N Input the NLth network layer and output the output matrix X of the NLth network layer N-L , and then continue to pass backward, repeat the above process iteratively, N>L≥1; m represents the output matrix X of the last network layer (the Nth network layer 20N) N The number of times the data is passed forward to the NLth network layer, m=0, 1, 2, 3, …
[0052] The parameter value range of the second threshold value T2 of the present invention is: 1>T2≥0.6, and those skilled in the art can set it according to actual conditions.
[0053] Preferably, in this embodiment, the input conversion vector H of the last network layer (the Nth network layer 20N) is N-1 m With the output conversion vector H N m The difference between the score N The second threshold is set to 0.8.
[0054] When the input transformation vector H of the last network layer (the Nth network layer 20N) N-1 m With the output conversion vector H N m The difference between the score N If it is greater than 0.8, the output matrix X of the last network layer (the Nth network layer 20N) is triggered. N Pass forward to the NLth network layer.
[0055] The output matrix X of the last network layer (the Nth network layer 20N) is NWhen the number of times m is forwarded to the NLth network layer reaches the trigger upper limit M (i.e. m=M); Then the input conversion vector H of the last network layer is no longer calculated N-1 m With the output conversion vector H N m The difference between the score N To avoid falling into an infinite loop, the output matrix X of the last network layer 20N is N As the output matrix of the hidden layer 200. Preferably, the trigger upper limit M is set to 5.
[0056] The output layer 300 is used to input the output matrix X of the last network layer N (output matrix of hidden layer 200), and the output matrix X of the last network layer N The output is the inference matrix Y.
[0057] The input matrix X' of the hidden layer 200 of the present invention is input to the hidden layer 200, and in the process of being sequentially transmitted backward through the N layers of the hidden layer 200, the output matrix X of the kth network layer is realized by calculating whether the difference between the input conversion vector and the output conversion vector of the kth network layer is less than the first threshold value. k Jump to the last network layer, feedforward accelerate the backward transmission, omit the calculation process of multiple intermediate network layers, save calculation time, and solve the problem of redundant calculation of existing neural networks for simple inputs.
[0058] The present invention calculates the input conversion vector H of the last network layer N-1 m With the output conversion vector H N m The difference between the score N Is it greater than the second threshold, the information conversion effect of the last network layer of the hidden layer 200 is evaluated. When the input conversion vector H of the last network layer N-1 m With the output conversion vector H N m The difference between the score N When it is greater than the second threshold, the output matrix X of the last network layer is triggered. N Pass it forward to the NLth network layer, and the output matrix X of the last network layer N The input of the NLth network layer continues to be passed backward, thereby realizing a more sufficient data processing capability of the hidden layer 200, improving the processing capability of the neural network system for complex data, and significantly improving the accuracy and application effect of the reasoning (prediction) results of the neural network system without increasing the number of parameters.
[0059] According to an embodiment of the present invention, a neural network reasoning method is provided, comprising the following method steps: Step S1 : The input layer 100 inputs an original matrix X, processes the original matrix X into an input matrix X′ of the hidden layer 200 , and transmits the matrix to the hidden layer 200 .
[0060] Step S2: The hidden layer 200 transmits the input matrix X' of the hidden layer 200 backward in sequence through the N network layers of the hidden layer 200, where N≥5.
[0061] Step S21, when the input matrix X' of the hidden layer 200 is transmitted to the k-th network layer 20k through the hidden layer 200, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer, N-2≥k>2.
[0062] The present invention compresses the input matrix of the k-th network layer into the input conversion vector of the k-th network layer, and compresses the output matrix of the k-th network layer into the output conversion vector of the k-th network layer by a compression averaging method.
[0063] Step S22: Calculate the difference between the input conversion vector and the output conversion vector of the kth network layer.
[0064] The present invention calculates the difference value between the input conversion vector and the output conversion vector of the kth network layer through the cosine similarity algorithm.
[0065] Specifically, the difference between the input conversion vector and the output conversion vector of the kth network layer is calculated as follows: score k = cosine(H k m , H k-1 m ); Among them, score k represents the difference between the input conversion vector and the output conversion vector of the kth network layer, H k-1 m represents the input transformation vector of the kth network layer, H k m Represents the output transformation vector of the k-th network layer.
[0066] When the difference between the input transformation vector and the output transformation vector of the kth network layer is k When it is greater than or equal to the first threshold T1, the output matrix of the kth network layer is sequentially transmitted to the next network layer until the last network layer (the Nth network layer 20N).
[0067] When the difference between the input conversion vector and the output conversion vector of the kth network layer is less than the first threshold T1, the output matrix of the kth network layer jumps to the last network layer and is used as the input matrix of the last network layer. Otherwise, the output matrix of the kth network layer is sequentially passed to the next network layer.
[0068] Step S23: compress the input matrix of the last network layer into the input conversion vector of the last network layer, and compress the output matrix of the last network layer into the output conversion vector of the last network layer.
[0069] The present invention compresses the input matrix of the last network layer into the input conversion vector of the last network layer, and compresses the output matrix of the last network layer into the output conversion vector of the last network layer through a compression and averaging method.
[0070] Step S24: Calculate the difference between the input conversion vector and the output conversion vector of the last network layer.
[0071] The present invention calculates the difference value between the input conversion vector and the output conversion vector of the last network layer through the cosine similarity algorithm.
[0072] Specifically, the difference between the input transformation vector and the output transformation vector of the last network layer is calculated as follows: score N = cosine(H N m , H N-1 m ); Among them, score N Represents the difference between the input conversion vector and the output conversion vector of the last network layer, H N-1 m represents the input transformation vector of the last network layer, H N m represents the output conversion vector of the last network layer, m represents the number of times the output matrix of the last network layer is passed forward to the NLth network layer, m=0, 1, 2, 3, ….
[0073] When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold T2; The output matrix of the last network layer is triggered to be forwarded to the NLth network layer, and then continued to be passed backward, repeating steps S21 to S24, repeating the above process, N>L≥1.
[0074] In the present invention, when the number of times the output matrix of the last network layer is forwarded to the NLth network layer reaches the triggering upper limit; The difference between the input conversion vector and the output conversion vector of the last network layer is no longer calculated.
[0075] Step S3: the output matrix of the last network layer is input into the output layer 300, and the output matrix of the last network layer is output as an inference matrix.
[0076] According to an embodiment of the present invention, a computer storage medium is provided for storing computer-executable instructions. The computer-executable instructions are used to execute a neural network reasoning method provided by the present invention.
[0077] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A neural network system combining feedforward acceleration and feedback thinking, characterized in that: The neural network system comprises: an input layer, a hidden layer and an output layer; The input layer is used to input the original matrix and process it into the input matrix of the hidden layer; The hidden layer includes N network layers, which are used to sequentially pass the input matrix of the hidden layer backward through the N network layers, where N≥5; When the input matrix is transmitted to the k-th network layer through the hidden layer, the input matrix of the k-th network layer is compressed and converted into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed and converted into the output conversion vector of the k-th network layer, N-2≥k>2; When the difference between the input conversion vector and the output conversion vector of the k-th network layer is less than the first threshold T1, the output matrix of the k-th network layer jumps to the last network layer; otherwise, the output matrix of the k-th network layer is sequentially passed to the next network layer; The input matrix of the last network layer is compressed and converted into the input conversion vector of the last network layer, and the output matrix of the last network layer is compressed and converted into the output conversion vector of the last network layer; When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold T2; The output matrix of the last network layer is then triggered to be forwarded to the NLth network layer, and then it is further forwarded, and the above process is repeated, N>L≥1; The output layer is used to input the output matrix of the last network layer, and output the output matrix of the last network layer as an inference matrix.
2. The neural network system according to claim 1, characterized in that By means of compression averaging, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer; And, the input matrix of the last network layer is compressed into the input transformation vector of the last network layer, and the output matrix of the last network layer is compressed into the output transformation vector of the last network layer.
3. The neural network system according to claim 1, characterized in that Calculate the difference between the input conversion vector and the output conversion vector of the kth network layer by using the cosine similarity algorithm; And, the difference between the input transformation vector and the output transformation vector of the last network layer.
4. The neural network system according to claim 1, characterized in that When the number of times the output matrix of the last network layer is forwarded to the NLth network layer reaches the trigger upper limit; The difference between the input conversion vector and the output conversion vector of the last network layer is no longer calculated.
5. A neural network reasoning method, characterized in that: The reasoning method comprises the following method steps: S1, the input layer inputs the original matrix, processes the original matrix into the input matrix of the hidden layer, and transmits it to the hidden layer; S2, the hidden layer transmits the input matrix of the hidden layer backward through the N network layers of the hidden layer in sequence, where N≥5; S21, when the input matrix is transmitted to the k-th network layer through the hidden layer, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer, N-2≥k>2; S22, calculating the difference between the input conversion vector and the output conversion vector of the k-th network layer; When the difference between the input conversion vector and the output conversion vector of the k-th network layer is less than the first threshold T1, the output matrix of the k-th network layer jumps to the last network layer and is used as the input matrix of the last network layer and input into the last network layer; otherwise, the output matrix of the k-th network layer is sequentially passed to the next network layer; S23, compressing the input matrix of the last network layer into the input conversion vector of the last network layer, and compressing the output matrix of the last network layer into the output conversion vector of the last network layer; S24, calculating the difference between the input conversion vector and the output conversion vector of the last network layer; When the difference between the input conversion vector and the output conversion vector of the last network layer is greater than the second threshold T2; The output matrix of the last network layer is then triggered to be forwarded to the NLth network layer, and then continued to be backwards, repeating steps S21 to S24, N>L≥1; S3. The output matrix of the last network layer is input into the output layer, and the output matrix of the last network layer is output as the inference matrix.
6. The inference method according to claim 5, characterized in that: In step S21, the input matrix of the k-th network layer is compressed into the input conversion vector of the k-th network layer, and the output matrix of the k-th network layer is compressed into the output conversion vector of the k-th network layer by a compression averaging method; In step S23, the input matrix of the last network layer is compressed into the input conversion vector of the last network layer, and the output matrix of the last network layer is compressed into the output conversion vector of the last network layer by the compression averaging method.
7. The inference method according to claim 5, characterized in that: In step S22, Calculate the difference between the input conversion vector and the output conversion vector of the kth network layer by using the cosine similarity algorithm; In step S24, the difference between the input conversion vector and the output conversion vector of the last network layer is calculated by using the cosine similarity algorithm.
8. The inference method according to claim 5, characterized in that: When the number of times the output matrix of the last network layer is forwarded to the NLth network layer reaches the trigger upper limit; The difference between the input conversion vector and the output conversion vector of the last network layer is no longer calculated.
9. A computer storage medium, characterized in that The computer storage medium is used to store computer-executable instructions; the computer-executable instructions are used to execute the reasoning method described in any one of claims 6 to 8.