Lightweight neural network fault diagnosis method and system based on tensor manifold

Abstract

A light-weight neural network fault diagnosis method and system based on tensor manifold, the method includes treating the fault diagnosis image as a vector, reconstructing it into a first multi-order tensor under constraints, and representing the first multi-order tensor in the form of tensor string decomposition; under the condition that the tensor rank remains unchanged, the first multi-order tensor is iterated multiple times by the gradient descent method of the tensor manifold to obtain the first contracted multi-order tensor after data dimension reduction; reconstructing the weight matrix in the fault diagnosis neural network into a second multi-order tensor under constraints, and representing the second multi-order tensor in the form of tensor string decomposition to obtain a new neural network with factor tensors as weights; training the new neural network with the first contracted multi-order tensor as input data; obtaining a new fault diagnosis image and converting it into the first contracted multi-order tensor after data dimension reduction, and inputting the trained neural network to obtain the corresponding fault diagnosis result. The present invention can enhance the computing parallel capability.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision intelligent fault diagnosis, and in particular relates to a lightweight neural network fault diagnosis method and system based on tensor manifold. Background Art

[0002] In cutting-edge equipment such as aircraft engines, large aircraft, and nuclear power, the internal systems are extremely complex and mutually influential. The safe and stable operation of the equipment is of great significance. Timely repair of hidden faults is an important means to ensure the normal operation of complex equipment. Traditional maintenance methods have problems such as low work efficiency, scattered detection quality, single means, and difficulty in ensuring timeliness. The vision-based intelligent fault diagnosis solution can accurately judge the operating status of the equipment through image acquisition equipment and pattern recognition algorithms, and promptly feedback equipment abnormalities, which has practical and urgent significance for efficient fault diagnosis of complex equipment.

[0003] The visual system is the core part of intelligent fault diagnosis equipment. The visual system uses computers to simulate human visual functions, extract information from images or image sequences, and identify the motion and shape of objects to replace the human eye to make measurements and judgments. Compared with traditional manual detection, machine vision has the advantages of fast speed, high accuracy, long stable working time, large spectral response range, stable and reliable detection results, and convenient information integration. The rapid development of artificial intelligence technology, especially deep neural networks, has a significant role in promoting the research of machine vision. Although the combination of machine vision and deep neural networks has great application potential in the field of fault diagnosis, the excessive network scale does not match the limited hardware platform, and gradually becomes a bottleneck for accelerating the application of technology. The first is the explosive expansion of the dimension of raw image data brought about by the performance improvement of CCD and CMOS sensors. Usually, higher resolution can provide more information data for tasks such as fault diagnosis, but from the perspective of feature data, the useful information in the image is often extremely sparse relative to the amount of data in the original image. That is to say, although more raw data implies more useful feature data, the expanded raw data dimension brings great difficulty to feature extraction, which is the so-called "dimensionality disaster" problem. Secondly, although deep neural networks have made breakthrough progress in machine vision, their success is to some extent supported by large high-performance hardware platforms (such as GPUs), which makes it difficult to directly deploy them on intelligent fault diagnosis equipment in industrial application scenarios. In addition, the data structure of deep neural networks is itself a large redundant structure. The dimensionality disaster causes the scale of the network to continue to increase to cope with the expansion of the raw data as input, which seriously restricts the development of embedded fault diagnosis equipment. The above-mentioned raw data dimensionality disaster problem and the large scale of deep neural networks are unavoidable obstacles in the development of high-performance intelligent fault diagnosis equipment. Summary of the invention

[0004] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a lightweight neural network fault diagnosis method and system based on tensor manifolds, which reduces the dimension of feature data and the neural network weights, so that the calculation between feature data and neural network weights is completed in a manifold manner, and the number of layers of the neural network does not change, thereby making all data including weights and feature data lightweight, reducing storage pressure, enhancing the parallel capability of computing methods, and improving acceleration potential.

[0005] In order to achieve the above object, the present invention has the following technical solutions:

[0006] A lightweight neural network fault diagnosis method based on tensor manifold, comprising the following steps:

[0007] The acquired fault diagnosis image of any size is regarded as a vector, reconstructed into a first multi-order tensor under constraints, and the first multi-order tensor is represented as a tensor manifold in the form of tensor string decomposition;

[0008] Under the condition that the tensor rank remains unchanged, the first multi-order tensor is iterated multiple times by using the gradient descent method of the tensor manifold to obtain the first contracted multi-order tensor after data dimension reduction;

[0009] The weight matrix in the fault diagnosis neural network is reconstructed into a second multi-order tensor under constraints, and the second multi-order tensor is represented by a tensor manifold in the form of tensor string decomposition to obtain a new neural network with factor tensors as weights;

[0010] Using the first contracted multi-order tensor as input data to train the new neural network, and obtaining a trained neural network;

[0011] A new fault diagnosis image is obtained and converted into a first contracted multi-order tensor after data dimension reduction, and input into the trained neural network to obtain a fault diagnosis result corresponding to the obtained fault diagnosis image.

[0012] Preferably, in the step of regarding the acquired fault diagnosis image of any size as a vector, reconstructing it into a first multi-order tensor under constraints, and representing the first multi-order tensor as a tensor manifold in the form of tensor string decomposition:

[0013] Treat fault diagnosis images of any size as vectors In constraint Reconstructed into a d-order tensor And the d-order tensor The tensor manifold in the form of tensor string decomposition is represented as:

[0014]

[0015] In the formula, the factor tensor All i (x) is the tensor rank, calculate × 1 Refers to tensor contraction calculation, adjacent and After calculation, the rank r can be eliminated i (x) .

[0016] Preferably, in the step of iterating the first multi-order tensor multiple times by using the gradient descent method of the tensor manifold under the condition that the tensor rank remains unchanged to obtain the first contracted multi-order tensor after data dimension reduction:

[0017] In tensor rank r i (x) Under the same condition, the d-order tensor Both belong to European space The Riemann manifold below:

[0018]

[0019] Solving for a d-order tensor by solving an unconstrained optimization problem on the manifold

[0020]

[0021] The first contraction multi-order tensor obtained by iterative solution is a tensor of order d The best estimate of .

[0022] Furthermore, under the condition that the tensor rank remains unchanged, the first multi-order tensor is iterated multiple times by the gradient descent method of the tensor manifold to obtain the first contracted multi-order tensor after data dimension reduction:

[0023] Using the gradient descent method on tensor manifolds to derive the d-order tensor To iterate:

[0024]

[0025] Where k is the number of iterations, η k is the step size of this iteration, represents the Euclidean gradient, is the Riemannian manifold cost function, Refers to gradient projection, Refers to the shrink operation.

[0026] Preferably, the weight matrix in the fault diagnosis neural network is reconstructed into a second multi-order tensor under constraints, and the second multi-order tensor is represented by a tensor manifold in the form of tensor string decomposition to obtain a new neural network with factor tensors as weights, including:

[0027] Fault Diagnosis Neural Network The weight matrix in In constraint and Reconstructed into a d-order tensor The tensor manifold in the form of tensor string decomposition is expressed as:

[0028]

[0029] Get the factor tensor A new neural network with weights

[0030] Preferably, the step of using the first contracted multi-order tensor as input data to train the new neural network to obtain a trained neural network comprises:

[0031] The first shrink multi-order tensor It is expressed as:

[0032]

[0033] Among them, the matrix is a factor tensor Reconstruction, represents the strong Kronecker product;

[0034] The second multi-order tensor Expressed as:

[0035]

[0036] In the formula, the matrix is a factor tensor Reconstruction of

[0037] Will and Compute between factors by position:

[0038]

[0039] In the formula, And there are and r i (y) =r i (W) r i (x) , |·| is the core contraction product, and the result is Still a tensor manifold.

[0040] Furthermore, the strong Kronecker product is, let the block matrix That is, A contains r1×r2 sub-matrices of size n1×n2, and B contains r2×r3 sub-matrices of size m1×m2. So That is, C contains r1×r3 sub-matrices of size n1m1×n2m2, which is:

[0041]

[0042] in, and

[0043] Furthermore, the kernel contraction product is: let the block matrix If A|·|B=C, then That is, C contains r1s1×r2s2 matrices of size n×l, which are:

[0044]

[0045] In the formula,

[0046] A lightweight neural network fault diagnosis system based on tensor manifold, comprising:

[0047] Industrial cameras, used to acquire images of diagnostic targets;

[0048] An FPGA image processing board, used to implement the tensor manifold-based lightweight neural network fault diagnosis method described in any one of claims 1 to 8, and to obtain corresponding fault diagnosis results according to the diagnostic target image captured by the industrial camera;

[0049] And, the host computer is used to display the fault diagnosis results.

[0050] Preferably, the FPGA image processing board adopts Xilinx K7 series FPGA as the core processor, and the FPGA hardware architecture is divided into two parts: tensor string decomposition of data dimension reduction and tensor string decomposition of fault diagnosis neural network. A top-down design approach is adopted to construct parameterized and arbitrarily configurable underlying basic functional units. Starting from the design of strong Kronecker product and kernel contraction product sub-functions related to tensor string decomposition, an FPGA design framework of the data dimension reduction module and the overall compressed deep neural network is built.

[0051] Compared with the prior art, the present invention has at least the following beneficial effects:

[0052] Based on the tensor manifold in the form of tensor string decomposition, the original fault image data is reduced in dimension. On the one hand, the features of the fault image are preliminarily extracted to facilitate the subsequent deep neural network modeling, and on the other hand, the purpose of compressing data capacity and reducing data storage space consumption is achieved. Secondly, the typical deep neural network is compressed using the tensor manifold in the form of tensor string decomposition. On the one hand, the typical neural network consumes less storage space and computing resources, and on the other hand, the compressed deep neural network matches the input data of the tensor manifold dimension reduction. After that, the input data after the tensor manifold dimension reduction is used to train the tensor manifold lightweight deep neural network to obtain the trained tensor manifold lightweight deep neural network. Finally, after the new fault diagnosis data is reduced in dimension by the tensor manifold, the trained tensor manifold lightweight deep neural network is input to obtain the corresponding fault diagnosis result. The present invention is suitable for fault diagnosis scenarios with large fault image size, limited storage space, and high computing efficiency requirements. It can provide technical support for vision-based fault diagnosis automation equipment, has far-reaching significance for solving the dimensionality disaster problem, and has broad practical significance for the practical application of large-scale deep neural networks.

[0053] Compared with the prior art, the lightweight neural network fault diagnosis system based on tensor manifolds in the present invention has an FPGA hardware architecture divided into two parts: tensor string decomposition for data dimensionality reduction and tensor string decomposition for fault diagnosis neural network. According to the concurrency of the tensor manifold calculation process in the form of tensor strings and the parallel framework of FPGA, a high-parallelism and high-throughput pipeline logic structure is designed, providing a general hardware design paradigm for the embedded concurrent acceleration design of large-scale neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 This is the tensor network schematic for matrix-vector multiplication;

[0056] Figure 2 This is the schematic diagram of the contraction model in Riemann optimization;

[0057] Figure 3 Schematic diagram of the hardware structure of an image processing board of a fault diagnosis system according to an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a topological structure of a power supply system of a fault diagnosis system according to an embodiment of the present invention;

[0059] Figure 5 is a schematic diagram of the power-on timing of the fault diagnosis system according to an embodiment of the present invention;

[0060] Figure 6 It is a schematic diagram of the Gigabit network interconnection structure of the fault diagnosis system according to an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of LVDS interface interconnection of a fault diagnosis system according to an embodiment of the present invention;

[0062] Figure 8 is a schematic diagram of serial port interconnection of a fault diagnosis system according to an embodiment of the present invention;

[0063] Fig. 9 is a schematic diagram of hardware interconnection of a fault diagnosis system according to an embodiment of the present invention;

[0064] Fig.10 is a structural diagram of a tensor manifold data dimension reduction hardware system of a fault diagnosis system according to an embodiment of the present invention;

[0065] Fig.11 is a hardware structure diagram of a strong Kronecker product module of a fault diagnosis system according to an embodiment of the present invention;

[0066] Fig.12 It is a hardware structure diagram of a core shrinking and integrating module of a fault diagnosis system according to an embodiment of the present invention;

[0067] Fig.13 It is a block diagram of the FPGA hardware acceleration structure of the fault diagnosis system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also obtain other embodiments without making creative work.

[0069] In view of the problems existing in the prior art, the present invention provides a lightweight neural network fault diagnosis method and system based on tensor manifolds. In the traditional neural network compression method, each factor of the tensor string decomposition is regarded as an independent layer of neural network weights, and the feature data is not decomposed, that is, the process of calculating the feature data and all the factors of the tensor string one by one is mapped to the process of calculating the feature data and multiple neural network weights one by one. The lightweight neural network fault diagnosis method based on tensor manifolds proposed by the present invention, on the one hand, reduces the dimension of the feature data, and on the other hand, reduces the dimension of the neural network weights, and then completes the calculation between the feature data and the neural network weights in a manifold manner, and the number of layers of the neural network does not change, so that all data (including weights and feature data) are all lightweight processed, reducing storage pressure, and the calculation method has strong parallel capability and higher acceleration potential.

[0070] Specifically, the lightweight neural network fault diagnosis method based on tensor manifold in the embodiment of the present invention mainly includes:

[0071] 1. Dimensionality reduction of input data based on Riemann optimization;

[0072] 1.1 Treating fault diagnosis images of any size as vectors In constraint Reconstructed into a d-order tensor and will The tensor manifold in the form of tensor string decomposition is represented as:

[0073]

[0074] Among them, the factor tensor All i (x) is the tensor rank; all The initial value of is arbitrary. All the shrinkage and multiplication can be restored to x in size, but the element value is not equal to x. Calculating × 1 Refers to tensor contraction calculation, adjacent and After calculation, the rank r can be eliminated i (x) , with the help of Figure 1 The tensor network diagram can be more clearly represented, where the horizontal line connecting two blocks represents a contraction calculation × 1 ;

[0075] 1.2 In rank r i (x) Under the same conditions, all All belong to European space The Riemann manifold below:

[0076]

[0077] Solution It can be viewed as an unconstrained optimization problem on the manifold:

[0078]

[0079] Therefore, the iterative solution to this problem is is the optimal estimate of the vector x.

[0080] Using the gradient descent method on tensor manifold To iterate:

[0081]

[0082] Where k is the number of iterations, η k is the step size of this iteration, represents the Euclidean gradient, is the Riemannian manifold cost function, Refers to gradient projection, Refers to the contraction operation, the so-called "contraction" operation is an operation that maps points in the tangent space to the manifold, which is the key to Riemannian gradient descent, such as Figure 2 As shown. Recorded as Complete data dimensionality reduction.

[0083] 2. Data structure design and calculation method design of fault diagnosis neural network;

[0084] 2.1 Fault Diagnosis Neural Network The weight matrix in In constraint and Reconstructed into a d-order tensor The tensor manifold in the form of tensor string decomposition is expressed as:

[0085]

[0086] Get the factor tensor A new neural network with weights

[0087] 2.2 Tensor Manifold As input data, the new neural network defined above Conduct training.

[0088] Among them, the tensor manifold Input neural network When calculating, first Rewritten as:

[0089]

[0090] Among them, the matrix is the step 1.1 Reconstruction, represents the strong Kronecker product; secondly, the tensor manifold represented in step 2.1 is Rewritten as:

[0091]

[0092] Among them, the matrix It is described in 2.1 ; finally and Calculating between factors by position gives:

[0093]

[0094] in, And there are and r i (y) )r i (W) r i (x) , |·| is the core contraction product, It can be used as the input for the next layer and calculated in the same way until the neural network calculation is completed.

[0095] The strong Kronecker product is described as follows: That is, A contains r1×r2 sub-matrices of size n1×n2, and B contains r2×r3 sub-matrices of size m1×m2. So That is, it contains r1×r3 sub-matrices of size n1m1×n2m2, namely:

[0096]

[0097] in, and

[0098] The kernel contraction product is described as follows: If A|·|B=C, then That is, it contains r1s1×r2s2 matrices of size n×l, namely:

[0099]

[0100] in,

[0101] If you need other data for the neural network Continue training and return to step 1.1; if the neural network If no more training is needed, proceed to the next step.

[0102] 2.3 New fault diagnosis data Use the method from step 1.1 to step 1.2 to obtain Input it into the neural network trained in step 2.2 Get the fault diagnosis result of z.

[0103] Simulation validation of data dimensionality reduction and compression of deep neural networks

[0104] Through the above-mentioned lightweight neural network fault diagnosis method based on tensor manifolds of the present invention, we can obtain: a complete mathematical model and calculation method from the collected high-dimensional image data to the characteristics of the fault diagnosis result output by the neural network. The effectiveness of the entire theoretical framework can be verified with the help of simulation calculation tools such as MATLAB, Python, TensorFlow, etc. At the same time, a specific test of transmission line insulator fault detection was carried out. Through the above method, the VGG16 model used was compressed from 500M to 65M, and the recognition accuracy of the compressed deep neural network reached 95%, which is basically consistent with the recognition accuracy of the original model.

[0105] Another embodiment of the present invention further provides a lightweight neural network fault diagnosis system based on tensor manifolds, such as Fig. 9 As shown, including:

[0106] Industrial cameras, used to acquire images of diagnostic targets;

[0107] An FPGA image processing board is used to implement the tensor manifold-based lightweight neural network fault diagnosis method, and obtain corresponding fault diagnosis results according to the diagnostic target image collected by the industrial camera;

[0108] And, the host computer is used to display the fault diagnosis results.

[0109] In a possible implementation, the FPGA image processing board implements the algorithm in the form of the architecture of Xilinx's new generation product XC7K410T, and satisfies the data interaction functions of different interfaces by expanding the commonly used peripheral interfaces. Figure 3As shown in the figure, the board is divided into FPGA minimum system, power system, clock distribution and interface driver according to its function. The board uses Xilinx K7 series FPGA as the core processor, and the power supply is 220VAC. The voltage is changed to 12V / 3A through a 220VAC adapter as the primary power supply of the board. It is connected to an external 8Gb 64bit DDR3, two Gigabit Ethernets, and supports currently commonly used interfaces such as serial ports, LVDS interfaces, etc. In addition, the commercial standard universal interface PMOD is integrated in the interface form for subsequent expansion.

[0110] FPGA minimum system: The minimum system consists of an XC7K410T FPGA, DDR3 memory, and FLASH memory. The XC7K410T has 55650 Slices logic resources, 715×36Kb Block RAM resources, and 1540 DSP48 resources (each DSP slice contains a pre-adder, a 25×18 multiplier, an adder, and an accumulator), which can meet the pre-conditioning circuit and FFT operation requirements of current typical image formats. The FPGA is connected to a 1GB 64-bit DDR3 memory to implement fast data storage, and a 512Mb FLASH memory to implement program loading and parameter storage.

[0111] Power supply system: The power supply system mainly includes secondary power supply and tertiary power supply. The secondary power supply and tertiary power supply use DC-DC power supply chips and LDO to realize the power supply design of the general part and high-speed part respectively. The power supply system topology is as follows Figure 4 As shown. According to the power-on sequence requirements of FPGA, 1.0V->1.8V->3.3V / 2.5V and AVTT->AVCC, the power-on timing of the board is positioned as 1.0V->1.8V->3.3V / 2.5V->AVTT->AVCC. The power-on timing is controlled by the PG output terminal and EN control input terminal of the power chip. The power-on timing is as follows: Figure 5 shown.

[0112] Clock network: The clock network is implemented using TI's CDCE62005 multi-channel clock frequency synthesizer. The multi-channel clock frequency synthesizer supports 2 differential clock inputs or 2 single-ended clock inputs, with an input frequency range of 40kHz to 1500MHz; 1 passive crystal input, with an input frequency range of 2MHz to 42MHz; the output supports 5 differential outputs, 10 single-ended outputs or differential single-ended mixed output modes, with an output frequency range of 4.25MHz to 1.5GHz; the input and output support multiple level logics (LVDS, LVPECL, LVCMOS). CDCE62005 has an SPI interface to implement PLL and clock frequency configuration. The SPI interface is implemented by FPGA logic and configures CDCE62005. The clock network allocation is shown in Table 1.

[0113] Table 1

[0114] name frequency Accuracy Level Logic use SGMIICLK 125MHz ±100PPM LJCB or LVDS XC7K410T SGMII SerDes driver clock SRIOCLK 125MHz ±100PPM LJCB or LVDS XC7K410T SRIO SerDes driver clock SYSCLK 200MHz ±100PPM LJCB or LVDS XC7K410T system SerDes driver clock

[0115] Interface driver: Interface driver mainly includes Ethernet interface, LVDS interface, serial port, etc. The Ethernet interface uses Marvell's 88E1111 to implement two 10 / 100 / 1000Mb / s Ethernet interfaces and supports SGMII / RGMII / MII / GMII interface standards. Figure 6 As shown. The ICS844021 clock chip provides a high-quality, low-jitter 125MHz LVDS clock for the FPGA's GTX. The LVDS interface will use two LVDS019 chips to implement the driver circuit design of the 8-receive 8-transmit LVDS interface, with a signal rate of up to 100Mbps, a transmit group delay of 1.4ns, a receive group delay of 1.9ns, and a 3.3V power supply. The LVDS interface interconnection is as follows Figure 7 The serial port uses two interface chips, LV31T and LV32A, with strong driving capability to realize 6-way RS422 interface, supporting a maximum transmission rate of 10Mb / s. Figure 8 shown.

[0116] Data Dimensionality Reduction and Compression Neural Network Embedded Transplantation Based on FPGA

[0117] The FPGA hardware architecture is divided into two parts: tensor string decomposition for data dimension reduction and tensor string decomposition for fault diagnosis neural network. A top-down design approach is adopted to build parameterized and arbitrarily configurable underlying basic functional units. Starting from the design of strong Kronecker product and kernel contraction product sub-functions related to tensor string decomposition, the FPGA design framework of the data dimension reduction module and the overall compressed deep neural network is gradually built.

[0118] FPGA Design of Tensor String Decomposition for Data Dimensionality Reduction

[0119] In a possible embodiment, the present invention adopts Fig.10 The hardware structure implements the data dimension reduction tensor string decomposition algorithm, which is mainly composed of input memory, gradient descent module (cost function submodule, gradient calculation submodule, error calculation submodule, deterministic subdecomposition tensor submodule), buffer and control unit.

[0120] When the system is running, under the control of the control unit, the system inputs high-dimensional data and the initial sub-decomposition tensor factors for decomposition operations. In the gradient descent module, the initial decomposition tensor factors are first subjected to tensor operations and compared with the input data, and then the cost loss function is calculated, and then the gradient of the cost function is calculated, and the result is sent to the error calculation module. The error calculation module is not a typical gradient value multiplied by the learning rate, but a projected gradient value of the gradient value multiplied by the learning rate. The reason is that it is essentially a gradient on the Riemann space (non-Euclidean), and its direction is along the curved geodesic, unlike the gradient direction in the Euclidean space, which is a straight line. Therefore, it is necessary to use the Euclidean space gradient, projection, contraction and other methods to approximate. Since the gradient of the error calculation module uses the projection value, the result needs to be "contracted" to the correct sub-decomposition tensor in the sub-decomposition tensor determination module. The whole process is iteratively carried out under the control of the control unit. When the minimum error or the maximum number of iterations is reached, the sub-decomposition tensor after the high-dimensional data is decomposed can be output to the buffer for output, completing the entire data dimensionality reduction work.

[0121] The data dimension reduction tensor string decomposition function module is designed as a reconfigurable parameterized module, which can be an independent module or a submodule of a more complex module and used in conjunction with other submodules. In other words, it can be packaged into an IP core (Intellectual Property block) and used as a submodule in a more complex system (the input layer of a deep neural network).

[0122] FPGA Design of Compressed Model of Fault Diagnosis Neural Network

[0123] The weight compression of deep neural networks mainly uses the strong Kronecker product The kernel shrinks and the product |·| is calculated.

[0124] In a possible embodiment, the present invention adopts Fig.11 The hardware structure shown implements the strong Kronecker product Strong Kronecker product The hardware module consists of a main controller, a Kronecker product module, and a data / address manager. Its main function is to traverse the A block matrix and the B block matrix. It can generate read addresses for accessing A-RAM and B-RAM, and receive data read from A-RAM and B-RAM, and then send these data to the Kronecker product module. The calculation results of the Kronecker product module will be output to the cache area, and the data / address manager will allocate a suitable address to store it in the result matrix C_RAM. In the hardware design of the strong Kronecker product, the parameters r1, r2, m, n, l, etc. of the operation matrix dimension are parameterized to specify the dimension of the input block matrix, so that the strong Kronecker product module is suitable for different block matrix dimensions. The data fetching logic of the main controller of the strong Kronecker product is similar to the data fetching logic of matrix multiplication. The implementation idea is to design two layers of loops. The first layer of loops traverses each row of the block matrix A, and the second layer of loops traverses each column of the block matrix B. The block matrices fetched from the first two loops are sent to the Kronecker product module to calculate the Kronecker product of the two sub-matrices. Next, the following implementation strategy is adopted for the control logic of the Kronecker product: still design two loops, the first layer of loop fixes the sub-matrix A ii The first element a ii , traverse the submatrix B ii For each element of ii The sub-block matrix b ii , traverse the molecular matrix A ii For each block element, since the data obtained are all scalars, the matrix elements taken out in the first two cycles are sent to the multiplier module to calculate the multiplication of two scalars. The multiplier here is implemented using the IP core provided by Xilinx. After the Kronecker product calculation is completed, the calculation result is output to the output buffer along with the write address and write enable signal. Then the data of the output buffer is input into the data / address manager to assign the correct address to it, and finally it is assigned to the correct position of the result matrix C_RAM.

[0125] The kernel contraction product |·| is a generalized block matrix multiplication operation. The submodule is proposed to use Fig.12The hardware structure shown. The core shrink-and-product hardware module consists of a main controller, a matrix multiplication (MM) module, and a data / address manager (Data / Address Manager); its main function is to traverse the A block matrix and the B block matrix. It can generate read addresses to access A-RAM and B-RAM, and receive data read from A-RAM and B-RAM, and then send these data to the matrix multiplication module. The calculation results of the matrix multiplication module will be output to the cache area, and the appropriate address will be allocated in the data / address manager to store in the result matrix C_RAM. In the hardware design of the core shrink-and-product, the parameters r1, r2, m, n, l and other parameters of the operation matrix dimension are parameterized to specify the dimension of the input block matrix, so that the core shrink-and-product module is suitable for different block matrix dimensions. The FPGA implementation idea of ​​the core shrink-and-product is to design two layers of loops. The first layer of loop fixes the sub-block matrix A of the block matrix A. ii , traverse each block matrix of the block matrix B, and the second layer loop fixes the sub-block matrix B of the block matrix B ii , traverse each block matrix of the block matrix A, and send the sub-matrices taken out in the first two loops to the matrix multiplication module to calculate the multiplication of the two matrices. The FPGA implementation idea of ​​the matrix multiplication sub-module is to design two layers of loops. The first layer loops through each row of matrix A, and the second layer loops through each column of matrix B. The vectors taken out in the first two loops are sent to the multiply-accumulate (MAC) module to calculate the dot product of the two vectors, and then the calculation results are output to the output buffer along with the write address and write enable signal. Then the data of the output buffer is input to the data / address manager to assign the correct address to it, and finally it is assigned to the correct position of the result matrix C.

[0126] In order to realize the general FPGA computing architecture, all functional modules of the system adopt parameterized design. By modifying the design parameters, different functional modules can adapt to different application scenarios. According to the principle of FPGA parameterized design, in different sub-functional modules, the parameter keyword is used to define the parameters that need to be reconfigured, and the default values ​​of these parameters are given. After the sub-functional module parameterized design and operation logic simulation are successful, the parameter configuration process can be started. The specific process is as follows: In the design of different sub-functional modules, the parameters that need to be reconfigured are defined by the parameter keyword, and the default values ​​are given. When the overall module instantiates the sub-module, the default parameters in the sub-module are changed by passing parameters in the form of defparam, so as to realize the function of unified parameter management in the overall module.

[0127] Fault Diagnosis Neural Network Model Compression

[0128] After the functional submodule design is completed, the overall design of FPGA hardware acceleration of the lightweight neural network fault diagnosis method based on tensor string decomposition can be carried out, such as Fig.13 As shown, it mainly consists of three functional modules (tensor string decomposition module, pooling calculation module, fully connected calculation module), off-chip memory (SDRAM), and I / O port for input data.

[0129] The three functional modules can be further subdivided. The tensor string decomposition module is mainly composed of the initial sub-decomposition factor G, the gradient descent module (cost function and error calculation), and the determination of the sub-decomposition tensor G. The tensor string decomposition first accepts the input high-order tensor data under the control of the stage controller, and then sets the initial values ​​of multiple decomposition factors G through the initial sub-decomposition tensor G module. Since the initial value is not the result of the final tensor string decomposition, it needs to be corrected by the gradient descent method to obtain the determined sub-decomposition tensor, and the determined sub-decomposition tensor is cached and prepared to be input into the pooling module. The pooling module is mainly composed of an internal control logic module, a comparison / accumulator, and an output cache block. The module first receives the start signal of the stage controller, reads the pooling result of the second stage, and then compares and accumulates the input data under the control of its internal control logic module. Finally, the pooling result is output to the output cache area for use in the next stage. The parameterization design of the pooling module is a reconfigurable hardware IP core, which is convenient for hardware debugging of deep neural network algorithms. Before the fully connected module, the input data still needs to be decomposed into tensor strings. The design of this module is the same as the tensor string decomposition module mentioned above. The fully connected layer module mainly consists of an internal control module, a strong Kronecker product The specific fully connected operation process is as follows: after the tensor string decomposition is completed, the stage controller receives the end signal of the tensor string decomposition module and starts the fully connected module through the control logic. The fully connected module first receives the sub-decomposition tensor with lower complexity after the tensor string decomposition, and then performs the strong Kronecker operation and the core shrinking and product operation in series under the control of the control logic module. These two operations are parameterized in the sub-modules mentioned above and encapsulated into IP cores.

[0130] The above four modules are the four basic operation modules of the deep neural network compression model. The present invention sets them as reconfigurable IP cores to facilitate the deep expansion and hyperparameter adjustment of the deep neural network compression model. Finally, a deep neural network compression model with arbitrary expansion depth can be realized. At the same time, this module can become an independent module or a submodule of a more complex module and be used in conjunction with other submodules. In other words, it can be packaged into an IP core (Intellectual Property block) and used as a submodule in a more complex system.

[0131] Finally, a high-parallelism, high-throughput FPGA parallel framework is obtained to realize an embedded online fault diagnosis visual system for complex application scenarios. In the above-mentioned transmission line insulator fault detection test, the execution efficiency and overall power consumption of the adopted FPGA acceleration solution are better than those of GPU.

[0132] In summary, the present invention aims at the problem of dimensionality disaster of original data and the problem of over-scale of deep neural network, and proposes a lightweight neural network fault diagnosis method and system based on tensor manifold. The method and system are applied to fault diagnosis scenarios with large fault image size, limited storage space and high computational efficiency requirements, and can provide technical support for vision-based fault diagnosis automation equipment and promote industrial application.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to impose any limitation on the technical solution of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solution can also be subjected to several simple modifications and substitutions, and these modifications and substitutions are also within the scope of protection covered by the claims.

Claims

1. A lightweight neural network fault diagnosis method based on tensor manifold, characterized in that: The following steps are involved: The acquired fault diagnosis image of any size is regarded as a vector, reconstructed into a first multi-order tensor under a first constraint condition, and the first multi-order tensor is represented as a tensor manifold in the form of tensor string decomposition; Under the condition that the tensor rank remains unchanged, the first multi-order tensor is iterated multiple times by using the gradient descent method of the tensor manifold to obtain the first contracted multi-order tensor after data dimension reduction; The weight matrix in the fault diagnosis neural network is reconstructed into a second multi-order tensor under the second constraint condition, and the second multi-order tensor is represented by a tensor manifold in the form of tensor string decomposition to obtain a new neural network with factor tensors as weights; Using the first contracted multi-order tensor as input data to train the new neural network, and obtaining a trained neural network; A new fault diagnosis image is obtained and converted into a first contracted multi-order tensor after data dimension reduction, and input into the trained neural network to obtain a fault diagnosis result corresponding to the obtained fault diagnosis image.

2. The tensor manifold-based lightweight neural network fault diagnosis method according to claim 1 is characterized in that: In the step of treating the acquired fault diagnosis image of any size as a vector, reconstructing it into a first multi-order tensor under a first constraint, and representing the first multi-order tensor as a tensor manifold in the form of tensor string decomposition: Treat fault diagnosis images of any size as vectors , in the first constraint Reconstructed as Rank Tensor , and Rank Tensor The tensor manifold in the form of tensor string decomposition is represented as: In the formula, the factor tensor ,all is the tensor rank, and is calculated Refers to tensor contraction calculation, adjacent and After calculation, the rank can be eliminated .

3. The method for lightweight neural network fault diagnosis based on tensor manifold according to claim 2 is characterized in that: In the step of performing multiple iterations on the first multi-order tensor using the gradient descent method of the tensor manifold under the condition that the tensor rank remains unchanged to obtain the first contracted multi-order tensor after data dimension reduction: In tensor rank Under unchanged conditions, Rank Tensor Both belong to European space The Riemann manifold below: Solve by solving an unconstrained optimization problem on the manifold Rank Tensor : The first contraction multi-order tensor obtained by iterative solution yes Rank Tensor The best estimate of .

4. The tensor manifold-based lightweight neural network fault diagnosis method according to claim 3 is characterized in that: In the step of performing multiple iterations on the first multi-order tensor using the gradient descent method of the tensor manifold under the condition that the tensor rank remains unchanged to obtain the first contracted multi-order tensor after data dimension reduction: Using the gradient descent method on tensor manifold Rank Tensor To iterate: In the formula, is the number of iterations, is the step size of this iteration, represents the Euclidean gradient, is the Riemannian manifold cost function, Refers to gradient projection, Refers to the shrink operation.

5. The method for lightweight neural network fault diagnosis based on tensor manifold according to claim 1 is characterized in that: The method of reconstructing the weight matrix in the fault diagnosis neural network into a second multi-order tensor under the second constraint condition, and representing the second multi-order tensor in a tensor manifold in the form of tensor string decomposition to obtain a new neural network with factor tensors as weights includes: Fault Diagnosis Neural Network The weight matrix in In the second constraint and Reconstructed as Rank Tensor , and then expressed as a tensor manifold in the form of tensor string decomposition: Get the factor tensor A new neural network with weights .

6. The tensor manifold-based lightweight neural network fault diagnosis method according to claim 5 is characterized in that: The step of using the first contracted multi-order tensor as input data to train the new neural network to obtain the trained neural network includes: The first shrink multi-order tensor It is expressed as: Among them, the matrix is a factor tensor Reconstruction, represents the strong Kronecker product; The second multi-order tensor Expressed as: In the formula, the matrix is a factor tensor Reconstruction of Will and Compute between factors by position: In the formula, , and there are and , is the core contraction product, and the result is Still a tensor manifold.

7. The tensor manifold-based lightweight neural network fault diagnosis method according to claim 6 is characterized in that: The strong Kronecker product is, let the block matrix , ,Right now Included The size is The sub-matrix of Included The size is If the sub-matrix of ,So ,Right now Include The size is The sub-matrix of is: in, , ,and .

8. The method for lightweight neural network fault diagnosis based on tensor manifold according to claim 6 is characterized in that: The kernel contraction product is, let the block matrix , ,like ,So ,Right now Include The size is The matrix is: In the formula, , .

9. A lightweight neural network fault diagnosis system based on tensor manifold, characterized in that: include: Industrial cameras, used to acquire images of diagnostic targets; An FPGA image processing board, used to implement the tensor manifold-based lightweight neural network fault diagnosis method described in any one of claims 1 to 8, and to obtain corresponding fault diagnosis results according to the diagnostic target image captured by the industrial camera; And, the host computer is used to display the fault diagnosis results.

10. The light-weight neural network fault diagnosis system based on tensor manifold according to claim 9 is characterized in that: The FPGA image processing board uses Xilinx K7 series FPGA as the core processor. The FPGA hardware architecture is divided into two parts: tensor string decomposition for data dimension reduction and tensor string decomposition for fault diagnosis neural network. A top-down design approach is adopted to construct parameterized and arbitrarily configurable underlying basic functional units. Starting from the design of strong Kronecker product and kernel contraction product sub-functions related to tensor string decomposition, the FPGA design framework of the data dimension reduction module and the overall compressed deep neural network is constructed.

Images