A Medical Image Classification Method Combining Wavelet Transform and Tensor Network

Abstract

The present invention relates to the technical field of medical image classification, and specifically discloses a medical image classification method combining wavelet transform and tensor network. A coarsening network is proposed, which combines wavelet transform and MERA, that is, encoding D4 wavelet into MERA to form a waveletMERA model with fixed internal parameters, and a tensor classification network like a fully connected layer is also constructed. The present invention uses the MNIST dataset, the Covid-19 dataset and the LIDC dataset for multi-dimensional verification. The results show that the accuracy of waveletMERA is stably high, and it has better coarsening ability than the deep neural network of CNNs, enabling waveletMERA to reduce the number of model parameters to a greater extent while ensuring accuracy. The results show that waveletMERA is not only superior to the current mainstream deep neural networks in classification, but also superior to ordinary wavelet transform in data preprocessing. Moreover, waveletMERA also has the interpretability advantage of the tensor network itself.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image classification, and particularly to a medical image classification method combining wavelet transform and tensor network. Background Art

[0002] In recent decades, machine learning has developed vigorously, and many algorithms have emerged, which have all been proven to be good enough in their respective eras, such as Naive Bayes ( Bayes), kernel methods, decision trees, random forests, and neural networks (NNs). In recent years, deep neural networks have achieved amazing success. Convolutional Neural Networks (CNNs) are one of the most successful deep neural networks. Its convolutional layer can extract important features from the original data, and then the pooling layer compresses the data. Finally, the fully connected layer is input to obtain the prediction result.

[0003] In the field of medical image analysis, CNNs models and their variants are widely used in colonoscopy, HEp-2 cell image classification, breast mass classification, biomarker detection of cerebral small vessel diseases, and skin cancer classification. In the past two years, the novel coronavirus (Covid-19 virus) has spread rampantly around the world, which has made people pay more attention to the field of intelligent medicine. Researchers are thinking about how to use machines to assist humans in defeating diseases. Precise computer-aided medical image analysis tools can assist doctors in identifying and locating lesions, improving the diagnosis efficiency and accuracy. Medical images include X-rays, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasonic waves, etc. It is not easy to use them to train deep learning models. First of all, medical images generally have various quality distortions and degradations, such as noise, blurring, and compression effects. Also, different from natural images, labeling a large number of medical images is both expensive and time-consuming. Therefore, the medical images available for training deep neural network models are always limited. Finally, the medical image classification task requires the model to have high accuracy and at the same time requires the prediction results of the model to be well-founded.

[0004] Therefore, CNNs and their variants have inevitable drawbacks when used in medical image classification tasks. As the number of model layers deepens, the distortion of medical images will lead to more biases; small data sets usually cause overfitting in the training of deep neural networks; in classical machine learning algorithms, interpretability and high accuracy often come at the expense of each other. The non-linear operations in CNNs bring excellent accuracy, but also limit their interpretability. Therefore, there is a desire to find a method that can take into account the advantages of CNNs and make up for their disadvantages. Summary of the Invention

[0005] The present invention provides a medical image classification method combining wavelet transform and tensor network, and the technical problem to be solved is that existing medical image classification methods cannot take into account high accuracy, avoid overfitting and interpretability in small data sets.

[0006] To solve the above technical problems, the present invention provides a medical image classification method combining wavelet transform and tensor network, including the steps:

[0007] S1. Flatten a two-dimensional medical grayscale image with N pixels into a one-dimensional vector v(x) of length N;

[0008] S2. Feature-map each element x in the one-dimensional vector v(x) i into a quantum state in a Hilbert space, obtaining N quantum states;

[0009] S3. Perform L times of coarse-graining processing on the N quantum states through a coarse-graining network combining wavelet transform and tensor network, and output one term containing only a single |1> state each time, so as to obtain L terms containing only a single |1> state as the coarse-graining output;

[0010] S4. Feature-map the coarse-graining output into multiple eigenvalues and connect them through tensor product and input them into the trained tensor classification network to obtain the predicted classification result.

[0011] Further, in the step S3, the coarse-graining network is constructed based on coarse-graining units, and each coarse-graining unit includes two disentanglers and an isometry; the two disentanglers are used to input four adjacent quantum states obtained in step S2 connected through tensor product, and each obtains one term containing only a single |1> state through disentangling; the isometry is used to perform a coarse-graining operation on the two terms containing only a single |1> state output by the two disentanglers to obtain a corresponding term containing only a single |1> state.

[0012] Further, use U and V to represent the disentangler and the isometry respectively, then U and V are defined as:

[0013]

[0014]

[0015] Among them, θ u =-π / 6,θ v =π / 12.

[0016] Furthermore, the coarse-grained network is composed of a multi-layer network structure consisting of disentanglers and isometry, each layer of the network structure contains more than two disentanglers and more than one isometry, each disentangler has two input indicators and two output indicators, and each isometry has two input indicators and one output indicator; the last disentangler of each layer of the network structure is connected to the first disentangler through an additional isometry;

[0017] In a non-top-level network structure, two equally spaced input indicators are connected to an output indicator of two adjacent disentanglers, and two output indicators of a disentangler are connected to an input indicator of two adjacent equally spaced ones. Each equally spaced output indicator in the same layer is connected to an input indicator of a disentangler in the next layer of the network structure.

[0018] In the bottom network structure, the four input indices of each two adjacent disentanglers are used to input each four adjacent quantum states of the N quantum states obtained in step S2 connected by tensor products;

[0019] Each disentangler is used to preserve the entanglement of the two quantum states under the tensor product connection of the input, and to disentangle the two quantum states from the quantum states of other disentanglers input. Each disentangler outputs the first term and the second term containing only a single |1> state from two output indicators, respectively, where the first term and the second term are the results obtained when the two quantum states are in the last two digits and the first two digits of the input in the scaling function operation, respectively.

[0020] Each isometry is used to perform a coarse-grained operation on the two terms containing only a single |1> state under the tensor product connection of the input, and obtain the result of the D4 wavelet transform of the four adjacent quantum states input to the two disentanglers connected to it.

[0021] Further, in step S2, the i-th element x in the one-dimensional vector v(x) i is mapped to:

[0022] |φ(x i )>=|0>+x i |1>.

[0023] Further, in step S3, the tensor product of four adjacent quantum states is The process of coarse-graining by the adjacent first disentangler, second disentangler, and the first equidistant connection to both the first disentangler and the second disentangler includes the steps:

[0024] S31. The first disentangler disentangles the tensor product and outputs the second term containing only a single |1> state into the first equidistant; the second disentangler disentangles the tensor product and outputs the first term containing only a single |1> state into the first equidistant;

[0025] S32. The first equidistant performs a tensor product operation on the two input terms each containing only a single |1> state, and obtains a term containing only a single |1> state and inputs it into a disentangler in the next-layer network structure connected to the first equidistant.

[0026] Furthermore, the first disentangler disentangles the tensor product according to its own definition, which is expressed by the formula:

[0027]

[0028] where, (x 2i+1 sinθ u +x 2i cosθ u )|1> and (x 2i+1 cosθ u -x 2i sinθ u )|1> are respectively the first and second terms containing only a single |1> state output by the first disentangler;

[0029] Similarly, the first and second terms containing only a single |1> state output by the second disentangler are respectively (x 2i+3 sinθ u +x 2i+2 cosθ u )|1> and (x 2i+3 cosθ u -x 2i+2 sinθ u )|1>:

[0030] Then, the first equidistant performs a tensor product on the second term output by the first disentangler and the first term output by the second disentangler according to its own definition to obtain a term containing only a single |1> state, which is expressed by the formula:

[0031]

[0032] where, acting on the original input data (x 2i x 2i+1 x2i+2 x 2i+3 The coefficient on

[0033]

[0034] Furthermore, the step S4 specifically includes the steps:

[0035] S41. Retain the coefficients of all terms in the |1> state in the coarsened output obtained in step S3, and use the deviation normalization method to normalize these L coefficient values to the interval [0, 1] to obtain L normalized values;

[0036] S42. Perform feature mapping on the L normalized values using the following formula:

[0037]

[0038] where x refers to any one of the L normalized values, and φ(x) represents the eigenvalue obtained after mapping x;

[0039] S43. Connect the L eigenvalues obtained in step S42 through tensor product to obtain the global feature mapping corresponding to the medical two-dimensional grayscale image;

[0040] S44. Input the global feature mapping into the trained tensor classification network to predict the classification result of the medical two-dimensional grayscale image.

[0041] Furthermore, the tensor classification network adopts the matrix product state and is represented by a linear decision function as:

[0042]

[0043] where f j (x) represents the prediction result of the tensor classification network for the j-th sample, Φ(x) j represents the global feature mapping of the j-th sample in the dataset, W represents the weight matrix, which consists of L weight tensors n = 0, 1, 2,...., L. After being trained by the dataset, W remains fixed.

[0044] Furthermore, during the process of training the tensor classification network, the cross-entropy loss function is used to calculate the distance between the prediction result of the tensor classification network and the true label, and the backpropagation algorithm is used to update the weight tensors in the tensor classification network; the cross-entropy loss function is expressed as:

[0045]

[0046] where y j represents the label of the j-th sample, the positive class is 1, and the negative class is 0; pj represents the probability that the j-th sample is predicted as the positive class.

[0047] A medical image classification method combining wavelet transform and tensor network provided by the present invention proposes a coarsening network that performs excellently on medical images. This coarsening network combines wavelet transform and MERA (Multi-scale Entanglement Renormalization Ansatz), that is, encodes the D4 wavelet into MERA to form a model with fixed internal parameters, which the present invention names waveletMERA. The present invention also constructs a trainable tensor network (i.e., tensor classification network) like a fully connected layer. The present invention uses the MNIST dataset, the Covid-19 dataset, and the LIDC dataset for multi-dimensional verification. The results show that the accuracy of waveletMERA is stably high, and it has better coarsening ability than the deep neural network of CNNs. This ability enables waveletMERA to reduce the number of model parameters to a greater extent while ensuring accuracy. The results show that waveletMERA is not only superior to the current mainstream deep neural networks in classification but also superior to ordinary wavelet transform in data preprocessing. Moreover, waveletMERA also has the interpretability advantage of the tensor network itself, which can provide more targeted and efficient diagnostic assistance for doctors, alleviating the problems of medical tension and doctor-patient conflicts. Description of the Drawings

[0048] Figure 1 is a graphical symbol representation diagram of tensors and tensor operations provided by an embodiment of the present invention;

[0049] Figure 2 is a structural diagram of CNNs and MERA provided by an embodiment of the present invention;

[0050] Figure 3 is a relationship diagram of the original data, scaling function, and wavelet function in the D4 wavelet transform provided by an embodiment of the present invention;

[0051] Figure 4 is a diagram showing four tensor network representations of an 8-bit quantum multi-body state provided by an embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of the properties of two tensors of MERA provided by an embodiment of the present invention;

[0053] Figure 6 is a decay curve diagram of the ability of MPS and MERA to capture the correlation between data as the distance x increases provided by an embodiment of the present invention;

[0054] Figure 7It is a quantum circuit diagram composed of unitary gates for implementing wavelet transform provided by an embodiment of the present invention;

[0055] Figure 8 is provided by an embodiment of the present invention Figure 7 The unitary gate group in the circuit in can be combined to form disentanglers U and isometries V diagrams as shown in formulas (9) and (8);

[0056] Figure 9 is |0> + x provided by an embodiment of the present invention 2i |1> and |0> + x 2i+1 Schematic diagrams of the first and last two bits of |1> during a single-scale function operation;

[0057] Figure 10 It is a process display diagram of four adjacent data being coarsened by a single coarsening unit provided by an embodiment of the present invention;

[0058] Figure 11 It is a schematic diagram of unfolding a medical two-dimensional grayscale image into a vector using the "zigzag" flattening method provided by an embodiment of the present invention;

[0059] Figure 12 It is a flowchart of a medical image classification method combining wavelet transform and tensor network provided by an embodiment of the present invention;

[0060] Figure 13 It is a structural diagram of a tensor classification network (MPS) provided by an embodiment of the present invention;

[0061] Figure 14 It is a comparison diagram before and after a lung CXR image passes through a 4-layer wavelet MERA diagram provided by an embodiment of the present invention;

[0062] Figure 15 It is a schematic diagram of the confusion matrix of wavelet MERA on the Covid-19 test set provided by an embodiment of the present invention;

[0063] Figure 16 It is a process display diagram of preprocessing the Covid-19 dataset provided by an embodiment of the present invention;

[0064] Figure 17 It is a schematic diagram of the confusion matrix of wavelet MERA on the Covid-19-mask test set provided by an embodiment of the present invention;

[0065] Figure 18 It is a schematic diagram of the confusion matrix of wavelet MERA on the LIDC test set provided by an embodiment of the present invention. Detailed implementation manners

[0066] The embodiments of the present invention will be specifically described below in conjunction with the accompanying drawings. The given examples are only for illustrative purposes and should not be construed as limiting the present invention. The included drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0067] As a powerful numerical tool in the fields of quantum many-body physics and quantum information science, tensor networks (TNs) have been used in the research combining quantum physics and machine learning and have achieved vigorous development in recent years. Both TNs and NNs are composed of simple units (tensors or neurons) to achieve complex functions. As an extension of matrices, tensors can represent high-dimensional data features such as text, images, audio, and video in the real world. The graphical symbols of tensors and tensor operations are as Figure 1 shown in (a): the graphical symbol representations of the scalar S, vector V i , matrix M ij , third-order tensor T ijk ; (b): the graphical symbol representation of matrix multiplication or tensor contraction; (c): the graphical symbol representation of the trace of the product of two matrices.

[0068] TNs is a tensor-based data analysis method that can solve the "curse of dimensionality" problem faced by tensors when their order increases. However, different from NNs, TNs perform linear operations in high-dimensional spaces, which makes them easier to perform theoretical analysis and promotes the development of more interpretable algorithms. Here, the interpretability of TNs is emphasized because they naturally combine the quantum probability interpretation based on statistical physics and quantum field theory, and the operations on TNs are transparent and specific. In the process of the development of the intelligent medical industry, TNs have shown amazing effects.

[0069] A key motivation for the development of tensor networks is the idea of coarse-graining, which is called the renormalization group in physics. Representative tensor networks with the idea of coarse-graining such as the tree tensor network (TTN) and the multi-scale entanglement renormalization ansatz (MERA) are shown in Figure 4 (c) and 4(d) respectively. Surprisingly, CNNs have the "genes" of physics. Their convolutional operations contain the idea of coarse-graining, and CNNs are very similar in structure to the MERA tensor network, as Figure 2 shown in Figure 2The upper part is the process of performing convolution operations on lung images, and the lower part is the coarse-graining process of MERA. The original data needs to be feature-mapped before being input into the tensor network. Therefore, in this example, a lightweight method based on MERA is developed, which not only benefits from the interpretability of TNs but also makes up for the shortcomings brought about by the excessive complexity of CNNs models.

[0070] Wavelets and wavelet transforms are one of the most important developments in the field of signal and image processing in recent decades. The D4 wavelet has four scaling function coefficients h and four wavelet function coefficients g as follows:

[0071]

[0072] The scaling function and the wavelet function are given by the inner product of the function coefficients (h and g) and four input data. The scaling function is shown in Equation (2), and the wavelet function is shown in Equation (3):

[0073] s(x) = h0x 2i + h1x 2i+1 + h2x 2i+2 + h3x 2i+3 , (2)

[0074] w(x) = g0x 2i + g1x 2i+1 + g2x 2i+2 + g3x 2i+3 , (3)

[0075] where i ∈ {i|1 ≤ i < (N / 2), i ∈ Z}, and here N represents the total number of input data. When performing wavelet transform, the values of the scaling function and the wavelet function are calculated iteratively. In each iteration, i in the above formula increases by 1. The scaling function can capture the distribution characteristics of the data and simulate the distribution of the original data with a general amount of data. The wavelet function can capture the differences between the data, as Figure 3 shown. There is an edge problem with the D4 wavelet: when i = (N / 2) - 1, the wavelet transform should act on the input data x N-2 、x N-1 、x N 、x N+1 , but x N and x N+1 do not exist. There are two solutions to this edge problem: (1) regard the original data as periodic, that is, end-to-end connected, and the data following x N-1 becomes x0 and x1; (2) regard both the head and the tail ends of the original data as mirror images, so that the input data becomes x N-2 、x N-1 、x N-1 、xN-2 .

[0076] TNs can solve the "curse of dimensionality" problem encountered when using classical computers to simulate quantum many-body systems. An N-bit quantum many-body state can be represented as

[0077]

[0078] The dimension of its corresponding Hilbert Space is d N . As Figure 4 (a) shows, the coefficient of the quantum state can be represented as a tensor of order N, and the dimension of each order is d. Then the number of parameters of this coefficient tensor increases exponentially with the increase of N. When N is large enough, this tensor can no longer be processed by classical computers.

[0079] Therefore, some TNs, such as Matrix Product State (MPS), are used to approximate high-order tensors. MPS is a tensor decomposition method that can approximate any high-order tensor using a three-order tensor chain, as Figure 4 (b). The MPS form of the N-order coefficient tensor can be represented as:[[]]

[0080]

[0081] where represents each third-order tensor that makes up the MPS, i n is the physical index of each tensor, and α n is called the virtual index, connecting the tensors in the MPS. The dimension of the index α n is χ, which determines the approximate representation ability of the MPS. The number of parameters of the tensor is reduced from exponential-level d N to polynomial-level N·d·χ 2 . When χ = d N2 , the approximation of the MPS is exact and can represent any tensor. The ability of the MPS to capture the correlation between data decays exponentially with the increase of the distance between two data, as Figure 4 shown: (a)[[]] Tensor image representation of; (b) Matrix Product State (MPS); (c) Tree Tensor Network (TTN); (d) Multi-scale Entanglement Ansatz (MERA).

[0082] MERA is also a type of tensor network. Similar to MPS, it can approximately represent a high-order tensor, as shown in Figure 4 (d), and is designed to implement the real-space renormalization group idea in a powerful numerical algorithm. Compared with MPS, MERA can capture the correlations between data that are far apart. As Figure 6 shown, that is, the ability of MERA to capture the correlations between data decays as a power law with the increase in the distance between the two data.

[0083] MERA has two types of tensors: disentanglers and isometries, as shown in Figure 5 (a) and Figure 5 (b) respectively. For a standard binary MERA, both of these two tensors need to satisfy specific restrictions. The disentangler must be unitary: The isometry needs to satisfy: The disentangler preserves the entanglement between the data input to the same disentangler and disentangles the data input to different disentanglers; then the isometry layer coarsens the output of the disentanglement layer. The characteristics of the disentangler and the isometry enable MERA to capture all the entanglements at the same scale in the same layer.

[0084] Compared with one-dimensional single-layer tensors such as MPS, the hierarchical tensor network can better represent the correlations between linear layers. TTN is also a hierarchical tensor network, which consists of some isometry tensors as shown in Figure 4 (c). Due to the characteristics of the isometry, TTN can also complete the coarsening or renormalization of the original data, and each layer of TTN can reduce the data volume of the original data by half. However, compared with MERA, TTN has obvious disadvantages. For example, the adjacent data at i4 and i5 in Figure 4 (c) will only meet at the topmost tensor of TTN. This requires the topmost tensor to capture the short-range entanglement between some adjacent data, which limits its ability to capture long-range entanglement, while the disentanglement layer of MERA can make up for this deficiency of TTN.

[0085] The idea of the renormalization group has influenced the development of wavelet transforms, and wavelets have been proven to be useful tools for the application of the renormalization group. MERA and wavelet transforms are both closely related to the renormalization group. The wavelet transform can be implemented using a quantum circuit composed of some unitary gates, as shown in Figure 7 shown. Each layer of the circuit has two sub-layers, which are composed of unitary gates u(θ1) and u(θ2) respectively. When the index leg in the upper left of each unitary gate u(θ2) contracts with the |1> state and θ1 = -π / 6, θ2 = π / 12, the circuit part in the gray shaded area can implement the scaling function of the D4 wavelet transform.

[0086] Figure 7 The single-qubit gates in the shown circuit are defined as follows:

[0087]

[0088] Figure 7 The gray shaded area in... is expressed by the formula as follows:

[0089]

[0090] The two terms (sinθ2cosθ2) in the original single-qubit gates are retained. Similarly, when contracting with the |0> state, Figure 7 The gray shaded area in... implements the wavelet function of the D4 wavelet transform. The output of half of each layer of the circuit corresponds to the result of the scaling function and is passed upward as the input of the next layer of the circuit.

[0091] Based on these theoretical foundations, the embodiments of the present invention provide a medical image classification method combining wavelet transform and tensor network, referring to Figure 12 , specifically including the steps:

[0092] S1. Flatten a two-dimensional medical grayscale image with N pixels into a one-dimensional vector v(x) of length N;

[0093] S2. Feature map each element x in the one-dimensional vector v(x) i into a quantum state in a Hilbert space to obtain N quantum states;

[0094] S3. Perform L times of coarse-graining processing on the N quantum states through a coarse-graining network combining wavelet transform and tensor network, and output one term containing only a single |1> state each time, so as to obtain L terms containing only a single |1> state as the coarse-graining output;

[0095] S4. Feature map the coarse-graining output into multiple eigenvalues and connect them through tensor product and input them into the trained tensor classification network to obtain the predicted classification result.

[0096] The coarse-graining network in step S3 combines the D4 wavelet transform and MERA, which is called waveletMERA in this example. Its structure is as Figure 8 (b) shown, where the light gray indices represent the first output bit of each layer of waveletMERA. waveletMERA is based on Figure 8(a) The coarse-grained unit shown on the right is constructed, and the coarse-grained unit is transformed from the unitary gate group in the D4 wavelet transform shown on the left of 8(a). Specifically, the coarse-grained unit includes two disentanglers and an isometry; the two disentanglers are used to input four adjacent quantum states of the N quantum states obtained in step S2 connected by tensor product, and each obtains a term containing only a single |1> state through disentanglement; the isometry is used to perform a coarse-grained operation on the two terms containing only a single |1> state output by the two disentanglers, and obtain a corresponding term containing only a single |1> state.

[0097] The coarse-grained network consists of an M-layer network structure. Figure 8 (b) is an example of M=2 layers. Each layer of the network structure contains more than 2 disentanglers and more than 1 isometry, each disentangler has two input indicators and two output indicators, and each isometry has two input indicators and one output indicator; the last disentangler of each layer of the network structure is connected to the first disentangler through an additional isometry.

[0098] In a non-top-level network structure, two equally spaced input indicators are each connected to an output indicator of two adjacent disentanglers, and two output indicators of a disentangler are each connected to an input indicator of two adjacent equally spaced ones. Each equally spaced output indicator in the same layer is connected to an input indicator of a disentangler in the next layer of network structure. In the lowest-level network structure, four input indicators of each adjacent two disentanglers are used to input each four adjacent quantum states of the N quantum states obtained in step S2 connected by tensor product.

[0099] Each disentangler is used to preserve the entanglement of the two quantum states under the tensor product connection of the input, and to disentangle the two quantum states from the quantum states of other disentanglers input. Each disentangler outputs the first term and the second term containing only a single |1> state from two output indicators, respectively, where the first term and the second term are the results obtained when the two quantum states are in the last two digits and the first two digits of the input in the scaling function operation, respectively.

[0100] Each isometry is used to perform a coarse-grained operation on two items containing only a single |1> state under the tensor product connection of the input, and obtain the result of the D4 wavelet transform of the four adjacent quantum states input to the two disentanglers connected to it.

[0101] It is easy to see that after the input data passes through M layers of waveletMERA, the length is reduced from N to N / 2 M =L.

[0102] In this example, the disentangler U and isometry V in waveletMERA can be defined as:

[0103]

[0104]

[0105] where θ u = -π / 6, θ v = π / 12.

[0106] In step S2, the i-th element x in the one-dimensional vector v(x) i (the i-th pixel point in the image) is mapped to a quantum state |φ(x i )〉 in the Hilbert space:

[0107] |φ(x i )〉 = |0〉 + x i |1〉, (10)

[0108] where the Dirac symbol "|〉" represents a vector, |0〉 represents the vector (10) T , |1〉 represents the vector (01) T . After the feature mapping, the adjacent inputs are connected by the tensor product. The D4 wavelet acts on four adjacent data each time. Referring to Figure 10 , for the tensor product composed of any four adjacent quantum states the process of coarse-graining by the adjacent first disentangler (located Figure 10 on the left), the second disentangler (located Figure 10 on the right), and the first isometry to which both the first disentangler and the second disentangler are connected includes the steps:

[0109] S31. The first disentangler disentangles the tensor product and outputs the second term containing only a single |1〉 state to the first isometry; the second disentangler disentangles the tensor product and outputs the first term containing only a single |1〉 state to the first isometry;

[0110] S32. The first isometry performs a tensor product operation on the two input terms each containing only a single |1〉 state, and outputs the term containing only a single |1〉 state to the disentangler in the next-layer network structure connected to this first isometry.

[0111] More specifically, the first disentangler contracts the tensor product according to its own definition, expressed as

[0112]

[0113] When there are terms containing multiple |1> states in the input vector, wavelet MERA no longer encodes the D4 wavelet transform. Therefore, the terms of all |0> states and those containing multiple |1> states are removed, and only the terms containing a single |1> state are retained. Equation (11) can be further derived as follows:

[0114]

[0115] It is obvious that the result obtained from Equation (12) is an entangled state. Therefore, two cases need to be considered, as Figure 9 shown. For four adjacent input data |φ(x 2i )>, |φ(x 2i+1 )>, |φ(x 2i+2 )>, |φ(x 2i+3 )〉, when i = 0, |φ(x2)> and |φ(x3)> are in the last two positions in a single scaling function operation; when i = 1, |φ(x2)> and |φ(x3)> are in the first two positions. Therefore, the result of Equation (12) contains the results obtained from the two scaling function operations of the adjacent inputs |φ(x 2i )> and |φ(x 2i+1 )>. is the result obtained when |φ(x 2i )> and |φ(x 2i+1 ) are the last two inputs in the scaling function operation (as Figure 9 (a) shown), is the result obtained when |φ(x 2i )> and |φ(x 2i+1 )> are the first two inputs (as Figure 9 (b) shown).

[0116] The disentanglement process of the first disentangler is summarized by the formula as:

[0117]

[0118] where, (x 2i+1 sinθ u +x 2i cosθ u )|1> and (x 2i+1 cosθ u -x 2i sinθ u )|1> are respectively the first and second terms containing only a single |1> state output by the first disentangler;

[0119] |φ(x 2i+2 )> and |φ(x 2i+3) Similarly, the first and second terms of the output of the second disentangler that contain only a single |1> state are respectively (x 2i+3 sinθ u +x 2i+2 cosθ u )|1>, (x 2i+3 cosθ u -x 2i+2 sinθ u )|1>.

[0120] Then, according to its own definition, the first isometry performs a tensor product on the second term of the output of the first disentangler and the first term of the output of the second disentangler to obtain a term that contains only a single |1> state, which is expressed by the formula as:

[0121]

[0122] Among them, the coefficient acting on the original input data (x 2i x 2i+1 x 2i+2 x 2i+3 ) is:

[0123]

[0124] Therefore, the coefficient of the |1> state in formula (13) is equal to the numerical result obtained by the original data through the D4 wavelet transform scaling function shown in formula (2).

[0125] After knowing how to use waveletMERA to implement the D4 wavelet transform, the number of layers of waveletMERA can be adjusted according to the task to achieve the best balance between classification accuracy and the number of parameters.

[0126] It should be noted that in this example, step S1 adopts the "zigzag" flattening method to make adjacent pixels in the space as close to each other as possible, as Figure 11 shown, the dot represents the starting point of flattening, and the arrow represents the ending point.

[0127] The purpose of step S3 is to coarsen the original input data for input into the trained tensor classification network to obtain the predicted classification result (step S4). Specifically, step S4 includes the steps:

[0128] S41. Retain the coefficients of all terms with |1> states in the coarsened output obtained in step S3, and use the deviation normalization method to normalize these L coefficient values to the interval [0,1] to obtain L normalized values;

[0129] S42. Perform feature mapping on the L normalized values:

[0130] S43. Connect the L eigenvalues obtained in step S42 through tensor product to obtain the global feature map corresponding to the medical two-dimensional grayscale image;

[0131] S44. Input the global feature map into the trained tensor classification network to predict the classification result of the medical two-dimensional grayscale image.

[0132] The feature mapping process of each pixel in the image provides non-linear expansion for the tensor network, similar to the activation function in machine learning, enabling the tensor network model to solve complex problems. The sine local feature mapping is selected in step S42:

[0133]

[0134] s = 1,..., d. When d = 2, we get:

[0135]

[0136] where x refers to any one of the L normalized values, and φ(x) represents the eigenvalue obtained after mapping x.

[0137] Perform the feature mapping shown in formula (16) on the values obtained in step S41 and connect them through tensor product to get:

[0138]

[0139] It should be noted that during the training process, the processing of each sample in the dataset is the same as steps S1 - S43, and steps S1 - S4 are implemented by the network model shown in Figure 12 . However, before applying this model, it is necessary to train the tensor classification network to fix its parameters.

[0140] In this example, the tensor classification network adopts the matrix product state MPS, as shown in Figure 13 . Except that the physical index of the central tensor is used as the output index of the network, the index leg of each input vector φ(x n ) is connected to the physical index leg of each 3rd-order tensor in the MPS, indicating that it will be contracted. The MPS used in this example first performs horizontal contraction between the input data and the MPS, and then performs vertical contraction between the weight tensors. It can be expressed as a linear decision function:

[0141]

[0142] where f j (x) represents the prediction result of the tensor classification network for the jth sample, and Φ(x) jrepresents the global feature mapping of the j-th sample in the dataset (the obtaining process is the same as steps S1 to S43), W represents the weight matrix, which consists of L weight tensors where n = 0, 1, 2,..., L. After being trained with the dataset, W remains fixed.

[0143] During the process of training the tensor classification network, the cross-entropy loss function is used to calculate the distance between the prediction result of the tensor classification network and the true label, and the backpropagation algorithm is used to update the weight tensors in the tensor classification network; the cross-entropy loss function is expressed as:

[0144]

[0145] where y j represents the label of the j-th sample, with the positive class being 1 and the negative class being 0; p j represents the probability that the j-th sample is predicted as the positive class.

[0146] In this example, the MNIST dataset, the Covid-19 dataset, and the LIDC dataset are used to verify the classification ability of the method proposed in this example, and a comparative experiment is conducted with other TNs (including MPS and TTN), classical NNs, and some state-of-the-art models. First, the classification accuracy and the number of parameters of waveletMERA are compared with four other models in the MNIST dataset. Then, in the other three medical image datasets, this example compares the number of parameters, accuracy, sensitivity, specificity, balanced accuracy (BA), and the approximate time for a single iteration, and plots the confusion matrix of waveletMERA on each dataset, which consists of four metrics: True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN), to assist in evaluating the classification ability of the waveletMERA model. Accuracy is the ratio of the number of samples correctly classified by the model to the total number of samples (see formula (20)). Specificity is defined as the ratio of the actual number of negative samples in the sample to the number of predicted negative results (see formula (21)). Sensitivity is a measure of the proportion of actual positive cases predicted as positive (see formula (22)). BA can help determine whether the proportion of positive and negative samples in the dataset is balanced (see formula (23)).

[0147]

[0148]

[0149]

[0150]

[0151] To make the experimental results more comparable, the dimension of the virtual index is set to χ = 6 for all TNs used in the experiment. All experiments were conducted on an Inter(R) Core(TM) i7-10700 CPU@2.90GHz and simulated by Python.

[0152] Table 1. Comparative test results on the MNIST dataset

[0153]

[0154] The MNIST dataset is widely used to verify the basic classification ability of machine learning models and contains a large number of grayscale images of handwritten digits with a size of 28×28. Currently, a method has proposed a trainable network that combines a classical neural network with a tensor network, called Hybrid Tensor Network (HTN), and compared the classification accuracy and the number of parameters of several TNs and classical NNs on the MNIST dataset. In this example, the waveletMERA tensor network proposed in this paper is added, as shown in Table 1.

[0155] Since the original size of the MNIST data samples is not large, only one layer of coarse-graining operation is performed on it in this example. The number of parameters of waveletMERA on the MNIST dataset is 2.9×10 4 , and the classification accuracy on the test set can reach 96%. Compared with the MPS without using waveletMERA, the accuracy remains unchanged when the number of parameters is halved. Whether it is the currently relatively successful tensor networks TTN and HTN, or the mature neural network FCN, the number of parameters is much higher than that of waveletMERA, and the accuracy is almost the same.

[0156] Covid-19 pneumonia has emerged recently, and there is no suitable and available dataset in the existing public network resources. Against this backdrop, a research team from Qatar University in Doha and the University of Dhaka in Bangladesh, along with collaborators from Pakistan and Malaysia, worked with doctors to create this CXR image database of Covid-19 positive cases and healthy and viral pneumonia images. In this example, a balanced subset consisting of some Covid-19 positive cases and healthy images was selected from the above database to train waveletMERA and other models for comparative experiments. Specifically, this subset contains 5,112 lung CXR images, among which 2,597 are healthy images and 2,515 are Covid-19 positive case images. Additionally, another 1,091 healthy images and 1,101 Covid-19 positive case images were selected as the test set, totaling 2,192 images. The ratio of positive and negative samples in both the training set and the test set is close to 1:1, with a size of 128×128, and there are no duplicate images.

[0157] First, in this example, with the number of layers of MERA in waveletMERA (i.e., the number of wavelet transforms) as the variable and the number of parameters and the classification accuracy on the test set as the two measurement indicators, a comparative experiment was conducted using the Covid-19 dataset, and the results are shown in Table 2. The results indicate that for the Covid-19 dataset with a size of 128×128, when the number of layers is 4, a balance can be achieved between the reduction in the number of parameters and the decline in accuracy. On average, it takes 1 second for each lung CXR image to pass through 4-layer waveletMERA. Figure 14 Shows the CXR images of the lungs before and after 4-layer waveletMERA transformation.

[0158] Table 2. Number of parameters and accuracy that can be achieved with different numbers of waveletMERA layers on the Covid-19 dataset

[0159]

[0160] In addition to waveletMERA, six network structures were experimented with in this example, including those using the control variable method for comparative experiments, such as: MPS, db2+MPS, MLP, db2+MLP, and the current state-of-the-art models that can classify Covid-19 positive cases from lung CXR images, such as: patch-GTNC and Fused-DenseNet-Tiny. The MPS model used in the experiment is the same as the MPS that acts as the trainable network (fully connected layer) in waveletMERA, aiming to verify the effectiveness of the coarsening process in waveletMERA. db2+MPS replaces waveletMERA with the db2 wavelet transform function in the pywavelets library to compare waveletMERA with ordinary wavelet transforms. db2+MLP serves as the control group for ordinary wavelets + classical neural networks. The db2 wavelet transform function in the Pywavelets library has the same scaling function and wavelet function as the D4 wavelet, and the mirror method is used to handle edge problems. Fused-DenseNet-Tiny currently performs well in the task of classifying COVID-19 from CXR images, with high model accuracy and few parameters, but the model is very abstract and invisible. Table 3 shows the results of the comparative experiment, where the time for each iteration of waveletMERA only includes the training phase, Figure 15 and gives the confusion matrix of waveletMERA on the Covid-19 test set.

[0161] Table 3. Comparative experiment results on the Covid-19 dataset

[0162]

[0163] Compared with MPS, the 4-layer wavelet MERA not only reduces the number of parameters by more than one million, but also significantly improves all indicators, proving that the coarsening of wavelet MERA can extract more valuable information from the original image; due to the different ways of dealing with edge problems, the number of parameters of db2 + MPS is slightly larger than that of wavelet MERA, but the classification effect is significantly worse than that of wavelet MERA, thus proving that the ability of wavelet MERA to extract features is stronger than that of ordinary wavelets. Patch-GTNC uses "patches" to retain the global structural information of the original image and is currently a tensor network model with better performance in the task of classifying Covid-19 positive cases based on X-ray images. However, both the number of parameters and the classification performance are inferior to wavelet MERA. MLP and db2 + MLP, as classical control methods, have fewer parameters than MPS and wavelet MERA, but all indicators are inferior to the tensor network method. The lightweight deep neural network Fused-DenseNet-Tiny is comparable to wavelet MERA in all indicators, but the number of parameters is still very large.

[0164] Of course, the credibility of classifying and identifying COVID-19 through CXR images also needs to be considered. Due to the particularity of medical images, the dataset may come from the same hospital, clinic, CT scanner, or even the same doctor. Inevitably, these information will become the basis for the network to make classification decisions, rather than fully focusing on the lung region of the CXR image. Therefore, the classification decisions made based on image features outside the lung region may be unreliable. According to the work of Tartaglione, the independent lung region can be segmented from the CXR image as the training data for the COVID-19 image classifier, thus removing the redundant features in the original dataset. In this example, the U-Net image segmentation network is selected and trained using the Lung-CXR dataset. Through this method, a U-Net model that can segment the lung region from the CXR image is obtained, with an accuracy of 96%. Using this model to segment the CXR images in the Covid-19 dataset, binary segmentation results can be obtained, and then the original pixels in the segmented lung region are retained to obtain the training data for this experiment, as shown in Figure 16 , where: (a) the original lung X-ray image in the Covid-19 dataset; (b) the binary lung image after segmentation using U-net; (c) the original pixels retaining the lung region.

[0165] After the above processing of the original Covid-19 dataset, a total of 704 healthy lung images and 691 Covid-19 positive case images were finally retained as the training set, and 291 healthy images and 308 Covid-19 positive case images were used as the test set. In this example, this dataset is called the Covid-19-mask dataset. Similar to the Covid-19 dataset, the ratio of positive to negative samples is close to 1:1, and the size is 128×128. Through the experiments in Table 2, in this example, the number of layers of waveletMERA was also set to 4 on this dataset, and the same comparison experiments as those on the Covid-19 dataset were carried out. As shown in Table 4, the time for each iteration of waveletMERA only includes the training phase. The confusion matrix of waveletMERA on the Covid-19-mask test set is as Figure 17 shown.

[0166] Table 4. Comparison experiment results on the Covid-19-mask dataset

[0167]

[0168] The number of pictures in Covid-19-mask is less than that in Covid-19, which is more in line with the actual situation of medical image datasets. In the experiment, overfitting occurred in MPS, db2+MPS, MLP, and db2+MLP. The performance of Patch-GTNC and Fused-DenseNet-Tiny on the Covid-19-mask dataset was significantly worse than that on the Covid-19 dataset, indicating that the classification decisions made by these two models on the Covid-19 dataset do not completely rely on the pixels in the lung region, and the sensitivity of these two models is much higher than the specificity, which will cause a high missed diagnosis rate and delay the treatment opportunity. waveletMERA not only has the lowest number of parameters among the 6 models, but also can achieve an accuracy of 99.50%, and the difference between sensitivity and specificity is not significant.

[0169] In addition to the dataset related to the novel coronavirus, this example also verified the effectiveness of the waveletMERA model on other medical datasets. The LIDC dataset is a dataset containing clinical chest CT scans with annotated lesion areas. All lung nodules in the dataset are sized between 3mm and 33mm. In this example, according to the malignancy degree annotated by doctors for each image, the LIDC dataset is divided into two categories: benign and malignant. Among them, the training set consists of 2031 malignant images and 1928 benign images, and the test set consists of 840 malignant images and 858 benign images. They are all 64×64 grayscale images, and the ratio of positive to negative samples is close to 1:1. In the experiment, the number of layers of waveletMERA in this example is set to 2, and the coarsening time for each image on average is 0.2 seconds, which can achieve the optimal balance between the number of parameters and the accuracy. In the control experiment, the number of transformation times of the db2 wavelet function is also set to 2 times. In Table 5, this example presents the comparison experiment results of waveletMERA with 6 other models. The time for each iteration of waveletMERA only includes the training phase. Among them, the LoTeNet* model adds convolutional operations and fully connected layers on the basis of the LoTeNet model. The confusion matrix of waveletMERA on the LIDC test set is as Figure 18 shown.

[0170] Table 5. Comparison experiment results on the LIDC dataset

[0171]

[0172] Compared with the Covid-19 dataset, the classification difficulty of the LIDC dataset is higher. WaveletMERA can still maintain its advantages of having fewer parameters and higher accuracy, reducing the number of parameters by more than 200,000, and achieving an accuracy of 98.47% on the test set. On the contrary, the accuracies of the other 6 models are all lower than 80%.

[0173] In summary, a medical image classification method combining wavelet transform and tensor network provided by an embodiment of the present invention proposes a coarsening network that performs excellently on medical images. This coarsening network combines wavelet transform and MERA, that is, encodes the D4 wavelet into MERA to form a model with fixed internal parameters, which the present invention names waveletMERA. The present invention also constructs a trainable tensor network (i.e., a tensor classification network) like a fully connected layer. The present invention uses the MNIST dataset, the Covid-19 dataset, and the LIDC dataset for multi-dimensional verification. The results show that the accuracy of waveletMERA is stably high, and it has better coarsening ability than the deep neural network of CNNs. This ability enables waveletMERA to reduce the number of model parameters to a greater extent while ensuring accuracy. The results show that waveletMERA is not only superior to the current mainstream deep neural networks in classification but also superior to ordinary wavelet transform in data preprocessing. Moreover, waveletMERA also has the interpretability advantage of the tensor network itself, which can provide more targeted and efficient diagnostic assistance for doctors, alleviating the problems of medical tension and doctor-patient conflicts.

[0174] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A medical image classification method combining wavelet transform and tensor network, characterized in that, Includes steps: S1. Flatten a medical two-dimensional grayscale image with N pixels into a one-dimensional vector v(x) with a length of N; S2. Each element x in the one-dimensional vector v(x) i is feature-mapped to a quantum state in the Hilbert space, obtaining N quantum states; in step S2, the i-th element x in the one-dimensional vector v(x) i is mapped to: |φ(x i )> = |0> + x i |1>; S3, performing L times of coarse-graining processing on N quantum states through a coarse-graining network combining wavelet transform and tensor network, and outputting an item containing only a single |1> state each time, thereby obtaining L items containing only a single |1> state as coarse-graining output; in the step S3, the coarse-graining network is constructed based on a coarse-graining unit, and the coarse-graining unit includes two disentanglers and an isometry; the two disentanglers are used to input four adjacent quantum states of the N quantum states obtained in step S2 connected by tensor product, and each obtains a term containing only a single |1> state through disentanglement; the isometry is used to perform coarse-graining operation on the two items containing only a single |1> state output by the two disentanglers, and obtain a corresponding term containing only a single |1> state; S4. Map the coarse-grained output feature into multiple feature values ​​and connect them through tensor products and input them into the trained tensor classification network to obtain the predicted classification result.

2. The medical image classification method combining wavelet transform and tensor network according to claim 1, wherein Let U and V represent the disentangler and isometry respectively, then U and V are defined as: where θ u = -π / 6, θ v = π / 12.

3. The medical image classification method combining wavelet transform and tensor network according to claim 2, characterized in that: The coarse-grained network is composed of a multi-layer network structure of disentanglers and isometry, each layer of the network structure contains more than two disentanglers and more than one isometry, each disentangler has two input indicators and two output indicators, and each isometry has two input indicators and one output indicator; the last disentangler of each layer of the network structure is connected to the first disentangler through an additional isometry; In a non-top-level network structure, two equally spaced input indicators are connected to an output indicator of two adjacent disentanglers, and two output indicators of a disentangler are connected to an input indicator of two adjacent equally spaced ones. Each equally spaced output indicator in the same layer is connected to an input indicator of a disentangler in the next layer of the network structure. In the bottom network structure, the four input indices of each two adjacent disentanglers are used to input each four adjacent quantum states of the N quantum states obtained in step S2 connected by tensor products; Each disentangler is used to preserve the entanglement of the two quantum states under the tensor product connection of the input, and to disentangle the two quantum states from the quantum states of other disentanglers input. Each disentangler outputs the first item and the second item containing only a single |1> state from two output indicators, respectively, where the first item and the second item are the results obtained when the two quantum states are in the last two digits and the first two digits of the input in the scaling function operation, respectively. Each isometry is used to perform a coarse-grained operation on the two terms containing only a single |1> state under the tensor product connection of the input, and obtain the result of the D4 wavelet transform of the four adjacent quantum states input to the two disentanglers connected to it.

4. A medical image classification method combining wavelet transform and tensor network according to claim 1, characterized in that, In step S3, the tensor product formed by four adjacent quantum states The process of coarse-graining by an adjacent first disentangler, a second disentangler, and a first isometry to which both the first disentangler and the second disentangler are connected includes the steps: S31. The first disentangler performs disentanglement on the tensor product and outputs it to the second to the first isometry that only contains a single |1> state; The second disentangler performs disentanglement on the tensor product and outputs the first term to the first isometry containing only a single |1> state; S32. The first isometric operation performs a tensor product operation on two input items that only contain a single |1> state, and obtains an item that only contains a single |1> state, which is input into a disentangler in the next layer of the network structure connected by the first isometric connection.

5. A medical image classification method combining wavelet transform and tensor network according to claim 4, characterized in that The first disentangler performs disentanglement on the tensor product according to its own definition which is expressed by the formula as follows: where, (x 2i+1 sinθ u +x 2i cosθ u )|1> and (x 2i+1 cosθ u -x 2i sinθ u )|1> are respectively the first term and the second term containing only a single |1> state output by the first disentangler; Similarly, the first and second terms that only contain a single |1> state output by the second disentangler are respectively (x 2i+3 sinθ u +x 2i+2 cosθ u )|1> and (x 2i+3 cosθ u -x 2i+2 sinθ u )|1>: the first isometry performs a tensor product on the second term output by the first disentangler and the first term output by the second disentangler according to its own definition to obtain a term containing only a single |1> state, which is expressed by the formula: Among them, the coefficients acting on the original input data (x 2i x 2i+1 x 2i+2 x 2i+3 ) are as follows:

6. A medical image classification method combining wavelet transform and tensor network according to any one of claims 1 to 5, characterized in that, The specific steps of step S4 include: S41. Retain the coefficients of all terms with |1> states in the coarsened output obtained in step S3, and use the deviation normalization method to normalize these L coefficient values to the interval [0,1] to obtain L normalized values; S42. Perform feature mapping on the L normalized values using the following formula: where x refers to any one of the L normalized values, and φ(x) represents the eigenvalue obtained after mapping x; S43. Connect the L eigenvalues obtained in step S42 through tensor product to obtain the global feature mapping corresponding to the medical two-dimensional grayscale image; S44. Input the global feature mapping into the trained tensor classification network to predict the classification result of the medical two-dimensional grayscale image.

7. A medical image classification method combining wavelet transform and tensor network according to claim 1, characterized in that, The tensor classification network adopts the matrix product state and is represented by a linear decision function: Among them, f j (x) represents the prediction result of the tensor classification network for the j-th sample, and Φ(x) j represents the global feature map of the j-th sample in the dataset. W represents the weight matrix, which is composed of L weight tensors where n = 0, 1, 2,..., L. After being trained by the dataset, W remains fixed.

8. A medical image classification method combining wavelet transform and tensor network according to claim 7, characterized in that, During the process of training the tensor classification network, the cross-entropy loss function is used to calculate the distance between the prediction result of the tensor classification network and the true label, and the backpropagation algorithm is used to update the weight tensor in the tensor classification network; the cross-entropy loss function is expressed as: where y j represents the label of the j-th sample, with the positive class being 1 and the negative class being 0; p j represents the probability that the j-th sample is predicted as the positive class.

Images