A COVID-19 medical imaging detection and classification method and device
Through a convolutional neural network optimized based on particle swarm algorithm, the problem of time-consuming and subjective errors in medical image detection in COVID-19 is solved, and efficient and accurate image classification is achieved, with a wide range of application and low cost.
Patent Information
- Application Number
- CN202111140377.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-09-28
AI Technical Summary
The existing medical imaging detection methods for COVID-19 have problems such as long time and subjective diagnostic errors, especially when CT images are manually diagnosed, it is difficult to quickly and accurately identify the lesions.
A convolutional neural network optimized based on particle swarm algorithm is adopted to obtain and process medical image data sets, select hyperparameters and train convolutional neural networks to achieve efficient classification of COVID-19 images.
It improves the efficiency and accuracy of COVID-19 image detection, reduces costs, solves the problems of time-consuming and subjective errors in manual diagnosis, and achieves efficient and low-cost lesion recognition.
Smart Images

Figure CN113989543B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a COVID-19 medical imaging detection classification method and device, and belongs to the field of medical imaging detection and deep learning technology. Background Art
[0002] Corona Virus Disease 2019 (COVID-19) is an infectious disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), referred to as "new crown pneumonia". Since the end of 2019, COVID-19 has broken out on a large scale in countries around the world. The current mainstream detection and diagnosis methods mainly include nucleic acid testing and manual diagnosis of medical images. However, since nucleic acid testing takes a long time and nucleic acid testing kits are in short supply at the beginning of the outbreak, and manual diagnosis of medical images has the problems of long analysis time, hidden lesions are difficult to detect, and subjective diagnostic errors are inevitable, new COVID-19 detection methods are urgently needed.
[0003] As the name of COVID-19 suggests, the lungs are the main part attacked by the new coronavirus. Compared with the lungs of normal people, the lungs of patients with COVID-19 have "lesions" like cloudy frosted glass, which are medically called "ground glass shadows". However, the lung is an internal organ, and it is difficult for doctors to directly observe the appearance and location of the lesions, so imaging examinations are needed. According to the research of the Infectious Diseases Group of the Radiology Branch of the Chinese Medical Association and other departments (2020), lung imaging manifestations appear earlier than clinical symptoms, and lung CT is highly sensitive for the early diagnosis of COVID-19 patients. The changes in clinical symptoms and changes in lung CT lesions are temporally consistent, which is a simpler and faster indicator for judging the changes in the disease. However, since manual detection methods are time-consuming and subjective diagnostic errors are inevitable, deep learning image processing related technologies can be combined to screen out diseased lungs from CT, and then track down COVID-19 infected people, so as to isolate and treat patients. Summary of the invention
[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a COVID-19 medical image detection and classification method and device. With the help of deep learning, efficient automatic identification of COVID-19 medical images can be achieved. Deep learning combined with swarm intelligence optimization algorithm can further improve the image classification accuracy.
[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:
[0006] In a first aspect, the present invention provides a COVID-19 medical image detection and classification method and device, comprising:
[0007] Obtain medical imaging datasets for COVID-19 detection and diagnosis;
[0008] Processing medical image data sets, and dividing the processed medical image data sets;
[0009] Based on the particle swarm algorithm to optimize the convolutional neural network, first select the hyperparameters that can be optimized in the convolutional neural network, and then determine the optimal solution of the hyperparameters to be optimized in this experiment;
[0010] The medical imaging dataset is divided by training the convolutional neural network optimized by the particle swarm algorithm to batch recognize images and classify them correctly.
[0011] Furthermore, the medical image data set is processed, including: selecting images with clear and identifiable lesion features by naked eyes.
[0012] Furthermore, the processed medical imaging dataset is divided into:
[0013] The labeled images are divided into training set, test set and validation set in a ratio of 3:1:1;
[0014] The processed medical imaging datasets are divided into two categories, one is the COVID dataset and the other is the NonCOVID dataset.
[0015] Furthermore, the convolutional neural network is optimized based on the particle swarm algorithm, including:
[0016] (1) In the particle swarm algorithm, each particle can be regarded as a point in the solution space. If the size of the particle group is N, the position of the i-th (i=1, 2, ..., N) particle can be recorded as X i , the speed is recorded as V i , the "best" position it has ever experienced is recorded as p best (i), the position of the “best” particle in the population is denoted by g best (i), particle i updates its velocity and position according to the following formula:
[0017]
[0018] Among them, V' i and X' i are the updated speed and position, ω is the inertia weight; c1, c2 are acceleration constants; r is a random number on [0,1]. In addition, when the particle continuously adjusts its position according to the speed, it is affected by the maximum speed V max When V i Exceed V max When V max ;
[0019] (2) During the particle search process, the fitness value Fitness i To achieve the minimum, that is, the mean square error is minimized, and its formula is as follows:
[0020]
[0021] Among them, N is the number of training set samples, M is the number of output network neurons, is the actual state value of the jth output network node of the ith sample. When the error between the actual output and the expected output is less than the expected error value or the training reaches the maximum number of iterations, the algorithm terminates, that is, the training ends.
[0022] (3) The process of basic particle swarm algorithm is as follows:
[0023] ① According to the initialization process, the random position and speed of the particle group are initially set;
[0024] ② Calculate the fitness value of each particle;
[0025] ③For each particle, compare its fitness value with the best position p it has ever experienced best The fitness value of is compared, if it is better, it is taken as the current best position;
[0026] ④For each particle, compare its fitness value with the best position g experienced globally best The fitness value of is compared, if it is better, it is taken as the current global best position;
[0027] ⑤ Update the particle's speed and position according to the speed update formula and position update formula;
[0028] ⑥ If the end condition is not met, usually a sufficiently good adaptation threshold or a preset maximum number of generations (G max ), then return to step ②.
[0029] Furthermore, the convolutional neural network adopts a CNN network structure.
[0030] Furthermore, when selecting hyperparameters in the convolutional neural network, modeling is chosen based on the LeNet-5 network, wherein the LeNet-5 network has a total of 7 layers, including 3 convolutional layers, 2 pooling layers and 2 fully connected layers.
[0031] Furthermore, the hyperparameters that need to be optimized in the convolutional neural network include the size and number of convolution kernels, the pooling size, and the pooling step size.
[0032] In a second aspect, the present invention provides a COVID-19 medical image detection and classification device, comprising:
[0033] A medical imaging dataset acquisition unit, used to acquire medical imaging datasets for COVID-19 detection and diagnosis;
[0034] A medical image data set processing and division unit, used for processing the medical image data set and dividing the processed medical image data set;
[0035] The hyperparameter selection and determination unit is used for the convolutional neural network optimized based on the particle swarm algorithm. It first selects the hyperparameters that can be optimized in the convolutional neural network, and then determines the optimal solution of the hyperparameters that need to be optimized in this experiment;
[0036] The training recognition unit is used to train the divided medical imaging dataset based on the convolutional neural network optimized by the particle swarm algorithm, batch recognize the images and classify them correctly.
[0037] In a third aspect, the present invention provides a COVID-19 medical image detection and classification device, including a processor and a storage medium;
[0038] The storage medium is used to store instructions;
[0039] The processor is used to operate according to the instructions to execute the steps of any of the methods described above.
[0040] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] The present invention utilizes a convolutional neural network based on a particle swarm algorithm to improve the efficiency of identifying whether a person is infected with COVID-19, and solves the shortcomings of manual diagnosis of whether a CT image is infected with the 2019 coronavirus pneumonia, which takes a long time and has subjective diagnostic errors. The method has low cost, high accuracy, and a wide range of applications. The present invention uses a particle swarm algorithm to optimize a convolutional neural network, and then trains COVID-19 CT images through the convolutional neural network, so that the network can efficiently identify and classify whether clinical test subjects are infected with the 2019 coronavirus pneumonia in batches. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a flowchart of a COVID-19 medical imaging detection and classification method provided by an embodiment of the present invention;
[0044] Figure 2 This is an example image of a COVID-19 dataset provided by an embodiment of the present invention;
[0045] Figure 3is a CT_COVID example diagram provided by an embodiment of the present invention;
[0046] Figure 4 It is a CT_NonCOVID example diagram provided by an embodiment of the present invention;
[0047] Figure 5 is a flow chart of a general form of a particle swarm algorithm provided by an embodiment of the present invention;
[0048] Figure 6 is a graph of training and testing accuracy results provided by an embodiment of the present invention;
[0049] Figure 7 is a graph of training and test loss results provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0051] Example 1
[0052] This embodiment introduces a COVID-19 medical image detection and classification method and device, including:
[0053] Obtain medical imaging datasets for COVID-19 detection and diagnosis;
[0054] Processing medical image data sets, and dividing the processed medical image data sets;
[0055] Based on the particle swarm algorithm to optimize the convolutional neural network, first select the hyperparameters that can be optimized in the convolutional neural network, and then determine the optimal solution of the hyperparameters that need to be optimized in this experiment;
[0056] The medical imaging dataset is divided by training the convolutional neural network optimized by the particle swarm algorithm to batch recognize images and classify them correctly.
[0057] The application process of the COVID-19 medical image detection and classification method and device provided in this embodiment specifically involves the following steps:
[0058] 1. Prepare the dataset:
[0059] The dataset used in this article comes from the virus information database of the National Center for Bioinformatics, which has been certified by the National Professional Research Center. It contains 5,000 COVID-19 CT images and 5,000 normal CT images. When processing the dataset, first select images with clear and visible lesion features from the dataset. When dividing the dataset, the labeled images are allocated to the training set, test set, and validation set in a ratio of 3:1:1. The image distribution is shown in Table 1:
[0060] Table 1
[0061] Type NonCOVID-19 COVID-19 Total Train 3000 3000 6000 Val 1000 1000 2000 Test 1000 1000 2000
[0062] Finally, all processed datasets are divided into two categories, one is the COVID dataset and the other is the NonCOVID dataset. The COVID dataset includes 3,000 training images, 1,000 verification images, and 1,000 test images; the NonCOVID dataset includes 3,000 training images, 1,000 verification images, and 1,000 test images. Figure 2 shows an example of COVID-19 dataset images, Figure 3 and Figure 4 Example images of CT_COVID and CT_NonCOVID are shown respectively.
[0063] 2. Particle swarm algorithm optimizes convolutional neural networks and the selection of hyperparameters in convolutional neural networks.
[0064] (1) Particle Swarm Optimization of Convolutional Neural Networks
[0065] 1) In the particle swarm algorithm, each particle can be regarded as a point in the solution space. If the size of the particle group is N, the position of the i-th (i=1, 2, ..., N) particle can be recorded as X i , the speed is recorded as V i , the "best" position it has ever experienced is recorded as p best (i), the position of the “best” particle in the population is denoted by g best (i). Particle i updates its velocity and position according to the following formula:
[0066]
[0067] Among them, V' i and X' i are the updated speed and position, ω is the inertia weight, c1 and c2 are acceleration constants, and r is a random number on [0,1]. In addition, when the particle continuously adjusts its position according to the speed, it is affected by the maximum speed V max When V i Exceed V maxWhen V max .
[0068] 2) During the particle search process, the fitness value Fitness i To achieve the minimum, that is, the mean square error is minimized, and its formula is as follows:
[0069]
[0070] Among them, N is the number of training set samples, M is the number of output network neurons, is the actual state value of the jth output network node of the ith sample. When the error between the actual output and the expected output is less than the expected error value or the training reaches the maximum number of iterations, the algorithm terminates and the training ends.
[0071] 3) The process of basic particle swarm algorithm is:
[0072] ① According to the initialization process, the random position and speed of the particle group are initially set;
[0073] ② Calculate the fitness value of each particle;
[0074] ③For each particle, compare its fitness value with the best position p it has ever experienced best The fitness value of is compared, if it is better, it is taken as the current best position;
[0075] ④For each particle, compare its fitness value with the best position g experienced globally best The fitness value of is compared, if it is better, it is taken as the current global best position;
[0076] ⑤ Update the particle's speed and position according to the speed update formula and position update formula;
[0077] ⑥ If the end condition is not met, usually a sufficiently good adaptation threshold or a preset maximum number of generations (G max ), then return to step ②.
[0078] Figure 5 The flowchart of the general form of particle swarm algorithm.
[0079] The mathematical description of the particle swarm optimization algorithm is: the D-dimensional space of the problem to be solved by particle search, the size of the population is N, and generally the solution effect of the particle swarm algorithm becomes better with the increase of the population size, but at the same time, as the population expands, the solution time will also increase. For this reason, the population size selected in the present invention is 4. In addition, the maximum number of iterations is set to 14, the inertia weight ω = 0.5, and the learning factor c1 = c2 = 0.5.
[0080] (2) Selection of hyperparameters in convolutional neural networks
[0081] The difference between a CNN neural network and a common neural network is that a convolutional neural network contains a feature extractor consisting of a convolutional layer and a subsampling layer. In the convolutional layer of a convolutional neural network, a neuron is only connected to some neurons in the adjacent layers. In a convolutional layer of a CNN, it usually contains several feature planes, each of which consists of some neurons arranged in a matrix. Neurons in the same feature plane share weights, namely, convolution kernels.
[0082] This embodiment chooses to model on the basis of LeNet-5 when selecting hyperparameters in the convolutional neural network. Taking LeNet-5 as an example, combined with experimental data, the influence of CNN hyperparameters on network performance is explained, and then the influence of CNN network structure on the network is explained. The LeNet-5 network has a total of 7 layers, including 3 convolutional layers, 2 pooling layers, and 2 fully connected layers. The first convolutional layer uses 6 5×5 kernels, and the activation function is ReLU; the pooling size of the first pooling layer is 2×2, and the step size is 2×2; the second convolutional layer uses a 5×5 convolution kernel, and the activation function is ReLU; the pooling size of the second pooling layer is 2×2, and the step size is 2×2; the next convolutional layer uses 120 2×2 convolution kernels, and the activation function is ReLU; the first fully connected layer has 84 nodes; the second fully connected layer has 2 nodes. On the MNIST dataset, the hyperparameters selected in LeNet-5 are shown in the following table:
[0083] Table 2
[0084] symbol Hyperparameters lr Learning Rate n Number of convolution kernels in C1 sf Convolution kernel size in C1 sp Pooling size in P1 l Pooling step size in P1 Batchsize Batch size
[0085] When Epoch=1, the accuracy of LeNet-5 on the MNIST dataset is shown in Table 3. Epoch represents the number of times the image is traversed, also known as "period".
[0086] Table 3
[0087]
[0088]
[0089] The learning rate of CNN determines the speed of network learning and affects the performance of the network. The learning rates of LeNet-5-1, LeNet-5-2 and LeNet-5-3 are different, and the recognition accuracy is also different. As the learning rate increases, the CNN learns faster, but the network performance will first improve and then decrease.
[0090] The number of convolution kernels affects the classification accuracy of the model. LeNet-5-1 and LeNet-5-4 have different numbers of convolution kernels, and their recognition accuracy is also different. Experimental results verify that for models of the same depth, the larger the number of feature planes, the better the classification performance, but it will increase the computational cost. The size of the convolution kernel will directly affect the number and capacity of parameters of the convolutional neural network. LeNet-5-1 and LeNet-5-5 have different sizes of convolution kernels, and their recognition accuracy is also different.
[0091] The pooling layer is usually connected after the convolution layer to reduce information redundancy. The pooling sizes of LeNet-5-7, LeNet-5-1 and LeNet-5-6 gradually increase, and they are different, and the recognition accuracy is also different. It can be found that as the pooling size increases, the network performance gradually decreases. Therefore, the pooling size in the network structure is generally taken as a smaller value, usually 1*1.
[0092] From LeNet-5-1 and LeNet-5-8, we can see that different pooling steps lead to different recognition accuracy. In addition, as the pooling step increases, the image will lose information, which will reduce the final recognition accuracy.
[0093] The larger the batch size, the more memory is required. In actual situations, the batch size will be limited by the video memory, so it is necessary to balance the relationship between the two. At the same time, the batch size affects the model classification accuracy and training time. By observing LeNet-5-9, LeNet-5-1 and LeNet-5-10, it can be found that as the batch size increases, the recognition accuracy of the network model gradually decreases.
[0094] When training CNN, the convolution kernel is generally initialized in the form of a random number matrix. During the network training process, the convolution kernel will learn to obtain reasonable weights.
[0095] 3. Convolutional neural network training based on particle swarm optimization algorithm on COVID-19 medical imaging dataset and its results
[0096] Taking lenet-5 as an example, its output is the accuracy and loss of the input image in the test set after a large amount of training on the training set. An accuracy rate higher than 90% means that the particle swarm algorithm is used to optimize the convolutional neural network, which can be used to efficiently identify whether clinical test subjects are infected with the 2019 novel coronavirus in batches.
[0097] The present invention uses the particle swarm algorithm to optimize the convolutional neural network, and then trains the COVID-19 CT images through the convolutional neural network, so that the network can efficiently identify and classify whether clinical test subjects are infected with the 2019 coronavirus disease pneumonia in batches.
[0098] The beneficial effects of the present invention are: the efficiency of identifying whether a person is infected with COVID-19 is improved by using a convolutional neural network based on a particle swarm algorithm, and the shortcomings of manual diagnosis of whether a CT image is infected with 2019 coronavirus pneumonia, which takes a long time and has subjective diagnostic errors, are solved. The method has low cost, high accuracy and a wide range of applications.
[0099] Example 2
[0100] This embodiment provides a COVID-19 medical image detection and classification device, including:
[0101] A medical imaging dataset acquisition unit, used to acquire medical imaging datasets for COVID-19 detection and diagnosis;
[0102] A medical image data set processing and division unit, used for processing the medical image data set and dividing the processed medical image data set;
[0103] The hyperparameter selection and determination unit is used for the convolutional neural network optimized based on the particle swarm algorithm. It first selects the hyperparameters that can be optimized in the convolutional neural network, and then determines the optimal solution of the hyperparameters that need to be optimized in this experiment;
[0104] The training recognition unit is used to train the divided medical imaging dataset based on the convolutional neural network optimized by the particle swarm algorithm, batch recognize the images and classify them correctly.
[0105] Example 3
[0106] This embodiment provides a COVID-19 medical image detection and classification device, including a processor and a storage medium;
[0107] The storage medium is used to store instructions;
[0108] The processor is configured to operate according to the instructions to perform the steps of any of the following methods:
[0109] Obtain medical imaging datasets for COVID-19 detection and diagnosis;
[0110] Processing medical image data sets, and dividing the processed medical image data sets;
[0111] Based on the particle swarm algorithm to optimize the convolutional neural network, first select the hyperparameters that can be optimized in the convolutional neural network, and then determine the optimal solution of the hyperparameters that need to be optimized in this experiment;
[0112] The medical imaging dataset is divided by training the convolutional neural network optimized by the particle swarm algorithm to batch recognize images and classify them correctly.
[0113] Example 4
[0114] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of any of the following methods are implemented:
[0115] Obtain medical imaging datasets for COVID-19 detection and diagnosis;
[0116] Processing medical image data sets, and dividing the processed medical image data sets;
[0117] Based on the particle swarm algorithm to optimize the convolutional neural network, first select the hyperparameters that can be optimized in the convolutional neural network, and then determine the optimal solution of the hyperparameters that need to be optimized in this experiment;
[0118] The medical imaging dataset is divided by training the convolutional neural network optimized by the particle swarm algorithm to batch recognize images and classify them correctly.
[0119] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A COVID-19 medical image detection and classification method, characterized in that: include: Obtain medical imaging datasets for COVID-19 detection and diagnosis; Processing medical image data sets, and dividing the processed medical image data sets; Based on the particle swarm algorithm to optimize the convolutional neural network, first select the hyperparameters that can be optimized in the convolutional neural network, and then determine the optimal solution for the hyperparameters that need to be optimized through experimental comparison; The medical imaging datasets that have been divided by training convolutional neural networks optimized by particle swarm algorithm can recognize images in batches and classify them correctly. Optimize convolutional neural networks based on particle swarm optimization, including: (1) In the particle swarm algorithm, each particle can be regarded as a point in the solution space. If the size of the particle group is N, then The position of a particle can be written as , the speed is recorded as , the "best" position it has ever experienced is recorded as , the position of the "best" particle in the population is recorded as ,particle Update your speed and position according to the following formula: ; in, and are the updated speed and position respectively, is the inertia weight; is the acceleration constant; for In addition, when the particle continuously adjusts its position according to the speed, it is affected by the maximum speed. restrictions; when Exceed When is limited to ; (2) During the particle search process, the fitness value needs to be To achieve the minimum, that is, the mean square error is minimized, and its formula is as follows: ; Among them, N is the number of training set samples, M is the number of output network neurons, is the actual state value of the jth output network node of the ith sample. When the error between the actual output and the expected output is less than the expected error value or the training reaches the maximum number of iterations, the algorithm terminates, that is, the training ends. (3) The process of basic particle swarm algorithm is: ① According to the initialization process, the random position and speed of the particle group are initially set; ② Calculate the fitness value of each particle; ③For each particle, compare its fitness value with the best position it has ever experienced The fitness value of is compared, if it is better, it is taken as the current best position; ④For each particle, compare its fitness value with the best position experienced globally The fitness value of is compared, if it is better, it is taken as the current global best position; ⑤ Update the particle's speed and position according to the speed update formula and position update formula; ⑥ If the end condition is not met, usually a sufficiently good adaptation threshold or a preset maximum number of generations is reached , then return to step ②.
2. The COVID-19 medical imaging detection and classification method according to claim 1, characterized in that: Process medical imaging datasets, including selecting clear images with lesion features that can be distinguished by the naked eye.
3. The COVID-19 medical imaging detection and classification method according to claim 1, characterized in that: The processed medical imaging dataset is divided into: The labeled images are divided into training set, test set and validation set in a ratio of 3:1:1; The processed medical imaging datasets are divided into two categories, one is the COVID dataset and the other is the NonCOVID dataset.
4. The COVID-19 medical imaging detection and classification method according to claim 1, characterized in that: The convolutional neural network adopts a CNN network structure.
5. The COVID-19 medical imaging detection and classification method according to claim 1, characterized in that: When selecting hyperparameters in the convolutional neural network, modeling is performed based on the LeNet-5 network, wherein the LeNet-5 network has a total of 7 layers, including 3 convolutional layers, 2 pooling layers, and 2 fully connected layers.
6. The COVID-19 medical imaging detection and classification method according to claim 1, characterized in that: The hyperparameters that need to be optimized in the convolutional neural network include the size and number of convolution kernels, the pooling size, and the pooling step size.
7. A COVID-19 medical image detection and classification device, using the COVID-19 medical image detection and classification method according to claim 1, characterized in that: include: A medical imaging dataset acquisition unit, used to acquire medical imaging datasets for COVID-19 detection and diagnosis; A medical image data set processing and division unit, used for processing the medical image data set and dividing the processed medical image data set; The hyperparameter selection and determination unit is used for the convolutional neural network optimized based on the particle swarm algorithm. It first selects the hyperparameters that can be optimized in the convolutional neural network, and then determines the optimal solution of the hyperparameters that need to be optimized in this experiment; The training recognition unit is used to train the divided medical imaging dataset based on the convolutional neural network optimized by the particle swarm algorithm, batch recognize the images and classify them correctly.
8. A COVID-19 medical image detection and classification device, characterized by: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
