An intelligent fault diagnosis method based on data privacy protection

Abstract

This invention belongs to the field of federated learning technology, specifically disclosing an intelligent fault diagnosis method based on data privacy protection, comprising the following steps: Step 1, each source client initializes its local model. Step 2, each local model is trained, and the parameters of the feature extractor and two classifiers are updated. Step 3, the global model of the central server receives the parameters of all local models, evaluates the usability of the local models, and then weights and combines the parameters of the local models to form the parameters of the global model. Step 4, the global model is downloaded to each source client, and each client inputs task data into the global model. The global model updates its local models according to its classification loss. Step 5, steps 2-4 are repeated a set number of times, and the final global model is downloaded to the target client. This invention provides an intelligent fault diagnosis method based on data privacy protection that enables local models to identify fault samples with ambiguous class boundaries.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to an intelligent fault diagnosis method based on data privacy protection. Background Technology

[0002] Monitoring the stable operation of mechanical equipment is crucial for industrial production, effectively extending equipment lifespan, improving factory efficiency, and reducing industrial costs. With the development of digital manufacturing, many neural network frameworks have been rapidly developed and applied to equipment health maintenance, such as Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNNs), and Residual Neural Networks (RNNs). Unlike traditional manual diagnostic methods, data-driven deep learning models not only reduce the reliance on specialized technical personnel but also rapidly process large amounts of data with high accuracy, finding widespread application in precision manufacturing, automotive, shipbuilding, and aerospace industries.

[0003] While existing deep learning models have demonstrated satisfactory capabilities in diagnosing faults in critical equipment components, they primarily rely on two sets of assumptions: 1) The training model and test data must strictly adhere to the same distribution, with minimal bias between the data sets; 2) Target domain samples carrying fault label information exist. However, in real-world industrial scenarios, operating conditions are often time-varying and uncertain. Samples not only differ significantly from source domain data but are also of poor quality and lack corresponding labels. This necessitates that deep learning models possess greater robustness.

[0004] Considering that transfer learning aims to learn common-sense knowledge in various scenarios to help algorithm models exhibit strong performance in more application scenarios, transfer learning can enable neural network models to have better learning performance and the ability to effectively capture complex features when facing multi-source stimuli or variable operating conditions. It also possesses strong generalization performance from related but different application scenarios to new ones. Fallahdizcheh et al. proposed a deep learning model that uses degradation index to predict data, focusing on transferring degradation information from related processes to the target process for prediction. Xing et al. discussed the problem of low damage to key components in actual equipment operation and proposed an adaptive knowledge transfer scheme under the continuous machine learning paradigm. Li et al. combined conditional data comparison with unsupervised network prediction consistency theory to propose a new fault diagnosis method to address the problem of insufficient data and limited target domain samples in actual operating conditions. Although transfer learning does not require training and test data to simultaneously satisfy independent and identical distributions, users are often unwilling to share their product information due to data protection considerations, leading to a lack of necessary training data for fault diagnosis models. Due to the widespread adoption of data privacy protection concepts, the datasets available for developers to train and validate algorithms are even more limited, severely hindering the widespread application of artificial intelligence technology in the industrial field.

[0005] To meet the aforementioned data privacy protection requirements, McMahan et al. proposed a promising federated transfer learning scheme. For example... Figure 1 As shown, the FTL scheme does not require acquiring the private data of the source client for learning. Instead, it trains the framework and adjusts parameters by sharing the model parameters or high-dimensional features of the client. In this way, the data of each client is no longer exchanged, and the high-dimensional features or model parameters extracted by the network model itself have an encryption function, protecting the data information. However, federated learning does not consider the possibility of the network misclassifying fault samples with blurred class boundaries during the client training process. This increases the difficulty and time of training and affects the cross-domain diagnostic performance of the global model. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent fault diagnosis method based on data privacy protection that enables local models to identify fault samples with fuzzy class boundaries.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides an intelligent fault diagnosis method based on data privacy protection, comprising the following steps:

[0009] Step 1: Each source client initializes the local model.

[0010] Step 2: Train each local model and update the parameters of the feature extractor and the two classifiers.

[0011] Step 3: The global model of the central server receives the parameters of all local models, evaluates the availability of the local models, and then weights and combines the parameters of the local models to form the parameters of the global model, and updates the global model.

[0012] Step 4: Download the global model to each source client and provide task feedback. Each client inputs task data into the global model, which is then validated to obtain the classification loss of the global model in the source client. This classification loss is then used to update the local model in the source client.

[0013] Step 5: Repeat steps 2-4 a set number of times to download the final global model to the target client. This application aims to learn different techniques from privacy data using each source client model, with a central server determining the importance of parameters for each client model. As training steps are accumulated, the central model gradually achieves indirect learning from various source client samples and evaluation of client models, discarding model parameters with poor diagnostic performance.

[0014] Furthermore, in step 3, the method for evaluating the usability of the local model is as follows:

[0015]

[0016] In the formula, λ i,j This represents the evaluation coefficient of the i-th client in the j-th round of federated communication. and Let represent the final diagnostic accuracy and training accuracy of the i-th client in the j-th round of source client training.

[0017] Furthermore, in step 3, the method for weighted combination of the parameters of the local model is as follows:

[0018]

[0019]

[0020] In the formula, and N represents the feature extractor parameters and classifier parameters of the global model in the central server, respectively. c This represents the number of local models uploaded. For the Nth c The parameters of the feature extractor in a local model and These are the parameters of the two classifiers in the i-th local model.

[0021] Furthermore, in step 2, the training of the local model includes the following steps:

[0022] Step 2.1: Input source domain data, source domain data labels, and source domain client tasks into the local model, train the local model once, calculate the loss function, and optimize the parameters of the feature extractor and the two classifiers based on the distance loss and classification loss.

[0023] Step 2.2: Train the local model once. During the training process, freeze the parameters of the feature extractor in the local model, calculate the loss function, and optimize the parameters of the classifier in reverse based on the classification loss and the difference loss of the classifier.

[0024] Step 2.3: Train the local model once. During the training process, freeze the parameters of the classifier, calculate the loss function, and optimize the parameters of the feature extractor based on the distance loss and the difference between the classifier and the classifier.

[0025] Step 2.4, repeat steps 2.1-2.3 until the local model reaches the set number of training iterations.

[0026] Furthermore, in step 2, the classification loss is:

[0027]

[0028] In the formula, o s To extract client source domain sample features, y s The meaning of the sample prediction label is: n is the number of training samples, c is the fault class, and I[·] represents the probability score of the classifier in judging the failure type of the sample.

[0029] Furthermore, in step 2, the distance loss is:

[0030]

[0031] ∏(P1,P2) is the set of all joint distributions of the two distributions P1 and P2, where x and y are the parameters of the two distributions, respectively.

[0032] Furthermore, in step 2, the difference loss of the classifier is:

[0033]

[0034] In the formula, P 1C and P 2C P1 and P2 represent the predicted probabilities of class c faults under different classifiers, respectively.

[0035] In a second aspect, the present invention provides a microcontroller unit carrying an adaptive torque control protocol or program capable of executing the intelligent fault diagnosis method described in the first aspect.

[0036] Thirdly, the present invention provides an intelligent fault diagnosis device based on data privacy protection, including a main control module and multiple source clients, all source clients being connected to a central server, and the central server being connected to target clients; both the source clients and the central server include control units, and the control units are equipped with the microcontroller unit described in the second aspect; the main control module is a hardware device equipped with a microprocessor and has wireless communication capabilities for recording and transmitting model parameters.

[0037] Fourthly, the present invention provides an electronic device including one or more processors; and further including a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent fault diagnosis method described in any of the first aspects.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. While existing deep learning models achieve high diagnostic accuracy in addressing domain adaptation problems, they neglect the impact of learning differences between different classifiers on the feature classification of samples with ambiguous class boundaries, leading to an inability to accurately distinguish samples from the target domain. Considering this issue, this application maximizes the difference between the parameters of the two sets of classifiers in the local model of each source client to accurately identify ambiguous fault samples with class boundaries. This application proposes a differential learning strategy for client data. By freezing the feature extractor parameters and updating the classifier parameters, the difference in classifier outputs is maximized. By freezing the parameters of two sets of task-specific classifiers and updating the feature extractor parameters, the difference in classifier outputs is minimized. By batch freezing the parameter regions within the client model, the local model can gradually identify fault samples with ambiguous class boundaries, improving the fault diagnosis rate of the local model.

[0040] 2. The performance of the local model of each source client will be judged by the central server. The central server selectively learns the parameters of the client model based on the performance of the client model on the diagnostic task, rather than averaging the model parameters. The model with better training performance will be assigned more weight, which can improve the fault diagnosis rate of the global model.

[0041] 3. This application can achieve the convergence of source domain data distribution to target domain data distribution without prior knowledge of any data information of the target domain samples. The three sets of processes—local model training, global model updating, and task feedback—are continuously cyclical, gradually forming a global model that adapts to various tasks. Furthermore, the model for each source client is optimized, enabling better extraction of global features from the samples. Attached Figure Description

[0042] Figure 1 A schematic diagram of an existing federated learning scheme;

[0043] Figure 2 This is a schematic diagram of the federated transfer learning framework of Embodiment 1 of the present invention;

[0044] Figure 3 This is a schematic diagram of the client-side differential training scheme in Embodiment 1 of the present invention;

[0045] Figure 4 This is a schematic diagram of the federated dynamic communication process in Embodiment 1 of the present invention. Detailed Implementation

[0046] Example 1

[0047] A smart fault diagnosis method based on data privacy protection includes the following steps:

[0048] Step 1: Obtain the dataset; use vibration signals of various health states of bearings under different working conditions as the dataset, divide the vibration signals into a certain number of samples by equal window partitioning, and verify the samples.

[0049] The CWRU bearing dataset, as a publicly available dataset, is often used as a benchmark dataset for evaluating the quality of bearing fault diagnosis methods. A single fault was introduced into the test bearing through electrical discharge machining (EDM), resulting in four health states: normal (NC), inner ring fault (IF), ball fault (BF), and outer ring fault (OF). The damage diameters were 7 mils, 14 mils, and 21 mils, respectively, and the data acquisition rate was 48 k samples / second. During the testing process, fault samples were collected for each bearing fault at four different motor speeds (1797, 1772, 1750, and 1730 rpm) and corresponding loads (0, 1, 2, and 3 HP). A total of 100 samples were collected for each fault health state at different motor speeds, and each time-domain sample contained 2400 data points. The specific experimental data, including the speed and load information corresponding to the fault type, are shown in Table 1.

[0050] Table 1 Dataset Information

[0051]

[0052] The cylindrical roller bearing (CRB) dataset was obtained from a specially designed rotating machinery fault diagnosis test bench. Vibration signals from the bearing were transmitted to an LMS data acquisition instrument via a vibration acceleration sensor mounted on the outer end of the bearing housing. The experimental motor speeds were set to 1500 r / min, 2000 r / min, 2500 r / min, and 3000 r / min, with different loads applied at each speed, as shown in Table 1. The roller bearing type was N205EU, and the sampling frequency was set to 25.6 kHz. The experimental bearings included four health states: normal (NC), inner ring fault (IF), ball fault (BF), and outer ring fault (OF). Each fault type included three damage sizes: 0.2 mm, 0.4 mm, and 0.6 mm. A total of 200 samples were collected for each fault health state at different motor speeds, with each time-domain sample containing 6400 data points.

[0053] Step 2: Each source client initializes the model parameters during local model training. Since each model has different diagnostic tasks and training samples, the initial parameters are randomized. The federated transfer learning framework proposed in this application is as follows: Figure 2As shown, the framework mainly consists of various clients and a central server. The clients have the same network structure as the central server, but they have two sets of task-specific classifiers, while the central server only has one. The client's feature extractor uses a three-layer fully connected neural network with dimensions of 600, 200, and 150, consistent with the central server. The client classifier uses a two-layer fully connected (FC) network with dimensions of 100 and 10. The network parameter settings are shown in Table 2, and the detailed experimental transfer tasks are shown in Table 3. The numbers in Table 3 represent the data collected at the corresponding motor speeds.

[0054] Table 2 Parameter Settings

[0055]

[0056] Table 3 Experimental Task Settings

[0057]

[0058] During the model initialization phase, since each client has different diagnostic tasks, there is no universal model framework to propagate. Therefore, the parameters of each local model are randomly initialized, while the central server model does not participate in the initialization task at this time.

[0059] Step 3 involves training each local model and updating the parameters of the feature extractor and both classifiers. When the data distributions of the client source domain and target domain are inconsistent but the diagnostic task is the same, this neural network learning mode is called domain adaptation. While existing deep learning models have high diagnostic accuracy in solving domain adaptation problems, they neglect the impact of learning differences between different classifiers on the feature classification of samples with ambiguous class boundaries, leading to an inability to accurately distinguish samples from the target domain. Considering these issues, each local client model strives to maximize the difference between the parameters of the two sets of classifiers to accurately identify ambiguous fault samples with class boundaries, such as... Figure 3 As shown. Local model training includes the following steps:

[0060] Step 3.1 is the first step in differential training of the local model on each source client. The local model is trained once by inputting source domain data, source domain data labels, and source domain client tasks, and then the loss function is calculated. The cross-entropy loss is used as the classification loss for the source domain samples, and its expression is as follows:

[0061]

[0062] In the formula, o s To extract client source domain sample features, y s The meaning of the sample prediction label is: n is the number of training samples, c is the fault class, and I[·] represents the probability score of the classifier in judging the failure type of the sample.

[0063] Using Wasserstein distance as the domain distance loss, its expression is as follows:

[0064]

[0065] In the formula, ∏(P1,P2) is the set of all joint distributions of the two distributions P1 and P2, and x and y are the parameters of the two distributions, respectively.

[0066] The parameters of the feature extractor and two classifiers are optimized based on classification loss and distance loss, adapting each local model to the diagnostic task and ensuring that both classifiers can accurately classify all source domain samples. The specific optimization function is as follows:

[0067]

[0068] L c (X s ,Y s ) = L C1 (o s ,y s )+L C2 (o s y s )

[0069] In the formula, E refers to the feature extractor, C1 and C2 refer to the two classifiers, and δ1 is the first coefficient of the differential training.

[0070] Step 3.2: The local model performs the second stage of differential training; the source domain data, source domain data labels, and source domain client task are input into the local model, and the local model is trained once. During training, the feature extractor parameters in the local model are frozen, the loss function is calculated, and the parameters of the local model are updated; the differential loss of the classifier is calculated as follows:

[0071]

[0072] In the formula, P 1C and P 2C P1 and P2 represent the predicted probabilities of class c faults under different classifiers, respectively.

[0073] Then, based on the classification loss and the classifier's difference loss, the classifier's parameters are optimized in reverse. By training the classifier, the difference in the classifier's output is increased. The specific optimization function is as follows:

[0074]

[0075] Among them, P 1C and P 2CP1 and P2 represent the predicted probabilities of class C faults under different classifiers, respectively, and δ2 is the second empirical coefficient for differential training.

[0076] Step 3.3, the third stage of differential training is performed on the local model; the source domain data, source domain data labels, and source domain client tasks are input into the local model, and the local model is trained once. During the training process, the classifier parameters are frozen, the loss function is calculated, and the parameters of the local model are updated; based on the distance loss and the difference between the classifiers, the parameters of the feature extractor are updated to reduce the difference in the classifier output results.

[0077]

[0078] In the formula, δ3 and δ4 are the third and fourth empirical coefficients in model training, respectively.

[0079] Step 3.4: Each source client begins task training and learning in a different location, repeating steps 3.1-3.3 until the local model reaches the set number of training iterations. Each local model adapts to the local diagnostic task, striving to reduce the domain distribution distance between the source client's source domain samples and the client's task samples, completing the differential training of the local model in this stage. The source client ultimately obtains an extractor that accurately extracts sample features and two sets of task classifiers capable of identifying ambiguous fault samples at class boundaries, achieving client-side differential training. Then, each source client uploads its local model to the central server. Next, federated information begins to interact between individual clients and the global model, such as... Figure 4 As shown.

[0080] Step 4: The central server's global model receives parameters from all local models and evaluates the availability of the local models.

[0081]

[0082] In the formula, λ i,j This represents the evaluation coefficient of the i-th client in the j-th round of federated communication. and Let represent the final diagnostic accuracy and training accuracy of the i-th client in the j-th round of source client training.

[0083] After evaluating each source client, the central server weights and combines the parameters of each local model to form the parameters of the central server's global model; the weighting method is as follows:

[0084]

[0085]

[0086] In the formula, and N represents the feature extractor parameters and classifier parameters of the global model in the central server, respectively. c This represents the number of local models uploaded. For the Nth c The parameters of the feature extractor in a local model and These are the parameters of the two classifiers in the i-th local model. After the central server receives the weighted parameters of each source client model for the first time, it completes the parameter update and initialization.

[0087] Step 5: Download the updated global model to each source client. Each client inputs task data into the global model. The global model performs feedback diagnostics and validation for each source client's task, obtaining the classification loss of the global model in the source client. This classification loss is then used to update the local model in the source client. Finally, each source client contains the trained local model, the updated global model, and the feedback validation loss of the global model.

[0088] Step 6: Repeat steps 2-5 to gradually form a global model adapted to various tasks. Optimize the model for each source client to better extract global features from the samples. After the federated dynamic interaction process reaches the initially set maximum number of rounds, the global model will terminate the learning of diagnostic knowledge and be downloaded to the target client for final verification of the target task. The following methods will be used for comparison:

[0089] (1) Baseline Method: The baseline method serves as a control group for experimental validation. In this scheme, source clients are exempt from differential training, and the global model treats all source clients equally during the federated interaction process. Therefore, the global model is the average of all source client models.

[0090] (2) PM-NoDT: Considering the advantages of source-client differential training, the proposed model removes client differential training (PM-NoDT) as a comparative method for experimental verification. This approach is consistent with the training and parameter settings of the method in this paper.

[0091] (3) PM-NoCE: In order to verify the efficiency of global model parameter difference learning, the model removes the client evaluation mechanism (PM-NoCE) as a benchmark and conducts experimental verification of the proposed method.

[0092] The experimental results for fault diagnosis rate are shown in Table 4. The proposed federated transfer learning scheme was validated on five target tasks set up on the CWRU and CRB datasets. It can be seen that the diagnostic results of the federated transfer learning scheme gradually improve with the increase of training steps. Specifically, the model showed good domain adaptability for target tasks 1 and 2 in the early stages of training. After 50 training rounds, the diagnostic results of the federated learning scheme reached over 98%, meeting the expected results for the target tasks.

[0093] Since the target domain samples do not interfere with the training process of the proposed scheme, and the target task parameter information provided by the local model is extremely limited, these limitations lead to a sharp decline in the diagnostic performance of the global model when solving more complex target tasks, although the final fault diagnosis rate is still above 80%. Considering the randomness of the global model training process, each federated learning scheme is tested 5 times for different target tasks, and the average diagnosis is taken. The baseline method shows the worst diagnostic performance in the diagnosis of each target task, indicating that the model cannot effectively solve the domain adaptation problem when the target task is more complex. Although the diagnostic results of the schemes PM-NoDT and PM-NoCE are similar to the proposed federated learning scheme on some diagnostic tasks, the diagnostic results of the two schemes differ greatly on different target tasks, and the model generalization is poor. The results show that the proposed federated learning model is more effective than other comparative schemes in solving unlabeled target sample tasks. It is worth noting that the PM-NoDT method achieves competitive diagnostic performance in Task 1 and Task 2, and also shows high diagnostic accuracy in the diagnosis of target tasks; however, as the diagnostic task becomes more complex, the diagnostic performance of PM-NoDT drops sharply. Meanwhile, despite the increasing difficulty of the diagnostic tasks, the model's diagnostic results remained relatively stable. Overall, the proposed federated transfer learning scheme still achieves excellent results even when the global model cannot obtain target client samples for training, and it also exhibits significant advantages on source client tasks.

[0094] Table 4 Experimental Results

[0095]

[0096]

[0097] Example 2

[0098] This embodiment provides a microcontroller unit carrying an adaptive torque control protocol or program capable of executing the intelligent fault diagnosis method described in Embodiment 1.

[0099] This embodiment also provides an intelligent fault diagnosis device based on data privacy protection, including a main control module and multiple source clients. All source clients are connected to a central server, and the central server is connected to target clients. Both the source clients and the central server include control units, and the control units are equipped with the aforementioned microcontrollers. The main control module is a hardware device equipped with a microprocessor and has wireless communication capabilities for recording and transmitting model parameters.

[0100] This embodiment also provides an electronic device, including one or more processors; it also includes a storage device storing one or more programs thereon, which, when executed by one or more processors, cause one or more processors to implement any of the intelligent fault diagnosis methods described in Embodiment 1.

Claims

1. A method for intelligent fault diagnosis based on data privacy protection, characterized in that, Includes the following steps: Step 1: Obtain the dataset. Use vibration signals of various health states of bearings under different working conditions as the dataset. Each source client performs initialization operations on the local model. Step 2: Train each local model and update the parameters of the feature extractor and the two classifiers; Step 3: The global model of the central server receives the parameters of all local models, evaluates the availability of the local models, and then weights and combines the parameters of the local models to form the parameters of the global model. Step 4: Download the global model to each source client. Each client inputs task data into the global model. The global model is validated to obtain the classification loss of the global model in the source client. Use this classification loss to update the local model in the source client. Step 5: Repeat steps 2-4 a set number of times to download the final global model to the target client; In step 2, local model training includes the following steps: Step 2.1: Input source domain data, source domain data labels, and source domain client tasks into the local model, train the local model once, calculate the loss function, and optimize the parameters of the feature extractor and the two classifiers based on the distance loss and classification loss. Step 2.2: Train the local model once. During the training process, freeze the parameters of the feature extractor in the local model, calculate the loss function, and optimize the parameters of the classifier in reverse based on the classification loss and the difference loss of the classifier. Step 2.3: Train the local model once. During the training process, freeze the parameters of the classifier, calculate the loss function, and optimize the parameters of the feature extractor based on the distance loss and the difference between the classifier and the classifier. Step 2.4, repeat steps 2.1-2.3 until the local model reaches the set number of training iterations.

2. The data privacy protection based intelligent fault diagnosis method of claim 1, wherein, In step 3, the method for evaluating the usability of the local model is as follows: In the formula, This represents the evaluation coefficient of the i-th client in the j-th round of federated communication. and Let represent the final diagnostic accuracy and training accuracy of the i-th client in the j-th round of source client training.

3. The intelligent fault diagnosis method based on data privacy protection as described in claim 2, characterized in that, In step 3, the method for weighted combination of the parameters of the local model is as follows: In the formula, and N represents the feature extractor parameters and classifier parameters of the global model in the central server, respectively. c This represents the number of local models uploaded. For the Nth c The parameters of the feature extractor in a local model and These are the parameters of the two classifiers in the i-th local model.

4. The intelligent fault diagnosis method based on data privacy protection as described in claim 1, characterized in that, In step 2, the classification loss is In the formula, o s To extract client source domain sample features, y s The meaning of the sample prediction label is: n is the number of training samples, c is the fault class, and I[·] represents the probability score of the classifier in judging the failure type of the sample.

5. The intelligent fault diagnosis method based on data privacy protection as described in claim 4, characterized in that, In step 2, the distance loss is: Let x and y be the set of all joint distributions of the two distributions P1 and P2, where x and y are the parameters of the two distributions, respectively.

6. The intelligent fault diagnosis method based on data privacy protection as described in claim 5, characterized in that, In step 2, the difference loss of the classifier is: In the formula, P 1C and P 2C P1 and P2 represent the predicted probabilities of class c faults under different classifiers, respectively.

7. A microcontroller unit, characterized in that, It carries an adaptive torque control protocol or program capable of executing the intelligent fault diagnosis method according to any one of claims 1-6.

8. An intelligent fault diagnosis device based on data privacy protection, characterized in that, The system includes a main control module and multiple source clients, all of which are connected to a central server. The central server is connected to target clients. Both the source clients and the central server include control units, and each control unit is equipped with the microcontroller unit described in claim 7. The main control module is a hardware device equipped with a microprocessor and has wireless communication capabilities for recording and transmitting model parameters.

9. An electronic device comprising one or more processors; further comprising a storage device having one or more programs stored thereon, characterized in that, When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent fault diagnosis method as described in any one of claims 1-6.

Images