Oil pumping unit indicator diagram classification method based on ECAVFormer

Abstract

In order to solve the problems of low precision and unbalanced samples in the existing pumping unit indicator diagram classification method, the invention provides a pumping unit indicator diagram classification method based on ECAVFormer, and the method comprises the following steps: (1) preprocessing original indicator diagram data, unifying the image size, and dividing the original indicator diagram data into a training set and a verification set according to the ratio of 7: 3; (2) reasonably setting training parameters according to the performance of hardware equipment; and (3) the ECAVFormer model is researched and realized, and the pre-training weight is loaded. And (4) training the model through an Adam optimizer and a Focal-Loss loss function, and dynamically adjusting the learning rate by using a cosine annealing algorithm. And (5) comprehensively evaluating the performance of the model through indexes such as the accuracy, the macro recall rate and the macro F1 score. According to verification of the embodiment, the accuracy of indicator diagram classification can be effectively improved, reliable technical support is provided for fault diagnosis of the oil pumping unit, and the method has extremely high application value.

Description

Technical Field

[0001] The invention belongs to the technical field of classification of pumping unit indicator diagrams, and in particular relates to a novel classification method of pumping unit indicator diagrams based on ECAV_Former. Background Art

[0002] As a non-renewable energy source, oil is not only the primary energy source of the world's industry today, but also plays a vital role as a strategic resource. The oil pump is a widely used oil production equipment in the oil production industry at home and abroad. It occupies a dominant position in the use of oil production machinery. It has the characteristics of simple structure, convenient maintenance and management, and low comprehensive mining cost. However, due to the harsh working environment of the oil well, it is easily affected by high-condensation and high-viscosity media such as sand, wax, gas, and water around the oil field. Failures such as wax deposition, sand production, and valve leakage may occur at any time during the oil production process. As a key equipment in the oil recovery process, the working condition of the oil pump will directly affect the oil output rate, recovery efficiency, and the profitability of the entire oil industry. The failure of the oil pump may cause production interruption, which in turn brings huge economic losses to oil production companies. Therefore, improving the fault diagnosis capability of the oil pump to ensure its continuous and reliable operation is of great strategic and economic significance to the oil industry.

[0003] The current method for diagnosing oil pumping unit faults is mainly based on the analysis of the indicator diagram. This method records the displacement and load data of each stroke of the oil pumping unit and converts it into an indicator diagram. It uses the characteristics of the load changing with the displacement in the working principle to provide a high-precision and low-cost fault detection method for the oil extraction industry. If an abnormal phenomenon occurs during the operation of the oil pumping unit, the potential problem can be discovered in time based on the indicator diagram, thereby avoiding further deterioration of the fault and ensuring the normal operation of the oil well.

[0004] Traditional methods for fault diagnosis of oil pumping units, such as expert system method, electrical parameter method, support vector machine method, etc., have their own advantages, but also have certain limitations. The expert system method relies on expert knowledge, which is not only inflexible but also poorly interpretable. These reasons limit its effective application in a wider range of scenarios and different groups of technicians; the electrical parameter method uses the underground propagation characteristics of current to obtain information to analyze the working conditions of the oil pumping unit, but it can only focus on the electrical performance of the motor. It cannot detect the mechanical parts of the oil pumping unit, such as the fatigue degree of the sucker rod and the sealing performance of the pump valve, and cannot fully evaluate the overall working conditions of the oil pumping unit. It is easy to misjudge or miss the fault of the oil pumping unit due to missing information; the support vector machine method has high requirements for data preparation and parameter adjustment. When processing large-scale data, the computational complexity will increase dramatically, which requires high computing resources, and its internal complex mathematical principles make it difficult to intuitively explain the decision-making process. It is difficult for operators to know exactly how the model makes diagnostic conclusions, which seriously hinders the actual promotion of this method.

[0005] With the introduction of deep learning technology, the classification of dynamometer diagrams has ushered in higher accuracy and efficiency. Deep learning technology can process complex dynamometer diagram data, identify subtle signs of faults, and provide engineers and operators with more reliable fault diagnosis tools. This will significantly reduce the maintenance cost of pumping units, improve resource utilization, and extend the service life of equipment, and is expected to help the oil industry improve the sustainability and efficiency of production in practical applications. Therefore, in-depth research on pumping unit fault diagnosis methods based on deep learning and the establishment of a new generation of intelligent diagnosis models are of great significance to improving the accuracy and efficiency of pumping unit fault detection, ensuring the stable operation of oil fields, and promoting the digital transformation of the oil industry. Summary of the invention

[0006] The present invention is devoted to studying and realizing a new type of pumping unit indicator diagram classification method based on deep learning technology, namely, the ECAV_Former method, which aims to make up for the defects and shortcomings in the current pumping unit indicator diagram classification method, thereby promoting the further development of pumping unit fault diagnosis technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The classification method of pumping unit indicator diagram based on ECAV_Former is a new deep learning method. Its main research contents and characteristics are as follows:

[0009] The ECAV_Former model is created based on a variety of ideas, and its core lies in the self-attention mechanism and parallel computing. The self-attention mechanism can calculate the degree of association between each position in the input sequence and other positions, so that the model can capture long-distance dependencies in long sequence data. In the present invention, the attention weight is calculated by linear transformation of the query, key, and value in the attention mechanism module. For example, for the feature sequence after the image block is embedded, the present invention can automatically learn the correlation between each image block feature and other image block features, while the traditional convolutional neural network (CNN) is limited to a fixed convolution kernel receptive field. This mechanism enables the model to associate and integrate image features on a global scale, which helps to process the complex structures in the image and the relationship between objects. The present invention abandons the sequential structure of the traditional recurrent neural network (RNN), so that the model can be trained in parallel. In the present invention, multiple modules (Block) can be calculated simultaneously, which greatly improves the training efficiency. The attention mechanism and multilayer perceptron (MLP) operations within each module are independent of the sequence order, and can fully utilize the parallel computing capabilities of modern graphics processing units (GPUs) to accelerate the model training process.

[0010] The classification method of pumping unit indicator diagram based on ECAV_Former mainly includes the following modules:

[0011] Efficient Channel Attention (ECA), PatchEmbed, a sequence of multiple modules (Block), a normalization layer (norm), a pre-output layer (pre_logits), and an output head layer (head). Among them, the ECA layer plays a key role in optimizing the utilization of feature channel information in the model. First, the global average pooling layer compresses the spatial dimension of the feature map to obtain the channel feature vector, and then the one-dimensional convolution layer captures the channel dependency and generates weights in a lightweight and efficient manner. Finally, the Sigmoid activation function is used for normalization to obtain attention weights with a suitable value range, and then these weights are multiplied by the original channel features one by one. This process enables the model to focus more on valuable information and helps to effectively identify the features of each dynamometer diagram in the dynamometer diagram classification task. The PatchEmbed layer is designed to process the input image data in blocks and convert it into a suitable feature representation so that it can be processed by the subsequent architecture. The sequence of multiple blocks is one of the core parts of the ECAV_Former model, which is used to perform multiple feature transformations and extractions on the input features. In each Block, "norm1" first performs layer normalization. By normalizing the feature dimensions, the training process is made more efficient and stable and the model convergence can be accelerated. "Query, Key, Value" in the "attn" module generates query, key and value vectors with the help of linear transformation, aiming to lay the foundation for the multi-head attention mechanism. "proj" restores the dimension of the weighted features. "drop_path" currently exists in the form of "Identity", which can randomly discard the entire "Block" under a specific strategy to enhance the generalization and robustness of the model. "norm2" performs layer normalization again to optimize data for subsequent "mlp" processing. "fc1" in "mlp" expands the feature dimension and adds nonlinear expression: "act" uses the "GELU'" activation function to introduce nonlinearity. "fc2" compresses the feature dimension to extract key information. "drop" does not enable drop but has the function of preventing overfitting. After the feature processing of multiple blocks, the final features are subjected to a layer normalization operation to further stabilize the numerical distribution of the output features of the entire model, so that the features are in a more appropriate numerical range when entering the subsequent classification and other task-related layers, which helps to improve the performance and stability of the model. The pre_logits layer first keeps the feature dimension unchanged through a linear layer (fully connected layer), and then uses the Tanh activation function to perform a nonlinear transformation on its output, mapping the feature value to the (-1, 1) interval, and obtaining the pre-output feature representation to prepare suitable feature inputs for the final classification or other downstream tasks.As the last layer of the model, the head layer converts the previously obtained feature dimensions into dimensions corresponding to the number of target categories through a linear transformation (fully connected layer) to achieve the final prediction output.

[0012] A classification method for pumping unit indicator diagram based on ECAV_Former mainly includes the following steps:

[0013] (1) Data acquisition and preprocessing. The original dynamometer data comes from the suspension point of the pumping unit, covering the displacement and load data within an up and down stroke cycle. Since most of the existing dynamometer data are concentrated in the normal working state and the data distribution is uneven, this situation can easily lead to overfitting in the model training process. In order to effectively deal with this problem, a series of preprocessing operations are required before data partitioning to expand the data set and standardize the data format.

[0014] ① Image size unification. All input images are uniformly converted to 224×224 pixels. This operation can keep the data in a consistent dimension when input into the model, which is convenient for feature extraction and processing of the model.

[0015] ② Dataset expansion. A variety of data enhancement techniques are used to expand the data set, such as image binarization, random cropping, random horizontal flipping, center cropping, etc., to enrich the diversity of data and reduce the model's excessive dependence on specific data patterns, thereby improving the model's generalization ability.

[0016] ③ Dataset construction: 12 classic types of dynamometer diagrams were selected for model training, and divided into training set and validation set in a ratio of 7:3.

[0017] (2) Data loader creation. Determine the number of worker threads used when loading data based on factors such as the set batch size and the number of available central processing unit (CPU) cores. Create a training data loader and a validation data loader and set the corresponding batch size, whether to shuffle the order, whether to fix the memory, the number of worker threads, and a custom sorting function.

[0018] (3) Model creation and weight loading. Call the function to create an ECAV_Former model instance and pass in the model parameters. If the pre-trained weight path is specified, first check whether the weight file exists. If so, load the weight into the model.

[0019] (4) Model training and verification.

[0020] ① Training phase. Perform cyclic training from the specified starting round to the set total rounds, pass in parameters such as the model, optimizer, training data loader, device, and current round, and obtain the loss and accuracy of this round of training. Create an Adam optimizer from the model's trainable parameters, pass in parameters such as the learning rate, betas parameters, and weight decay coefficient to configure the optimizer, and use the cosine annealing learning rate scheduling strategy to update the learning rate. In order to solve the problem of unbalanced dynamometer sample, the Focal-Loss loss function is used, and its formula is as follows:

[0021] FL(p t )=-(1-p t ) γ ·log(p t ) (1)

[0022] Among them, P t Represents the predicted probability of a certain category, P t When it is close to 1, it means that the input dynamometer image is easy to classify. γ is an adjustment factor used to balance the influence of easy-to-classify samples and hard-to-classify samples on the loss function. When γ is large, easy-to-classify samples are reduced, while the loss of hard-to-classify samples is amplified more.

[0023] ② Verification phase: Pass in the model, verification data loader, device, current round and other parameters to get the verification loss, verification accuracy, verification macro F1 score, verification macro recall and confusion matrix.

[0024] ③ Model evaluation indicators. Select a variety of image classification model evaluation indicators to evaluate the performance of each model, namely, accuracy, macro recall, macro F1 score, and confusion matrix.

[0025] A. Accuracy refers to the ratio of the number of correctly classified dynamometer samples to all samples for the input dynamometer. The formula is as follows:

[0026]

[0027] Where m is the total number of samples in the dataset D, y i is the sample x i The true label of f(x), I(·) is the indicator function, when the condition in the brackets is true, the value of I is 1, otherwise it is 0. i ) is the model f for sample x i prediction results.

[0028] B. Macro-averaging refers to the arithmetic average of each statistical indicator value of all indicator diagram categories. The calculation formulas for macro recall and macro F1 score are as follows:

[0029]

[0030] Where R i is the recall rate of the i-th class sample, F i is the F1 score of the i-th class sample.

[0031] C. The confusion matrix shows the classification of each category by the model in the form of a matrix. The rows of the confusion matrix represent the true labels, the columns represent the labels predicted by the model, and the value in each cell represents the number of samples of the true label and the predicted label. It contains four main entries: True Positive (TP), False Negative (FN), False Positive (FP), and True Negative (TN).

[0032] (5) Model saving and result visualization. After the training cycle is completed, the model weights of the last round are saved to the specified path. In addition, the model evaluation data is saved, and the verification loss curve, verification accuracy curve, confusion matrix, etc. are plotted, and the images are saved to the specified result folder.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] (1) The ECAV_Former model can calculate the degree of correlation between each element in the input feature without being restricted by distance. It can efficiently and accurately capture the long-distance dependency between image block features and better understand the overall structure of the image and the complex relationship between images.

[0035] (2) The ECAV_Former model abandons the sequential structure of traditional deep learning models and adopts the characteristics of parallel training. Multiple blocks can be operated simultaneously, which greatly improves the parallel computing capability while ensuring efficient feature extraction;

[0036] (3) The ECAV_Former model incorporates an efficient channel attention mechanism that can adaptively assign weights to different channels, highlight the feature information of important channels, and suppress those irrelevant or interfering channel features.

[0037] In summary, the present invention is novel and reasonable in design, can accurately and efficiently make up for the deficiencies of traditional oil pump fault diagnosis models, and provide a reliable solution for the operation and maintenance of oil pumps in the petroleum industry.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart for realizing the method of the present invention.

[0040] Figure 2 It is a structural diagram of the model of the present invention.

[0041] Figure 3 This is a comparison chart of the accuracy of the model of the present invention and other models.

[0042] Figure 4 This is the loss graph of the model of the present invention and other models.

[0043] Figure 5 This is a comparison chart of the confusion matrix between the model of the present invention and other models. DETAILED DESCRIPTION

[0044] In order to make the technical solution of the present invention clearer, the technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

[0045] Example 1

[0046] (1)Reference Figure 1 This embodiment classifies the dynamometer diagram of a pumping unit in Shengli Oilfield based on the ECAV_Former method, and its main steps are as follows:

[0047] Step 1: Collect displacement and load data of the upper and lower strokes at the suspension point of the pumping unit;

[0048] Step 2: Convert the data collected in step 1 into a dynamometer diagram, and perform binarization, image size unification, random cropping, random horizontal flipping, and center cropping operations on the image;

[0049] Step 3: Divide the processed data set into a training set and a validation set according to a ratio of 7:3;

[0050] Step 4: Input the training set divided in step 3 into the ECAV_Former model structure, perform model training, and build an oilfield pumping unit indicator diagram classification model;

[0051] Step 5: Optimize the parameters of the model through the FocalLoss loss function and the Adam optimizer so that it can better adapt to the training data and improve the performance on the test data.

[0052] Step 6: Repeat step 5 until the model reaches the maximum number of iterations and each evaluation indicator tends to be stable, and save the final model weights;

[0053] Step 7: Input the validation set data into the trained ECAV_Former model, extract image features, and output the classification results of the dynamometer diagram.

[0054] (2) The data of this embodiment comes from an oil production plant in Shengli Oilfield. The original dynamometer data is the displacement and load data collected at the suspension point of the pumping unit within one cycle of the up and down strokes. After the operation of step 2, the dynamometer is divided into 12 categories, namely: normal operation, insufficient liquid supply, gas influence, upper impact pump, lower impact pump, plunger out of pump working cylinder, slow closing of floating valve, floating valve, fixed valve, sand influence, gas lock and inertia influence. The data set in this embodiment contains a total of 10,907 dynamometer pictures, of which the training set is divided into 7,557 pictures and the test set is divided into 3,350 pictures. The proportion of each type of sample is shown in Table 1.

[0055] Table 1 Sample table of various types of indicator diagrams

[0056]

[0057] (3)Reference Figure 2 The specific process of step 4 of this embodiment is as follows:

[0058] ① Input image processing and feature enhancement. The input pumping unit indicator diagram first enters the "ECA" module. This step aims to focus on the key channel information of the image and accurately capture the subtle feature differences contained in the indicator diagram by adaptively adjusting the weights of each channel. Specifically, first perform global average pooling through AdaptiveAvgPool2d to extract global features from each channel. Then, use the Conv1d one-dimensional convolution operation to capture the correlation between different channels and help the model focus on key feature areas. Finally, the channel weights are generated through the Sigmoid activation function and reapplied to the input feature map to enhance the information expression ability of the key channels.

[0059] ② Feature embedding. The image processed by the ECA module will enter the "Patch+Position Embedding" module. First, use "Conv2d" to divide the input image into multiple small blocks (patches) and extract the local features of each patch. Then, flatten its 2D features into a one-dimensional vector to adapt it to the input format of the Transformer. Next, by adding position embedding (Position Embedding) to the features of each patch, the spatial position information in the image is encoded. Finally, some features are randomly discarded through Dropout to enhance the generalization ability of the model.

[0060] ③ Feature extraction and modeling. The embedded features are then passed to the "Transformer Encoder" module. In each "Encoder block", the input features are first normalized through the "Layer Norm" to stabilize the model training process and accelerate convergence. Next, enter the "Multi-Head Attention" module, and use the multi-head attention mechanism to capture the global dependencies between different regions in the image and learn the interactive information between features. After passing through the "Dropout / DropPath" layer, the robustness of the model is further enhanced to prevent overfitting. Finally, the extracted features are deeply modeled through the "MLP Block" module to further enhance the discriminative ability of feature expression, so that it can distinguish different categories more accurately. Multiple "Encoderblock" modules are stacked layer by layer to gradually extract deeper and richer information from the image.

[0061] ④ Classification head. The features output from the "Transformer Encoder" are input to the "MLP Head" module for classification. First, the "Extra Class Token" is extracted from the output, which is a feature vector representing the category information. Then, it is further processed through the "Pre-Logits" layer to generate the final feature representation. Finally, the "Linear" layer is used as a classifier to map the feature vector to a specific dynamometer category label.

[0062] ⑤Output: The model outputs a category label, indicating the specific category to which the input pumping unit indicator diagram belongs.

[0063] (4) Combined with the results of optimizing the initial training parameters of the model by the particle swarm optimization algorithm and the actual situation, the main hyperparameter settings of this embodiment and the basis for setting are shown in Table 2. Specifically, the learning rate is set to 0.0001, mainly considering that complex models require a lower learning rate to ensure the stability of training, avoid gradient explosion and loss oscillation, and at the same time, a lower learning rate helps to avoid overfitting and improve the generalization ability of the model; the batch size is set to 32, which provides a good balance between computing efficiency and generalization ability, which can adapt to the memory limitation and provide stable and meaningful gradient optimization; the number of training rounds is set to 50, which is enough for the model to fully converge on the data set and avoid overfitting; the optimizer selects Adam, which has the advantages of adaptive learning rate adjustment, faster convergence speed, insensitivity to initial learning rate, and more adaptability to sparse data and high noise gradients compared to the SGD optimization algorithm; the loss function uses FocalLoss, which is used to solve the problem of class imbalance and replaces the traditional cross entropy loss function for better results; the training device uses CUDA, which significantly improves the computing efficiency and speeds up the training of the model compared to the CPU.

[0064] Table 2 Main hyperparameter settings of the model

[0065]

[0066] (5) In order to evaluate the performance of the proposed method on the dynamometer classification task, the following evaluation indicators are used: accuracy, loss value, macro recall, macro F1 score and confusion matrix. The evaluation results are shown in Table 3.

[0067] Table 3 Comparison of evaluation indexes of various models

[0068]

[0069] As shown in Table 3, the test accuracy, macro recall and macro F1 score of the Logistic Regression model are the lowest, which are 32.15%, 29.17% and 24.04% respectively, while the test accuracy, macro recall and macro F1 score of the ECAV_Former model are the highest, which are 91.28%, 91.39% and 90.70% respectively. The evaluation indexes of the SVM, LeNet, ResNet and DenseNet models are between the Logistic Regression and ECAV_Former models. Compared with other models, the test accuracy of the ECAV_Former model increased by about 1.7 to 59.13 percentage points, and the macro recall increased by about 1.49 ～ The macro F1 improved by about 1.48 to 66.66 percentage points, which shows that the model is more comprehensive and effective in dealing with various abnormal situations.

[0070] (6) The comparison of the parameter amount, memory size, training time and test time of the present invention and other models is shown in Table 4. As can be seen from Table 4, the Logistic Regression, SVM and LeNet models have fewer parameters, memory usage, training time and test time than the other three models. Among them, the parameter amount and memory usage of the Logistic Regression and SVM models are the same and the least, which are 36876 and 0.141MB respectively, while the training time of SVM is the shortest, which is 412.268s, and the test time of Logistic Regression is the shortest, which is 147.859s. The parameter amount, memory usage, training time and test time of the LeNet model are slightly higher than those of the Logistic Regression and SVM models, which are 62176, 0.237MB, 511.141s and 160.965s respectively. ResNet, DenseNet and ECAV_Former have more parameters, memory usage, training time and test time, among which the parameters and memory usage of the DenseNet model are less, which are 7978856 and 30.437MB respectively, but the training time and test time are the longest, which are 4307.154s and 740.954s respectively. The parameters, memory usage, training time and test time of the ResNet model are between DenseNet and ECAV_Former, which are 21290828, 81.218MB, 2270.342s and 516.330s respectively. The ECAV_Former model has the most parameters and memory usage, which are 88055056 and 335.90MB respectively, but the training time and test time are slightly shorter than the DenseNet model, which are 3446.287s and 679.900s respectively. Compared with other models, the present invention has increased parameters and memory usage, but the accuracy on the test set has been greatly improved.

[0071] Table 4 Comparison of model parameters

[0072]

[0073] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A classification method for pumping unit indicator diagram based on ECAV_Former, the main research content and characteristics are as follows: The ECAV_Former model is created based on a variety of ideas, and its core lies in the self-attention mechanism and parallel computing. The self-attention mechanism can calculate the degree of association between each position in the input sequence and other positions, so that the model can capture long-distance dependencies in long sequence data. In the present invention, the attention weight is calculated by linear transformation of the query, key, and value in the attention mechanism module. For example, for the feature sequence after the image block is embedded, the present invention can automatically learn the correlation between each image block feature and other image block features, while the traditional convolutional neural network (CNN) is limited to a fixed convolution kernel receptive field. This mechanism enables the model to associate and integrate image features on a global scale, which helps to process the complex structures in the image and the relationship between objects. The present invention abandons the sequential structure of the traditional recurrent neural network (RNN), so that the model can be trained in parallel. In the present invention, multiple modules (Block) can be calculated simultaneously, which greatly improves the training efficiency. The attention mechanism and multilayer perceptron (MLP) operations within each module are independent of the sequence order, and can fully utilize the parallel computing capabilities of modern graphics processing units (GPUs) to accelerate the model training process.

2. The method for classifying the pumping unit indicator diagram based on ECAV_Former according to claim 1 mainly comprises the following modules: Efficient Channel Attention (ECA), PatchEmbed, a sequence of multiple modules (Block), a normalization layer (norm), a pre-output layer (pre_logits), and an output head layer (head). The ECA layer plays a key role in optimizing the utilization of feature channel information in the model. First, the global average pooling layer compresses the spatial dimension of the feature map to obtain the channel feature vector, and then the one-dimensional convolution layer captures the channel dependency and generates weights in a lightweight and efficient manner. Finally, the Sigmoid activation function is used for normalization to obtain attention weights with a suitable value range, and then these weights are multiplied one by one with the original channel features. This process enables the model to focus more on valuable information and helps to effectively identify the features of each dynamometer diagram in the dynamometer diagram classification task. The PatchEmbed layer is designed to process the input image data in blocks and convert it into a suitable feature representation so that it can be processed by the subsequent architecture. The sequence of multiple blocks is one of the core parts of the ECAV_Former model, which is used to perform multiple feature transformations and extractions on the input features. In each Block, "norm1" first performs layer normalization. By normalizing the feature dimensions, the training process is made more efficient and stable and the model convergence can be accelerated. "Query, Key, Value" in the "attn" module generates query, key and value vectors with the help of linear transformation, aiming to lay the foundation for the multi-head attention mechanism. "proj" restores the dimension of the weighted attention features. "drop_path" currently exists in the form of "Identity", which can randomly discard the entire "Block" under a specific strategy to enhance the generalization and robustness of the model. "norm2" performs layer normalization again to optimize data for subsequent "mlp" processing. "fc1" in "mlp" expands the feature dimension and adds nonlinear expression. "act" uses the "GELU" activation function to introduce nonlinearity. "fc2" compresses the feature dimension to extract key information. "drop" does not enable drop but has the function of preventing overfitting. After the feature processing of multiple blocks, the final features are subjected to a layer normalization operation to further stabilize the numerical distribution of the output features of the entire model, so that the features are in a more appropriate numerical range when entering the subsequent classification and other task-related layers, which helps to improve the performance and stability of the model. The pre_logits layer first keeps the feature dimension unchanged through a linear layer (fully connected layer), and then uses the Tanh activation function to perform a nonlinear transformation on its output, mapping the feature value to the interval (-1, 1) to obtain the pre-output feature representation, and prepare suitable feature inputs for the final classification or other downstream tasks. The head layer is the last layer of the model. It converts the previously obtained feature dimensions into dimensions corresponding to the number of target categories through a linear transformation (fully connected layer) to achieve the final prediction output.

3. The method for classifying the pumping unit indicator diagram based on ECAV_Former according to claim 1, characterized in that: The following steps are involved: (1) Data acquisition and preprocessing. The original dynamometer data comes from the suspension point of the pumping unit, covering the displacement and load data within an up and down stroke cycle. Since most of the existing dynamometer data are concentrated in the normal working state and the data distribution is uneven, this situation can easily lead to overfitting in the model training process. In order to effectively deal with this problem, a series of preprocessing operations are required before data partitioning to expand the data set and standardize the data format. ① Image size unification. All input images are uniformly converted to 224×224 pixels. This operation can keep the data in a consistent dimension when input into the model, which is convenient for feature extraction and processing of the model. ② Dataset expansion. A variety of data enhancement techniques are used to expand the data set, such as image binarization, random cropping, random horizontal flipping, center cropping, etc., to enrich the diversity of data and reduce the model's excessive dependence on specific data patterns, thereby improving the model's generalization ability. ③ Dataset construction: 12 classic types of dynamometer diagrams were selected for model training, and divided into training set and validation set in a ratio of 7:

3. (2) Data loader creation. Determine the number of worker threads used when loading data based on factors such as the set batch size and the number of available central processing unit (CPU) cores. Create a training data loader and a validation data loader and set the corresponding batch size, whether to shuffle the order, whether to fix the memory, the number of worker threads, and a custom sorting function. (3) Model creation and weight loading. Call the function to create an ECAV_Former model instance and pass in the model parameters. If the pre-trained weight path is specified, first check whether the weight file exists. If so, load the weight into the model. (4) Model training and verification. ① Training phase. Perform cyclic training from the specified starting round to the set total rounds, pass in parameters such as the model, optimizer, training data loader, device, and current round, and obtain the loss and accuracy of this round of training. Create an Adam optimizer from the model's trainable parameters, pass in parameters such as the learning rate, betas parameters, and weight decay coefficient to configure the optimizer, and use the cosine annealing learning rate scheduling strategy to update the learning rate. In order to solve the problem of unbalanced dynamometer sample, the Focal-Loss loss function is used, and its formula is as follows: FL(p t )=-(1-p t ) γ ·log(p t ) (1) Among them, P t Represents the predicted probability of a certain category, P t When it is close to 1, it means that the input dynamometer image is easy to classify. γ is an adjustment factor used to balance the influence of easy-to-classify samples and hard-to-classify samples on the loss function. When γ is large, easy-to-classify samples are reduced, while the loss of hard-to-classify samples is amplified more. ② Verification phase: Pass in the model, verification data loader, device, current round and other parameters to get the verification loss, verification accuracy, verification macro F1 score, verification macro recall and confusion matrix. ③ Model evaluation indicators. This embodiment selects multiple model evaluation indicators for image classification to evaluate the performance of each model, namely, accuracy, macro recall, macro F1 score, confusion matrix and other evaluation indicators. A. Accuracy refers to the ratio of the number of correctly classified dynamometer samples to all samples for the input dynamometer. The formula is as follows: Where m is the total number of samples in the dataset D, y i is the sample x i The true label of f(x), I(·) is the indicator function, when the condition in the brackets is true, the value of I is 1, otherwise it is 0. i ) is the model f for sample x i prediction results. B. Macro-averaging refers to the arithmetic average of each statistical indicator value of all indicator diagram categories. The calculation formulas for macro recall and macro F1 score are as follows: Where R i is the recall rate of the i-th class sample, F i is the F1 score of the i-th class sample. C. The confusion matrix shows the classification of each category by the model in the form of a matrix. The rows of the confusion matrix represent the true labels, the columns represent the labels predicted by the model, and the value in each cell represents the number of samples of the true label and the predicted label. It contains four main entries: True Positive (TP), False Negative (FN), False Positive (FP), and True Negative (TN). (5) Model saving and result visualization. After the training cycle is completed, the model weights of the last round are saved to the specified path. In addition, the model evaluation data is saved, and the verification loss curve, verification accuracy curve, confusion matrix, etc. are plotted, and the images are saved to the specified result folder.

Images