A long tail data-based transformer-based oil extraction violation identification method

Abstract

The application discloses a kind of long tail data based on Transformer's oil production violation identification method, belong to computer vision field, including the following steps: collecting the video stream of oilfield construction work site, making oil production work site violation data set;Using Grabcut-based enhancement algorithm to enhance the middle tail class data in training set;Build the oil production violation identification model based on Transformer network;Design the overall sample sampling strategy of oil production violation identification model;Design the training weight distribution strategy of oil production violation identification model;Training and optimization are carried out to the oil production violation identification model;Real-time acquisition of monitoring field work video, based on the oil production violation identification model of training completion to field work video identification, obtain violation identification classification result, and store identification and determination information.The application has good identification ability to violation behavior, improves the production management efficiency of oilfield construction work site.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision, and particularly relates to a Transformer-based oil production violation identification method under long-tail data. BACKGROUND

[0002] Oilfield construction operation procedures are complex and have many links, so field safety production is of great significance. The traditional oil production operation site supervision mechanism mainly uses cameras to remotely view in an artificial manner to ensure the normal operation of the production operation site and to reduce part of the manpower and resources to a certain extent. With the increase in the unit price of computer hardware computing power and the development of soft facilities such as convolutional neural networks and ViT models, large-scale video monitoring using computers has become a reality. However, some violations such as not wearing goggles, smoking, and using mobile phones are not easy to detect using target detection, pose estimation, or personnel tracking due to small targets, complex environments, and low incidence. Therefore, addressing this issue is a key to improving oil production site management efficiency and greatly reducing labor and resource expenditures in the construction operation site, and is also a key to reducing production management costs and improving management quality in the construction operation site.

[0003] In order to train a good model, data collected from the oilfield construction operation site is needed to form a data set. However, in real-world applications, each class in the collected data set often exhibits natural imbalance and long-tail distribution, i.e., a small number of classes (head classes) account for most of the data, while most classes (middle and tail classes) have very few samples. This is mainly due to the low incidence, difficulty in collecting, and high cost of collecting data for some classes. In order to collect a balanced data set, it will result in the entire project spending huge financial and human resources just in the data collection link. If the original convolutional neural network or ViT model is directly trained on such a long-tail data set, the overall performance of the model will be biased towards the head classes, and the recognition accuracy of the tail classes and the overall will be significantly reduced. In order to focus most of the attention on algorithm design and optimization and save project expenditures, a good solution is to optimize and improve the model, and then train it on the long-tail data set. SUMMARY

[0004] In order to solve the problem of long-tail distribution of data collected from the actual oilfield construction operation site and the difficulty in identifying some violations, the application proposes a Transformer-based oil production violation identification method under long-tail data, which constructs a violation identification model based on a Transformer network, so that the violation identification model can be trained directly on a long-tail data set and obtain similar performance in each class, improving the classification efficiency while reducing labor and resource expenditures.

[0005] The technical solution of the application is as follows:

[0006] A long tail data based on Transformer oil production violation identification method, comprising the following steps:

[0007] Step 1, collect the video stream of oilfield construction site, make oil production site violation data set;

[0008] Step 2, using Grabcut-based enhancement algorithm to enhance the tail class data in the training set;

[0009] Step 3, build oil production violation identification model based on Transformer network;

[0010] Step 4, design the overall sample sampling strategy of oil production violation identification model;

[0011] Step 5, design the training weight distribution strategy of oil production violation identification model;

[0012] Step 6, training and optimization of oil production violation identification model;

[0013] Step 7, monitoring real-time acquisition of field work video, based on the trained oil production violation identification model to identify the field work video, get the violation identification classification result, and store the identification and judgment information.

[0014] Further, the specific process of step 1 is: first, frame the video stream of oilfield construction site to get frame image data, use Labelimg tool to label the frame image data, get frame image and corresponding label file; Then use python script to cut and classify the violation labeled data to make violation data set, and divide the data set into training set, validation set and test set in random form according to fixed proportion.

[0015] Further, the specific process of step 2 is:

[0016] Step 2.1, apply Grabcut-based enhancement algorithm on the tail class data in the training set;

[0017] First, sample images from the same violation category, then use Grabcut-based enhancement algorithm to model the background and foreground of the image and extract the foreground, and then use Border Matting algorithm to smooth the boundary information of the segmentation; Finally, use the data enhancement methods of horizontal flip, rotation and scaling to enhance the extracted foreground;

[0018] The specific process of the enhanced algorithm based on Grabcut is as follows: first, the model parameters are initialized, then the image is modeled using a Gaussian mixture model, and then the image is segmented using a graph cut algorithm; in this process, each pixel in the image is regarded as a node, and the adjacent pixels are connected to form edges; then, according to the Gaussian mixture model established in the foregoing, the probability of each node belonging to the foreground or background is calculated, and these probabilities are used as the weights of the edges, and the minimum cut algorithm is used to divide the image into foreground and background two parts;

[0019] Step 2.2, a part of the oil production operation site related background image data is collected, the enhanced foreground image is fused with the new background image to obtain the enhanced training set, and 1-3 foreground images are randomly stored in a single background.

[0020] Further, in step 3, the specific content of the oil production violation recognition model based on the Transformer network is as follows: an RGB image with an input size of 224x224x3 is input, 768 16x16 convolution kernels are used to perform convolution operation on the input image, and an image data with an output size of 14x14x768 is output; the data is flattened, and an output size of 196x768 is output; a trainable parameter Class Token with a size of 1x768 is spliced, and the output data has a size of 197x768, and the Class Token stores the feature information of the violation target; then, the position encoding is added, and the output data has a size of 197x768; then, it passes through the Dropout layer and enters the 12 Encoder Block layers; then, it passes through the Layer Norm layer, and the violation image classification features in the ClassToken are extracted and sent to the MLP Head classification network to obtain the final violation classification result of the input image;

[0021] Among them, the specific content of a single Encoder Block layer is as follows: the data input into the Encoder Block layer is first normalized by the Layer Norm layer, enters the multi-head attention layer, and then the Dropout layer is used to obtain the output; the output is added to the initial input; then, it enters the Layer Norm layer, the MLP Block layer and the Dropout layer in turn, and the output is added to the input of the Layer Norm layer again to obtain the final output.

[0022] Further, the specific process of step 4 is as follows:

[0023] Step 4.1, the class balanced sampling probability and the instance balanced sampling probability are calculated using the general formula (1), q [0, 1], when q=0 is substituted into, the class balanced sampling probability is obtained When q = 1 is entered, the example balanced sampling probability is obtained Formula (1) is as follows:

[0024]

[0025] Wherein, p j represents the probability of sampling samples in class j; i, j represent class indexes; n i is the number of samples in class i; C is the total number of classes;

[0026] Step 4.2, introduce the sampling balance parameter λ, integrate the class balanced sampling and the example balanced sampling, as the sampling strategy of the oil violation identification model based on the Transformer network, the overall sampling strategy formula is as follows:

[0027]

[0028] Wherein, represents the overall sampling strategy; T is the total number of rounds, and t is the current round.

[0029] Further, the specific process of step 5 is as follows:

[0030] Step 5.1, using cross-entropy loss function as the first part of training weight distribution, the feature information of the head class is extracted;

[0031]

[0032] Wherein, assuming that the class label space of the training set is c∈{1, 2,..., C}, C is the total number of classes, for the image with class label c, the model prediction output is defined as y=[y1, y2,..., y i ,..., y C ];

[0033] Step 5.2, using class difficulty to distribute weight; the precision measurement uses F β score or only uses the recall rate R c of class c;

[0034] When considering the recall rate and the precision at the same time, F β score is used for calculation, and the difficulty d of class label c at this time is only considered when the precision is used, and the recall rate R c is used for calculation, and the difficulty d c of class label c at this time is d c (t)=1-R β (t);

[0035] F β score calculation formula is as follows:

[0036]

[0037] wherein c is the class label index, R c and P c represent the recall and precision, respectively;

[0038] The recall R c is calculated as follows:

[0039] R c = n c / N c #(5)

[0040] wherein N c represents the total number of samples of class c in the validation set, and n c represents the number of samples of class c in the validation set that are correctly classified by the model;

[0041] The corresponding weight distribution strategy is:

[0042] w c (t) = softmax [δ + d c (t)]#(6)

[0043] wherein δ = c / C is used to control the degree of weighting for different classes;

[0044] Step 5.3, finally use the following weight distribution strategy to train the model:

[0045]

[0046] wherein the predicted output of the model is y, c is the class label index of the dataset, and t is the current round.

[0047] Further, the specific process of step 6 is as follows:

[0048] Step 6.1, set the training parameters of the network; the training parameters include the maximum number of iterations, the learning rate, the optimizer momentum, and the weight;

[0049] Step 6.2, use the enhanced illegal training set and validation set to train the oil extraction violation recognition model based on the Transformer network. After the end of each round of epoch training, the corresponding class difficulty is calculated in real time, and the weight of different violation items is allocated accordingly, until the training is completed, and the trained oil extraction violation recognition model is obtained;

[0050] Step 6.3, evaluate the trained model using the test set, and use the overall precision as the evaluation index. When the overall precision index rises and tends to be stable, it proves that the current model training result is good, and the current model is output as the final oil extraction violation recognition model; otherwise, the model is retrained.

[0051] Further, the specific process of step 7 is: using the same processing as step 1, the video stream to be detected is subjected to frame division and cutting processing; the processed frame data is input into the trained oil extraction violation recognition model to obtain the violation recognition classification result; at the same time, the oil extraction violation recognition software system is developed based on the oil extraction violation recognition model; first, the above oil extraction violation recognition model is constructed using the PyTorch deep learning framework and training is completed, and then the entire software system is deployed in the computer in the well site monitoring room and connected with the well site server; the server will store the video data stream collected by the operation site monitoring camera in real time, and the computer will continuously read the video stream from the server and complete the violation recognition task; the software system will remind and visually display in real time when detecting the violation behavior, which is convenient for the operation supervisor to handle in real time; the detection data is image data including a single violation category, and the computer returns the violation recognition data and judgment information to the server for storage backup, so that the management personnel can view and handle.

[0052] The beneficial technical effects brought by the present application are:

[0053] The present application mainly aims at the problem that the data collected in the actual oil extraction operation site is long-tailed, reduces the time and financial cost of data collection when constructing the violation recognition model, and can focus the team's attention on algorithm design and optimization, thereby improving the efficiency of production operation; the present application has the characteristics of easy training and easy optimization, and can directly train on the long-tailed data set on the basis of improving the model, thereby alleviating the model recognition imbalance problem brought by the long tail and achieving similar performance on each category; the present application uses computer vision and deep learning related technologies, without the need to install additional hardware devices such as sensors, thereby reducing the hardware cost, and collecting data through the existing monitoring camera in the operation site, and deploying the corresponding software on the server. The present application has the characteristics of accurate and rapid recognition under long-tailed data, has good recognition ability for violation behaviors that are difficult to recognize by ordinary algorithms, and improves the production management efficiency of the oilfield operation site. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 A long-tailed data-based oil extraction violation recognition method flowchart based on Transformer.

[0055] Figure 2 A schematic diagram of the overall network structure of the long-tailed data-based oil extraction violation recognition model based on Transformer.

[0056] Figure 3 A schematic diagram of the overall network structure of the long-tailed data-based oil extraction violation recognition model based on Transformer. Figure 2 A schematic diagram of the structure of a single Encoder Block layer in the network structure. DETAILED DESCRIPTION

[0057] The application will be described in further detail below in conjunction with the accompanying drawings and specific embodiments:

[0058] As shown in the drawings, Figure 1 The long-tail data-based oil production violation identification method based on the Transformer of the application includes the following steps:

[0059] Step 1, collect the video stream of the oilfield construction site to make the oil production operation site violation data set. The specific process is:

[0060] First, the video stream of the oilfield construction site is subjected to frame processing to obtain frame image data, and the Labelimg tool is used to mark the violation of the obtained frame image data to obtain frame image and corresponding label file; then, the python script is used to cut and classify the violation marked data to make the violation data set, and the data set is divided into training set, validation set and test set in a random form according to a fixed ratio.

[0061] In the embodiment of the application, the video stream data containing violation items with a collection time length of 90 min is subjected to frame processing to obtain image data, 8000 images are randomly selected from the image data as the original data set, the Labelimg is used to mark the violation of the obtained original data set to obtain image and corresponding label file, and the python script is used to cut and classify the violation marked data to make the violation data set. The actual obtained data set basically conforms to the long-tail distribution (some categories have more samples, and the remaining categories have much fewer samples), and the data set is randomly divided into training set, validation set and test set according to 8:1:1, and finally 6400 training images, 800 validation images and 800 test images are obtained. In the embodiment of the application, the violation items include not wearing a safety helmet, not wearing goggles, using a mobile phone, smoking, not wearing work shoes, not wearing industrial gloves, not wearing a red work uniform, no command in the operation site, not wearing a raincoat, not holding a ladder, operation personnel shuttle between hydraulic tongs and drilling tools, not wearing a safety belt and non-work personnel, etc.

[0062] Step 2, use the Grabcut-based enhancement algorithm to enhance the tail class data in the training set. The specific process is:

[0063] Step 2.1, apply the Grabcut-based enhancement algorithm to the tail class data in the training set. The specific process is:

[0064] First, sample images from the same violation category, then model the background and foreground of the image using the Grabcut-based enhancement algorithm and extract the foreground, and then use the Border Matting algorithm to smooth the boundary information of the segmentation, etc. Post-processing operations, this process can also be artificially intervened, the purpose is to increase the diversity of the foreground; Finally, use traditional data enhancement methods such as horizontal flip, rotation, scaling, etc. To enhance the extracted foreground, so as to increase the diversity of the tail class samples in the data set;

[0065] The specific process of the Grabcut-based enhancement algorithm is: first, initialize the model parameters, then use the Gaussian Mixture Model (GMM) to model the image, this model can be used to describe the distribution of foreground and background pixels. Next, use the graph cut algorithm to segment the image. In this process, each pixel in the image is treated as a node, and adjacent pixels are connected to form edges. Then, according to the Gaussian mixture model established in the previous step, calculate the probability of each node belonging to the foreground or background, and use these probabilities as edge weights, and use the minimum cut algorithm to divide the image into foreground and background two parts.

[0066] Step 2.2, collect a part of the related background image data of the oil extraction operation site, fuse the enhanced foreground image with the new background image to obtain an enhanced training set, and randomly store 1-3 foreground images in a single background.

[0067] Step 3, construct an oil extraction violation recognition model based on a Transformer network.

[0068] The overall structure of the oil extraction violation recognition model based on the Transformer network constructed by the present application is as follows: Figure 2As shown, in the model, an RGB image with an input size of 224x224x3 is convolved with 768 16x16 convolution kernels (Conv2d) (step = 16) to obtain image data with an output size of 14x14x768; the data is flattened to obtain an output size of 196x768; a Class Token with a size of 1x768 is spliced, the Class Token is a trainable parameter added (the feature information of the illegal target is stored in the Class Token), and the size of the output data is 197x768; then, the position encoding Position Embedding is added, the symbol in the figure is an addition operation, the size remains unchanged, and the size of the output data is 197x768; then, the data passes through the Dropout layer and enters the Encoder Block layer in the Transformer model, and the model includes 12 Encoder Block layers; then, the data passes through the Layer Norm layer, and the illegal image classification feature in the Class Token with a size of 1x768 is extracted and sent to the MLP Head classification network (including the pre-Logit layer and the Linear layer) to obtain the final illegal classification result of the input image.

[0069] Figure 2 The specific structure of a single Encoder Block layer in the model is as shown in Figure 3 The data input into the Encoder Block layer is first normalized by the Layer Norm layer, enters the Multi-Head Attention (multi-head attention) layer, then passes through the Dropout layer, and the obtained output is added to the initial input; then, the Layer Norm layer, the MLP (multi-layer perception) Block layer (wherein the MLP Block is a group layer structure composed of the Linear, the GELU activation function, the Dropout, the Linear, and the Dropout) and the Dropout layer are sequentially entered again, and the obtained output is added to the input of the Layer Norm layer to obtain the final output.

[0070] Step 4, the overall sample sampling strategy of the oil extraction violation recognition model based on the Transformer network is designed. The specific process is as follows:

[0071] Step 4.1, the class balance sampling probability and the instance balance sampling probability are calculated by using the general formula (1), q [0, 1], when q = 0 is substituted, the class balance sampling probability is obtained When q = 1 is substituted, the instance balance sampling probability is obtained The formula (1) is as follows:

[0072]

[0073] where p j denotes the probability of sampling a sample in class j; i, j denote class indices; n i is the number of samples in class i; C is the total number of classes;

[0074] Step 4.2, introduce the sampling balance parameter λ (λ = C), integrate class balance sampling and instance balance sampling as the sampling strategy of the oil violation identification model based on the Transformer network, make full use of the advantages of different sampling and achieve smooth progressive effect, the overall sampling strategy formula is as follows:

[0075]

[0076] where, denotes the overall sampling strategy; T is the total number of epochs, t is the current epoch.

[0077] Step 5, design the training weight distribution strategy of the oil violation identification model based on the Transformer network. The specific steps are as follows:

[0078] Step 5.1, use the cross-entropy loss function as the first part of the training weight distribution, so that the model can fully extract the feature information of the head class;

[0079]

[0080] where, suppose the class label space of the training set is c ∈ {1, 2,..., C}, C is the total number of classes, for the image with class label c, the model's predicted output is defined as y = [y1, y2,..., y i ,..., y C ].

[0081] Step 5.2, then use the class difficulty to distribute the weight. The accuracy can use F β score (β represents a natural number, usually take 0.5, 1 or 2) or just use the recall rate R c of class c;

[0082] When considering the recall rate and the accuracy at the same time, use F β score calculation, at this time the difficulty of class label c When only considering the accuracy, use the recall rate R c calculation, at this time the difficulty of class label c d c (t) = 1-R c (t);

[0083] F β The score calculation formula is as follows:

[0084]

[0085] Wherein, c is the class label index, R c and P c represent the recall and precision respectively.

[0086] The recall R c is calculated as follows:

[0087] R c = n c / N c #(5)

[0088] Wherein, N c represents the total number of c-class samples in the verification set, and n c represents the number of c-class samples correctly classified by the model in the verification set.

[0089] The corresponding weight allocation strategy is:

[0090] w c (t) = softmax [delta + d c (t)]#(6)

[0091] Wherein, delta = c / C is to control the weighting degree of different classes.

[0092] Step 5.3, finally use the following weight allocation strategy to train the model:

[0093]

[0094] Wherein, the model predicts the output as y, c is the data set class label index, and t is the current round epoch.

[0095] Step 6, train and optimize the oil violation identification model. The specific steps are:

[0096] Step 6.1, set the training parameters of the network; the training parameters include the maximum number of iterations, the learning rate, the optimizer momentum and the weight.

[0097] In the embodiment of the application, the maximum number of iterations is set to 200; the learning rate adopts a cosine scheduling strategy, and is initially 1e-3 and gradually decays to 1e-5; the stochastic gradient descent method (SGD) is adopted, the optimizer momentum is set to 0.9, and the weight decay is 5e-5; the pre-training model uses vit_base_patch16_224.pth; at the same time, the number of identified categories is modified to the total number of violations in the data set.

[0098] Step 6.2, the enhanced illegal training set and the validation set are used to train the oil extraction illegal identification model based on the Transformer network, and at the end of each epoch training, the corresponding class difficulty is calculated in real time, and the weight of different illegal items is allocated accordingly until the training is completed, and the trained oil extraction illegal identification model is obtained.

[0099] Step 6.3, the test set is used to evaluate the trained model, and Top-1 Acc (overall accuracy) and Many-shot (head class) Acc and Few-shot (tail class) Acc are used as evaluation indexes. When the overall accuracy index rises and tends to be stable, it proves that the current model training result is good, and the current model is output as the final oil extraction illegal identification model; otherwise, the model is retrained.

[0100] Step 7, real-time monitoring is performed on the field work video, and the field work video is identified based on the trained oil extraction illegal identification model to obtain the illegal identification classification result, and the identification and judgment information is stored; the specific process is as follows:

[0101] The same processing as step 1 is adopted to frame and cut the video stream to be detected, and the processed frame data is input into the trained oil extraction illegal identification model to obtain the illegal identification classification result.

[0102] The oil extraction illegal identification software system is developed based on the oil extraction illegal identification model. First, the PyTorch deep learning framework is used to build the above-mentioned oil extraction illegal identification model and complete the training, and then the entire software system is deployed in the computer in the well site monitoring room and connected with the well site server. The server will store the video data stream collected by the operation site monitoring camera in real time, and the computer will continuously read the video stream from the server and complete the illegal identification task. When the software system detects illegal behavior, it will remind and visually display in real time, which is convenient for the operation supervisor to handle in real time. The detection data is image data including a single illegal category, and the computer returns the illegal identification data and judgment information to the server for storage backup for management personnel to view and handle.

[0103] In order to prove the feasibility and superiority of the present application, and better simulate the real world, a comparative experiment is carried out on the illegal data set test set. Table 1 shows the comparison results of the recognition accuracy of the present application and the prior art, wherein CE is the recognition model using cross entropy function, CS_CE is the recognition model using cost-sensitive cross entropy function, CB_SM and CB_Focal (gamma = 2) are the recognition models using class balance function, and Focal (gamma = 2) is the recognition model using Focal loss.

[0104] Table 1 Comparison of recognition accuracy of the present application and prior art

[0105]

[0106] These commonly used latest methods such as CE, Focal loss, class-balanced loss (CB_SM and CB_Focal) and cost-sensitive cross-entropy loss (CS_CE) are selected as comparison objects. As can be seen from Table 1, the backbone of the application is used to achieve a total accuracy of 73.13%, which performs well. At the same time, it is noted that the cost-sensitive cross-entropy brings better generalization ability to the model due to the reverse allocation of weights, but the overall performance is not good.

[0107] Of course, the above description is not a limitation on the present application, and the present application is also not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A method for identifying oil production violations based on a Transformer under long tail data, characterized in that, Comprising the following steps: Step 1, collecting the video stream of the oilfield construction site, making the oil production site violation data set; Step 2, using Grabcut-based enhancement algorithm to enhance the tail class data in the training set; the specific process is: Step 2.1, apply Grabcut-based enhancement algorithm on the tail class data in the training set; First, sample images from the same violation category, then use Grabcut-based enhancement algorithm to model the background and foreground of the image and extract the foreground, and then use Border Matting algorithm to smooth the boundary information of the segmentation; Finally, use the data enhancement methods of horizontal flip, rotation and scaling to enhance the extracted foreground; The specific process of Grabcut-based enhancement algorithm is: first, initialize the model parameters, then use Gaussian mixture model to model the image, next, use graph cut algorithm to segment the image; In this process, each pixel in the image is regarded as a node, and the adjacent pixels are connected to form an edge; Then, according to the Gaussian mixture model established in the previous step, calculate the probability of each node belonging to the foreground or background, and use the probability as the weight of the edge, and use the minimum cut algorithm to divide the image into foreground and background two parts; Step 2.2, collect a part of the oil production construction site related background image data, fuse the enhanced foreground image with the new background image to get the enhanced training set, and randomly store 1-3 foreground images in a single background; Step 3, build an oil production violation recognition model based on the Transformer network; Step 4, design the overall sample sampling strategy of the oil production violation recognition model; Step 5, design the training weight distribution strategy of the oil production violation recognition model; Step 6, train and optimize the oil production violation recognition model; Step 7, monitor the real-time acquisition of the field work video, and based on the trained oil production violation recognition model, recognize the field work video to get the violation recognition classification result, and store the recognition and judgment information. 2.The method of claim 1, wherein, The specific process of step 1 is: first, frame the video stream of the oilfield construction site to get frame image data, use Labelimg tool to label the frame image data, get frame image and corresponding label file; Then use python script to cut and classify the violation labeled data to make violation data set, and divide the data set into training set, validation set and test set in random form according to fixed proportion. 3.The method of claim 1, wherein, In step 3, the specific content of the oil extraction violation identification model based on the Transformer network is as follows: an RGB image with an input size of 224x224x3 is input, 768 16x16 convolution kernels are used to perform convolution operation on the input image, and an image data with an output size of 14x14x768 is output; the data is flattened, and an output with a size of 196x768 is output; a trainable parameter Class Token with a size of 1x768 is spliced, and the output data has a size of 197x768, and the Class Token stores the feature information of the violation target; then, the position encoding is added, and the output data has a size of 197x768; then, it passes through the Dropout layer and enters the 12 Encoder Block layers; then, it passes through the Layer Norm layer, and the violation image classification features in the Class Token are extracted and sent to the MLP Head classification network to obtain the final violation classification result of the input image. The specific content of a single Encoder Block layer is as follows: the data input into the Encoder Block layer is first normalized by the Layer Norm layer, enters the multi-head attention layer, and then passes through the Dropout layer; the output obtained is added to the initial input; then, it enters the Layer Norm layer, the MLP Block layer and the Dropout layer in sequence, and the output obtained is added to the input of the Layer Norm layer again to obtain the final output. 4.The method of claim 1, wherein, The specific process of step 4 is as follows: Step 4.1, calculate the class-balanced sampling probability and instance-balanced sampling probability by using the general formula (1), When q = 0 is substituted, the class-balanced sampling probability is obtained ; when is substituted, the instance-balanced sampling probability is obtained ; the formula (1) is as follows: （1） wherein, denotes the probability of a sampled sample in class j; , both denote the class index; is the number of samples in class C is the total number of classes. Step 4.2, introducing the sampling balance parameter , integrating the class balance sampling and instance balance sampling as the sampling strategy of the Transformer network-based oil production violation identification model, and the overall sampling strategy formula is as follows: （2） wherein, denotes the overall sampling strategy; T is the total number of rounds, t is the current round. 5.The method of claim 4, wherein, The specific process of step 5 is as follows: Step 5.1, using a cross-entropy loss function As a first part of the training weight assignment, feature information of the head class is extracted; （3） where the class label space of the training set is assumed to be , C is the total number of classes, and for an image with class label , the model's predicted output is defined as ; Step 5.2, use class difficulty to assign weights; precision measure uses fraction or just use class recall ; When considering both recall and precision, use the score calculation where the difficulty of the class label is taken into account ; when considering only precision, use the recall calculation where the difficulty of the class label is taken into account ; when considering only recall, use the precision calculation where the difficulty of the class label is taken into account ; The formula for calculating fractions is as follows: （4） wherein, is a class label index, and represent recall and precision, respectively; Recall rate The formula is as follows: （5） wherein, denotes the total number of samples of class c in the validation set, denotes the number of samples of class c in the validation set that are correctly classified by the model; The corresponding weight allocation strategy is as follows: （6） wherein, is to control the degree of weighting for different categories; Step 5.3, the following weight allocation strategy is used for training the model: （7） where the model's predicted output for a data point is , c is the data set class label index, and t is the current epoch.

6. The method for identifying oilfield violations based on Transformer under long-tail data according to claim 1, characterized in that, The specific process of step 6 is as follows: Step 6.1, set the training parameters of the network; the training parameters include the maximum number of iterations, the learning rate, the optimizer momentum and the weight; Step 6.2, train the oil extraction violation identification model based on the Transformer network using the enhanced violation training set and the validation set; after each round of epoch training, the corresponding class difficulty is calculated in real time, and the weights of different violation items are allocated accordingly, until the training is completed, and the trained oil extraction violation identification model is obtained; Step 6.3, evaluate the trained model using the test set, and use the overall precision as the evaluation index; when the overall precision index rises and tends to be stable, it proves that the current model training result is good, and the current model is output as the final oil extraction violation identification model; otherwise, the model is retrained. 7.The method of claim 1, wherein, The specific process of the step 7 is that: using the same processing as step 1, the video stream to be detected is subjected to frame division and cutting; the processed frame data is input into the trained oil extraction violation identification model to obtain a violation identification classification result; at the same time, an oil extraction violation identification software system is developed based on the oil extraction violation identification model; first, the PyTorch deep learning framework is used to build the oil extraction violation identification model and complete the training, and then the entire software system is deployed in a computer in the well site monitoring room and connected with a well site server; the server stores the video data stream collected by the operation site monitoring camera in real time, and the computer continuously reads the video stream from the server and completes the violation identification task; when the software system detects a violation behavior, it will remind and visually display in real time, facilitating real-time processing by the operation supervisor; the video stream to be detected is image data including a single violation category, and the computer returns the violation identification data and judgment information to the server for storage backup for the management personnel to view and process.

Images