LDAR target classification detection and automatic processing method based on deep learning

Abstract

The invention relates to the technical field of detection and machine learning fusion, and discloses an LDAR target classification detection and automatic processing method based on deep learning, and the method comprises the steps: building a deep learning model for leakage detection and repair sealing point recognition according to an actual condition; model training and preparation and manual processing of test data; the model is used for processing training data, and a high leakage component test recognition result graph is obtained through continuous parameter adjustment and iterative training; combining a trained result with a coding rule and related data of a background database to obtain a latest sealing point code; and according to the position processed by the model, the processing work of the final LDAR sealing point is automatically completed in combination with a traditional image processing technology. The method has the advantages that LDAR target batch positioning recognition, coding and processing can be achieved, the sealing point processing efficiency is improved, and meanwhile the LDAR filing work complexity is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of fusion of detection and machine learning, and specifically to a LDAR target classification detection and automatic processing method based on deep learning. Background Art

[0002] At present, Leak Detection and Repair (LDAR) is the internationally common VOCs unorganized control technology, which is widely used in petrochemical, chemical and other industries to reduce VOCs emissions by mitigating equipment leakage, while reducing the risk of enterprise safety accidents. Project filing is the first and most critical step in the implementation of LDAR. Its scientificity, completeness and accuracy determine the overall implementation quality of LDAR.

[0003] In existing research, LDAR work mainly includes project sealing point identification, sealing concentration definition, equipment component identification, leaking component repair and data recording. Among them, project sealing point identification is the core of LDAR file building. At present, the sealing files are manually identified and located one by one, and then labeling is performed. This operation consumes a lot of manpower and material resources, and at the same time, due to manual processing errors, it brings many inconveniences to subsequent work.

[0004] In order to solve the sealing point archiving problem in leak detection and repair, it is necessary to design a deep learning-based LDAR target classification detection and automatic processing method to realize batch positioning, identification, encoding and processing of LDAR targets, improve the sealing point processing efficiency, and reduce the complexity of LDAR archiving work. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a LDAR target classification detection and automatic processing method based on deep learning, which has the advantages of realizing batch positioning, identification, encoding and processing of LDAR targets, improving the efficiency of sealing point processing, and reducing the complexity of LDAR archiving work, thereby solving the problems in the above-mentioned background technology.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose of realizing batch positioning, identification, coding and processing of LDAR targets, improving the efficiency of sealing point processing, and reducing the complexity of LDAR archiving work, the present invention provides the following technical solutions: a LDAR target classification detection and automatic processing method based on deep learning, comprising the following steps:

[0009] S1: Establish a deep learning model for leak detection and repair sealing point identification based on actual conditions;

[0010] Preferably, the deep learning model in S1 includes:

[0011] Iuput input layer, the image is input into the network, the size is 640×640×3, the original image is read, and the image is initialized;

[0012] Backbone network, after a series of convolutional layers, C2f, and FDCA for feature extraction, the output size of each convolutional layer gradually decreases, while the depth of the feature increases;

[0013] In the head network, the features are gradually restored to their original size through upsampling, and feature information of different scales is merged through multiple Concat operations, and these features are further processed through the convolution layer again;

[0014] Segmentation layer: The final feature map is converted into a prediction result of the same size as the input image through a convolutional layer to produce the final segmentation output;

[0015] Output layer, the network finally outputs the classification results for each pixel.

[0016] The process of FDCA includes input data in the format of C×H×W;

[0017] The FDCA process includes:

[0018] The input data is in the format of C×H×W;

[0019] The data passes through a group convolution layer with an output size of ×H×W;

[0020] Perform data normalization and Relu activation function processing, and keep the output as ×H×W;

[0021] Scale transformation, the output size is C×KH×KW;

[0022] Extract attention feature map;

[0023] The processed feature maps are merged into feature maps through channel feature dimension expansion and feature fusion operations, and intermediate attention features are calculated;

[0024] The final output result is in the C×H×W feature vector format.

[0025] The calculation formula for receptive field attention feature extraction is as follows:

[0026] F = Spftmax(g i*i (AvgPool(X(C,H,W))))*ReLU(BatchNorm(g k*k (X(C,H,W))))

[0027] Among them, C, H, and W represent the number of channels, the height and width of the feature map respectively, X is the input feature map, and g i*i represents the grouped convolution of size i*i, K represents the convolution kernel size, BatchNorm represents data normalization, AvgPool represents the average pooling operation, and σ represents the nonlinear activation function, which can be Sigmoid or ReLU;

[0028] There are three ways to extract attention feature maps:

[0029] The first processing method: After average pooling, convolution, Sigmoid activation function and channel feature scale expansion, a feature map of size C×KH×KW is obtained. The calculation expression is as follows:

[0030] F1=σ(Conv 1*1 D k (X(H,W)))

[0031] Among them, D k For average pooling, Conv 1*1 It is 1*1 convolution calculation, σ represents Sigmoid activation function;

[0032] The second processing method: After high average pooling and Sigmoid activation function, wide average pooling, convolution, Sigmoid activation function and high-width feature fusion, we finally get a feature map of size C×KH×KW. The feature map calculation expression is as follows:

[0033] F2=X C (KH, KW)*σ(F KH (f KH ))*σ(F KW (f KW ))

[0034] f=δ(F1([z h , z w ]))

[0035] Among them, X is the input feature vector, F KH and F KW It uses 1*1 convolution to convert f KH and f KW Transformed into a tensor with the same channels C as X, σ represents the Sigmoid activation function, f is the intermediate feature map containing the horizontal and vertical spatial dimensions, θ is the nonlinear activation function, [...] represents the link operation along the spatial dimension;

[0036] The third processing method: After average pooling, deviation operation, convolution operation, Sigmoid activation function and deviation space feature extraction, a feature map of size C×KH×KW is obtained. The deviation extraction calculation formula is as follows:

[0037]

[0038] Among them, U represents the mean of all channels C under the X feature vector, X is the input feature vector, and the calculation formula of the deviation attention feature map is as follows:

[0039] F3=σ(Conv 3*3 (D(C,KH,KW)))

[0040] Among them, σ represents the Sigmoid activation function, Conv 3*3 Indicates a convolution operation with a kernel of 3*3.

[0041] The attention mechanism is directly added to the middle layer of the network. The calculation formula of the intermediate attention feature is as follows:

[0042]

[0043] Among them, F is a sequential attention model, including a combination of linear, Conv2d, and MaxPool2d operations, R is a nonlinear activation function, M is the number of the initial sequential model, takes the intermediate feature map as input, and feeds it into another set of attention feature extraction, and finally, the output of Y can be defined as multiplicative fusion, which is calculated as follows:

[0044]

[0045] Among them, Y c is the output feature map, σ represents the activation function, which can be Sigmoid or ReLU;

[0046] S2: Preparation and manual processing of model training and testing data;

[0047] Preferably, the manual processing software used in S2 is Labelmg software, and the file is saved after each picture is processed. The training data is manually annotated using Labelmg software, and the Java development language is used to complete automatic coding and layout of lines and text boxes.

[0048] S3: Use the model to process the training data, and obtain a higher leakage component test recognition result graph through continuous parameter adjustment and iterative training;

[0049] Preferably, the training program in S3 is developed in Python language and Pytorch framework, and the detailed steps are as in S1, and the data in S2 is trained to automatically complete the sealing point coding, and further includes combining the drawing information, coding rules and sealing point information in the background database, cyclically identifying the sealing device components in the drawing, and finally completing the identification of all component categories and automatic layout annotation.

[0050] S4: Using the training results, combined with the encoding rules and related data of the background database, the latest sealing point encoding is obtained;

[0051] S5: Based on the position processed by the model and combined with traditional image processing technology, the final LDAR sealing point processing is automatically completed.

[0052] Preferably, S5 further includes using the training results in S3 to classify and identify components of a single image to be processed, and after identification, using traditional image processing technology to automatically complete the processing of the LDAR sealing device components, and determining the position of the center point of the device. The coordinates of the start and end of the sealing device are (x1, y1) and (x2, y2), then the center position is The center position is the starting point of the line connection; the text box position is determined, the length and width of the picture are (x, y), the length of the text box is L, the width is W, and the length of the line is A. The start and end coordinates of the text box (x3, y3), (x4, y4) unknown variables satisfy the following formula:

[0053]

[0054]

[0055]

[0056]

[0057] Condition 1 is met

[0058]

[0059]

[0060]

[0061]

[0062] Condition 2 is met

[0063]

[0064]

[0065]

[0066]

[0067] Condition 3 is met

[0068]

[0069]

[0070]

[0071]

[0072] Satisfying condition 4 is

[0073] The seal number is input by filling the 4-digit seal point code obtained in S4 into the center of the text box; the line connection is to determine the position of the end point in the rectangular text box according to different conditions, and connect the starting point and the end point of the line. The coordinates of the end point of the line for conditions 1 and 2 are The coordinates of the end points of the lines for conditions 3 and 4 are Save the line and text box starting point positions to the database.

[0074] (III) Beneficial effects

[0075] Compared with the prior art, the present invention provides a LDAR target classification detection and automatic processing method based on deep learning, which has the following beneficial effects:

[0076] 1. By building a deep learning framework, the leaked images were manually processed for component classification, and then the framework was used to train the processed data. In the process, novel modules such as C2f and FDCA were used to enhance the feature extraction capability and improve the model's recognition rate of complex image features;

[0077] Moreover, combining feature information of different scales through the Concat operation helps to capture the details and contextual information in the image and improve the segmentation effect.

[0078] By combining upsampling with convolution, the size is gradually restored while feature processing is performed, which optimizes the quality of feature reconstruction and makes the final output more accurate.

[0079] 2. Combine the training results with traditional databases and image processing to realize batch LDAR device component annotation processing, which is conducive to improving the efficiency of leakage equipment image management and reducing maintenance costs.

[0080] 3. In the present invention, by adopting the FDCA method, attention is improved compared with the CA, CBAM, and CFA processing methods.

[0081] FDCA emphasizes the differentiation of features, and by comparing the similarities and differences between different features, it enhances the network's attention to important features. Compared with CA and CFA, which mainly focus on the weighting of channel features, FDCA pays more attention to the comparison between features and can better capture complex feature relationships.

[0082] FDCA can also perform contrastive learning between different domains, such as comparing features in different regions of the same image. This cross-domain contrast mechanism can effectively extract useful information and reduce the interference of redundant features, thereby improving overall attention.

[0083] In addition, FDCA has adaptive characteristics and can dynamically adjust the attention weights according to the characteristics of the input data. This allows FDCA to more flexibly enhance or suppress features when processing different types of data, thereby improving the overall performance of the network.

[0084] Through contrastive learning, FDCA can more effectively identify and exclude the influence of noise features. CA, CBAM and CFA may be affected by noise, while FDCA enhances the robustness to noise through feature contrast and improves the ability to extract effective features.

[0085] FDCA also performs well in feature fusion. Through contrastive learning, the model takes into account more contextual information when integrating features, thereby improving the comprehensiveness and accuracy of feature representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 is a flow chart of the method of the present invention;

[0087] Figure 2 It is a training model diagram of the present invention;

[0088] Figure 3 is a flow chart of FDCA of the present invention;

[0089] Figure 4 is a flow chart of the batch processing of sealing points of the present invention;

[0090] Figure 5 It is a flow chart of automatic processing of sealing point numbering and connecting lines of the present invention;

[0091] Figure 6 It is a diagram of labeled examples under different conditions of the present invention. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] The present invention provides a technical solution: a LDAR target classification detection and automatic processing method based on deep learning, comprising the following steps:

[0094] S1: Establish a deep learning model for leak detection and repair sealing point identification based on actual conditions;

[0095] S2: Preparation and manual processing of model training and testing data;

[0096] The manual processing software used by S2 is LabelImg software. After each image is processed, a file is saved, such as XX001.txt file; in addition, Label.txt is a classification file for all images.

[0097] S3: Use the model to process the training data, and obtain a higher leakage component test recognition result graph through continuous parameter adjustment and iterative training;

[0098] The training program in S3 is developed in Python and Pytorch framework. The detailed steps are the same as S1, and the data in S2 is trained.

[0099] S4: Using the training results, combined with the encoding rules and related data of the background database, the latest sealing point encoding is obtained;

[0100] In S4, the coding rules in the background database of the current image to be processed and the serial number of the current leaking component are obtained, such as the image code, component code and floor information.

[0101] The drawing file encoding rule is: 6-digit device code + 2-digit area code + 2-digit floor code + 4-digit picture number, for example XRTMT0-01-01-0006.

[0102] The sealing point code is: drawing code + 4-digit sealing point code, where the 4-digit sealing point code consists of 2-digit number + 1-digit fluid state + 1-digit equipment type. For example, the complete sealing point code is XRTMT0-01-01-0006-03LF, indicating that sealing point No. 3 of the drawing is a flange, the fluid state is lightweight liquid, and the text value marked in S5 is the 4-digit sealing gasket code.

[0103] S5: Based on the position processed by the model and combined with traditional image processing technology, the final LDAR sealing point processing is automatically completed.

[0104] In S5, the training results in S3 are used to classify and identify the components of a single image to be processed. After identification, traditional image processing technology is used to automatically complete the processing of the LDAR sealing device components.

[0105] 5.1 The center point position of the device is determined. The coordinates of the start and end of the sealing device are (x1, y1) and (x2, y2). The center position is This center position is the starting point for the line connection;

[0106] 5.2 Determine the position of the text box. The length and width of the image are (x, y), the length of the text box is L, the width is W, and the line length is A. The start and end coordinates of the text box (x3, y3), (x4, y4) unknown variables satisfy the following formula:

[0107]

[0108]

[0109]

[0110]

[0111] Condition 1 is met

[0112]

[0113]

[0114]

[0115] Condition 2 is met

[0116]

[0117]

[0118]

[0119] Condition 3 is met

[0120]

[0121]

[0122]

[0123] Satisfying condition 4 is

[0124] 5.3 Input the sealing gasket number, and fill in the 4-digit sealing point code obtained in S4 into the center of the text box;

[0125] 5.4 Line connection, determine the position of the end point in the rectangular text box according to different conditions, connect the starting point and the end point of the line, where the coordinates of the end point of the line for conditions 1 and 2 are The coordinates of the end points of the lines for conditions 3 and 4 are

[0126] 5.5 Save the line and text box starting point positions to the database.

[0127] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0128] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A LDAR target classification detection and automatic processing method based on deep learning, characterized in that: Consider integrating deep learning, traditional image processing and image database technology to automatically complete the batch labeling of LDAR sealing points. The following steps are involved: S1: Establish a deep learning model for leak detection and repair sealing point identification based on actual conditions; S2: Preparation and manual processing of model training and testing data; S3: Use the model to process the training data, and obtain a higher leakage component test recognition result graph through continuous parameter adjustment and iterative training; S4: Using the training results, combined with the encoding rules and related data of the background database, the latest sealing point encoding is obtained; S5: Based on the position processed by the model and combined with traditional image processing technology, the final LDAR sealing point processing is automatically completed.

2. The LDAR target classification detection and automatic processing method based on deep learning according to claim 1 is characterized in that: The deep learning model in S1 includes: Iuput input layer, the image is input into the network, the size is 640×640×3, the original image is read, and the image is initialized; Backbone network, after a series of convolutional layers, C2f, and FDCA for feature extraction, the output size of each convolutional layer gradually decreases, while the depth of the feature increases; In the head network, the features are gradually restored to their original size through upsampling, and feature information of different scales is merged through multiple Concat operations, and these features are further processed through the convolution layer again; Segmentation layer: The final feature map is converted into a prediction result of the same size as the input image through a convolutional layer to produce the final segmentation output; Output layer, the network finally outputs the classification results for each pixel.

3. The LDAR target classification detection and automatic processing method based on deep learning according to claim 2 is characterized in that: The FDCA process includes: The input data is in the format of C×H×W; The data passes through a group convolution layer, and the output size is CK 2 ×H×W; Perform data normalization and Relu activation function processing, and keep the output as CK 2 ×H×W; Scale transformation, the output size is C×KH×KW; Extract attention feature map; The processed feature maps are merged into feature maps through channel feature dimension expansion and feature fusion operations to calculate intermediate attention features; The final output result is in the C×H×W feature vector format.

4. The LDAR target classification detection and automatic processing method based on deep learning according to claim 3 is characterized in that: The calculation formula for receptive field attention feature extraction is as follows: F=Spftmax(g i*i (AvgPool(X(C,H,W))))*ReLU(BatchNorm(g k*k (X(c,H,W)))) Among them, C, H, and W represent the number of channels, the height and width of the feature map respectively, X is the input feature map, and g i*i represents the grouped convolution of size i*i, K represents the convolution kernel size, BatchNorm represents data normalization, AvgPool represents the average pooling operation, and σ represents the nonlinear activation function, which can be Sigmoid or ReLU.

5. The LDAR target classification detection and automatic processing method based on deep learning according to claim 3 is characterized in that: There are three ways to extract attention feature maps: The first processing method: After average pooling, convolution, Sigmoid activation function and channel feature scale expansion, a feature map of size C×KH×KW is obtained. The calculation expression is as follows: F1=σ(Conv 1*1 D k (X(H,W))) Among them, D k For average pooling, Conv 1*1 It is 1*1 convolution calculation, σ represents Sigmoid activation function; The second processing method: After high average pooling and Sigmoid activation function, wide average pooling, convolution, Sigmoid activation function and high-width feature fusion, we finally get a feature map of size C×KH×KW. The feature map calculation expression is as follows: F2=X C (KH,KW)*σ(F KH (f KH ))*σ(F KW (f KW )) f=6(F1([z h ,z w ])) Among them, X is the input feature vector, F KH and F KW It uses 1*1 convolution to convert f KH and f KW Transformed into a tensor with the same channels C as X, σ represents the Sigmoid activation function, f is the intermediate feature map containing the horizontal and vertical spatial dimensions, θ is the nonlinear activation function, [...] represents the link operation along the spatial dimension; The third processing method: After average pooling, deviation operation, convolution operation, Sigmoid activation function and deviation space feature extraction, a feature map of size C×KH×KW is obtained. The deviation extraction calculation formula is as follows: Among them, U represents the mean of all channels C under the X feature vector, X is the input feature vector, and the calculation formula of the deviation attention feature map is as follows: F3=σ(Conv 3*3 (D(C, KH, KW))) Among them, σ represents the Sigmoid activation function, Conv 3*3 Indicates a convolution operation with a kernel of 3*3.

6. The LDAR target classification detection and automatic processing method based on deep learning according to claim 3 is characterized in that: The attention mechanism is directly added to the middle layer of the network. The calculation formula of the intermediate attention feature is as follows: Among them, F is the sequential attention model, including the combination of linear, Conv2d, and MaxPool2d operations, R is the nonlinear activation function, M is the number of the initial sequential model, takes the intermediate feature map as input, and feeds it into another set of attention feature extraction. Finally, the output of Y can be defined as multiplicative fusion. Y is the final feature output after processing by the sequential attention model. It is obtained by combining the intermediate feature map with the additional attention features. The calculation formula is as follows: Among them, Y c is the output feature map, σ represents the activation function, which can be Sigmoid or ReLU.

7. The LDAR target classification detection and automatic processing method based on deep learning according to claim 1 is characterized in that: The manual processing software used in S2 is Labelmg software. After each picture is processed, the file is saved. The training data is manually annotated using Labelmg software, and the Java development language is used to complete automatic coding and layout of lines and text boxes.

8. The LDAR target classification detection and automatic processing method based on deep learning according to claim 1 is characterized in that: The training program in S3 is developed in Python language and Pytorch framework. The detailed steps are as in S1. The data in S2 is trained to automatically complete the sealing point coding. It further includes combining the drawing information, coding rules and sealing point information in the background database, cyclically identifying the sealing device components in the drawing, and finally completing the identification of all component categories and automatic layout annotation.

9. The LDAR target classification detection and automatic processing method based on deep learning according to claim 1 is characterized in that: S5 further includes using the training results in S3 to classify and identify components of a single image to be processed, and after identification, using traditional image processing technology to automatically complete the processing of the LDAR sealing device components, and determine the position of the center point of the device. The starting and ending coordinates of the sealing device are (x1, y1) and (x2, y2), then the center position is The center position is the starting point of the line connection; the text box position is determined, the length and width of the picture are (x, y), the length of the text box is L, the width is W, and the length of the line is A. The start and end coordinates of the text box (x3, y3), (x4, y4) unknown variables satisfy the following formula: Condition 1 is met Condition 2 is met Condition 3 is met Satisfying condition 4 is The seal number is input by filling the 4-digit seal point code obtained in S4 into the center of the text box; the line connection is to determine the position of the end point in the rectangular text box according to different conditions, and connect the starting point and the end point of the line. The coordinates of the end point of the line for conditions 1 and 2 are The coordinates of the end points of the lines for conditions 3 and 4 are Save the line and text box starting point positions to the database.

Images