Identification method of natural gas lifting and pressure operation based on image detection

Abstract

The present invention relates to the field of image recognition and processing technology, and discloses a method for identifying a natural gas lifting and pressure operation based on image detection, comprising the steps of: collecting a photo of a U-type pressure gauge before the lifting and pressure operation, identifying the scale and liquid level mark of the U-type pressure gauge in the photo through an RT-DETR model, and cutting out an initial area; removing image edge noise from the cut-out initial area using gradient operations and closed operations; using Hough transform to identify scale segments in the image with edge noise removed, and identifying numbers in the reading area to obtain pressure readings before the lifting and pressure operation; after performing the lifting and pressure operation, re-obtaining the pressure reading after the lifting and pressure operation; if the absolute value of the difference between the two pressure readings is greater than a set threshold, it is determined that the lifting and pressure operation pressure does not meet the standard, and the lifting and pressure operation needs to be performed again. The present invention monitors the lifting and pressure operation quality of operators in real time and accurately, ensures the gas safety of users, and avoids safety accidents.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and processing, and in particular to a method for identifying a natural gas lifting and pressure operation based on image detection. Background Art

[0002] Natural gas lifting and pressure operation usually uses Figure 2 The U-type pressure gauge shown in the figure is tested. During the test, first open the valve on the natural gas supply pipeline to allow natural gas to flow into the U-type pressure gauge, observe the reading pointed to by the liquid level mark ball in the U-type pressure gauge, and record the initial pressure reading; then, according to actual needs, gradually adjust the valve to control the flow and pressure of the natural gas until the target pressure is reached, then close the valve on the natural gas supply pipeline, and record the final pressure reading. If the absolute value of the difference between the two pressure readings is greater than 100Pa, that is, the height on one side of the U-type pressure gauge decreases by more than 50Pa, and the height on the other side increases by more than 50Pa, it means that the pressure does not meet the standard and the lifting and pressing operation is unqualified, and the lifting and pressing operation needs to be performed again.

[0003] In order to prevent operators from getting away with it due to multiple unqualified lifting and pressure operations, the existing technical solution uses the background manual review of the U-type pressure gauge to determine whether the lifting and pressure operation is qualified, so as to supervise the quality of the operators' work. However, the background cannot achieve real-time supervision, nor can it achieve full coverage of all lifting and pressure operations, which will leave certain hidden dangers to gas safety. Therefore, in order to supervise the operators in real time to complete the lifting and pressure operations in a qualified manner and ensure the gas safety of users, it is necessary to conduct more comprehensive supervision on whether the lifting and pressure operations are qualified. Summary of the invention

[0004] The purpose of the present invention is to monitor the quality of the lifting and pressing operations of operators in real time and accurately, to ensure the gas safety of users, to avoid safety accidents, and to provide an identification method for natural gas lifting and pressing operations based on image detection.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0006] The method for identifying natural gas lifting pressure operation based on image detection includes the following steps:

[0007] Step 1: collect a photo of the U-shaped pressure gauge before the lifting and pressing operation, identify the scale and liquid level mark of the U-shaped pressure gauge in the photo through the RT-DETR model, and cut out the initial area;

[0008] Step 2, using gradient operation and closing operation to remove image edge noise from the cut initial area;

[0009] Step 3, using Hough transform to identify the scale line segments in the image with edge noise removed, and identify the numbers in the reading area to obtain the pressure reading before the lifting and pressing operation;

[0010] Step 4, after the lifting and pressing operation is performed, steps 1 to 3 are repeated to obtain the pressure reading after the lifting and pressing operation; if the absolute value of the difference between the two pressure readings is greater than the set threshold, it is determined that the lifting and pressing operation pressure does not meet the standard and the lifting and pressing operation needs to be performed again.

[0011] Compared with the prior art, the present invention has the following beneficial effects: the present invention can identify whether the two U-type pressure gauge readings before and after the lifting and pressing operation are within the specified range, thereby supervising the operators to complete the specified work, ensuring gas safety, and improving the supervision work of the gas company. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 is a flow chart of the method of the present invention;

[0014] Figure 2 Intended for U-type pressure;

[0015] Figure 3 This is a schematic diagram of the network structure of the RT-DETR model of the present invention;

[0016] Figure 4 This is a schematic diagram of the backbone network structure of the present invention;

[0017] Figure 5 This is a schematic diagram of the BacisBlock_WDBB network structure of the present invention;

[0018] Figure 6 This is a schematic diagram of the WDBB network structure of the present invention;

[0019] Figure 7 A schematic diagram of the initial region of the present invention;

[0020] Figure 8 It is a schematic diagram of corrosion operation of the present invention;

[0021] Fig. 9 It is a schematic diagram of the expansion operation of the present invention;

[0022] Fig.10 Schematic diagram of gradient calculation of the present invention;

[0023] Fig.11 It is a schematic diagram of the closing operation of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0025] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0026] Embodiment 1:

[0027] The present invention is achieved through the following technical solutions: Figure 1 As shown, the method for identifying natural gas lifting pressure operation based on image detection identifies the pressure reading before the lifting pressure operation and the pressure reading after the lifting pressure operation. If the absolute value of the difference between the two pressure readings is greater than the set threshold, it is determined that the lifting pressure operation pressure does not meet the standard and the lifting pressure operation needs to be performed again. The method includes the following steps:

[0028] Step 1: Collect a photo of the U-type pressure gauge before the lifting and pressing operation, identify the scale and liquid level mark of the U-type pressure gauge in the photo through the RT-DETR model, and cut out the initial area.

[0029] Use a camera-enabled device to take photos of the U-type pressure gauge, such as a smart camera-enabled helmet, a smart phone, a wearable recorder, and other devices.

[0030] The scale and level mark of the U-type pressure gauge are identified by the improved RT-DETR (Real-Time DEtection TRansformer) model, and the origin (0,0) is defined at the upper left corner of the image. The positive direction of the x-axis is to the right of the origin, and the positive direction of the y-axis is to the bottom of the origin. Figure 7As shown in (a) in the figure, the scale and liquid level mark are marked respectively with the minimum circumscribed rectangular frame, and the coordinates of the scale are output as (x1, y1, w1, h1), and the coordinates of the liquid level mark are output as (x2, y2, w2, h2), where x1 and y1 are the horizontal and vertical coordinates of the center point of the minimum circumscribed rectangular frame of the scale, w1 and h1 are the width and height of the minimum circumscribed rectangular frame of the scale, x2 and y2 are the horizontal and vertical coordinates of the center point of the minimum circumscribed rectangular frame of the liquid level mark, and w2 and h2 are the width and height of the minimum circumscribed rectangular frame of the liquid level mark.

[0031] like Figure 3 The figure shows the structural diagram of the improved RT-DETR model. The RT-DETR model includes a backbone network (PResNet), an efficient hybrid encoder, an IoU-aware query selector, a decoder, and a detection head. After the original collected photos are input into the backbone network, the features are extracted, and after passing through the efficient hybrid encoder, IoU-aware query selector, decoder, and detection head in sequence, the coordinates of the scale and liquid level mark are output.

[0032] Improve the four residual convolution modules of the backbone network (PResNet), such as Figure 4 As shown in the figure, each residual convolution module is improved into a wide and diverse branch module, namely the first wide and diverse branch module, the second wide and diverse branch module, the third wide and diverse branch module, and the fourth wide and diverse branch module. The first wide and diverse branch module, the second wide and diverse branch module, the third wide and diverse branch module, and the fourth wide and diverse branch module are each composed of two BasicBlock_WDBB (BasicBlock Wide Diverse Branch Block). The improved backbone network includes a 3×3 convolution layer (Conv3×3), a maximum pooling layer (MaxPool), a first wide and diverse branch module, a second wide and diverse branch module, a third wide and diverse branch module, and a fourth wide and diverse branch module connected in sequence. Among them, the second wide and diverse branch module outputs a feature map F3, the third wide and diverse branch module outputs a feature map F4, and the fourth wide and diverse branch module outputs a feature map F5; then the feature maps F3, F4, and F5 enter the efficient hybrid encoder.

[0033] The network structure of each BasicBlock_WDBB is the same, such as Figure 5As shown in the figure, BasicBlock_WDBB includes two WDBBs connected in sequence, one ADD, one ReLU activation function, and a 1×1 convolution layer connected between the input and ADD. The feature map of the input BasicBlock_WDBB first enters the first WDBB with a kernel size k of 3, and then enters the second WDBB with a kernel size k of 3. At the same time, the feature map of the input BasicBlock_WDBB also enters the 1×1 convolution layer (conv1×1), and then the output of the second WDBB is fused with the output of the 1×1 convolution layer in ADD, and finally output through the ReLu activation function.

[0034] The network structure of each WDBB in BasicBlock_WDBB is the same, such as Figure 6 As shown in the figure, WDBB includes 6 branches. The first branch includes a 1×1 convolution layer and a BN layer connected in sequence; the second branch includes a 1×1 convolution layer, a BN layer, a k×k convolution layer, and a BN layer connected in sequence; the third branch includes a 1×1 convolution layer, a BN layer, an AVG layer, and a BN layer; the fourth branch includes a k×k convolution layer and a BN layer connected in sequence; the fifth branch includes a 1×k convolution layer and a BN layer connected in sequence; the sixth branch includes a k×1 convolution layer and a BN layer connected in sequence. The feature maps input into WDBB enter these 6 branches respectively, and the outputs of the 6 branches are fused in ADD and finally output through the SiLu activation function. Among them, ADD is a feature fusion method that adds feature maps but keeps the number of channels unchanged.

[0035] The improved backbone network decomposes and binds different convolution operations. WDBB enables the network to learn different feature representations and improve its detection performance, promoting better feature extraction and representation capabilities within the network. Finally, the RT-DETR model outputs the coordinates of the scale (x1, y1, w1, h1) and the coordinates of the liquid level mark (x2, y2, w2, h2).

[0036] Furthermore, taking (x1, y2) as the reference point, the initial area (x1, y2, w1, 2m×h2) in the photo is intercepted, where m is the integer ratio of the unit length l1 of the scale to the actual height l2 of the liquid level mark. The initial area is intercepted in order to obtain a complete reading, such as Figure 7 As shown, Figure 7 (a) is the minimum bounding rectangle of the scale and liquid level mark output by the RT-DETR model. Figure 7 (b) in the figure is the initial area cut out according to the coordinates (x1, y2, w1, 2m×h2).

[0037] Step 2: Use gradient operation and closing operation to remove image edge noise from the cut-out initial area.

[0038] The gradient operation of an image refers to the difference between the erosion operation result and the dilation operation result of the image to extract the structural edge in the image. Erosion operation and dilation operation are two operations in image morphology, and there is a dual operation relationship between the two. The erosion operation can extract the image trunk by shrinking the image, and the dilation operation can fill the image edge by expanding the image.

[0039] The operator for the erosion operation is , the structural element B eroded image I is recorded as: , the expression of the erosion operation is:

[0040]

[0041] Among them, B(x,y) represents the structural element B with point (x,y) as the reference point; I(x,y) represents the pixel value of image I at point (x,y). Figure 8 As shown in the figure, the reference point (x, y) of the structural element B is translated in the image I. If the structural element B is contained in the image I at this time, the pixel value of the image I at the point (x, y) remains unchanged, otherwise it is set to 0. When the reference point (x, y) traverses all the pixels in the image I, the resulting image R is called the eroded image of the structural element B on the image I. Figure 8 The numbers in represent the position reached by the reference point (x, y) of the structural element B when the image I and the structural element B meet the corrosion operation conditions during the corrosion operation.

[0042] The operator for the dilation operation is , the expanded image I of the structural element B is recorded as: , the expression of the expansion operation is:

[0043]

[0044] Among them, B(x,y) represents the structural element B with point (x,y) as the reference point; I(x,y) represents the pixel value of image I at point (x,y). Fig. 9 As shown in the figure, the reference point (x, y) of the structural element B is translated in the image I. If the intersection of the structural element B and the image I is not an empty set, the pixel value of the image I at the point (x, y) is set to 1 (the gray value is converted to 255), otherwise it remains unchanged. When the reference point (x, y) traverses all the pixels in the image I, the resulting image S is called the dilated image of the structural element B on the image I. Fig. 9 The numbers in represent the position reached by the reference point (x, y) of the structural element B when the image I and the structural element B satisfy the dilation operation conditions during the dilation operation.

[0045] The process of gradient calculation is as follows Fig.10 As shown, the mathematical expression of the gradient operation is:

[0046]

[0047] Among them, I g is the result of the gradient operation.

[0048] like Fig.11 As shown in the figure, the closing operation of the image means first dilating the image and then eroding the dilated image. The mathematical expression of the closing operation is:

[0049]

[0050] Among them, I c is the result of the closing operation.

[0051] from Fig.11 It can be seen that the closing operation of the image can fill the main holes, bridge the main gaps, and connect the separated objects. Therefore, the image edge noise removal method in this step first expands the background contour boundary through gradient operation to extract the edge noise, and then removes the edge noise in the image through closing operation. The specific process is as follows:

[0052] Step 2-1, read the edge contour of the initial area image I through gradient operation, and obtain the gradient operation result I g , and to I g After binarization, the contour image I is obtained. bc ;

[0053] Step 2-2: Fill the contour image I by closing operation bc Then we get Figure I bf :

[0054]

[0055] Step 2-3, compare the initial region image I with the image I bf Perform an AND operation, and then perform a difference operation on the result of the AND operation and the initial area image I to remove edge noise:

[0056]

[0057] Among them, I dt Represents an image with edge noise removed.

[0058] Step 3: Use Hough transform to identify the scale segments in the image with edge noise removed, and identify the numbers in the reading area to obtain the pressure reading before the lifting and pressing operation.

[0059] like Figure 2 or Figure 7As shown, the scale of the U-shaped pressure gauge is composed of multiple line segments, and in this embodiment, the line segment indicated by the scale with 100 as an integer multiple is longer, such as the line segment indicated by 1000Pa and 1100Pa is longer, and the line segment where the scale between two longer line segments is located is shorter, such as the line segment indicated by 1050Pa and 1150Pa is shorter. The scale line segments in the image with edge noise removed are identified using Hough transform, and the coordinates of the vertices at both ends of the line segment are expressed as {[(sx1,sy1),(ex1,ey1)]......[(sxn,syn),(exn,eyn)]}. If n line segments are identified in the image, (sxi,syi) represents the coordinates of the left vertex of the ith line segment, and (exi,eyi) represents the coordinates of the right vertex of the ith line segment, i=1,2,...,n.

[0060] Remove the line segment where exi is less than x1+w1 / 2, that is, if the liquid level mark is on the left side of the U-shaped pressure gauge, remove the scale line segment on the right side; if the liquid level mark is on the right side of the U-shaped pressure gauge, remove the scale line segment on the left side. This can reduce the amount of computational effort in image processing.

[0061] Then, after calculating |eyi-y2| for the remaining line segments, the calculation results are arranged in ascending order to obtain the sequence {d1, d2, ...}; then, according to the coordinates of the vertices at both ends of the line segment, the length Di of the line segment is calculated according to the following formula and then arranged in descending order to obtain the sequence {D1, D2, ...}, and the ordinate sequence {ey1, ey2, ...} of the right end vertex of each line segment is given according to the arrangement order of the sequence {D1, D2, ...}.

[0062]

[0063] Since m is taken according to the ratio of the unit length l1 of the scale and the actual height l2 of the liquid level mark, the purpose is to control the number of readings recognized by the reading area to 1-2. The Di obtained here is the length of the i-th line segment. After arranging in descending order, the scale line segment where the reading of the longest line segment is located, the first digit (when the number of scale digits is 1) or the first and second digits (when the number of scale digits is 2) is the scale line where the reading is located, and the corresponding ey1 or ey1, ey2 is the vertical coordinate of the line segment where the reading is located.

[0064] The RCNN model is trained to obtain the OC (Optical Character Recognition) model that can recognize Arabic numerals in images, and recognize the scale numbers in the image. If the number of scale numbers is 1, the scale number is recorded as Data, which means that the liquid level mark is close to the longer line segment indicated by the scale number. At this time, the reading of the liquid level mark is:

[0065]

[0066] Wherein, R represents the reading of the liquid level mark; Unit represents the unit reading of the U-shaped pressure gauge.

[0067] If the number of scale numbers is 2, and these two scale numbers are respectively denoted as Data1 and Data2, and Data1 > Data2, it indicates that the liquid level mark is close to the shorter line segment between the two longer line segments with scale numbers. At this time, if ey1 > ey2, the reading of the liquid level mark is:

[0068]

[0069] At this time, if ey1 < ey2, the reading of the liquid level mark is:

[0070]

[0071] Wherein, z represents the number of cells between the two line segments pointed to by the scale numbers.

[0072] Step 4, after performing the hanging pressure operation, repeat Steps 1 to 3 to obtain the pressure reading after the hanging pressure operation; if the absolute value of the difference between the two pressure readings is greater than the set threshold, it is determined that the hanging pressure operation pressure does not meet the standard, and the hanging pressure operation needs to be performed again.

[0073] In this embodiment, z = 2, Unit = 50 Pa. After calculating the absolute value of the difference between the two obtained readings R, if it is greater than the set threshold, such as greater than 50 Pa, it is determined that the hanging pressure operation pressure does not meet the standard, and the hanging pressure operation needs to be performed again.

[0074] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for identifying natural gas lifting and pressure operations based on image detection, characterized in that: The following steps are involved: Step 1: collect a photo of the U-shaped pressure gauge before the lifting and pressing operation, identify the scale and liquid level mark of the U-shaped pressure gauge in the photo through the RT-DETR model, and cut out the initial area; The RT-DETR model includes a backbone network, an efficient hybrid encoder, an IoU-aware query selector, a decoder, and a detection head connected in sequence; the backbone network includes a 3×3 convolutional layer, a maximum pooling layer, a first wide multivariate branch module, a second wide multivariate branch module, a third wide multivariate branch module, and a fourth wide multivariate branch module connected in sequence; wherein the second wide multivariate branch module outputs a feature map F3, the third wide multivariate branch module outputs a feature map F4, and the fourth wide multivariate branch module outputs a feature map F5; the feature maps F3, F4, and F5 are input into the efficient hybrid encoder; The first extensive multi-branch module, the second extensive multi-branch module, the third extensive multi-branch module, and the fourth extensive multi-branch module are each composed of two BasicBlock_WDBBs, and the network structure of each BasicBlock_WDBB is the same; BasicBlock_WDBB includes two WDBBs connected in sequence, one ADD, one ReLU activation function, and a 1×1 convolution layer connected between the input and the ADD; the feature map of the input BasicBlock_WDBB first enters the first WDBB, and then enters the second WDBB. At the same time, the feature map of the input BasicBlock_WDBB also enters the 1×1 convolution layer, and then the output of the second WDBB is fused with the output of the 1×1 convolution layer in the ADD, and finally output through the ReLu activation function; In the step 1, the coordinates of the scale and the liquid level mark of the U-shaped pressure gauge in the photo identified by the RT-DETR model are (x1, y1, w1, h1) and (x2, y2, w2, h2), respectively, wherein x1 and y1 are the horizontal coordinate and the vertical coordinate of the center point of the minimum circumscribed rectangular frame of the scale, respectively, w1 and h1 are the width and height of the minimum circumscribed rectangular frame of the scale, respectively, x2 and y2 are the horizontal coordinate and the vertical coordinate of the center point of the minimum circumscribed rectangular frame of the liquid level mark, respectively, and w2 and h2 are the width and height of the minimum circumscribed rectangular frame of the liquid level mark, respectively; with (x1, y2) as the reference point, the initial area of ​​(x1, y2, w1, 2m×h2) in the photo is intercepted, wherein m is the integer ratio of the unit length l1 of the scale to the actual height l2 of the liquid level mark; Step 2, using gradient operation and closing operation to remove image edge noise from the cut initial area; Step 3, using Hough transform to identify the scale line segments in the image with edge noise removed, and identify the numbers in the reading area to obtain the pressure reading before the lifting and pressing operation; Step 4, after the lifting and pressing operation is performed, steps 1 to 3 are repeated to obtain the pressure reading after the lifting and pressing operation; if the absolute value of the difference between the two pressure readings is greater than the set threshold, it is determined that the lifting and pressing operation pressure does not meet the standard and the lifting and pressing operation needs to be performed again.

2. The method for identifying natural gas lifting and pressure operation based on image detection according to claim 1 is characterized in that: The step 2 specifically includes the following steps: Step 2-1, read the edge contour of the initial area image I through gradient operation, and obtain the gradient operation result I g : Wherein, B(x,y) represents the structural element B with point (x,y) as the reference point; I(x,y) represents the pixel value of image I at point (x,y); An operator representing the dilation operation; An operator representing an erosion operation; Then to I g After binarization, the contour image I is obtained. bc ; Step 2-2: Fill the contour image I by closing operation bc Then we get Figure I bf : Step 2-3, compare the initial region image I with the image I bf Perform an AND operation, and then perform a difference operation on the result of the AND operation and the initial area image I to remove edge noise: Among them, I dt Represents an image with edge noise removed.

3. The method for identifying natural gas lifting and pressure operation based on image detection according to claim 2 is characterized in that: Step 3 specifically includes the following steps: Use the Hough transform to identify the scale line segments in the image with edge noise removed. The coordinates of the two end vertices of the line segment are represented as {[(sx1, sy1), (ex1, ey1)]......[(sxn, syn), (exn, eyn)]}. If n line segments are recognized in the image, then (sxi, syi) represents the coordinates of the left end vertex of the i-th line segment, and (exi, eyi) represents the coordinates of the right end vertex of the i-th line segment, where i = 1, 2,..., n; Remove the line segments where exi is less than x1 + w1 / 2. After calculating |eyi - y2| for the remaining line segments, sort the calculation results in ascending order to obtain the sequence {d1, d2,...}; then calculate the length Di of the line segment according to the coordinates of the two end vertices of the line segment using the following formula and sort it in descending order to obtain the sequence {D1, D2,...}, and give the ordinate sequence {ey1, ey2,...} of the right end vertex of each line segment according to the arrangement order of the sequence {D1, D2,...}; Use the RCNN model to identify the scale numbers in the image. If the number of scale numbers is 1, record this scale number as Data. At this time, the reading of the liquid level mark is: where, R represents the reading of the liquid level mark; Unit represents the unit reading of the U-shaped pressure gauge; If the number of scale numbers is 2, record these two scale numbers as Data1 and Data2 respectively, and Data1 > Data2. At this time, if ey1 > ey2, the reading of the liquid level mark is: At this time, if ey1 < ey2, the reading of the liquid level mark is: where, z represents the number of cells between the two line segments pointed to by the scale number.

Images