Oil pipe connection state determination method and device and electronic equipment

Abstract

The invention provides an oil pipe connection state determination method and device and electronic equipment, and relates to the technical field of monitoring, and the method comprises the steps: obtaining a target detection frame of an oil pipe in an oil discharge monitoring image under the condition that the opening and closing state of an oil discharge port is determined to be open; determining the number of the vertexes of the target detection frame falling into the coverage area of the oil discharge port in the oil discharge monitoring image; and determining the connection state of the oil pipe on the oil discharge port based on the number. According to the technical scheme provided by the invention, the problem of false alarm caused when the oil pipe connection state is detected by using detection frame IOU analysis can be avoided, and the false alarm rate of oil pipe connection state detection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a method, a device and an electronic device for determining the connection state of an oil pipeline. Background Art

[0002] During the oil unloading operation at the gas station, the staff will connect the oil pipe to the oil unloading port of the oil depot after completing the safe operation steps and start the oil unloading task. The oil loading and unloading process is an important part of the safety management of the gas station. Therefore, monitoring the oil unloading operation at the gas station is of great significance to ensure the safety of the gas station.

[0003] In related technologies, the connection status of the oil pipeline in the monitoring image is directly determined through a deep learning classification model, and the intersection over union (IOU) analysis of the detection frame is required during the detection process. However, in the oil unloading operation scenario, due to the influence of the camera's layout angle and the diverse shapes of the oil unloading port, the oil depot port may be tilted or blocked, which may cause false alarms during the IOU analysis of the detection frame. For example, Figure 1 and Figure 2 As shown, in Figure 1 In the figure, rectangular frame 11 is the target detection frame of the oil unloading port, and rectangular frame 12 is the target detection frame of the oil pipe. Figure 2 In the figure, rectangular frame 21 is the target detection frame of the oil unloading port, and rectangular frames 22, 23 and 24 are the target detection frames of the oil pipe. When the IOU analysis of the detection frames is used alone to determine the connection status of the oil pipe, it is easy to mistakenly report the unconnected state of the oil pipe as the connected state, resulting in a false alarm and affecting the safety monitoring of the oil unloading operation. Summary of the invention

[0004] The present invention provides a method, device and electronic device for determining the connection status of an oil pipe, so as to solve the problem that false alarms are easily generated when using detection frame IOU analysis to determine the connection status of an oil pipe in the prior art.

[0005] The present invention provides a method for determining a connection state of an oil pipe, comprising:

[0006] When it is determined that the switch state of the oil unloading port is on, a target detection frame of the oil pipe in the oil unloading monitoring image is obtained;

[0007] Determine the number of vertices of the target detection frame that fall within a coverage area of ​​the oil unloading port in the oil unloading monitoring image;

[0008] A connection state of the oil pipe at the oil unloading port is determined based on the number.

[0009] According to a method for determining a connection status of an oil pipe provided by the present invention, determining a connection status of the oil pipe at the oil unloading port based on the quantity includes:

[0010] When the number is 0, determining that the connection state of the oil pipe on the oil unloading port is not connected;

[0011] In the case where the number is one or two, determining an intersection degree characterization amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining a connection state of the oil pipe at the oil unloading port based on the intersection degree characterization amount;

[0012] When the number is greater than or equal to three, it is determined that the connection state of the oil pipe on the oil unloading port is connected.

[0013] According to a method for determining a connection state of an oil pipe provided by the present invention, when the number is one, determining an intersection degree representation amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining a connection state of the oil pipe at the oil unloading port based on the intersection degree representation amount, comprises:

[0014] Determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port, and determine the penetration depth representation of the target detection frame into the coverage area of ​​the oil unloading port;

[0015] When a first ratio of the area of ​​the intersection region to the area of ​​the target detection frame is greater than or equal to a first preset threshold, and the penetration depth characterization value is greater than or equal to a second preset threshold, determining that the connection state of the oil pipe at the oil unloading port is connected;

[0016] Among them, the intersection degree characterization quantity includes the intersection area and the penetration depth characterization quantity.

[0017] According to a method for determining a connection state of an oil pipe provided by the present invention, when the number is two, determining an intersection degree characterization amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining the connection state of the oil pipe at the oil unloading port based on the intersection degree characterization amount, comprises:

[0018] Determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port;

[0019] When a first ratio of the area of ​​the intersection region to the area of ​​the target detection frame is greater than or equal to a first preset threshold, determining that the connection state of the oil pipe on the oil unloading port is connected;

[0020] The intersection degree characterization quantity includes the area of ​​the intersection region.

[0021] According to a method for determining a connection state of an oil pipe provided by the present invention, the determining of the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port includes:

[0022] For each vertex in the target detection frame that falls within the coverage area of ​​the oil unloading port, determine the intersection point of two adjacent edges of the vertex in the target detection frame and the boundary of the coverage area of ​​the oil unloading port;

[0023] Deduplicating the boundary intersection points corresponding to all the vertices to obtain deduplicated boundary vertices;

[0024] Based on the boundary vertex and all the vertices, an intersection region between the target detection frame and the coverage region of the oil unloading port is determined, and the area of ​​the intersection region is determined to obtain the area of ​​the intersection region.

[0025] According to a method for determining a connection state of an oil pipe provided by the present invention, the step of determining a penetration depth characterization value of the target detection frame penetrating into a coverage area of ​​the oil unloading port comprises:

[0026] Determine an intersection point of a line between the center of gravity of the coverage area of ​​the oil unloading port and the midpoint of the target detection frame and a boundary of the coverage area of ​​the oil unloading port;

[0027] Determine a first vertical distance of the intersection from a first boundary of the target detection frame and a second vertical distance from a second boundary of the target detection frame; the first boundary and the second boundary are adjacent boundaries of vertices of the target detection frame that fall within the coverage area;

[0028] Determine a second ratio of the first vertical distance to the length of the second boundary and a third ratio of the second vertical distance to the length of the first boundary as the penetration depth characterization value;

[0029] The penetration depth characterization value being greater than or equal to the second preset threshold value includes:

[0030] The second ratio is greater than or equal to a third preset threshold, and the third ratio is greater than or equal to a fourth preset threshold; the second preset threshold includes the third preset threshold and the fourth preset threshold.

[0031] A method for determining a connection state of an oil pipe provided by the present invention further includes:

[0032] In response to a target marking operation for the oil unloading port in the oil unloading monitoring image, determining a coverage area of ​​the oil unloading port;

[0033] Performing edge expansion processing on the coverage area of ​​the oil unloading port to obtain a target detection frame image of the oil unloading port;

[0034] Inputting the target detection frame image into an oil unloading port state classification model to obtain the switch state of the oil unloading port output by the oil unloading port state classification model;

[0035] The oil unloading port state classification model is obtained by training an initial oil unloading port state classification model based on the oil unloading port sample image and the switch state label data corresponding to the oil unloading port sample image.

[0036] According to a method for determining the connection status of an oil pipeline provided by the present invention, the lower-layer residual block in the oil unloading port status classification model is obtained by weighted fusion of the results of discrete cosine transform weighted and down-sampling processing of the upper-layer residual block and the original lower-layer residual block, and the discrete cosine transform coefficient block obtained by weighted fusion is weighted fused with the original lower-layer residual block.

[0037] The present invention also provides a device for determining a connection state of an oil pipe, comprising:

[0038] A detection frame acquisition module, used to acquire a target detection frame of the oil pipe in the oil unloading monitoring image when determining that the switch state of the oil unloading port is on;

[0039] A vertex number determination module, used to determine the number of vertices of the target detection frame falling within the coverage area of ​​the oil unloading port in the oil unloading monitoring image;

[0040] A connection status determination module is used to determine the connection status of the oil pipe on the oil unloading port based on the quantity.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining the connection status of an oil pipeline as described above is implemented.

[0042] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for determining the connection status of an oil pipeline as described in any one of the above is implemented.

[0043] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining the connection status of an oil pipeline as described in any one of the above is implemented.

[0044] The method, device and electronic device for determining the connection status of the oil pipe provided by the present invention, when determining that the switch status of the oil unloading port is on, determine the number of vertices of the target detection frame of the oil pipe in the oil unloading monitoring image that fall within the coverage area of ​​the oil unloading port, and the number can reflect the degree of intersection between the area where the oil pipe is located and the coverage area of ​​the oil unloading port. The larger the number, the larger the area of ​​intersection, and the greater the degree of penetration of the oil pipe into the oil unloading port. The penetration degree can reflect the connection status of the oil pipe and the oil unloading port, and then the connection status of the oil pipe at the oil unloading port can be determined based on the number, thereby realizing the detection of the connection status of the oil pipe by utilizing geometric relationships, effectively avoiding the false alarm problem caused by using the detection frame IOU analysis, and reducing the false alarm rate of the oil pipe connection status detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 This is one of the schematic diagrams of a scenario in which a false alarm is generated when using the detection frame IOU analysis to detect the connection status of the oil pipeline in the prior art;

[0047] Figure 2 This is the second schematic diagram of a scenario in which a false alarm is generated when using the detection frame IOU analysis to detect the oil pipeline connection status in the prior art;

[0048] Figure 3 This is one of the flow charts of the method for determining the oil pipe connection state provided by an embodiment of the present invention;

[0049] Figure 4 It is one of the principle schematic diagrams of determining the connection state of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port in the embodiment of the present invention;

[0050] Figure 5 This is the second schematic diagram of the principle of determining the connection state of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port in the embodiment of the present invention;

[0051] Figure 6 This is the third principle schematic diagram of determining the connection state of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port in the embodiment of the present invention;

[0052] Figure 7 This is a fourth schematic diagram of the principle of determining the connection state of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port in the embodiment of the present invention;

[0053] Figure 8 This is a second flow chart of a method for determining a connection state of an oil pipe provided in an embodiment of the present invention;

[0054] Fig. 9 Schematic diagram of the principle of weighting the DCT coefficients of the output of the convolution block in an embodiment of the present invention;

[0055] Fig.10 It is a schematic diagram of the principle of obtaining a lower layer residual block based on an upper layer residual block in the oil unloading port state classification model in an embodiment of the present invention;

[0056] Fig.11 is a structural schematic diagram of a device for determining a connection state of an oil pipe provided in an embodiment of the present invention;

[0057] Fig.12 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] It should be noted that the serial numbers assigned to the objects described in the present invention, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0060] Combine the following Figure 3-Figure 10 The method for determining the connection status of an oil pipeline of the present invention is described. The method for determining the connection status of an oil pipeline can be applied to a monitoring device, and can also be applied to an electronic device such as a terminal device or a server that is communicatively connected to the monitoring device. Among them, the monitoring device can be deployed in a scene such as a gas station where oil unloading operations are required to collect oil unloading monitoring images; the terminal device can include a mobile phone, a computer, a tablet computer, a wearable device, etc.; the server can include an independent server, a cluster server or a cloud server, etc. The method for determining the connection status of an oil pipeline can also be applied to an oil pipeline connection status determination device set in an electronic device such as a terminal device or a server, and the oil pipeline connection status determination device can be implemented by software, hardware, or a combination of the two.

[0061] Figure 3 One of the flow charts of the method for determining the oil pipe connection state provided by an embodiment of the present invention is exemplarily shown, referring to Figure 3As shown, the method for determining the oil pipe connection status may include the following steps 310 to 330.

[0062] Step 310: When it is determined that the switch state of the oil unloading port is on, a target detection frame of the oil pipe in the oil unloading monitoring image is obtained.

[0063] In scenarios where oil unloading operations are required, such as gas stations, monitoring equipment can be deployed near the oil unloading port of the oil depot, and the camera of the monitoring equipment can be used to collect oil unloading monitoring images to monitor the oil unloading process in real time.

[0064] For the oil unloading monitoring image collected by the monitoring equipment, the target detection algorithm can be used to determine the target detection frame of the oil pipe in the oil unloading monitoring image, and the target detection frame is a rectangular frame. For example, the oil unloading monitoring image can be input into a trained oil pipe detection model, and the oil pipe detection model can be used to identify the oil pipe in the oil unloading monitoring image to obtain the target detection frame of the oil pipe in the oil unloading monitoring image. Among them, the oil pipe detection model can be obtained by training the initial oil pipe detection model based on the oil pipe sample image and the corresponding oil pipe label data. The initial oil pipe detection model can be a regional convolutional neural network (R-CNN), a YOLO (You Only Look Once) network, etc., and the present invention is not limited to this.

[0065] The switch state of the oil unloading port includes two states: open and closed. The switch state of the oil unloading port can be determined by analyzing the oil unloading monitoring image. For example, the oil unloading monitoring image can be input into a trained oil unloading port state classification model, and the oil unloading port state classification model can be used to detect the switch state of the oil unloading port. Among them, the oil unloading port state classification model can be obtained by training the initial oil unloading port state classification model based on the oil unloading port sample image and the switch state label data corresponding to the oil unloading port sample image. The initial oil unloading port state classification model can be a traditional convolutional neural network, a residual neural network (ResNet), etc.

[0066] For example, a switch status detection device, such as an infrared detection device, a switch button, etc., can also be arranged on the oil unloading port. When the oil unloading port is opened, the switch status detection device can be triggered to output a signal. When the electronic device detects the signal, it can determine that the oil unloading port is in the open state.

[0067] Step 320: Determine the number of vertices of the target detection frame that fall within the coverage area of ​​the oil unloading port in the oil unloading monitoring image.

[0068] The coverage area of ​​the oil unloading port in the oil unloading monitoring image can be understood as the image area surrounded by the boundary contour of the oil unloading port. Compared with the detection frame of the oil unloading port, the coverage area removes the interference of redundant information such as the background and can accurately reflect the regional position of the oil unloading port.

[0069] The coverage area of ​​the oil unloading port is a polygonal area. It is understandable that the shape design of the oil unloading port of different oil depots may be different, such as a quadrilateral, a pentagon, etc. It can be a regular polygonal area or an irregular polygonal area.

[0070] For example, the contour detection technology can be used to detect the contour of the oil unloading port in the oil unloading monitoring image, and then the coverage area of ​​the oil unloading port can be obtained. Alternatively, the user can mark the oil unloading port, select the polygonal area of ​​the oil unloading port along the contour of the oil unloading port, and issue it to obtain the coverage area of ​​the oil unloading port.

[0071] The target detection frame of the oil pipe is a standard rectangular frame including four vertices. The ray method can be used to determine whether each vertex falls into the coverage area of ​​the oil unloading port, and then the number of vertices falling into the coverage area can be counted.

[0072] Step 330: Determine the connection status of the oil pipe at the oil unloading port based on the quantity.

[0073] The number of target detection frames of the oil pipe falling into the coverage area of ​​the oil unloading port can reflect the degree of intersection between the area where the oil pipe is located and the coverage area of ​​the oil unloading port. The more the number, the larger the intersection area. Correspondingly, the greater the penetration degree of the oil pipe into the oil unloading port. The penetration degree can reflect the connection status of the oil pipe and the oil unloading port, and the connection status of the oil pipe at the oil unloading port can be further judged based on the number.

[0074] For example, when the number is 0, it means that the target detection frame of the oil pipe has no intersection with the coverage area of ​​the oil unloading port, which means that the oil pipe is not in contact with the oil unloading port. Based on this, it can be determined that the oil pipe is not connected to the oil unloading port. Figure 1 In the figure, the oil pipe is not connected to the oil unloading port, the rectangular frame 11 is the target detection frame of the oil unloading port, the rectangular frame 12 is the target detection frame of the oil pipe, and the dotted polygon 13 is the coverage area of ​​the oil unloading port. If the IOU analysis of the detection frame is used to determine whether the oil pipe is connected to the oil unloading port, it is easy to determine that it is connected, resulting in a false alarm. However, using the method of the embodiment of the present invention, the vertices of the target detection frame 12 of the oil pipe do not fall into the coverage area 13 of the oil unloading port, that is, the number is 0, then it can be directly determined that the oil pipe is not connected to the oil unloading port, avoiding the false alarm caused by the IOU analysis of the detection frame, reducing the false alarm rate, and improving the accuracy of the detection of the oil pipe connection state.

[0075] For another example, if there are four vertices in the target detection frame, and three or four vertices fall into the coverage area of ​​the oil unloading port, it means that the oil pipe has penetrated deep enough into the oil unloading port. This penetration can only be shown when the oil pipe is connected to the oil unloading port. In this case, it can be directly determined that the oil pipe and the oil unloading port are connected. For the case where one or two vertices of the target detection frame fall into the coverage area of ​​the oil unloading port, the intersection area of ​​the target detection frame and the coverage area can be analyzed to further determine the connection status of the oil pipe and the oil unloading port. For example, a threshold can be set. If the intersection area is greater than or equal to the threshold, the oil pipe is considered to be connected to the oil unloading port. The threshold can be obtained based on experience or through experiments.

[0076] The method for determining the connection status of an oil pipe provided by an embodiment of the present invention determines the number of vertices of a target detection frame of the oil pipe in the oil unloading monitoring image that fall within the coverage area of ​​the oil unloading port when it is determined that the switch state of the oil unloading port is on. The number can reflect the degree of intersection between the area where the oil pipe is located and the coverage area of ​​the oil unloading port. The larger the number, the larger the area of ​​intersection, and the greater the degree of penetration of the oil pipe into the oil unloading port. The penetration degree can reflect the connection status of the oil pipe and the oil unloading port, and then the connection status of the oil pipe at the oil unloading port can be determined based on the number, thereby realizing the detection of the connection status of the oil pipe by utilizing geometric relationships, effectively avoiding the false alarm problem caused by using the detection frame IOU analysis, reducing the false alarm rate of the oil pipe connection status detection, and improving the accuracy of the oil pipe connection status detection.

[0077] In an embodiment of the present invention, after obtaining the target detection frame of the oil pipeline in the oil unloading monitoring image, for each target detection frame, the connection status of the oil pipeline at the oil unloading port can be determined by determining the number of four vertices of the target detection frame that fall within the coverage area of ​​the oil unloading port.

[0078] Specifically, in an example embodiment, determining the connection status of the oil pipe at the oil unloading port based on quantity may include: when the quantity is 0, determining the connection status of the oil pipe at the oil unloading port as disconnected; when the quantity is one or two, determining a characterization quantity of the degree of intersection between the target detection frame and the coverage area of ​​the oil unloading port, and determining the connection status of the oil pipe at the oil unloading port based on the characterization quantity of the degree of intersection; when the quantity is greater than or equal to three, determining the connection status of the oil pipe at the oil unloading port as connected.

[0079] Among them, the ray method can be used to determine whether a point is within an area, that is, to draw a ray with the point as the endpoint in the horizontal right direction or horizontal left direction, and count the number of intersections between the ray and the polygonal area. If it is an odd number, it is determined that the point is within the polygonal area. If it is an even number, it is determined that the point is outside the polygonal area.

[0080] The intersection degree characterization quantity can characterize the penetration degree of the oil pipe into the oil unloading port area. The threshold of the intersection degree characterization quantity can be preset, and the determined intersection degree characterization quantity is compared with the threshold. If it is greater than or equal to the threshold, it can be determined that the connection state of the oil pipe at the oil unloading port is connected, otherwise it is not connected. The threshold can be determined based on experience or through experiments.

[0081] For example, the intersection degree characterization quantity may include at least one of the intersection area and the penetration depth characterization quantity. The intersection area is the intersection coverage area of ​​the target detection frame and the coverage area of ​​the oil unloading port; the penetration depth characterization quantity is used to characterize the depth of the vertex of the target detection frame protruding into the coverage area of ​​the oil unloading port, which can be the ratio between the vertical distance of the target intersection point from the first boundary and the length of the second boundary and the ratio between the vertical distance of the target intersection point from the second boundary and the length of the first boundary, or the distance of the target intersection point from the vertex falling into the coverage area. The first boundary and the second boundary are the adjacent boundaries of a vertex of the target detection frame falling into the coverage area; the target intersection point is the intersection of the line between the center of the target detection frame and the center of gravity of the coverage area and the boundary of the coverage area.

[0082] Specifically, Figure 4 One of the principle schematic diagrams of determining the connection status of the oil pipe using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port is shown, which is a case where no vertex of the target detection frame falls into the coverage area of ​​the oil unloading port, wherein the polygonal area 40 represents the coverage area of ​​the oil unloading port, and the rectangular frame 41 represents the target detection frame of the oil pipe. In this case, the oil pipe has no intersection with the oil unloading port, indicating that the oil pipe is not connected to the oil unloading port.

[0083] In an exemplary embodiment, when the number of vertices of the target detection frame that fall into the coverage area of ​​the oil unloading port is one, the intersection degree characterization quantity may include the intersection area area and the penetration depth characterization quantity. Accordingly, determining the intersection degree characterization quantity between the target detection frame and the coverage area of ​​the oil unloading port and determining the connection state of the oil pipe at the oil unloading port based on the intersection degree characterization quantity may include: determining the intersection area area of ​​the target detection frame and the coverage area of ​​the oil unloading port, and determining the penetration depth characterization quantity of the target detection frame probing into the coverage area of ​​the oil unloading port; when the first ratio of the intersection area area to the area of ​​the target detection frame is greater than or equal to the first preset threshold value, and the penetration depth characterization quantity is greater than or equal to the second preset threshold value, determining that the connection state of the oil pipe at the oil unloading port is connected.

[0084] Among them, the first preset threshold and the second preset threshold can be determined based on experience or by experiment. The penetration depth characterization amount can be the ratio between the vertical distance of the target intersection from the first boundary and the length of the second boundary, and the ratio between the vertical distance of the target intersection from the second boundary and the length of the first boundary, wherein the first boundary and the second boundary are adjacent boundaries of a vertex of the target detection frame falling into the coverage area. At this time, it can be understood that the second preset threshold can include preset thresholds corresponding to the two ratios. The penetration depth characterization amount can also be the distance of the target intersection from the vertex falling into the coverage area. At this time, it can be understood that the second preset threshold is the preset threshold corresponding to the distance. Among them, the target intersection is the intersection of the line between the center of the target detection frame and the center of gravity of the coverage area and the boundary of the coverage area.

[0085] For example, determining the area of ​​the intersection region between the target detection frame and the coverage area of ​​the oil unloading port may include: for each vertex in the target detection frame that falls within the coverage area of ​​the oil unloading port, determining the boundary intersection point between two adjacent edges of the vertex in the target detection frame and the coverage area of ​​the oil unloading port; deduplicating the boundary intersection points corresponding to all vertices to obtain deduplicated boundary vertices; determining the intersection region of the target detection frame and the coverage area of ​​the oil unloading port based on the boundary vertices and all vertices, and determining the area of ​​the intersection region to obtain the area of ​​the intersection region.

[0086] Among them, the area of ​​the intersection area can be calculated using a polygon area solution algorithm to obtain the area of ​​the intersection area. For example, the shoelace theorem (i.e., the Gaussian area formula) can be used to calculate the area of ​​the intersection area. Specifically, all vertices of the boundary vertices after deduplication and the target detection frame that fall into the coverage area of ​​the oil unloading port can form a point set R. All points in the point set R can be sorted in order by connecting them in sequence to form a polygon, and the intersection area S can be calculated according to the following shoelace theorem formula (1):

[0087]

[0088] Where n represents the number of points in the point set R, i represents the i-th point in R, (x i ,y i ) represents the coordinates of the ith point, (x i+1 ,y i+1 ) represents the coordinates of the i+1th point, (x i-1 ,y i-1 ) represents the coordinates of the i-1th point. When the subscripts i+1 of x and y are greater than n, there is x i+1 =x1,y i+1 =y1.

[0089] For example, Figure 5The second schematic diagram shows the principle of determining the connection status of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port, which is the case where one vertex of the target detection frame falls into the coverage area of ​​the oil unloading port. Figure 5 As shown, the polygonal area 50 represents the coverage area of ​​the oil unloading port, and the rectangular frame 51 represents the target detection frame of the oil pipe. The vertex P of the target detection frame 51 falls into the coverage area 50 of the oil unloading port. The two adjacent sides of the vertex P intersect with the boundary of the coverage area 50 of the oil unloading port at points A and B, and the boundary intersection points A and B are obtained. Only one vertex of the target detection frame 51 falls into the coverage area 50 of the oil unloading port. The boundary vertices after deduplication are still points A and B. Then, points A, B and vertex P are connected in sequence to form the intersection area of ​​the target detection frame 51 and the coverage area 50 of the oil unloading port, that is, triangle ABP. The area of ​​triangle ABP can be calculated using the polygon area solution algorithm to obtain the area of ​​the intersection area. For example, points A, B and vertex P can be used to form a point set R, and the area of ​​the intersection area can be determined using the above-mentioned shoelace theorem formula (1).

[0090] exist Figure 5 In the case shown, the connection state of the oil pipe and the oil unloading port can be judged by determining whether the area of ​​the triangle ABP meets the corresponding preset threshold requirement. If so, it is determined that the oil pipe and the oil unloading port are connected, otherwise they are not connected. Alternatively, the connection state of the oil pipe and the oil unloading port can be judged by determining whether the penetration depth characterization quantity of the vertex P into the coverage area of ​​the oil unloading port meets the corresponding preset threshold requirement. If so, it is determined that the oil pipe and the oil unloading port are connected, otherwise they are not connected. Alternatively, the connection state of the oil pipe and the oil unloading port can be judged by determining whether the area of ​​the triangle ABP and the penetration depth characterization quantity of the vertex P into the coverage area of ​​the oil unloading port both meet the corresponding preset threshold requirements. If both meet, it is determined that the oil pipe and the oil unloading port are connected, otherwise they are not connected.

[0091] For example, determining the penetration depth characterization amount of the target detection frame into the coverage area of ​​the oil unloading port may include: determining the intersection of the line between the center of gravity of the coverage area of ​​the oil unloading port and the midpoint of the target detection frame and the boundary of the coverage area of ​​the oil unloading port; determining the first vertical distance of the intersection from the first boundary of the target detection frame and the second vertical distance from the second boundary of the target detection frame, wherein the first boundary and the second boundary are adjacent boundaries of the vertices of the target detection frame falling into the coverage area; determining the second ratio of the first vertical distance to the length of the second boundary and the third ratio of the second vertical distance to the length of the first boundary as the penetration depth characterization amount. Accordingly, the penetration depth characterization amount being greater than or equal to the second preset threshold includes: the second ratio being greater than or equal to the third preset threshold, and the third ratio being greater than or equal to the fourth preset threshold; the second preset threshold includes the third preset threshold and the fourth preset threshold.

[0092] Among them, determining the first vertical distance of the intersection from the first boundary of the target detection frame and the second vertical distance from the second boundary of the target detection frame may include: determining the first projection point of the intersection on the first boundary of the target detection frame and the second projection point on the second boundary of the target detection frame; determining the distance between the first projection point and the target vertex to obtain the second vertical distance; determining the distance between the second projection point and the target vertex to obtain the first vertical distance; wherein the target vertex is a vertex of the target detection frame that falls within the coverage area.

[0093] For example, Figure 5 For example, the center of gravity P of the coverage area 50 of the oil unloading port can be determined first. a and the midpoint O of the target detection frame 51, and then determine the center of gravity P a The line P connecting the midpoint O a The intersection point P of O and the boundary of the coverage area 50 of the oil discharge port b . Determine the intersection point P respectively b The first projection point P on the first boundary of the target detection frame 51 w and the second projection point P on the second boundary h , then determine the first projection point P w The distance from the vertex P gives the second perpendicular distance D w =P w P, determine the second projection point P h The distance from the target vertex P, get the first vertical distance D h =P h P. Then calculate the first vertical distance D h The second ratio D of the length H of the second boundary h / H, calculate the second vertical distance D w The third ratio D of the length W of the first boundary w / W, D h / H is compared with the third preset threshold value, and D w / W is compared with the fourth preset threshold value. If D h / H is greater than or equal to the third preset threshold, and D w If / W is greater than or equal to the fourth preset threshold, it is determined that the oil pipe in the target detection frame 51 is connected to the oil unloading port, otherwise it is determined that it is not connected.

[0094] In this way, when the number of vertices of the target detection frame that falls into the coverage area of ​​the oil unloading port is one, the connection status of the oil pipeline and the oil unloading port can be detected using geometric methods based on the geometric area relationship between points, lines, and surfaces, which can effectively avoid the false alarm problem caused by simply using the detection frame IOU analysis.

[0095] In an exemplary embodiment, when the number of vertices of the target detection frame that fall within the coverage area of ​​the oil unloading port is two, the intersection degree characterization quantity may include the intersection area area. Accordingly, determining the intersection degree characterization quantity between the target detection frame and the coverage area of ​​the oil unloading port and determining the connection state of the oil pipe at the oil unloading port based on the intersection degree characterization quantity may include: determining the intersection area area of ​​the target detection frame and the coverage area of ​​the oil unloading port; when a first ratio of the intersection area area to the area of ​​the target detection frame is greater than or equal to a first preset threshold, determining the connection state of the oil pipe at the oil unloading port is connected; when a first ratio of the intersection area area to the area of ​​the target detection frame is less than a first preset threshold, determining the connection state of the oil pipe at the oil unloading port is not connected.

[0096] Among them, the method for determining the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port can be determined by referring to the intersection area calculation method of the above embodiment, such as using the above shoelace theorem formula (1) for calculation, which will not be repeated here.

[0097] Figure 6 The third schematic diagram shows the principle of determining the connection status of the oil pipe by using the number of vertices of the target detection frame of the oil pipe that fall into the coverage area of ​​the oil unloading port. It is a case where two vertices of the target detection frame fall into the coverage area of ​​the oil unloading port. In this case, the intersection area of ​​the two can be judged. If the ratio of the intersection area to the area of ​​the target detection frame is greater than or equal to the first preset threshold, it is determined that the oil pipe and the oil unloading port are connected, otherwise it is determined that they are not connected. For details, refer to Figure 6 As shown, the polygonal area 60 represents the coverage area of ​​the oil unloading port, the rectangular frame 61 represents the target detection frame of the oil pipe, the vertices P1 and P2 of the target detection frame 61 fall into the coverage area 60 of the oil unloading port, the two adjacent sides of the vertex P1 intersect with the boundary of the coverage area 60 of the oil unloading port at point E, and the two adjacent sides of the vertex P2 intersect with the boundary of the coverage area 60 of the oil unloading port at point F, then the intersection area shown in the shaded part can be determined, and the area of ​​the shaded part can be determined by using the intersection area area calculation method as described above to obtain the intersection area area of ​​the target detection frame 61 and the coverage area 60 of the oil unloading port. Then, the first ratio between the intersection area area and the area of ​​the target detection frame 61 is determined. If the first ratio is greater than or equal to the first preset threshold, it is determined that the connection state of the oil pipe on the oil unloading port is connected, otherwise it is not connected.

[0098] In this way, when the number of vertices of the target detection frame that fall into the coverage area of ​​the oil unloading port is two, the connection status of the oil pipeline and the oil unloading port can be detected using geometric methods based on the geometric area relationship between points, lines, and surfaces, which can effectively avoid the false alarm problem caused by simply using the detection frame IOU analysis.

[0099] In an example embodiment, Figure 7 The fourth schematic diagram shows the principle of determining the connection status of the oil pipe by using the number of vertices of the target detection frame of the oil pipe falling into the coverage area of ​​the oil unloading port, which is the case where three or more vertices of the target detection frame fall into the coverage area of ​​the oil unloading port. Figure 7 As shown, the polygonal area 70 represents the coverage area of ​​the oil unloading port, and the rectangular frame 71 represents the target detection frame of the oil pipe. The vertices P1, P2 and P3 of the target detection frame 71 fall into the coverage area 70 of the oil unloading port. In this case, it can be directly determined that the oil pipe is connected to the oil unloading port.

[0100] The method for determining the connection status of an oil pipeline provided in an embodiment of the present invention can, when determining that the switch state of the oil unloading port is on, detect the connection status of the oil pipeline and the oil unloading port based on the number of vertices of the target detection frame of the oil pipeline in the oil unloading monitoring image that fall within the coverage area of ​​the oil unloading port. Based on the actual scene of the oil unloading operation, the method is based on the geometric relationship between points, lines, and surfaces, and the distance and intersection relationship between the points, lines, and surfaces are judged. This can effectively avoid the false alarm problem caused by simply using the detection frame IOU analysis due to shooting angle deviation, shape difference, and occlusion in the actual scene.

[0101] In the methods of the above embodiments, the switch state of the oil unloading port can be determined first, and when the switch state of the oil unloading port is open, the connection state of the oil pipe and the oil unloading port can be determined. In the process of determining the switch state of the oil unloading port, the position of the coverage area of ​​the oil unloading port can be determined first, and then the switch state of the oil unloading port can be determined.

[0102] Since the shapes of the oil unloading ports at different gas stations are varied, and the angles of the surveillance cameras deployed at each gas station are not unique, if the detection model based on deep learning is directly used to determine the location of the oil unloading port, it is easy to be affected by factors such as environmental and scene changes, resulting in false detection or missed detection. In view of this, considering that the regional position of the oil unloading port in each gas station and the angle of the surveillance camera are fixed in the real scene, the embodiment of the present invention proposes an irregular fixed target pre-marking method, which pre-marks the coverage area (polygonal area) of the oil unloading port in the oil unloading monitoring image, and can determine the coverage area of ​​the oil unloading port in the oil unloading monitoring image around the clock, effectively avoiding the influence of factors such as backlight, blur, dim light at night, and weather changes.

[0103] In the process of judging the switch state of the oil unloading port, the classification model based on deep learning can be used to analyze the local image of the coverage area of ​​the oil unloading port, and the switch state of the oil unloading port can be judged according to the output result of the classification model. For example, the contour coordinates of the marked coverage area of ​​the oil unloading port can be obtained first, and the coverage area of ​​the oil unloading port can be expanded by the circumscribed rectangle, and the image after the expansion process can be input into the classification model of the oil unloading port state detection for classification, and the switch state of the oil unloading port can be determined according to the classification result. For example, when multiple consecutive frames of images show that the oil unloading port is in the open state, it is determined that the switch state of the oil unloading port is in the open state, and the detection of the connection state between the oil pipe and the oil unloading port is started at this time.

[0104] Based on this, in an exemplary embodiment of the present invention, Figure 8 The second flow chart of the method for determining the oil pipe connection state provided by the embodiment of the present invention is exemplarily shown. Figure 8 As shown, the method for determining the oil pipe connection state may further include a step of detecting the switch state of the oil unloading port, specifically including the following steps 810 to 830.

[0105] Step 810: In response to a target marking operation on an oil unloading port in an oil unloading monitoring image, determining a coverage area of ​​the oil unloading port.

[0106] After the electronic device acquires the oil unloading monitoring image collected by the monitoring device, the electronic device can display the oil unloading monitoring image on the display interface. The electronic device can receive a target marking operation of the user for the oil unloading port in the oil unloading monitoring image, and determine the coverage area of ​​the oil unloading port in response to the target marking operation.

[0107] For example, Figure 1 For example, the user can perform a target marking operation on the area where the oil unloading port is located in the oil unloading monitoring image, such as a stroke operation, to select the oil unloading port. The electronic device can determine the coverage area of ​​the oil unloading port shown in the polygonal area 13 based on the received target marking operation.

[0108] In this way, the irregular fixed target pre-marking method is used to determine the location of the coverage area of ​​the oil unloading port, which can avoid the problem of difficulty in obtaining the oil unloading port area due to factors such as backlighting, image blur, dim light at night, and weather changes.

[0109] Step 820: Expand the coverage area of ​​the oil unloading port to obtain a target detection frame image of the oil unloading port.

[0110] For example, Figure 1For example, the circumscribed rectangle of the polygonal area 13 can be determined first, and then the circumscribed rectangle can be expanded to form a regional image as shown in the rectangular frame 14, and the regional image 14 can be determined as the target detection frame image of the oil unloading port. In this way, the polygonal area 13 is first determined by the target marking operation, and then the circumscribed rectangle of the polygonal area 13 is expanded to obtain the target detection frame image of the oil unloading port. Compared with directly using the target detection algorithm to obtain the target detection frame image of the oil unloading port, it can avoid the problem that the target of the oil unloading port cannot be accurately detected by the target detection algorithm due to factors such as backlight, image blur, dim light at night, and weather changes.

[0111] Step 830: input the target detection frame image into the oil unloading port state classification model to obtain the switch state of the oil unloading port output by the oil unloading port state classification model.

[0112] The oil unloading port state classification model is obtained by training the initial oil unloading port state classification model based on the oil unloading port sample image and the switch state label data corresponding to the oil unloading port sample image. The initial oil unloading port state classification model can be a traditional CNN, ResNet or other neural network model.

[0113] For example, when the oil unloading port state classification model detects that the switch states of the oil unloading port corresponding to N consecutive frames of target detection frame images are all on, it is determined that the switch state of the oil unloading port is on. Wherein, N is a positive integer and can be preset.

[0114] For example, the oil unloading port state classification model may be a deep learning classification model improved based on discrete cosine transform (DCT), which can improve the robustness of the model for classifying the switch state of the oil unloading port.

[0115] Specifically, the lower residual block in the oil unloading port state classification model is obtained by weighted fusion of the results of DCT weighting and downsampling of the upper residual block and the original lower residual block, and weighted fusion of the DCT coefficient block obtained by weighted fusion with the original lower residual block.

[0116] Among them, the oil unloading port state classification model can be a new ResNet network that is improved based on DCT on the standard ResNet network. The original lower layer residual block is the residual convolution block in the standard ResNet network structure, and the lower layer residual block is the next residual convolution block of the upper layer residual block in the improved new ResNet network structure. It can be understood that the original lower layer residual block can be used as a data flow bridge between the upper layer residual block and the lower layer residual block in the improved new ResNet network structure.

[0117] Fig. 9The schematic diagram of the principle of weighting the DCT coefficients of the output of the convolution block is shown as an example, taking the residual network as an example of the oil discharge port state classification model. Fig. 9 As shown in the figure, for each residual convolution block of the original residual network, its output channel can be split, one branch performs normal convolution operation, and the other branch introduces DCT coefficients as the weight coefficients of the branch for DCT weighted processing. Finally, the feature maps obtained by the two branches are spliced, which can reduce the channel redundancy of the model and improve the high-frequency feature expression ability of the model. For the oil unloading port state classification model, obvious edge features in the image can be obtained. Moreover, in image processing, edge features belong to the high-frequency part in the image frequency domain. Based on this, the high-frequency part of the image is weighted using DCT coefficients, which can achieve effective extraction of oil unloading port features.

[0118] Combination Fig. 9 , Fig.10 The schematic diagram of the principle of obtaining the lower residual block based on the upper residual block in the oil unloading port state classification model is shown as an example. Taking the oil unloading port state classification model as a residual network as an example, refer to Fig. 9 As shown, the output of each upper residual block of the residual network and the output of the original lower residual block are weighted by DCT coefficients at the DCT layer. The DCT coefficient weighting can enhance the high-frequency part of the feature map output by each residual convolution block, which can lead to a high-frequency feature compensation branch. The features of the upper residual block after DCT coefficient weighting are downsampled again, and the processed results are weighted fused with the feature map of the original lower residual block after DCT coefficient weighting. The DCT coefficient block obtained by weighted fusion is further weighted fused with the output of the original lower residual block to obtain the lower residual block. The processing principle of DCT coefficient weighting of the residual block output at the DCT layer can be referred to. Fig. 9 Description.

[0119] Since the global average pooling (GAP) layer is a special case of DCT, weighting the DCT coefficients is equivalent to compensating for the high-frequency components required for model recognition on the basis of GAP, thereby improving the robustness of the model for classifying the switch state of the oil unloading port.

[0120] The method for determining the connection status of the oil pipe provided in the embodiment of the present invention can use the irregular fixed target pre-marking method to determine the location of the coverage area of ​​the oil unloading port, which can avoid the problem of difficulty in obtaining the oil unloading port area due to scenes and environmental factors such as backlight, image blur, dim light at night, weather changes, etc. At the same time, the switch status of the oil unloading port is detected by using the residual network model improved based on DCT, which can strengthen the extraction of the edge features of the oil unloading port and improve the accuracy of the recognition of the switch status of the oil unloading port.

[0121] The oil pipe connection state determining device provided by the present invention is described below. The oil pipe connection state determining device described below and the oil pipe connection state determining method described above can be referred to each other.

[0122] Fig.11 The schematic diagram of the structure of the oil pipe connection state determination device provided by the embodiment of the present invention is exemplarily shown. Fig.11 As shown, the oil pipeline connection status determination device may include: a detection frame acquisition module 1110, which is used to acquire a target detection frame of the oil pipeline in the oil unloading monitoring image when it is determined that the switch state of the oil unloading port is on; a vertex number determination module 1120, which is used to determine the number of vertices of the target detection frame that fall within the coverage area of ​​the oil unloading port in the oil unloading monitoring image; and a connection status determination module 1130, which is used to determine the connection status of the oil pipeline at the oil unloading port based on the number determined by the vertex number determination module 1120.

[0123] In an exemplary embodiment, the connection status determination module 1130 may include: a first determination unit, used to determine that the connection status of the oil pipeline at the oil unloading port is not connected when the number determined by the vertex number determination module 1120 is 0; a second determination unit, used to determine the intersection degree characterization amount between the target detection frame and the coverage area of ​​the oil unloading port when the number determined by the vertex number determination module 1120 is one or two, and determine the connection status of the oil pipeline at the oil unloading port based on the intersection degree characterization amount; a third determination unit, used to determine that the connection status of the oil pipeline at the oil unloading port is connected when the number determined by the vertex number determination module 1120 is greater than or equal to three.

[0124] In an exemplary embodiment, the second determination unit may include: a first determination subunit, used to determine the area of ​​the intersection region between the target detection frame and the coverage area of ​​the oil unloading port when the number determined by the vertex number determination module 1120 is one; a second determination subunit, used to determine the penetration depth characterization quantity of the target detection frame into the coverage area of ​​the oil unloading port; a third determination subunit, used to determine that the connection state of the oil pipe at the oil unloading port is connected when a first ratio of the intersection region area to the area of ​​the target detection frame is greater than or equal to a first preset threshold value and the penetration depth characterization quantity is greater than or equal to a second preset threshold value; wherein the intersection degree characterization quantity includes the intersection region area and the penetration depth characterization quantity.

[0125] In an exemplary embodiment, the second determination unit may include: a first determination subunit, used to determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port when the number determined by the vertex number determination module 1120 is two; a fourth determination subunit, used to determine that the connection state of the oil pipe at the oil unloading port is connected when a first ratio of the intersection area to the area of ​​the target detection frame is greater than or equal to a first preset threshold; wherein the intersection degree characterization quantity includes the intersection area.

[0126] In an exemplary embodiment, the first determination subunit is specifically used to: determine, for each vertex in the target detection frame that falls within the coverage area of ​​the oil unloading port, a boundary intersection point of two adjacent edges of the vertex in the target detection frame and the coverage area of ​​the oil unloading port; deduplicate the boundary intersection points corresponding to all vertices to obtain deduplicated boundary vertices; determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port based on the boundary vertices and all vertices, and determine the area of ​​the intersection area to obtain the area of ​​the intersection area.

[0127] In an exemplary embodiment, the second determination subunit is specifically used to: determine the intersection of a line between the center of gravity of the coverage area of ​​the oil unloading port and the midpoint of the target detection frame and a boundary of the coverage area of ​​the oil unloading port; determine a first vertical distance of the intersection from a first boundary of the target detection frame and a second vertical distance from a second boundary of the target detection frame, wherein the first boundary and the second boundary are adjacent boundaries of the vertices of the target detection frame that fall within the coverage area; determine a second ratio of the first vertical distance to the length of the second boundary and a third ratio of the second vertical distance to the length of the first boundary as a penetration depth characterization value; wherein the penetration depth characterization value being greater than or equal to the second preset threshold includes: the second ratio being greater than or equal to the third preset threshold, and the third ratio being greater than or equal to the fourth preset threshold; the second preset threshold includes the third preset threshold and the fourth preset threshold.

[0128] In an exemplary embodiment, the oil pipeline connection status determination device may also include: an oil unloading port area determination module, which is used to determine the coverage area of ​​the oil unloading port in response to the target marking operation on the oil unloading port in the oil unloading monitoring image; an edge expansion processing module, which is used to perform edge expansion processing on the coverage area of ​​the oil unloading port to obtain a target detection frame image of the oil unloading port; a switch state detection module, which is used to input the target detection frame image into the oil unloading port state classification model to obtain the switch state of the oil unloading port output by the oil unloading port state classification model; wherein the oil unloading port state classification model is obtained by training an initial oil unloading port state classification model based on the oil unloading port sample image and the switch state label data corresponding to the oil unloading port sample image.

[0129] In an exemplary embodiment, the lower residual block in the oil unloading port state classification model is obtained by weighted fusion of the result of discrete cosine transform weighted and down-sampling processing of the upper residual block and the original lower residual block, and the discrete cosine transform coefficient block obtained by weighted fusion is weighted fused with the original lower residual block.

[0130] Fig.12 An example of a structural diagram of an electronic device is shown in FIG. Fig.12 As shown, the electronic device may include: a processor 1210, a communication interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communication interface 1220 and the memory 1230 communicate with each other through the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the method for determining the oil pipe connection state provided by any of the above method embodiments.

[0131] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for determining the oil pipeline connection status provided by any of the above method embodiments.

[0133] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for determining the oil pipe connection status provided by any of the above method embodiments is implemented.

[0134] Illustratively, the computer-readable storage medium comprises a non-transitory computer-readable storage medium.

[0135] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a connection state of an oil pipe, characterized in that: include: When it is determined that the switch state of the oil unloading port is on, a target detection frame of the oil pipe in the oil unloading monitoring image is obtained; Determine the number of vertices of the target detection frame that fall within a coverage area of ​​the oil unloading port in the oil unloading monitoring image; A connection state of the oil pipe at the oil unloading port is determined based on the number.

2. The method for determining the oil pipe connection status according to claim 1, characterized in that: The step of determining the connection state of the oil pipe at the oil unloading port based on the quantity includes: When the number is 0, determining that the connection state of the oil pipe on the oil unloading port is not connected; In the case where the number is one or two, determining an intersection degree characterization amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining a connection state of the oil pipe at the oil unloading port based on the intersection degree characterization amount; When the number is greater than or equal to three, it is determined that the connection state of the oil pipe on the oil unloading port is connected.

3. The method for determining the oil pipe connection status according to claim 2, characterized in that: In the case where the number is one, determining an intersection degree representation amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining a connection state of the oil pipe at the oil unloading port based on the intersection degree representation amount, includes: Determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port, and determine the penetration depth representation of the target detection frame into the coverage area of ​​the oil unloading port; When a first ratio of the area of ​​the intersection region to the area of ​​the target detection frame is greater than or equal to a first preset threshold, and the penetration depth characterization value is greater than or equal to a second preset threshold, determining that the connection state of the oil pipe at the oil unloading port is connected; Among them, the intersection degree characterization quantity includes the intersection area and the penetration depth characterization quantity.

4. The method for determining the oil pipe connection status according to claim 2, characterized in that: In the case where the number is two, determining an intersection degree representation amount between the target detection frame and the coverage area of ​​the oil unloading port, and determining a connection state of the oil pipe at the oil unloading port based on the intersection degree representation amount includes: Determine the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port; When a first ratio of the area of ​​the intersection region to the area of ​​the target detection frame is greater than or equal to a first preset threshold, determining that the connection state of the oil pipe on the oil unloading port is connected; The intersection degree characterization quantity includes the area of ​​the intersection region.

5. The method for determining the oil pipe connection status according to claim 3 or 4, characterized in that: The determining of the intersection area of ​​the target detection frame and the coverage area of ​​the oil unloading port includes: For each vertex in the target detection frame that falls within the coverage area of ​​the oil unloading port, determine the intersection point of two adjacent edges of the vertex in the target detection frame and the boundary of the coverage area of ​​the oil unloading port; Deduplicating the boundary intersection points corresponding to all the vertices to obtain deduplicated boundary vertices; Based on the boundary vertex and all the vertices, an intersection region between the target detection frame and the coverage region of the oil unloading port is determined, and the area of ​​the intersection region is determined to obtain the area of ​​the intersection region.

6. The method for determining the oil pipe connection status according to claim 3, characterized in that: The step of determining a penetration depth characterization value of the target detection frame into the coverage area of ​​the oil unloading port includes: Determine an intersection point of a line between the center of gravity of the coverage area of ​​the oil unloading port and the midpoint of the target detection frame and a boundary of the coverage area of ​​the oil unloading port; Determine a first vertical distance of the intersection from a first boundary of the target detection frame and a second vertical distance from a second boundary of the target detection frame; the first boundary and the second boundary are adjacent boundaries of vertices of the target detection frame that fall within the coverage area; Determine a second ratio of the first vertical distance to the length of the second boundary and a third ratio of the second vertical distance to the length of the first boundary as the penetration depth characterization value; The penetration depth characterization value being greater than or equal to the second preset threshold value includes: The second ratio is greater than or equal to a third preset threshold, and the third ratio is greater than or equal to a fourth preset threshold; the second preset threshold includes the third preset threshold and the fourth preset threshold.

7. The method for determining the oil pipe connection status according to any one of claims 1 to 4, characterized in that: Also includes: In response to a target marking operation for the oil unloading port in the oil unloading monitoring image, determining a coverage area of ​​the oil unloading port; Performing edge expansion processing on the coverage area of ​​the oil unloading port to obtain a target detection frame image of the oil unloading port; Inputting the target detection frame image into an oil unloading port state classification model to obtain the switch state of the oil unloading port output by the oil unloading port state classification model; The oil unloading port state classification model is obtained by training an initial oil unloading port state classification model based on the oil unloading port sample image and the switch state label data corresponding to the oil unloading port sample image.

8. The method for determining the oil pipe connection state according to claim 7, characterized in that: The lower residual block in the oil unloading port state classification model is obtained by weighted fusion of the results of discrete cosine transform weighted and down-sampling processing of the upper residual block and the original lower residual block, and weighted fusion of the discrete cosine transform coefficient block obtained by weighted fusion with the original lower residual block.

9. A device for determining a connection state of an oil pipe, characterized in that: include: A detection frame acquisition module, used to acquire a target detection frame of the oil pipe in the oil unloading monitoring image when determining that the switch state of the oil unloading port is on; A vertex number determination module, used to determine the number of vertices of the target detection frame falling within the coverage area of ​​the oil unloading port in the oil unloading monitoring image; A connection status determination module is used to determine the connection status of the oil pipe on the oil unloading port based on the quantity.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the oil pipe connection status according to any one of claims 1 to 8 is implemented.

Images