A pipeline abnormal time positioning method, device and equipment and storage medium

Abstract

The application discloses a pipeline abnormal time positioning method, device and equipment and a storage medium. The method comprises the following steps: acquiring the confidence of each frame of image of a pipeline video to be detected, wherein the confidence comprises normal confidence and confidence of 16 kinds of defects; determining at least one abnormal interval according to the confidence of each frame of image, wherein the abnormal interval is composed of a first number of continuous abnormal frame images; determining a target defect set corresponding to each abnormal interval according to the confidence of 16 kinds of defects of each frame of image in each abnormal interval; and determining target time information corresponding to a target defect according to the target defect set corresponding to each abnormal interval. The embodiment of the application adopts a progressive weak supervision time positioning method, effectively locates a time interval through the guidance of defect classification and segmentation, and realizes more reliable abnormal classification in the abnormal interval.

Description

Technical Field

[0001] This invention relates to the field of underground pipeline network detection technology, and in particular to a method, apparatus, equipment, and storage medium for locating pipeline anomalies in a given time. Background Technology

[0002] Sewer pipes may seem insignificant in daily life, but they are one of the most important urban public facilities. Rainwater, domestic sewage, and industrial wastewater are collected through the sewer system and discharged into the sewage treatment system. The rupture or blockage of sewer pipes can have serious adverse effects on urban life. Therefore, regular inspection and maintenance of sewer pipes are particularly important.

[0003] Pipeline defects can be divided into two categories. The first category is ordinary single-frame defects, for which a single frame in a video is usually sufficient to capture the entire defect. The second category is continuous defects, which may extend along the pipeline for several meters or even tens of meters. In this case, a single frame in a video often cannot provide a complete picture of the defect. Existing pipeline defect detection methods have the following problems: the prediction results for each frame may have a certain degree of error; even if two frames with a certain time interval predict the same defect, it does not necessarily mean that the defect exists throughout the entire time period. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for locating pipeline anomalies in a given time period. This addresses the problem that existing pipeline defect detection methods neglect the importance of continuous defect intervals, enabling effective location of time intervals in pipeline anomaly detection and achieving a more accurate assessment of the severity of defects.

[0005] According to one aspect of the present invention, a method for locating pipeline anomalies in time is provided, the method comprising:

[0006] The confidence level of each frame of the video of the pipeline to be inspected is obtained, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects;

[0007] At least one abnormal interval is determined based on the confidence level of each frame image, wherein the abnormal interval consists of a first number of consecutive abnormal frame images;

[0008] The target defect set corresponding to each anomaly interval is determined based on the confidence level of 16 defects in each frame of image within each anomaly interval.

[0009] The target time information corresponding to the target defect is determined based on the target defect set corresponding to each abnormal interval.

[0010] According to another aspect of the present invention, a pipeline anomaly time location device is provided, the device comprising:

[0011] The acquisition module is used to acquire the confidence level of each frame of the video of the pipeline to be inspected, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects;

[0012] The first determining module is configured to determine at least one abnormal interval based on the confidence level of each frame image, wherein the abnormal interval is composed of a first number of consecutive abnormal frame images.

[0013] The second determination module is used to determine the set of target defects corresponding to each abnormal interval based on the confidence level of 16 defects in each frame of image in each abnormal interval.

[0014] The third determination module is used to determine the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the pipeline anomaly time location method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the pipeline anomaly time location method according to any embodiment of the present invention.

[0020] The technical solution of this invention involves acquiring the confidence level of each frame of a video image of the pipeline to be inspected, determining at least one abnormal interval based on the confidence level of each frame, determining the target defect set corresponding to each abnormal interval based on the confidence levels of 16 types of defects in each frame of each abnormal interval, and determining the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval. This invention solves the problem in existing pipeline defect detection methods that neglect the importance of continuous defect intervals, achieving effective location of time intervals in pipeline anomaly detection, and enabling a more accurate assessment of the severity of defects, which is beneficial for engineers to further repair and maintain the pipeline.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a pipeline anomaly time location method provided in Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of another pipeline anomaly time location method provided in Embodiment 1 of the present invention;

[0025] Figure 3 This is a schematic diagram of the pipeline anomaly time positioning device provided according to Embodiment 2 of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the pipeline anomaly time location method according to an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "target," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1This is a flowchart illustrating a pipeline anomaly time location method according to Embodiment 1 of the present invention. This embodiment is applicable to pipeline anomaly time location scenarios. The method can be executed by a pipeline anomaly time location device, which can be implemented in hardware and / or software. This device can be integrated into any electronic device that provides pipeline anomaly time location functionality. For example... Figure 1 As shown, the method includes:

[0031] S101. Obtain the confidence level of each frame of the video of the pipeline to be detected.

[0032] It should be clarified that the video of the pipeline to be inspected refers to a video of the pipeline that is to be inspected for defects. For example, it could be a video of a city's sewer system.

[0033] It is known that there are 16 existing classifications of pipeline anomalies (i.e., defects), which are, in order, SL (leakage), AJ (concealed connection), CR (foreign object insertion), TL (joint material detachment), TJ (disconnection), QF (undulation), CK (misalignment), FS (corrosion), BX (deformation), PL (crack), CJ (deposition), JG (scaling), ZW (obstacles), CQ (residual wall / dam root), SG (tree root), and FZ (scum).

[0034] Among them, SL (leakage) refers to water flowing into the pipe from outside the pipe or water leaking out of the pipe from inside the pipe. Since the phenomenon of water leaking out of the pipe is not easy to detect during pipe endoscopy, leakage mainly refers to water from underground (depending on the season) or from nearby leaking pipes flowing into the pipe wall, joints, and inspection well walls.

[0035] AJ (hidden connection) refers to a branch pipe being directly connected to the main pipe without passing through a manhole.

[0036] CR (foreign object insertion) refers to an object, not part of the pipe's perimeter, penetrating the pipe wall and entering the pipe. Intruding foreign objects include boulders in the backfill soil that crush the pipe, other structures passing through the pipe, and other pipelines crossing the pipe. CR differs from concealed branch pipe connections, which refer to drainage branch pipes being connected to the main drainage pipe without a manhole.

[0037] TL (joint material detachment) refers to the entry of joint materials such as rubber rings, asphalt, and cement into the pipeline. Rubber rings that enter the bottom of the pipeline can affect its flow capacity.

[0038] TJ (disconnection) refers to the incomplete connection or separation of the ends of two pipes. Due to settlement, the sleeve joints of the two pipes are not fully advanced or the joints are separated, making the adjacent pipes appear "full-moon shaped".

[0039] QF (fluctuation) refers to the subsidence of the interface, causing a significant change in the pipe slope and forming puddles. The causes of these fluctuations include both uneven pipe settlement and improper construction. When puddles (water accumulation) form due to settlement or other factors, the percentage of the actual water depth relative to the pipe's inner diameter should be recorded in the inspection record sheet.

[0040] CK (misalignment) refers to a lateral deviation between two pipe ends at the same interface, indicating they are not in the correct position on the pipeline. The joints of the two pipes are misaligned, making the adjacent pipes appear as a "crescent shape".

[0041] FS (corrosion) refers to the erosion and peeling of the inner wall of a pipe, resulting in pitting or exposed reinforcing steel. This occurs due to corrosion from harmful substances or wear and tear on the inner wall. Corrosion above the water surface in a pipe is primarily caused by hydrogen sulfide gas in drainage pipes. Corrosion at the bottom of the pipe is mainly caused by the combined effects of corrosive liquids and erosion.

[0042] BX (deformation) refers to the deformation of a pipe caused by external force, which alters the original shape of the pipe (only applicable to flexible pipes).

[0043] PL (rupture) refers to the rupture of a pipeline caused by external pressure exceeding its own bearing capacity. It can take three forms: longitudinal, circumferential, and combined.

[0044] CJ (sedimentation) refers to the accumulation of impurities at the bottom of a pipe. Organic or inorganic substances in the water deposit at the bottom of the pipe, forming sediment that reduces the pipe's cross-sectional area. Sediments include silt, broken bricks and stones, and solidified cement mortar.

[0045] JG (scale) refers to deposits on the inner wall of pipes. Dirt and debris in the water adhere to the inner wall of the pipe, forming deposits that reduce the cross-sectional area of ​​the pipe.

[0046] ZW (obstacles) refers to objects within a pipe that obstruct flow, including hard debris such as stones, planks, branches, abandoned tools, and fragments of broken pipes. Obstacles are external objects that enter the pipe, characterized by their obvious size and spatial presence. Foreign object penetration in structural defects refers to an external object penetrating the pipe wall and entering the pipe, damaging the pipe structure; the foreign object is located at the point of structural damage. A concealed branch connection refers to another drainage pipe not being connected to the main drainage pipe through a manhole as required by regulations, but instead being connected by drilling a hole. Deposition refers to the gradual accumulation of fine particulate matter in a pipe, forming a deposit covering a certain area. Scale is also fine particulate matter adhering to the pipe wall, and can exist on both the sidewalls and the bottom.

[0047] CQ (residual brick wall) refers to the temporary brick wall that was built to seal the pipeline during the water tightness test, and the residue that was not removed or was not completely removed after the test.

[0048] SG (tree root) refers to a single tree root or a group of tree roots that naturally grow into the pipeline. The entry of tree roots into the pipeline inevitably leads to damage to the pipeline structure and affects the pipeline's flow capacity. The impact on flow capacity is calculated as a functional defect, while the damage to the pipeline structure is calculated as a structural defect.

[0049] FZ (floating debris) refers to floating matter on the water surface inside the pipeline. This defect must be recorded in the inspection record sheet but is not included in the MI calculation.

[0050] Among them, longitudinal cracks, deformation, longitudinal corrosion, undulation, longitudinal leakage, deposition and scaling are continuous defects. These defects usually exist continuously in the pipeline and have a certain length along the pipeline.

[0051] In this embodiment, confidence level can be understood as probability. The confidence level of each frame of the pipeline video to be detected is the probability that each frame of the pipeline video to be detected is normal.

[0052] The confidence level includes: normal confidence level and confidence levels for 16 types of defects.

[0053] It should be explained that normal confidence level refers to the probability that each frame of the video of the pipeline under test is normal and without any defects.

[0054] It should be noted that the confidence level of a defect refers to the probability of each frame of the video of the pipeline under inspection containing the different types of defects and their specific characteristics.

[0055] Specifically, the video of the pipeline to be detected is input into the pipeline recognition and detection network, such as the VideoPipe pipeline recognition and detection network, which can take 25 frames per second. The network outputs the recognition and detection results of each frame of the image, thereby obtaining the confidence level of each frame of the video of the pipeline to be detected. The confidence level includes the normal confidence level and the confidence level of each of the 16 defects.

[0056] For example, the common segmentation model Mask R-CNN (Region-Convolutional Neural Network) can predict what defects are in an image and their locations, and give the probability of each defect occurring (i.e., the confidence level of each defect).

[0057] S102. Determine at least one abnormal interval based on the confidence level of each frame of image.

[0058] The abnormal interval consists of a first number of consecutive abnormal frame images.

[0059] It should be noted that an abnormal frame image refers to a frame image that has at least one defect.

[0060] It should be explained that the abnormal interval refers to the interval in which the pipeline video to be detected is abnormal, i.e., has a stable defect. Specifically, the abnormal interval can be an interval composed of a first number of consecutive abnormal frame images in the pipeline video to be detected. The first number can be a number of consecutive abnormal frame images preset according to the actual situation, for example, it can be 4. This embodiment of the invention does not limit this.

[0061] Specifically, after obtaining the confidence level of each frame of the video of the pipeline to be inspected, including the normal confidence level and the confidence levels of the 16 defects, at least one abnormal interval is determined based on the confidence level of each frame.

[0062] S103. Determine the target defect set corresponding to each abnormal interval based on the confidence level of the 16 defects in each frame of the image in each abnormal interval.

[0063] It should be noted that the target defect set refers to the set of defects that can characterize the abnormal interval. For example, the target defect set corresponding to the abnormal interval could be defect SG (tree root) and defect FZ (scum).

[0064] Specifically, after identifying at least one anomalous interval, the target defect set corresponding to each anomalous interval is determined based on the confidence levels of the 16 types of defects in each frame of the image within each anomalous interval. For example, the target defect set corresponding to that anomalous interval can be formed by sorting the top four defects by confidence level in each frame of the image within that anomalous interval.

[0065] S104. Determine the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval.

[0066] It should be noted that the target defect refers to the defect common to the target defect set corresponding to each anomaly interval. There can be one or more target defects.

[0067] It should be explained that the target time information refers to the time information of the target defect in the pipeline video to be inspected. For example, the target time information corresponding to defect SG (tree root) is between 2 minutes and 22 seconds and 2 minutes and 44 seconds in the pipeline video to be inspected.

[0068] Specifically, the target defects are determined based on the target defect set corresponding to each abnormal interval, and the target time information corresponding to the target defects is determined by extending outward.

[0069] The technical solution of this invention involves acquiring the confidence level of each frame of a video image of the pipeline to be inspected, determining at least one abnormal interval based on the confidence level of each frame, determining the target defect set corresponding to each abnormal interval based on the confidence levels of 16 types of defects in each frame of each abnormal interval, and determining the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval. This invention solves the problem in existing pipeline defect detection methods that neglect the importance of continuous defect intervals, achieving effective location of time intervals in pipeline anomaly detection, and enabling a more accurate assessment of the severity of defects, which is beneficial for engineers to further repair and maintain the pipeline.

[0070] Optionally, at least one anomalous interval is determined based on the confidence level of each frame of image, including:

[0071] The frames of the pipeline video to be inspected are grouped according to the type of pipeline inspection network.

[0072] Specifically, the video of the pipeline to be detected is input into the pipeline recognition and detection network. The number of grouped frames is selected according to the type of pipeline detection network used, and the recognition and detection results of each frame are output. The number of frames of the video of the pipeline to be detected is grouped.

[0073] In practice, because long videos can be over ten minutes long, with 60 seconds per minute and 25 frames per second, each video can have tens of thousands of frames, resulting in excessively dense content. Therefore, referencing the VideoPipe pipeline identification and detection network, the video to be detected is grouped into sets of 8 frames each.

[0074] Abnormal groups are determined based on the number of normal frame images within each group and the average normal confidence level.

[0075] It should be noted that a normal frame image refers to a frame image that is normal and has no defects.

[0076] It should be noted that the average normal confidence level refers to the average normal confidence level of each frame within each group.

[0077] For example, an anomaly group can be understood as a group in which the number of normal frame images is less than or equal to the number of abnormal frame images after grouping the number of frames of the pipeline video to be detected.

[0078] Specifically, the frames of the pipeline video to be detected are grouped, and the number of normal frame images and the average normal confidence level in each group are counted. The abnormal group is determined based on the number of normal frame images and the average normal confidence level in each group.

[0079] A predetermined number of consecutive abnormal groups are defined as the abnormal interval.

[0080] The preset quantity can be the number of abnormal groups that make up the abnormal interval, which is set in advance according to the actual situation. For example, it can be 12. This embodiment does not limit this.

[0081] Specifically, a predetermined number of consecutive abnormal groups are defined as abnormal intervals. For example, every 12 abnormal groups are set as a time period. At 25 frames per second, 12 abnormal groups are 96 frames, which is approximately 4 seconds. If 12 or more abnormal groups appear consecutively, this segment is considered a stable abnormal interval, and each abnormal interval is at least 4 seconds long.

[0082] Optionally, outlier groups are determined based on the number of normal frame images within each group and the average normal confidence level, including:

[0083] Abnormal and normal frame images are filtered from the pipeline video to be detected based on the confidence level of each frame.

[0084] Specifically, the pipeline video to be detected is input into the pipeline recognition and detection network, and the confidence score of each frame is output. The confidence score includes the normal confidence score and the confidence scores of 16 different defects. Abnormal and normal frame images are selected from the pipeline video to be detected based on the confidence score of each frame.

[0085] Obtain the number of normal frame images and the average of the normal confidence level.

[0086] Specifically, after filtering abnormal and normal frame images from the pipeline video to be detected based on the confidence level of each frame image, the number of normal frame images and the average value of normal confidence levels are obtained.

[0087] If the number of normal frame images in the target group is less than or equal to the number of abnormal frame images, then the target group is identified as an abnormal group.

[0088] The target group refers to the group used for anomaly group identification.

[0089] Specifically, if the number of normal frame images in the target group is less than or equal to the number of abnormal frame images, the target group is identified as an abnormal group; conversely, if the number of normal frame images in the target group is greater than the number of abnormal frame images, the target group is identified as a normal group.

[0090] Alternatively, if the average normal confidence level within the target group is less than the first confidence level threshold, then the target group is identified as an outlier group.

[0091] The first confidence threshold can be the average of the normal confidence levels within the target group, which is set in advance according to the actual situation. For example, it can be 0.5. This embodiment does not limit this.

[0092] Specifically, if the average normal confidence level within the target group is less than a pre-set first confidence level threshold, the target group is identified as an abnormal group.

[0093] Optionally, abnormal frame images are filtered from the pipeline video to be detected based on the confidence level of each frame image, including:

[0094] If the normal confidence level of the target image is less than the second confidence level threshold, the target image is determined to be an abnormal frame image.

[0095] It should be noted that the target image refers to the image used for judging positive and abnormal frames.

[0096] The target image is any frame from the video of the pipeline to be detected.

[0097] In this embodiment, the second confidence threshold can be the confidence level of the image set in advance according to the actual situation, for example, it can be 0.5. This embodiment does not limit this.

[0098] Specifically, if the normal confidence level of the target image is less than the second confidence level threshold, the target image is determined to be an abnormal frame image; conversely, if the normal confidence level of the target image is greater than or equal to the confidence level threshold, the target image is determined to be a normal frame image.

[0099] Alternatively, if the normal confidence level of the target image is less than the defect confidence level, then the target image is determined to be an abnormal frame image.

[0100] It should be noted that defect confidence refers to the confidence level of any type of defect in the target image.

[0101] Specifically, if the normal confidence level of the target image is less than the confidence level of any type of defect in the target image, the target image is determined to be an abnormal frame image; conversely, if the normal confidence level of the target image is greater than the confidence level of all types of defects in the target image, the target image is determined to be a normal frame image.

[0102] Optionally, the set of target defects corresponding to each anomaly interval is determined based on the confidence scores of 16 types of defects in each frame of the image within each anomaly interval, including:

[0103] Obtain the average confidence level of 16 defects for each frame of the image within the abnormal interval.

[0104] Specifically, the average confidence level of each of the 16 defects in each frame of the image within the anomaly interval is calculated.

[0105] Defects whose average confidence level is greater than the first threshold are identified as the first defect set.

[0106] The first threshold can be the average confidence level of each of the 16 defects in each frame of an image in the abnormal interval that is set in advance according to the actual situation. For example, it can be 0.27. This embodiment does not limit it.

[0107] It should be noted that the first defect set refers to the set of defects whose average confidence level in the abnormal interval is greater than the first threshold.

[0108] Specifically, defects whose average confidence level in the abnormal interval is greater than a first threshold are defined as the first defect set. For example, the first threshold can be set to 0.27, and the average confidence level of defect SG (tree root) in each frame of the abnormal interval is calculated to be 0.3, the average confidence level of defect FZ (scum) in each frame of the abnormal interval is 0.45, and the average confidence level of the remaining defects in each frame of the abnormal interval is 0.1. Then the first defect set includes defect SG (tree root) and defect FZ (scum).

[0109] Obtain the second set of defects corresponding to any frame image in the abnormal interval.

[0110] It should be explained that the second defect set refers to the set of the top few defects with the highest confidence among all defects present in any frame of the abnormal interval. For example, it can be the top 4. This embodiment of the invention does not limit this.

[0111] Specifically, the top few defects with the highest confidence levels among all defects present in any frame of an image within the anomaly interval are selected to form a corresponding second defect set. For example, if a frame of an image within the anomaly interval contains the top four defects with the highest confidence levels—namely, defect SG (tree root), defect AJ (hidden connection), TJ (disconnection), and defect SL (leakage)—then the second defect set would include defect SG (tree root), defect AJ (hidden connection), TJ (disconnection), and defect SL (leakage).

[0112] The target defect set corresponding to the abnormal interval is determined by the intersection of the first defect set and the second defect set.

[0113] In this embodiment, the intersection of the first defect set and the second defect set refers to selecting defects that coexist in both the first defect set and the second defect set.

[0114] Specifically, the target defect set corresponding to the abnormal interval is determined based on the intersection of the first defect set and the second defect set. For example, the first defect set could include defect SG (tree root) and defect FZ (scum); the second defect set could include defect SG (tree root), defect AJ (hidden connection), TJ (disconnection), and defect SL (leakage). Then, the target defect set corresponding to the abnormal interval, determined by the intersection of the first and second defect sets, includes defect SG (tree root).

[0115] Optionally, the target time information corresponding to the target defect is determined based on the target defect set corresponding to each abnormal interval, including:

[0116] The intersection of the target defect sets corresponding to adjacent abnormal intervals is determined as the target defect.

[0117] It should be noted that adjacent abnormal intervals refer to two adjacent abnormal intervals in the video of the pipeline to be inspected.

[0118] Specifically, the intersection of the target defect sets corresponding to adjacent abnormal intervals is determined as the target defect. For example, if abnormal interval 1 and abnormal interval 2 are adjacent, and the target defect set corresponding to abnormal interval 1 includes defect SG (tree root), defect AJ (concealed connection), and defect FZ (scum), and the target defect set corresponding to abnormal interval 2 includes defect SG (tree root), defect AJ (concealed connection), and defect SL (leakage), then the intersection of the target defect sets corresponding to abnormal interval 1 and abnormal interval 2, defect SG (tree root) and defect AJ (concealed connection), is determined as the target defect.

[0119] The initial and final times corresponding to adjacent abnormal intervals are determined as the target time information corresponding to the target defect.

[0120] It should be explained that the initial time refers to the start time of the adjacent abnormal interval with the earlier initial time, and the end time refers to the end time of the adjacent abnormal interval with the later initial time.

[0121] Specifically, the initial and ending times corresponding to adjacent abnormal intervals are determined as the target time information corresponding to the target defect. For example, if the duration of an abnormal interval is 4 seconds, and abnormal interval 1 and abnormal interval 2 are adjacent, with the start time of abnormal interval 1 being 1 minute 30 seconds and the end time being 1 minute 34 seconds, and the start time of abnormal interval 2 being 1 minute 34 seconds and the end time being 1 minute 38 seconds, then the target time information is from 1 minute 30 seconds to 1 minute 38 seconds.

[0122] Optionally, the target time information corresponding to the target defect is determined based on the target defect set corresponding to each abnormal interval, including:

[0123] The intersection of the target defect set corresponding to the first abnormal interval and the target defect set corresponding to the second abnormal interval, which are separated by at least one abnormal interval, is determined as the target defect.

[0124] It should be noted that the first abnormal interval refers to the abnormal interval with the earlier initial time among the two abnormal intervals, and the second abnormal interval refers to the abnormal interval with the later initial time among the two abnormal intervals.

[0125] The initial time of the first abnormal interval is earlier than the initial time of the second abnormal interval.

[0126] Specifically, the intersection of the target defect sets corresponding to the first abnormal interval (at least one abnormal interval apart) and the target defect sets corresponding to the second abnormal interval is determined as the target defect. For example, if there are two abnormal intervals between abnormal intervals 1 and 4, and the target defect set corresponding to abnormal interval 1 includes defect SG (tree root), defect AJ (concealed connection), and defect FZ (scum), while the target defect set corresponding to abnormal interval 4 includes defect SG (tree root), defect AJ (concealed connection), and defect SL (leakage), then the intersection of the target defect sets corresponding to abnormal intervals 1 and 4, defect SG (tree root) and defect AJ (concealed connection), is determined as the target defect.

[0127] In actual operation, the first abnormal interval and the second abnormal interval can be separated by an abnormal interval or by a normal interval (i.e., an interval where the video of the pipeline under inspection is normal and there are no stable defects). This embodiment does not limit this. Moreover, the fewer the intervals between the first abnormal interval and the second abnormal interval, the more accurate the time positioning result.

[0128] The initial time of the first abnormal interval is determined as the initial time of the target defect.

[0129] Specifically, the initial time of the first abnormal interval is determined as the initial time of the target defect. For example, the duration of the abnormal interval is 4 seconds, and there are two abnormal intervals between abnormal interval 1 and abnormal interval 4. The start time of abnormal interval 1 is 1 minute and 30 seconds, and the end time is 1 minute and 34 seconds. The start time of abnormal interval 4 is 1 minute and 42 seconds, and the end time is 1 minute and 46 seconds. Then the initial time of the target defect is 1 minute and 30 seconds.

[0130] The termination time of the second abnormal interval is determined as the termination time of the target defect.

[0131] Specifically, the termination time of the second abnormal interval is determined as the termination time of the target defect. For example, the duration of the abnormal interval is 4 seconds, and there are 2 abnormal intervals between abnormal interval 1 and abnormal interval 4. The start time of abnormal interval 1 is 1 minute and 30 seconds, and the end time is 1 minute and 34 seconds. The start time of abnormal interval 4 is 1 minute and 42 seconds, and the end time is 1 minute and 46 seconds. Then the termination time of the target defect is 1 minute and 46 seconds.

[0132] As an exemplary description of this embodiment, Figure 2 This is a flowchart of another pipeline anomaly time location method provided according to Embodiment 1 of the present invention. Figure 2 As shown, the process of another pipeline anomaly time location method specifically includes the following operations:

[0133] The video of the pipeline to be detected is input into the VideoPipe pipeline recognition and detection network at 25 frames per second. The confidence score of each frame is output. The specific output results are divided into two categories: the classification result of each frame (how many defects are contained in each frame) and the segmentation result of each frame (which defects are in each frame and where they are located).

[0134] The confidence threshold is set to 0.5. When the normal confidence is greater than or equal to 0.5, or when the normal classification has the highest confidence among all abnormal classifications, the frame image is identified as a normal frame image. Conversely, when the normal confidence is less than 0.5, or when the normal confidence is less than any defect confidence, the frame image is identified as an abnormal frame image.

[0135] Referring to the VideoPipe pipeline detection network, normal frame images are grouped into groups of 8 frames each (if other pipeline detection networks are used, the number of frames per group can also be different, depending on the type of pipeline detection network). If the number of normal frame images in 8 consecutive frames is greater than the number of abnormal frame images (i.e., the number of normal frame images > 4) and the average confidence score within the target group is greater than or equal to the first confidence score threshold, then this group is considered a normal group; otherwise, it is considered an abnormal group.

[0136] After identifying the anomalous groups, each time interval is defined as 12 anomalous groups, which is 96 frames, approximately 4 seconds. If 12 or more anomalous groups appear consecutively, this interval is considered a stable anomalous interval, with each anomalous interval lasting at least 4 seconds.

[0137] Find the target defect set that can characterize the abnormal interval: obtain the average confidence of the top 4 defects with the highest confidence in each frame of the abnormal interval, and determine the defects with an average confidence of greater than 0.27 as the first defect set; obtain the top 4 defects with the highest confidence among all defects in any frame of the abnormal interval (by performing an intersection operation on the top 4 defects with the highest confidence in the classification results of each frame and the segmentation results of each frame), and determine them as the second defect set; perform an intersection operation on the first defect set and the second defect set to determine the target defect set corresponding to the abnormal interval.

[0138] The target defect is determined based on the target defect set corresponding to each abnormal interval, and the target time information corresponding to the target defect is determined by expanding outward.

[0139] Example 2

[0140] Figure 3 This is a schematic diagram of a pipeline anomaly time location device provided in Embodiment 2 of the present invention. Figure 3As shown, the device includes: an acquisition module 201, a first determination module 202, a second determination module 203, and a third determination module 204.

[0141] The acquisition module 201 is used to acquire the confidence level of each frame of the video of the pipeline to be detected, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects;

[0142] The first determining module 202 is configured to determine at least one abnormal interval based on the confidence level of each frame image, wherein the abnormal interval is composed of a first number of consecutive abnormal frame images.

[0143] The second determining module 203 is used to determine the target defect set corresponding to each abnormal interval based on the confidence level of 16 defects in each frame of the image in each abnormal interval.

[0144] The third determining module 204 is used to determine the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval.

[0145] Optionally, the first determining module 202 includes:

[0146] A grouping unit is used to group the frames of the video of the pipeline to be detected according to the type of pipeline detection network;

[0147] The first determining unit is used to determine the abnormal group based on the number of normal frame images in each group and the average normal confidence level.

[0148] The second determining unit is used to determine a consecutive preset number of abnormal groups as an abnormal interval.

[0149] Optionally, the first determining unit includes:

[0150] A filtering subunit is used to filter abnormal frame images and normal frame images from the pipeline video to be detected based on the confidence level of each frame image;

[0151] The acquisition sub-unit is used to obtain the number of normal frame images and the average value of the normal confidence level;

[0152] A determination subunit is used to determine the target group as an abnormal group if the number of normal frame images in the target group is less than or equal to the number of abnormal frame images.

[0153] or;

[0154] If the average normal confidence level within the target group is less than the first confidence level threshold, then the target group is identified as an abnormal group.

[0155] Optionally, the filtering subunit is specifically used for:

[0156] If the normal confidence level of the target image is less than the second confidence level threshold, then the target image is determined to be an abnormal frame image, wherein the target image is any frame image in the pipeline video to be detected;

[0157] or;

[0158] If the normal confidence level of the target image is less than the defect confidence level, then the target image is determined to be an abnormal frame image.

[0159] Optionally, the second determining module 203 is specifically used for:

[0160] Obtain the average confidence level of 16 defects for each frame of the image within the anomaly interval;

[0161] Defects with an average confidence level greater than a first threshold are identified as the first defect set;

[0162] Obtain the second defect set corresponding to any frame image in the abnormal interval;

[0163] The target defect set corresponding to the abnormal interval is determined based on the intersection of the first defect set and the second defect set.

[0164] Optionally, the third determining module 204 is specifically used for:

[0165] The intersection of the target defect sets corresponding to adjacent abnormal intervals is determined as the target defect;

[0166] The initial and final times corresponding to adjacent abnormal intervals are determined as the target time information corresponding to the target defect.

[0167] Optionally, the third determining module 204 is further specifically used for:

[0168] The intersection of the target defect set corresponding to the first abnormal interval and the target defect set corresponding to the second abnormal interval, which are separated by at least one abnormal interval, is determined as the target defect, wherein the initial time of the first abnormal interval is earlier than the initial time of the second abnormal interval.

[0169] The initial time of the first abnormal interval is determined as the initial time of the target defect;

[0170] The termination time of the second abnormal interval is determined as the termination time of the target defect.

[0171] The pipeline anomaly time location device provided in the embodiments of the present invention can execute the pipeline anomaly time location method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0172] Example 3

[0173] Figure 4 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0174] like Figure 4 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0175] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0176] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as pipeline anomaly timing methods:

[0177] The confidence level of each frame of the video of the pipeline to be inspected is obtained, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects;

[0178] At least one abnormal interval is determined based on the confidence level of each frame image, wherein the abnormal interval consists of a first number of consecutive abnormal frame images;

[0179] The target defect set corresponding to each anomaly interval is determined based on the confidence level of 16 defects in each frame of image within each anomaly interval.

[0180] The target time information corresponding to the target defect is determined based on the target defect set corresponding to each abnormal interval.

[0181] In some embodiments, the pipeline anomaly timing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the pipeline anomaly timing method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the pipeline anomaly timing method by any other suitable means (e.g., by means of firmware).

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0186] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0187] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0188] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0189] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for locating pipeline anomalies by time, characterized in that, include: The confidence level of each frame of the video of the pipeline to be inspected is obtained, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects; At least one abnormal interval is determined based on the confidence level of each frame image, wherein the abnormal interval consists of a first number of consecutive abnormal frame images; The target defect set corresponding to each anomaly interval is determined based on the confidence level of 16 defects in each frame of image within each anomaly interval. The target time information corresponding to the target defect is determined based on the target defect set corresponding to each abnormal interval. Specifically, the set of target defects corresponding to each anomaly interval is determined based on the confidence levels of 16 types of defects in each frame of the image within each anomaly interval, including: Obtain the average confidence level of 16 defects for each frame of the image within the anomaly interval; Defects with an average confidence level greater than a first threshold are identified as the first defect set; Obtain the second defect set corresponding to any frame image in the abnormal interval; wherein, the second defect set refers to the set consisting of the top 4 defects with the highest confidence among all defects present in any frame image in the abnormal interval; The target defect set corresponding to the abnormal interval is determined based on the intersection of the first defect set and the second defect set.

2. The method according to claim 1, characterized in that, Determine at least one abnormal interval based on the confidence level of each frame of image, including: The frames of the video of the pipeline to be inspected are grouped according to the type of pipeline inspection network; Abnormal groups are determined based on the number of normal frame images and the average normal confidence level within each group; A predetermined number of consecutive abnormal groups are defined as the abnormal interval.

3. The method according to claim 2, characterized in that, Abnormal groups are determined based on the number of normal frame images within each group and the average normal confidence level, including: Abnormal and normal frame images are filtered from the pipeline video to be detected based on the confidence level of each frame image; Obtain the number of normal frame images and the average of the normal confidence level; If the number of normal frame images in the target group is less than or equal to the number of abnormal frame images, then the target group is determined to be an abnormal group. or; If the average normal confidence level within the target group is less than the first confidence level threshold, then the target group is identified as an abnormal group.

4. The method according to claim 3, characterized in that, Based on the confidence level of each frame, abnormal frame images are filtered from the pipeline video to be detected, including: If the normal confidence level of the target image is less than the second confidence level threshold, then the target image is determined to be an abnormal frame image, wherein the target image is any frame image in the pipeline video to be detected; or; If the normal confidence level of the target image is less than the defect confidence level, then the target image is determined to be an abnormal frame image.

5. The method according to claim 1, characterized in that, Based on the target defect set corresponding to each abnormal interval, determine the target time information corresponding to the target defect, including: The intersection of the target defect sets corresponding to adjacent abnormal intervals is determined as the target defect; The initial and final times corresponding to adjacent abnormal intervals are determined as the target time information corresponding to the target defect.

6. The method according to claim 1, characterized in that, Based on the target defect set corresponding to each abnormal interval, determine the target time information corresponding to the target defect, including: The intersection of the target defect set corresponding to the first abnormal interval and the target defect set corresponding to the second abnormal interval, which are separated by at least one abnormal interval, is determined as the target defect, wherein the initial time of the first abnormal interval is earlier than the initial time of the second abnormal interval. The initial time of the first abnormal interval is determined as the initial time of the target defect; The termination time of the second abnormal interval is determined as the termination time of the target defect.

7. A pipeline anomaly time location device, characterized in that, include: The acquisition module is used to acquire the confidence level of each frame of the video of the pipeline to be inspected, wherein the confidence level includes: normal confidence level and confidence level of 16 types of defects; The first determining module is configured to determine at least one abnormal interval based on the confidence level of each frame image, wherein the abnormal interval is composed of a first number of consecutive abnormal frame images. The second determination module is used to determine the set of target defects corresponding to each abnormal interval based on the confidence level of 16 defects in each frame of image in each abnormal interval. The third determination module is used to determine the target time information corresponding to the target defect based on the target defect set corresponding to each abnormal interval; Specifically, the second determining module is used for: Obtain the average confidence level of 16 defects for each frame of the image within the anomaly interval; Defects with an average confidence level greater than a first threshold are identified as the first defect set; Obtain the second defect set corresponding to any frame image in the abnormal interval; wherein, the second defect set refers to the set consisting of the top 4 defects with the highest confidence among all defects present in any frame image in the abnormal interval; The target defect set corresponding to the abnormal interval is determined based on the intersection of the first defect set and the second defect set.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pipeline anomaly time location method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the pipeline anomaly time location method according to any one of claims 1-6.

Images