Visual recognition method for water supply and drainage pipeline maintenance

Abstract

The present invention relates to the field of image processing technology, and in particular to a visual recognition method for water supply and drainage pipeline inspection, comprising: edge detection of an image frame to obtain a pipeline edge set; for any pipeline edge in the pipeline edge set, determining whether the reference edge is a matching edge that matches the pipeline edge according to the grayscale density of the area formed by the pipeline edge and the reference edge, the reference edge is located on one side of the pipeline edge and is closest to the pipeline edge; determining whether the pipeline edge is a target edge according to the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge that matches the pipeline edge relative to the pipeline edge, and obtaining a target edge set; determining the pipeline detection result according to the consistency of the distance between any two adjacent target edges in the target edge set. The present invention can accurately and efficiently detect pipeline misalignment defects in images displaying artifacts.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a visual recognition method for water supply and drainage pipeline maintenance. Background Art

[0002] Drainage pipes are pipe systems used to discharge wastewater, rainwater or other liquids. They are widely used in various places such as buildings, roads, and urban infrastructure. Pipe misalignment is a common internal defect of drainage pipes. Its structure is manifested in that the pipe joints are not properly aligned and the seal is poor. The incomplete structure at the misalignment causes the structural weakness of the pipe at that location, resulting in leakage, rupture, or even collapse of the pipe when the pressure or flow increases. Therefore, it is necessary to identify the misalignment defects of drainage pipes and notify maintenance personnel to repair them in time to avoid causing greater safety accidents.

[0003] Usually, the differences in the texture and pipe misalignment defects in the image are analyzed based on the relative position between the pipes, so as to identify and detect the pipe misalignment. However, due to the complex environment inside the pipeline, when the maintenance equipment equipped with the image acquisition device encounters an obstacle in the pipeline, it will shake, resulting in a blurry picture taken by the image acquisition device and artifacts. At this time, it will be impossible to identify the pipe misalignment defect by analyzing the relative position between the pipes. Summary of the invention

[0004] In order to solve the technical problem that it is impossible to accurately and efficiently detect pipeline misalignment defects in images showing artifacts, the purpose of the present invention is to provide a visual recognition method for water supply and drainage pipeline inspection and maintenance. The technical solution adopted is as follows:

[0005] Perform edge detection on the image frame to obtain a pipeline edge set, and the image frame displays the pipeline. For any pipeline edge in the pipeline edge set, determine whether the reference edge is a matching edge that matches the pipeline edge based on the grayscale density of the area formed by the pipeline edge and the reference edge, and the reference edge is located on one side of the pipeline edge and is closest to the pipeline edge. Determine whether the pipeline edge is a target edge based on the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge that matches the pipeline edge relative to the pipeline edge, and obtain a target edge set. Based on the consistency of the distance between any two adjacent target edges in the target edge set, the pipeline detection result characterizes whether the pipeline displayed by the image frame is misaligned.

[0006] In another possible example, the visual recognition method for water supply and drainage pipeline maintenance further includes: updating the matching edge according to a change in the distance between the matching edge and the reference edge.

[0007] In another possible example, determining whether the reference edge is a matching edge corresponding to the pipe edge based on the grayscale density of the area formed by the pipe edge and the reference edge includes: performing a sliding window operation on the area formed by the pipe edge and the reference edge according to a preset window in a normal direction perpendicular to the reference pixel point of the pipe edge, the preset window of each sliding window operation covering the reference pixel point; calculating the average grayscale difference within the preset window corresponding to each sliding window operation; for any reference edge, determining the grayscale density value of the area formed by the pipe edge and the reference edge according to the average grayscale difference of each sliding window operation; and determining the matching edge of the pipe edge based on the grayscale density value.

[0008] In another possible example, the method further includes: intercepting a curve on the edge of the pipe to obtain a curve segment, wherein the reference pixel point is a pixel point on the curve segment; and fitting the curve segment to obtain a normal direction of the curve segment.

[0009] In another possible example, for any reference edge, the gray density value is related to the variance value of the gray difference average value of each sliding window operation.

[0010] In another possible example, updating the matching edge according to the change in the distance between the matching edge and the reference edge includes: determining the shortest distance between each pixel point of the matching edge and the reference edge to obtain a first distance sequence; determining the difference in the shortest distance between any two adjacent pixel points of the matching edge and the reference edge to obtain a distance difference sequence; determining a variance difference according to the variance value of a positive number sequence in the distance difference sequence and the variance value of a negative number sequence in the distance difference sequence; and updating the matching edge according to the variance difference and a preset threshold.

[0011] In another possible example, determining the pipeline detection result based on the consistency of the distance between any two adjacent target edges in the target edge set includes: determining the shortest distance between each sampling pixel point of the first target edge and the second target edge among any two adjacent target edges to obtain a second distance sequence; determining a probability value characterizing the second target edge as a normal edge based on the variance value of the second distance sequence; and determining the pipeline detection result based on the probability value and a preset probability value.

[0012] In another possible example, determining whether the pipe edge is a target edge based on the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge that matches the pipe edge relative to the pipe edge includes: determining that the pipe edge is a target edge when the position direction of the matching edge that matches the pipe edge relative to the pipe edge is consistent with the movement direction of the image frame relative to the upstream image frame, and / or when the position direction of the matching edge that matches the pipe edge relative to the pipe edge is opposite to the movement direction of the image frame relative to the downstream image frame.

[0013] In another possible example, performing edge detection on the image frame to obtain the pipeline edge set includes: performing edge detection on the image frame based on an edge detection algorithm to obtain the pipeline edge set.

[0014] In another possible example, the image frame is obtained by de-framing a preset video stream, where the preset video stream is obtained by photographing a pipeline during a time period when the image acquisition device passes through an obstacle.

[0015] The present invention has the following beneficial effects:

[0016] Edge detection is performed on the image frame to obtain a set of pipe edges. On the one hand, due to jitter in actual scenes, the same pipe has a corresponding real pipe edge in the image frame and a pseudo pipe edge that matches the real pipe edge generated by jitter. According to the gray density characteristics, a matching edge that matches any pipe edge can be determined to form an edge group. There is also the problem that the uneven distribution of gray density caused by the dark light inside the pipe affects the accuracy of determining the matching edge according to the gray density. Therefore, the matching edge can also be accurately determined based on the above by the distance change characteristics between the pipe edge and the matching edge. Since the cause of the artifact is the jitter of the image acquisition device when passing through obstacles, etc., it can be determined whether the pipe edge is the target edge (that is, the real pipe edge) according to the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge relative to the pipe edge. There are also cases where the pipe edge in the image frame is the texture of the pipe itself. It is possible to determine whether the target edge is the pipe edge where the pipe is misaligned or the pipe texture edge based on the consistency of the distance between any two adjacent target edges. In this way, various types of edges can be accurately identified based on the performance characteristics in actual scenarios, thereby achieving accurate and efficient detection of pipe misalignment defects in images displaying artifacts. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A method flow chart of a visual recognition method for water supply and drainage pipeline maintenance provided by one embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a pipe edge at an interface of a normal pipeline provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram showing a pipeline misalignment at a pipeline interface and artifacts at a pipeline edge provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a visual identification device for water supply and drainage pipeline inspection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the visual recognition method for water supply and drainage pipeline inspection proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0023] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0024] A specific scheme of a visual recognition method for water supply and drainage pipeline inspection provided by the present invention is described in detail below in conjunction with the accompanying drawings.

[0025] See also Figure 1 , which shows a method flow chart of a visual recognition method for water supply and drainage pipeline maintenance provided by an embodiment of the present invention.

[0026] like Figure 1 As shown, the visual recognition method for water supply and drainage pipeline maintenance according to an embodiment of the present invention includes operations S110 to S140.

[0027] In operation S110, edge detection is performed on the image frame to obtain a pipeline edge set, and the image frame displays the pipeline.

[0028] Exemplarily, the image frame may be obtained by de-framing a preset video stream, where the preset video stream is obtained by capturing the pipeline by an image acquisition device during the time period when the image acquisition device passes through an obstacle. For example, some maintenance vehicles that can be used for maintenance work in pipelines are equipped with image acquisition devices such as cameras, and can be moved in the pipeline by control, and the video captured by the camera can be transmitted to an external storage device. When the maintenance vehicle moves through an obstacle in the pipeline, it will shake, so during the time period when it passes through the obstacle, the image captured by the camera will be blurred and artifacts will appear.

[0029] The preset video stream obtained by the camera or other image acquisition equipment shooting the pipeline during the time period when the obstacle passes can be obtained, for example, through the following embodiments: an obstacle detection sensor is installed on the maintenance vehicle, and the obstacle detection sensor can be an ultrasonic sensor. The obstacle detection sensor can detect obstacles in the pipeline and calculate the distance to the obstacle. When the obstacle detection sensor detects an obstacle that is too close, it can be determined that the maintenance vehicle is passing the obstacle. At this time, the time t when the maintenance vehicle starts to pass the obstacle can be obtained, and the time Δt (Δt = 1.5s) required for the maintenance vehicle to pass the obstacle is set. The video segment shot by the image acquisition device on the maintenance vehicle from time t to time t+Δt is obtained as the preset video stream obtained by the image acquisition device shooting the pipeline during the time period when the image acquisition device passes the obstacle.

[0030] Exemplarily, for example, edge detection may be performed on the image frame based on an edge detection algorithm to obtain a pipeline edge set.

[0031] The edge detection algorithm may include, for example, a Canny edge detection algorithm.

[0032] The Canny edge detection algorithm is a multi-stage image processing algorithm, including Gaussian blur, edge strength detection and edge non-maximum suppression, double threshold method and other steps: Gaussian blur is used to smooth the image using a Gaussian kernel to remove noise; the image gradient intensity is calculated to calculate the gradient intensity and direction of each pixel in the image; non-maximum suppression is used to retain the local maximum value in the image gradient; the double threshold method is used to determine the edge and eliminate noise.

[0033] In operation S120, for any one pipe edge in the pipe edge set, it is determined whether the reference edge is a matching edge that matches the pipe edge according to the grayscale density of an area formed by the pipe edge and the reference edge.

[0034] The reference edge is located on one side of the pipe edge and is closest to the pipe edge.

[0035] Unless otherwise specified below, the pipe edge and the matching edge are understood to be matching pipe edges and matching edges, that is, pipe edges and matching edges in one edge group.

[0036] Since the area where the artifact is located in the image frame appears as an uneven grayscale density distribution in the image, while the grayscale density distribution of the normal area without the artifact is consistent, the visual recognition method for water supply and drainage pipeline inspection in the embodiment of the present invention can accurately determine whether the reference edge on one side of the pipe edge is a matching edge that matches the pipe edge through the grayscale density characteristics on both sides of the pipe edge. The matching edge here is the edge in the artifact area (also called a pseudo-pipe edge).

[0037] According to another embodiment of the present invention, a visual recognition method for water supply and drainage pipeline inspection and maintenance can be implemented, for example, using the following embodiment to implement a specific example of determining whether a reference edge is a matching edge that matches the pipe edge based on the grayscale density of the area formed by the pipe edge and the reference edge: in the normal direction of the reference pixel point perpendicular to the pipe edge, a sliding window operation is performed on the area formed by the pipe edge and the reference edge according to a preset window, and the preset window of each sliding window operation covers the reference pixel point; the average grayscale difference within the preset window corresponding to each sliding window operation is calculated; for any reference edge, the grayscale density value of the area formed by the pipe edge and the reference edge is determined based on the average grayscale difference of each sliding window operation; and the matching edge of the pipe edge is determined based on the grayscale density value.

[0038] In another embodiment, the visual recognition method for water supply and drainage pipeline inspection according to the embodiment of the present invention also includes: intercepting a curve on the edge of the pipeline to obtain a curve segment, the reference pixel point is the pixel point on the curve segment; fitting the curve segment to obtain the normal direction of the curve segment.

[0039] Figure 2 The schematic diagram of the pipe edge at the interface of a normal pipeline is shown schematically.

[0040] like Figure 2 As shown, the two real pipe edges in the image frame are pipe edge A and pipe edge B. The center points of the two pipe edges are O A and O B For circular pipes, overlap means that there is no pipe misalignment at the interface of the two pipes.

[0041] Figure 3 The schematic diagram shows the existence of pipe misalignment at the pipe interface and artifacts at the pipe edge.

[0042] like Figure 3 As shown, the real edges of the two pipes in the image frame are pipe edge C1 and pipe edge D1, and the center points of the two pipe edges are O C1 and OD1 For circular pipes, there is a misalignment at the interface of the two pipes. Due to the shaking of the maintenance vehicle and other reasons, there are also artifacts in the image frame. Figure 3 In the example, the pseudo-pipeline edge corresponding to the real pipe edge C1 is pipe edge C2, and the pseudo-pipeline edge of the real pipe edge D2 is pipe edge D2.

[0043] Combination Figure 3 Taking the pipe edge D1 as an example, there are two reference edges on one side of the pipe edge D1 and closest to the pipe edge D1, namely the pipe edge C2 and the pipe edge D2. The curve segment l can be, for example, a portion of the pipe edge D1 located within the sliding window win, and the reference pixel point can be one pixel point or multiple pixel points of the curve segment l.

[0044] Figure 3 In the example, the normal direction of the reference pixel point and the normal direction of the curve segment l are direction x, and the normal direction of the reference pixel point perpendicular to the pipe edge D1 and the normal direction x perpendicular to the curve segment l are direction y.

[0045] The length of the curve segment l can be represented by the number of pixels, for example. The length of the curve l can take a value of 5, for example.

[0046] The curve segment 1 may be fitted by, for example, the least square method, and a straight line after fitting the curve segment 1 may be obtained. The normal direction of the curve segment 1 may be represented by, for example, the extension direction of the curve segment 1 after fitting.

[0047] For the convenience of the following description, the normal direction x perpendicular to the reference pixel point / curve segment l is also referred to as the sequence direction, that is, the direction y mentioned above.

[0048] For example, the reference pixel point or the nth pixel point on the curve segment l can be used as the center point, and the line segment formed by the center point and the pixel point with the shortest corresponding distance on the reference edge can be called a sequence segment. The grayscale density performance of the two sequence segments on both sides of the center point in the sequence direction y is analyzed. At this time, in the sequence direction y, two sequence segments D1 on both sides of the pipeline edge D1 in the sequence direction y can be obtained. a ,D b The sequence segment here is located in the area between the pipeline edge D1 and the reference edge, and the sequence segment can also be understood as the area formed by the pipeline edge and the reference edge.

[0049] The uniform grayscale density in the sequence segment is manifested in that the grayscale values ​​of adjacent pixels in the sequence segment are close, and the grayscale values ​​of the pixels in the sequence segment are relatively clustered; while the uneven grayscale density is manifested in that there may be abnormal points in the sequence segment or the grayscale difference between adjacent pixels is large. Therefore, the grayscale density value of the sequence segment can be determined by the variance of the average grayscale difference of the pixels in the sequence segment. a ,D b The gray density value process is as follows. a For example:

[0050] a) Setting the window size where the reference pixel is located. For example, a preset window of size 1*5 may be set, and the reference pixel may be located at the center of the preset window.

[0051] b) Calculate sequence segment D a The average grayscale difference of each pixel in the sequence segment D a The sliding window operation is performed on the sequence segment D, so that the average grayscale difference of each pixel in the preset window can be calculated, that is, the average grayscale difference in the preset window corresponding to each sliding window operation. The grayscale difference of the pixel in the preset window can be used as the average grayscale difference of the pixel to calculate the sequence segment D. a The average grayscale difference r of pixel i i The calculation formula (1) is as follows.

[0052]

[0053] In formula (1): h i Indicates the gray value of pixel i that needs to be calculated in the sequence segment, h v Represents the gray value of the vth pixel in the preset window where the pixel i is located, except the pixel i. i The smaller it is, the more uniform the grayscale of pixel i in the preset window.

[0054] c) Calculate sequence segment D a Gray density value. Through sequence segment D a The variance of the average grayscale difference of all pixels in the sequence segment D a The gray density value, at this time D a The gray density value f a =σ(r i ), f a The smaller the value, the greater the sequence segment D a The more uniform the grayscale density is, the more uniform the grayscale density is. σ() represents the variance calculation function.

[0055] The above calculation can obtain the sequence segment D on one side of the pipeline edge D1 in the sequence direction. a The gray density value f aSimilarly, the sequence segment D on the other side can be obtained b The gray density value f b .

[0056] d) According to the sequence segments D on both sides a ,D b The gray density value of the sequence segment on one side of the pipeline edge D1 is determined by the gray density value of the pipeline edge D1. If the gray density value of the sequence segment on one side of the pipeline edge D1 is larger, that is, the gray density is uneven, it can be considered that the sequence segment is located in the area where the artifact in the image frame is located, and the matching edge of the pipeline edge D1 is the reference edge on the same side of the sequence segment. a The gray density value f a and sequence segment D b The gray density value f b The size of f is taken as the larger value max , then f max The reference edge on the same side of the sequence segment is the matching edge of the pipeline edge D1, and the pipeline edge D1 and the matching edge can also be called a matching edge group. The edge group formed by the pipeline edge and its corresponding matching edge can be considered to include a real pipeline edge and a pseudo pipeline edge.

[0057] Since the light inside the pipeline is dim, grayscale mutations may occur in dark areas with smaller grayscale values, and the grayscale density distribution of the dark areas in the image frame is uneven. Both the artifact areas and the dark areas in the image frame show uneven grayscale density distribution. At this time, if the matching edge of the pipeline edge is determined only by the grayscale density values ​​on both sides of the pipeline edge, there may be matching errors.

[0058] In view of the above situation, according to another embodiment of the present invention, the visual recognition method for water supply and drainage pipeline maintenance may further include: updating the matching edge according to the change in the distance between the matching edge and the reference edge.

[0059] The pseudo-pipeline edge is generated by the real pipe edge, so there will be a certain misalignment between the real pipe edge and the pseudo-pipeline edge corresponding to the real pipe edge. Specifically, the distance between the real pipe edge and the pseudo-pipeline edge will gradually increase from the starting pixel point and then gradually decrease, and the distance between the edges will change consistently. If the matching edge of the pipe edge is generated by motion artifacts, the positional relationship between the pipe edge and the matching edge is a translation relationship in the image. The pipe edge and the matching edge are represented in the image as two similar circles with offset centers, that is, the shortest distance from each point on the pipe edge to the matching edge should be gradual.

[0060] Therefore, the visual recognition method for water supply and drainage pipeline inspection in an embodiment of the present invention can not only determine the matching edge of the pipeline edge by the gray density value as mentioned above, but also update the matching edge by the change in the distance between the pipeline edge and the matching edge, thereby improving the edge matching accuracy.

[0061] According to another embodiment of the present invention, a visual recognition method for water supply and drainage pipeline inspection can be implemented by, for example, using the following embodiments to implement a specific example of updating a matching edge according to a change in the distance between the matching edge and the reference edge:

[0062] a) Determine the shortest distance between each pixel of the matching edge and the reference edge to obtain a first distance sequence.

[0063] For example, the pixel point with the smallest distance between the pipe edge and the matching edge can be taken as the starting pixel point. If there are multiple starting pixels, any one can be selected and the direction can be selected arbitrarily to obtain the shortest distance from each pixel point on the pipe edge to the matching edge. According to the distance, the distance sequence between the pipe edge and the matching edge can be obtained, that is, the first distance sequence. For example, the first distance sequence is expressed by formula (2).

[0064] {hd1,hd2,…,hd j} (2)

[0065] hd1 represents the shortest distance from the starting pixel on the edge of the pipeline to the matching edge, hd j Indicates the shortest distance from the end pixel on the pipe edge to the matching edge.

[0066] b) Determine the difference in the shortest distance between any two adjacent pixels of the matching edge and the reference edge to obtain a distance difference sequence.

[0067] For example, the difference in the shortest distance between any two adjacent pixels and the reference edge is expressed by formula (3).

[0068] Δhd j-1 =hd j-1 -hd j (3)

[0069] For example, the distance difference sequence is expressed by formula (4).

[0070] {Δhd1, Δhd2,…, Δhd j-1} (4)

[0071] c) determining a variance difference value according to a variance value of a positive number sequence in the distance difference value sequence and a variance value of a negative number sequence in the distance difference value sequence, wherein the second evaluation data includes the variance difference value.

[0072] The distance variation between the matching edge and the reference edge is characterized by the variance difference mentioned above.

[0073] Since the distance change trend is gradually increasing from the starting pixel point and then gradually decreasing, there is a maximum point in the distance difference sequence at this time, so the direction of the distance difference sequence changes only once, and the number of direction changes s of the distance difference sequence is also the change trend of the distance.

[0074] For example, the distance difference sequence is: 1, 2, 5, 8, -1, -4, -9, then s is 1.

[0075] The distance difference sequence can be divided into two segments according to the changing trend of the distance difference sequence, that is, the distance difference sequence is divided into two segments B1 and B2 by taking the direction change point of the distance difference sequence as the dividing point. These two segments correspond to the positive sequence and the negative sequence in the distance difference sequence respectively. Based on this, the variance values ​​gh1 of the two segments can be calculated respectively v =σ(B1),gh2 v =σ(B2).

[0076] The closer the variance values ​​of the positive sequence in the distance difference sequence and the variance values ​​of the negative sequence in the distance difference sequence are, the more similar the shapes of the pipeline edge and the matching edge are, the more the spatial relationship between the pipeline edge and the matching edge conforms to the relationship between the pipeline edge and the corresponding motion artifact, and the greater the probability that the matching edge is a motion artifact of the pipeline edge.

[0077] d) Update the matching edge according to the variance difference and the preset threshold.

[0078] The variance difference can represent the probability G that the matched edge is the real pipeline edge. v , the following formula (5) is G v The calculation method of .

[0079]

[0080] In formula (5), exp represents the natural exponential function, and s represents the number of direction changes of the distance difference sequence.

[0081] The preset threshold GH can be, for example, 0.5. The matching edge corresponding to the variance difference value greater than the preset threshold can be considered as a pseudo-pipeline edge that matches the pipe edge. The matching edge corresponding to the variance difference value less than or equal to the preset threshold can be considered to be caused by a grayscale mutation in the dark area and can therefore be removed.

[0082] In operation S130, whether the pipe edge is a target edge is determined based on the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge relative to the pipe edge, and a target edge set is obtained, where the edge group includes the pipe edge and the matching edge corresponding to the pipe edge.

[0083] It should be noted that the cause of the artifacts in the image frame is the blurred image caused by the shaking of the camera due to the sudden change in the movement direction of the maintenance vehicle. Therefore, when the position direction of the matching edge that matches the pipe edge relative to the pipe edge is consistent with the movement direction of the image frame relative to the upstream image frame, and / or when the position direction of the matching edge that matches the pipe edge is opposite to the movement direction of the image frame relative to the downstream image frame, the pipe edge can be determined as the target edge.

[0084] Take a maintenance vehicle as an example. For example, the tilt sensor on the maintenance vehicle can measure the tilt angle of the maintenance vehicle relative to the horizontal plane. The moving direction of the maintenance vehicle can be obtained through the data of the tilt sensor. Therefore, the data of the tilt sensor on the maintenance vehicle can obtain the moving direction of the image frame at the moment relative to the upstream image frame and / or the downstream image frame.

[0085] In summary, the real pipeline edges in the image frame, that is, all target edges in the image frame, can be obtained.

[0086] In operation S140 , a pipeline detection result is determined according to the consistency of distances between any two adjacent target edges in the target edge set.

[0087] The pipeline detection result indicates whether the pipeline displayed in the image frame is misaligned.

[0088] The target edge in the target edge set in the image frame obtained above is the real pipe edge, which may be the texture inside the pipe (that is, the pipe texture edge) or the pipe misalignment defect. The pipe texture edge appears as a circle, which appears as a set of concentric circles in the image frame. The characteristic of concentric circles is that each circle has the same center, so the distance between each circle and the adjacent circles is consistent. The pipe misalignment defect is caused by improper alignment of the pipe connection. The circle formed by the pipe misalignment defect will be offset relative to the position of the circle formed by the pipe texture. Therefore, the visual recognition method for water supply and drainage pipeline inspection in the embodiment of the present invention can determine whether the target edge is the pipe edge with pipe misalignment or the pipe texture edge through the consistency of the distance between any two adjacent target edges.

[0089] According to another embodiment of the present invention, a visual recognition method for water supply and drainage pipeline inspection can be implemented by, for example, using the following embodiments to implement a specific example of determining a pipeline inspection result based on the consistency of the distance between any two adjacent target edges in the target edge set:

[0090] a) Among any two adjacent target edges, determine the shortest distance between each sampling pixel point of the first target edge and the second target edge to obtain a second distance sequence.

[0091] One of any two adjacent target edges is a first target edge, and the other is a second target edge. For example, the first target edge and the second target edge are adjacent.

[0092] Q pixels with the same interval can be selected as sampling pixels on the first target edge v. Q can be 8, for example. Taking the length of the first target edge as W as an example, the interval between any two adjacent sampling pixels on the first target edge v is W / Q.

[0093] b) Determine the variance value of the second distance sequence.

[0094] The distance consistency can be characterized, for example, by a variance value of the second distance sequence.

[0095] σ(d -v+1 ) characterizes the variance value of the second distance sequence between the first target edge v and the second target edge v+1.

[0096] σ(d -v+1 ) indicates that the smaller the value is, the more consistent the distance between the first target edge v and the second target edge v+1 is.

[0097] c) determining, according to the variance value of the second distance sequence, a probability value characterizing that the second target edge is a normal edge.

[0098] P v Characterizes the probability value of the target edge being a normal edge, P v For example, it can be determined by the following formula (6):

[0099] P v = norm{σ(d -v+1 )} (6)

[0100] In the above formula (6), norm{·} represents the norm or vector size of ·.

[0101] The probability threshold PT may take a value of 0.5, for example.

[0102] The variance value σ(d -v+1), the smaller the variance of , the more consistent the distance between the first target edge and the second target edge based on the sampling pixels, and the greater the possibility that the second target edge belongs to a normal edge. v When P is greater than the probability threshold, the second target edge can be considered as a pipeline texture edge; v When the probability is less than or equal to the probability threshold, the second target edge can be considered as a normal edge, that is, a pipeline misalignment occurs, thereby determining the pipeline detection result of the image frame.

[0103] In summary, the visual recognition method for water supply and drainage pipeline inspection in the embodiment of the present invention can perform edge detection on the image frame to obtain a set of pipe edges. On the one hand, due to jitter in the actual scene, the same pipe has a corresponding real pipe edge in the image frame and a pseudo pipe edge that matches the real pipe edge generated by jitter. According to the gray density characteristics, a matching edge that matches any pipe edge can be determined to form an edge group. There is also the problem that the uneven distribution of gray density caused by the dark light inside the pipeline affects the accuracy of determining the matching edge according to the gray density. Therefore, the matching edge can also be accurately determined based on the above by the distance change characteristics between the pipe edge and the matching edge. Since the cause of the artifact is the jitter of the image acquisition device when passing through obstacles, etc., it can be determined whether the pipe edge is the target edge (that is, the real pipe edge) according to the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge relative to the pipe edge. There are also cases where the pipe edge in the image frame is the texture of the pipe itself. It is possible to determine whether the target edge is the pipe edge where the pipe is misaligned or the pipe texture edge based on the consistency of the distance between any two adjacent target edges. In this way, various types of edges can be accurately identified based on the performance characteristics in actual scenarios, thereby achieving accurate and efficient detection of pipe misalignment defects in images displaying artifacts.

[0104] like Figure 4 As shown, an embodiment of the present invention further provides a schematic diagram of a visual identification device for water supply and drainage pipeline inspection, including: a first module 410, a second module 420, a third module 430, and a fourth module 440.

[0105] The first module 410 is used to perform edge detection on the image frame to obtain a pipeline edge set, and the image frame displays the pipeline.

[0106] The second module 420 is used to determine, for any one of the pipeline edges in the pipeline edge set, whether the reference edge is a matching edge that matches the pipeline edge according to the grayscale density of the area formed by the pipeline edge and the reference edge, and the reference edge is located on one side of the pipeline edge and is closest to the pipeline edge.

[0107] The third module 430 is used to determine whether the pipeline edge is a target edge according to the movement direction of the image frame relative to the upstream image frame and / or the downstream image frame and the relative position of the matching edge matching the pipeline edge relative to the pipeline edge, and obtain a target edge set.

[0108] The fourth module 440 is used to determine the pipeline detection result according to the consistency of the distance between any two adjacent target edges in the target edge set, and the pipeline detection result represents whether the pipeline displayed in the image frame is misaligned.

[0109] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory is used to store an executable program code, and the processor is used to store the memory and the processor. The memory is used to store the executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the above-mentioned visual recognition method for water supply and drainage pipeline maintenance. The electronic device can specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the visual recognition method for water supply and drainage pipeline maintenance provided in the above embodiment.

[0110] In other embodiments, a computer program product is also provided. When the computer program product is run on an electronic device such as a computer, the electronic device executes the above-mentioned related steps to implement a visual recognition method for water supply and drainage pipeline inspection provided in the above embodiment.

[0111] In other embodiments, a computer-readable storage medium is also provided, in which a computer program code is stored. When the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement the visual recognition method for water supply and drainage pipeline inspection provided in the above-mentioned embodiment.

[0112] Among them, the provided system, device, computer program product, and computer-readable storage medium are all used to execute the visual recognition method for water supply and drainage pipeline inspection provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the visual recognition method for water supply and drainage pipeline inspection provided above, and will not be repeated here.

[0113] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A visual recognition method for water supply and drainage pipeline maintenance, characterized in that: The method comprises: Perform edge detection on the image frame to obtain a pipeline edge set, wherein the image frame displays the pipeline; the image frame is obtained by performing frame de-framing processing on a preset video stream, wherein the preset video stream is obtained by photographing the pipeline during a time period when the image acquisition device passes through an obstacle; For any one of the pipeline edges in the pipeline edge set, determining whether the reference edge is a matching edge that matches the pipeline edge according to the grayscale density of the area formed by the pipeline edge and the reference edge, the reference edge being located on one side of the pipeline edge and being closest to the pipeline edge; the matching edge is an edge within the artifact area; updating the matching edge according to a change in the distance between the matching edge and the reference edge; In a case where the position direction of the matching edge matched with the pipeline edge relative to the pipeline edge is consistent with the movement direction of the image frame relative to the upstream image frame, and / or in a case where the position direction of the matching edge matched with the pipeline edge relative to the pipeline edge is opposite to the movement direction of the image frame relative to the downstream image frame, determining the pipeline edge as a target edge, and obtaining a target edge set; According to the consistency of the distance between any two adjacent target edges in the target edge set, a pipeline detection result is determined, and the pipeline detection result indicates whether the pipeline displayed by the image frame is misaligned.

2. The visual recognition method for water supply and drainage pipeline maintenance according to claim 1 is characterized in that: The determining, according to the grayscale density of the area formed by the pipe edge and the reference edge, whether the reference edge is a matching edge corresponding to the pipe edge comprises: In a normal direction perpendicular to a reference pixel point of the pipe edge, a sliding window operation is performed on an area formed by the pipe edge and the reference edge according to a preset window, wherein the preset window of each sliding window operation covers the reference pixel point; Calculating the average grayscale difference within the preset window corresponding to each sliding window operation; For any reference edge, determine the grayscale density value of the area formed by the pipe edge and the reference edge according to the average grayscale difference of each sliding window operation; A matching edge of the pipe edge is determined according to the gray density value.

3. The visual recognition method for water supply and drainage pipeline maintenance according to claim 2 is characterized in that: The method further comprises: A curve is intercepted on the edge of the pipeline to obtain a curve segment, wherein the reference pixel point is a pixel point on the curve segment; The curve segment is fitted to obtain the normal direction of the curve segment.

4. The visual recognition method for water supply and drainage pipeline maintenance according to claim 2 is characterized in that: For any reference edge, the grayscale density value is related to the variance value of the grayscale difference average value of each sliding window operation.

5. The visual recognition method for water supply and drainage pipeline maintenance according to claim 1, characterized in that: The updating of the matching edge according to the change in the distance between the matching edge and the reference edge comprises: Determine the shortest distance between each pixel point of the matching edge and the reference edge to obtain a first distance sequence; Determine the difference between the shortest distances between any two adjacent pixel points of the matching edge and the reference edge to obtain a distance difference sequence; Determine the variance difference value according to the variance value of the positive number sequence in the distance difference value sequence and the variance value of the negative number sequence in the distance difference value sequence; The matching edge is updated according to the variance difference and a preset threshold.

6. The visual recognition method for water supply and drainage pipeline maintenance according to claim 1, characterized in that: The determining of the pipeline detection result according to the consistency of the distance between any two adjacent target edges in the target edge set includes: Among any two adjacent target edges, determine the shortest distance between each sampling pixel point of the first target edge and the second target edge to obtain a second distance sequence; Determining, according to the variance value of the second distance sequence, a probability value characterizing that the second target edge is a normal edge; The pipeline detection result is determined according to the probability value and a preset probability value.

7. The visual recognition method for water supply and drainage pipeline maintenance according to claim 1, characterized in that: The performing edge detection on the image frame to obtain a pipeline edge set comprises: Perform edge detection on the image frame based on an edge detection algorithm to obtain a pipeline edge set.

Images