Wire detection method, device and equipment and storage medium

Abstract

The embodiment of the invention discloses a wire detection method and device, equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, the second binary image is obtained by carrying out hole filling processing on the first binary image obtained by carrying out wire segmentation on the original image, and the plurality of line segment groups are determined according to the plurality of first straight line segments obtained by carrying out straight line detection on the second binary image; each line segment group and the corresponding adjacent collinear line segment group are connected to obtain a second straight line segment, one or more target wire segments are determined according to the first straight line segment and the second straight line segment, and the situation that two straight lines are mistakenly detected from a thick wire in the binary image can be effectively reduced by performing hole filling processing on the first binary image; and the filling of the second straight line segment can effectively supplement the missing electric wire, thereby effectively improving the detection effect of the electric wire in the binary image and improving the three-dimensional reconstruction quality of the electric wire.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a wire detection method, device, equipment and storage medium. Background Art

[0002] The existing three-dimensional wire generation is generally achieved by performing wire segmentation processing on multiple original images of a complete flight, and then performing straight line detection on the binary image corresponding to the wire segmentation processing result to obtain the two-dimensional straight line segments on each binary image, and then performing three-dimensional reconstruction of the wire based on the two-dimensional straight line segments to realize wire vectorization.

[0003] When performing line detection on a binary image, it is generally performed based on line detection methods such as Hough line detection and LSD (Line Segment Detector). However, Hough line detection obtains the equation of a line, but cannot determine the length of the line. Line detection methods such as LSD can detect the angle and length of a line, but it is easy to detect two straight lines when the binary image is thick, and it is easy to mistakenly generate two three-dimensional wires in the three-dimensional reconstruction of the wire. The detection effect of wires in binary images is poor, which affects the quality of three-dimensional reconstruction of wires. Summary of the invention

[0004] The embodiments of the present application provide a wire detection method, device, equipment and storage medium to solve the technical problem in the related art that the wire detection effect in the binary image is poor, which affects the quality of the three-dimensional reconstruction of the wire, effectively improve the wire detection effect in the binary image, and improve the quality of the three-dimensional reconstruction of the wire.

[0005] In a first aspect, an embodiment of the present application provides a wire detection method, comprising:

[0006] Performing hole filling processing on the first binary image obtained by performing wire segmentation according to the original image to obtain a second binary image;

[0007] Performing straight line detection on the second binary image to obtain one or more first straight line segments, and determining one or more line segment groups based on the first straight line segments, wherein the line segment groups include one first straight line segment or multiple continuous first straight line segments;

[0008] Determine adjacent collinear line segment groups of each of the line segment groups, and connect the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment;

[0009] One or more target electric wire segments are determined according to the first straight line segment and the second straight line segment.

[0010] In a second aspect, an embodiment of the present application provides a wire detection device, including a hole filling module, a line segment group determination module, a straight line completion module and a result determination module, wherein:

[0011] The hole filling module is configured to perform hole filling processing on the first binary image obtained by performing wire segmentation on the original image to obtain a second binary image;

[0012] The line segment group determination module is configured to perform straight line detection on the second binary image to obtain one or more first straight line segments, and determine one or more line segment groups based on the first straight line segments, wherein the line segment group includes one first straight line segment or multiple continuous first straight line segments;

[0013] The line completion module is configured to determine adjacent collinear line segment groups of each of the line segment groups, and connect the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment;

[0014] The result determination module is configured to determine one or more target electric wire segments according to the first straight line segment and the second straight line segment.

[0015] In a third aspect, an embodiment of the present application provides a wire detection device, including: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the wire detection method as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the wire detection method as described in the first aspect when executed by a computer processor.

[0019] The embodiment of the present application obtains a second binary image by performing hole filling processing on a first binary image obtained by performing wire segmentation according to an original image, determines a plurality of line segment groups according to a plurality of first straight line segments obtained by performing straight line detection on the second binary image, connects each line segment group and a corresponding adjacent collinear line segment group to obtain a second straight line segment, and determines one or more target wire segments according to the first straight line segments and the second straight line segments. By performing hole filling processing on the first binary image, the situation in which thicker wires in the binary image are mistakenly detected as two straight lines can be effectively reduced, and the filling of the second straight line segments can effectively supplement the missing wires, effectively improving the wire detection effect in the binary image and improving the quality of three-dimensional reconstruction of the wires. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a wire detection method provided by an embodiment of the present application;

[0021] Figure 2is a flow chart of another wire detection method provided by an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of a candidate line segment determination process provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a first binary image provided in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a second binary image provided in an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of a center line provided in an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a first straight line segment provided in an embodiment of the present application;

[0027] Figure 8 is a schematic diagram of another first straight line segment provided in an embodiment of the present application;

[0028] Fig. 9 is a schematic diagram of wires in an original image provided by an embodiment of the present application;

[0029] Fig.10 is a schematic diagram of a first straight line segment fused with a second straight line segment provided in an embodiment of the present application;

[0030] Fig.11 is a schematic diagram of wires in another original image provided by an embodiment of the present application;

[0031] Fig.12 is a structural schematic diagram of a wire detection device provided in an embodiment of the present application;

[0032] Fig.13 It is a structural schematic diagram of a wire detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0034] Figure 1 A flow chart of a wire detection method provided in an embodiment of the present application is given. The wire detection method provided in an embodiment of the present application can be executed by a wire detection device, which can be implemented in hardware and / or software and integrated in a wire detection device.

[0035] The following description is made by taking the wire detection device executing the wire detection method as an example. Figure 1 , the wire detection method comprises:

[0036] S110: Perform hole filling processing on the first binary image obtained by performing wire segmentation on the original image to obtain a second binary image.

[0037] The original image provided by this solution can be obtained by shooting the target area, and the target area can be understood as the area where the wires need to be reconstructed in three dimensions. For example, a drone equipped with a camera flies over the target area and continuously shoots multiple images, and multiple images of the complete flight are used as the original image. The original image provided by this solution can display a partial area of ​​the target area, and there is overlap of some target areas in adjacent original images. The image corresponding to the entire target area can be displayed by splicing all the original images. Among them, one or more wires can be captured in part or all of the original images.

[0038] Exemplarily, multiple original images of the target area, 3D reconstruction intermediate results of 3D reconstruction of these original images (the 3D reconstruction intermediate results can be applied to the subsequent line matching of the target wire segments corresponding to the multiple original images), and the first binary image obtained by wire segmentation of these original images are obtained. Each original image corresponds to a first binary image, for example, a mask image (mask) corresponding to the original image is obtained by wire segmentation of the original image using a pre-trained wire segmentation model, and the mask image is used as the first binary image corresponding to the original image. Optionally, for the pixel points in the original image that are considered to belong to the wire pixel, the color value of the pixel in the first binary image can be set to the first color value (for example, set to the color value corresponding to white, such as 255), and for the pixel points in the original image that are considered not to belong to the wire pixel, the color value of the pixel in the first binary image can be set to the second color value (for example, set to the color value corresponding to black, such as 0). In one embodiment, after the wire segmentation is performed on the original image, when it is determined according to the wire segmentation result that there is no wire pixel in the original image, the original image can be discarded to reduce the influence of the original image without wire on the wire detection, thereby improving the wire detection efficiency.

[0039] Furthermore, a hole filling process is performed on each first binary image, and the first binary image after the hole filling process is used as the second binary image. The hole filling process can be understood as filling the gaps in the range surrounded by the wire pixel points in the first binary image (the range of non-wire pixel points surrounded by the wire pixel points), for example, changing the color value corresponding to the gap in the range surrounded by the wire pixel points from the second color value to the first color value. When there is no gap in the first binary image, the second binary image obtained based on the first binary image is the same as the first binary image. Figure 1 To.

[0040] Optionally, the first binary image may be subjected to hole detection according to a set disparity hole filling tool (e.g., a disparity hole filling function filterSpeckles), and the first binary image may be subjected to hole filling processing according to a set first color value. This solution effectively fills the gaps in the range surrounded by the wire pixel points in the first binary image by performing hole filling processing on the first binary image, fills the gaps in the adhesion areas in the wire segmentation result, reduces the situation where the wire pixel point range corresponding to one wire is mistakenly generated into multiple straight lines when performing straight line detection on these pixel points, and improves the accuracy of wire detection.

[0041] S120: Perform straight line detection on the second binary image to obtain one or more first straight line segments, and determine one or more line segment groups based on the first straight line segments, where the line segment group includes one first straight line segment or multiple continuous first straight line segments.

[0042] Exemplarily, straight line detection is performed on each second binary image to obtain one or more first straight line segments in each second binary image. When the straight line detection is performed on the second binary image, the detection can be performed based on a set straight line length threshold, and the length of the detected first straight line segment is within the straight line length threshold. For example, when the length of a wire in the second binary image exceeds the straight line length threshold, the wire pixel point range corresponding to the wire is detected to obtain multiple first straight line segments, and these first straight line segments are connected end to end in sequence, that is, these first straight line segments are continuous, and when the length of a wire in the second binary image is within the straight line length threshold, the wire pixel point range corresponding to the wire is detected to obtain a first straight line segment.

[0043] After completing the straight line detection of the second binary image, a line segment group is generated according to the continuity of the first straight line segments in the second binary image. A line segment group may include one or more first straight line segments. When a line segment group includes multiple first straight line segments, these first straight line segments are continuous first straight line segments. For example, two first straight line segments connected at their ends are considered to be continuous first straight line segments, that is, the first straight line segments in the same line segment group are the first straight line segments corresponding to the same electric wire.

[0044] S130: Determine adjacent collinear line segment groups of each line segment group, and connect the line segment group and the adjacent collinear line segment group to obtain a second straight line segment.

[0045] Exemplarily, for each line segment group on the same second binary graph, adjacent collinear line segment groups are determined from other line segment groups in the second binary graph. The adjacent collinear line segment groups can be understood as the collinear line segment groups closest to the current line segment group. Each line segment group can correspond to at most one adjacent collinear line segment group, or there may be no adjacent collinear line segment group. For example, when there is no collinear line segment group with the line segment group on the second binary graph, there is no adjacent collinear line segment group for the line segment group in the current second binary graph. Optionally, the slope and vertical distance between two collinear line segment groups are within a set range.

[0046] In one embodiment, when determining adjacent collinear line segment groups of a line segment group, the determination can be performed according to a set direction (for example, in the direction from small to large of the image coordinates). Once the adjacent collinear line segment groups are determined, the line segment groups will no longer participate in the determination of adjacent collinear line segment groups of other line segment groups.

[0047] Furthermore, after the adjacent collinear line segment group of the line segment group is determined, the line segment group is connected to the corresponding adjacent collinear line segment group to obtain a second straight line segment. For example, the closest endpoints of the line segment group and the corresponding adjacent collinear line segment group are connected to obtain the second straight line segment.

[0048] It needs to be explained that when the wires in an original image are merged with the background (for example, the wires are close in color to the background and cannot be successfully identified as wire pixels) or are blocked by obstructions, multiple line segment groups will be detected based on the second binary image of the wires. However, the corresponding straight line segments of the parts of the wires that are merged with the background or blocked cannot be correctly detected. This solution supplements the straight line segments that are merged with the background or blocked by the background by connecting the line segment groups and the adjacent colinear line segment groups to generate the second straight line segments, thereby completing the missing supplement processing of the straight line segments and improving the generation accuracy and completeness of the three-dimensional wires.

[0049] S140: Determine one or more target electric wire segments according to the first straight line segment and the second straight line segment.

[0050] Exemplarily, after the first straight line segment and the second straight line segment of the second binary image are determined, one or more target wire segments on the corresponding original image can be determined based on all the first straight line segments and the second straight line segments on the second binary image. For example, the endpoint coordinates of one or more target wire segments on the original image are determined based on the endpoint coordinates of the first straight line segment and the second straight line segment. In one embodiment, when the second straight line segment is not determined in the second binary image (for example, when the wires in the original image are not fused with the background or blocked by an obstruction), the one or more target wire segments are determined based on the first straight line segment.

[0051] In a possible embodiment, after determining one or more target wire segments based on the first straight line segment and the second straight line segment, the wire detection method provided by the present solution can also perform line matching based on the target wire segments corresponding to multiple original images, and generate three-dimensional wires based on the line matching results.

[0052] Exemplarily, after determining the target line segments corresponding to each original image, line matching (epipolar matching) is performed on the target wire line segments corresponding to these original images according to the endpoint coordinate information corresponding to these target line segments and the camera posture corresponding to each original image, and three-dimensional reconstruction is performed according to the line matching results to generate three-dimensional wires in the electronic map. This solution generates three-dimensional wires according to the line matching results of line matching of target wire line segments corresponding to multiple original images. Since the target wire line segments undergo hole filling and missing supplementation processing during the determination process, the wire detection effect in the binary image is more accurate, and the quality of three-dimensional reconstruction of wires is effectively improved.

[0053] In the above, a second binary image is obtained by performing hole filling processing on the first binary image obtained by performing wire segmentation according to the original image, a plurality of line segment groups are determined according to a plurality of first straight line segments obtained by performing straight line detection on the second binary image, each line segment group and a corresponding adjacent collinear line segment group are connected to obtain a second straight line segment, and one or more target wire segments are determined according to the first straight line segments and the second straight line segments. By performing hole filling processing on the first binary image, the situation in which thicker wires in the binary image are mistakenly detected as two straight lines can be effectively reduced, and the filling of the second straight line segments can effectively supplement the missing wires, effectively improving the wire detection effect in the binary image, and improving the quality of three-dimensional reconstruction of the wires.

[0054] Based on the above embodiments, Figure 2 A flow chart of another wire detection method provided in an embodiment of the present application is given, which is a specific embodiment of the above-mentioned wire detection method. Figure 2 , the wire detection method comprises:

[0055] S210: Perform hole filling processing on the first binary image obtained by performing wire segmentation on the original image to obtain a second binary image.

[0056] S220: performing centerline extraction on the second binary image according to a set skeleton extraction method to obtain one or more centerlines, and performing straight line detection on the centerlines to obtain one or more first straight line segments.

[0057] Exemplarily, centerline extraction is performed on each second binary image obtained above according to a set skeleton extraction method to determine one or more centerlines in each second binary image. Optionally, the skeleton extraction method provided in this solution can be based on skeleton extraction functions such as opencv thinning and skimage thinning.

[0058] Furthermore, straight line detection is performed on the one or more center lines determined above to obtain one or more first straight line segments. Optionally, straight line detection can be performed on the center lines determined above to obtain the first straight line segment corresponding to each center line by using a gradient detection method. This solution extracts the center lines in the second binary image by a skeleton extraction method, and performs straight line detection on the center lines to obtain the corresponding first straight line segments, accurately determines the first straight line segments in the second binary image, and effectively improves the detection effect of the wires in the binary image.

[0059] S230: Determine one or more line segment groups according to the first straight line segment, where the line segment group includes one first straight line segment or multiple continuous first straight line segments.

[0060] S240: The first straight line segment at the first end of the line segment group is used as the current straight line segment, and the first straight line segments at the second ends of other line segment groups that meet the set collinearity condition are determined as candidate straight line segments.

[0061] Exemplarily, for each line segment group in the second binary image, the first straight line segment at the first end of the current line segment group is used as the current straight line segment. The first straight line segment at the first end and the first straight line segment at the second end of the line segment group are the first straight line segments at the two far ends of the line segment group. For example, the first straight line segment at the tail of the line segment group (the first straight line segment with the largest horizontal coordinate or vertical coordinate) can be used as the first straight line segment at the first end, and the first straight line segment at the head of the line segment group (the first straight line segment with the smallest horizontal coordinate or vertical coordinate) can be used as the first straight line segment at the second end.

[0062] In one embodiment, when there is no first straight line segment satisfying the set collinearity condition in other line segment groups, the determination of the candidate straight line segment corresponding to the current line segment group can be skipped. For example, when the current line segment is the first straight line segment at the rightmost end of the second binary image, or when there is no other line segment group in the extension direction of the current line segment, the determination of the candidate straight line segment corresponding to the current line segment group can be skipped.

[0063] For each current straight line segment, poll the first straight line segment at the second end of each other line segment group to determine whether the first straight line segment at the second end of each other line segment group and the current straight line segment meet the set collinearity condition, and determine the first straight line that meets the set collinearity condition in the first straight line segment at the second end of each other line segment group as a candidate straight line segment for the current straight line segment.

[0064] In one possible embodiment, Figure 3 As shown in a schematic diagram of a candidate line segment determination process, the wire detection method provided by the present solution includes:

[0065] S241: Determine the perpendicular distance and angle information between the current straight line segment and the first straight line segment at the second end of each of the other line segment groups.

[0066] S242: The first straight line segment whose vertical distance is within the set distance range and whose angle information is within the set angle range is taken as a candidate straight line segment.

[0067] Exemplarily, for each current straight line segment, the perpendicular distance between the current straight line segment and the first straight line segment at the second end of each of the other line segment groups and the angle information (the angle formed by the current straight line segment and the first straight line segment) are determined, and the first straight line segment whose perpendicular distance is within a set distance range and whose angle information is within a set angle range is determined as a candidate straight line segment for the current straight line segment.

[0068] For example, the set distance range can be set to, for example, 3 pixels, the set angle range can be set to 3°, the first straight line segment at the tail of the current line segment group can be used as the current straight line segment, the vertical distance and angle information between the current straight line segment and the first straight line segment at the head of each other line segment group can be calculated, and the first straight line segment with a vertical distance within 3 pixels and an angle information within 3° can be used as a candidate straight line segment. This solution can accurately determine the candidate straight line segment that may be connected to the current straight line segment from the first straight line at the second end of each other line segment group according to the vertical distance and angle information between the current straight line segment and the first straight line segment at the second end of each other line segment group, thereby improving the accuracy of subsequent completion of the straight line segment that is fused with the background or blocked, and improving the accuracy of wire detection.

[0069] S250: Determine distance information between the current straight line segment and each candidate straight line segment, and determine a target straight line segment from the candidate straight line segments according to the distance information.

[0070] Exemplarily, for each current straight line segment and the corresponding candidate straight line segment, distance information between each candidate straight line segment and the current straight line segment is determined, wherein the distance information may reflect the distance between the candidate straight line segment and the current straight line segment.

[0071] Further, the candidate straight line segment corresponding to the minimum distance information is used as the target straight line segment corresponding to the current straight line segment. The target straight line segment can be understood as two first straight line segments that belong to the same electric line as the current straight line segment and are close in distance. Optionally, when it is determined that there is a candidate straight line segment, the candidate straight line segment can be directly determined as the target straight line segment.

[0072] In a possible embodiment, the wire detection method provided by the present solution, when determining the distance information between the current straight line segment and each candidate straight line segment, and determining the target straight line segment from the candidate straight line segments based on the distance information, may be: determining the distance information between the first end of the current straight line segment and the second end of each candidate straight line segment, and determining the candidate straight line segment corresponding to the minimum distance information as the target straight line segment.

[0073] Exemplarily, for each current straight line segment and the corresponding candidate straight line segment, the distance information between the first end of the current straight line segment and the second end of each candidate straight line segment is determined, the minimum distance information among multiple distance information is determined, and the candidate straight line segment corresponding to the minimum distance information is determined as the target straight line segment. Optionally, the first end of the current straight line segment may be the tail of the current straight line segment, and the second end of the candidate straight line segment may be the head of the candidate straight line segment, that is, the distance between the two endpoints is calculated according to the coordinate information of the tail endpoint of the current straight line segment and the coordinate information of the head endpoint of the candidate straight line segment, and the distance is the distance information between the first end of the current straight line segment and the second end of the candidate straight line segment. This solution determines the candidate straight line segment with the smallest distance information between the second end and the first end of the current straight line segment as the target straight line segment, accurately determines the target straight line segment with the highest possibility of connection with the current straight line segment, improves the accuracy of subsequent completion of straight line segments that are fused with the background or blocked, and improves the accuracy of wire detection.

[0074] S260: Connect the current straight line segment and the target straight line segment to obtain a second straight line segment.

[0075] Exemplarily, for each current straight line segment, after determining the target straight line segment corresponding to the current straight line segment, the current straight line segment and the corresponding target straight line segment are connected to obtain a second straight line segment. For example, the endpoint of the current straight line segment at the first end (the rightmost end or the end point) and the endpoint of the target straight line segment at the second end (the leftmost end or the end point) are connected to obtain the second straight line segment.

[0076] Among them, the target straight line segment can be understood as two first straight line segments that belong to the same electric wire as the current straight line segment and are close in distance. Since the electric wire is fused with the background or blocked by an obstruction at the connection position of the current straight line segment and the target straight line segment in the original image, the current straight line segment and the target straight line segment are disconnected when the first straight line segment is determined based on the second binary image. At this time, the angle information and vertical distance of the current straight line segment and the target straight line segment are closer to each other than the other first straight line segments, and the distance between the two is closer to each other than the other candidate straight line segments. Based on this, the target straight line segment can be accurately determined according to the distance information between the current straight line segment and each candidate straight line segment, and the current straight line segment and the target straight line segment can be connected to obtain the second straight line segment fused with the background or blocked, so as to complete the straight line segment fused with the background or blocked, and improve the accuracy of wire detection.

[0077] S270: Merge the second straight line segment into the first straight line segment, and determine one or more target electric line segments according to the first straight line segment, wherein the target electric line segments include the first straight line segment merged with the second straight line segment and the first straight line segment not merged with the second straight line segment.

[0078] Exemplarily, for each second straight line segment, the second straight line segment is merged into the connected first straight line segment (corresponding to the current straight line segment or the target straight line segment). At this time, the length of the first straight line segment merged with the second straight line segment is the sum of the original length of the first straight line segment and the length of the second straight line segment. The second binary image includes the first straight line segment merged with the second straight line segment and the first straight line segment not merged with the second straight line segment. Further, one or more target wire segments can be determined based on the first straight line segment. For example, the endpoint coordinates of each first straight line segment in the second binary image can be used as the endpoint coordinates of each target wire segment. This solution reduces the situation of generating shorter target wire segments based on the second straight line segment by merging the second straight line segment into the first straight line segment and determining the target wire segments according to each first straight line segment, effectively reducing the influence of the noise generated by the shorter target wire segments on the accuracy of three-dimensional reconstruction when the target wire segments are subsequently used to perform three-dimensional reconstruction of the wire, thereby improving the quality of wire detection and subsequent three-dimensional reconstruction of the wire.

[0079] In a possible embodiment, when the wire detection method provided by the present solution merges the second straight line segment into the first straight line segment, it can be: determining the similarity between the second straight line segment and the adjacent first straight line segment, and merging the second straight line segment into the first straight line segment corresponding to the maximum similarity.

[0080] Exemplarily, for each second straight line segment, the similarity between the second straight line segment and two adjacent first straight line segments (i.e., the current straight line segment and the target straight line segment corresponding to the second straight line segment) is determined, and the second straight line segment is merged into the first straight line segment corresponding to the maximum similarity. Optionally, the similarity between the second straight line segment and the first straight line segment can be represented by the similarity between the slope of the second straight line segment and the slope of the first straight line segment, that is, the closer the slope of the second straight line segment is to the slope of the first straight line segment, the higher the corresponding similarity. This solution reduces the influence of the noise generated by the shorter target wire segment on the accuracy of the three-dimensional reconstruction when the target wire segment is used to perform three-dimensional reconstruction of the wire in the subsequent process by merging the second straight line into the adjacent first straight line segment with the maximum similarity, while ensuring the accuracy of the generated first straight line segment, thereby improving the quality of wire detection and subsequent three-dimensional reconstruction of the wire.

[0081] like Figure 4 A schematic diagram of a first binary image is provided and Figure 5 A schematic diagram of a second binary image is provided, wherein Figure 4is a binary image corresponding to a part of the original image. After the original image is segmented by wires to obtain the first binary image, it is assumed that there are 5 wires AE in the original image. After the original image is segmented by wires, there are 5 connected domains S1-S5 corresponding to the wires AE in the first binary image. The color values ​​corresponding to the pixels in these connected domains are 255, and the color values ​​of the pixels in other positions are 0. Since the image of wire D in the original image is relatively coarse, the connected domain corresponding to it has a hole area in the middle during the wire segmentation process, and the color value of the pixels in the hole area is 0. If the first binary image is directly detected by straight line detection, and the wire detection result is generated according to the straight line detection result, two straight lines will be detected at the wire D in the end, and two wires will be detected at the position of the wire D based on these two lines, resulting in wire detection errors.

[0082] After obtaining the first binary image of the original image, this solution will perform hole filling processing on the first binary image to obtain a second binary image. For example, the color value of the pixel point corresponding to the hole area formed at the wire D is modified to 255 to achieve the filling processing of the hole area. Figure 4 The first binary image provided is processed by hole filling to obtain the following Figure 5 A second binary image is provided, and it can be seen that the hollow area formed at the wire D in the second binary image is filled.

[0083] like Figure 6 A schematic diagram of the center line is provided and Figure 7 As shown in the schematic diagram of the first straight line segment provided, skeleton extraction methods such as opencv thinning and skimage thinning are used to extract the Figure 5 The centerline of the second binary image provided is extracted, and the result is as follows Figure 6 The center lines L1-L5 of the connected domains S1-S5 are shown in FIG. 1 and FIG. 2 . Further, the center lines L1-L5 are detected by gradient detection to obtain the following: Figure 7 The center lines L1 - L5 shown correspond to multiple first straight line segments, wherein each center line corresponds to multiple first straight line segments.

[0084] Figure 8 A schematic diagram of another first straight line segment provided in an embodiment of the present application, Fig. 9 A schematic diagram of wires in an original image provided in an embodiment of the present application, Figure 8 It shows all the first straight line segments in the second binary image corresponding to the complete area in the original image. Fig. 9 Shows the wires in the original image and Figure 8The corresponding relationship between the first straight line segments in the second binary image is provided. After the second binary image is subjected to straight line detection to obtain a plurality of first straight line segments, the continuous first straight line segments are formed into corresponding line segment groups. It is assumed that the line segment groups G11, G12 and G13 are three line segment groups corresponding to different regions of the electric wire A, wherein the line segment groups G11 and G13 include a plurality of first straight line segments, and the line segment group G12 has only one first straight line segment. Figure 8 and Fig. 9 It can be seen that there are gaps between the line segment groups G11 and G12, and between G12 and G13 due to background fusion or occlusion or no occlusion.

[0085] Fig.10 A schematic diagram of a first straight line segment fused with a second straight line segment provided in an embodiment of the present application, Fig.11 A schematic diagram of another wire in an original image provided in an embodiment of the present application, Fig.10 All the first straight line segments in the second binary image corresponding to the complete area in the original image (including the first straight line segments fused with the second straight line segments and the first straight line segments not fused with the second straight line segments) are displayed. Fig.10 Shows the wires in the original image and Fig.10 The corresponding relationship between each first straight line segment in the second binary image provided. Determine Figure 8 The adjacent collinear line segment groups of each line segment group in , and connect each line segment group and the adjacent collinear line segment group to obtain a second straight line segment, and merge the second straight line segment into the connected first straight line segment with the highest similarity. Figure 8 The line segment group G11 in the second binary graph is selected, and the first straight line segment at the end of the line segment group G11 is used as the current straight line segment. The vertical distance and angle information between the first straight line at the head of the other line segment groups in the second binary graph and the current straight line segment are determined, and the first straight line segment whose vertical distance is within the set distance range and whose angle information is within the set angle range is selected as the candidate straight line segment. Assume Figure 8 The first straight line segment corresponding to the line segment group G12 and the first straight line at the head of the line segment group G13 are determined as candidate straight line segments. The distance information D1 between the head of the first straight line segment corresponding to G12 and the tail of the current straight line segment, as well as the distance information D2 between the head of the first straight line at the head of G13 and the tail of the current straight line segment are further calculated. It can be seen that the distance information D1 is smaller than the distance information D2, and the distance information D2 is the minimum distance information. The first straight line segment corresponding to the line segment group G12 can be determined as the target straight line segment corresponding to the current straight line segment, and the current straight line segment and the target straight line segment are connected to obtain the second straight line segment. At the same time, since the similarity between the second straight line segment and the current straight line segment is higher than the similarity between the second straight line segment and the target straight line segment, the second straight line segment is merged into the current straight line segment. Combined Figure 8 and Fig. 9It can be seen that the second straight line segment is successfully added to the gap between the line segment groups G11 and G12, and between G12 and G13, which was previously separated due to background fusion or occlusion and no occlusion, and is fused into one of the connected first straight line segments. At this time, the endpoint coordinates of each first straight line segment in the second binary image can be used as the endpoint coordinates of each target wire segment to obtain multiple target wire segments.

[0086] In the above, the second binary image is obtained by performing a hole filling process on the first binary image obtained by performing wire segmentation according to the original image, a plurality of line segment groups are determined according to a plurality of first straight line segments obtained by performing line detection on the second binary image, and each line segment group and the corresponding adjacent collinear line segment group are connected to obtain a second straight line segment, and one or more target wire segments are determined according to the first straight line segment and the second straight line segment. By performing a hole filling process on the first binary image, the situation where the thicker wire in the binary image is mistakenly detected as two straight lines can be effectively reduced, and the filling of the second straight line segment can effectively supplement the missing wire, effectively improve the wire detection effect in the binary image, and improve the quality of three-dimensional reconstruction of the wire. At the same time, the target straight line segment is accurately determined according to the distance information between the current straight line segment and each candidate straight line segment, and the second straight line segment that is fused or blocked by the background is accurately restored to improve the accuracy of wire detection. In addition, the second straight line segment is fused into the first straight line segment, which effectively reduces the influence of the noise generated by the shorter target wire segment on the accuracy of three-dimensional reconstruction when the target wire segment is used to perform three-dimensional reconstruction of the wire, and improves the quality of wire detection and subsequent three-dimensional reconstruction of the wire.

[0087] Fig.12 A schematic diagram of the structure of a wire detection device provided in an embodiment of the present application is given. Fig.12 The wire detection device includes a hole filling module 31, a line segment group determination module 32, a straight line completion module 33 and a result determination module 34.

[0088] Among them, the hole filling module 31 is configured to perform hole filling processing on the first binary image obtained by wire segmentation according to the original image to obtain a second binary image; the line segment group determination module 32 is configured to perform straight line detection on the second binary image to obtain one or more first straight line segments, and determine one or more line segment groups based on the first straight line segments, and the line segment group includes a first straight line segment or multiple continuous first straight line segments; the line completion module 33 is configured to determine the adjacent collinear line segment groups of each line segment group, and connect the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment; the result determination module 34 is configured to determine one or more target wire segments based on the first straight line segment and the second straight line segment.

[0089] In the above, a second binary image is obtained by performing hole filling processing on the first binary image obtained by performing wire segmentation according to the original image, a plurality of line segment groups are determined according to a plurality of first straight line segments obtained by performing straight line detection on the second binary image, each line segment group and a corresponding adjacent collinear line segment group are connected to obtain a second straight line segment, and one or more target wire segments are determined according to the first straight line segments and the second straight line segments. By performing hole filling processing on the first binary image, the situation in which thicker wires in the binary image are mistakenly detected as two straight lines can be effectively reduced, and the filling of the second straight line segments can effectively supplement the missing wires, effectively improving the wire detection effect in the binary image, and improving the quality of three-dimensional reconstruction of the wires.

[0090] In a possible embodiment, when the line segment group determination module 32 performs line detection on the second binary image to obtain one or more first line segments, the method includes:

[0091] Extracting the center line of the second binary image according to the set skeleton extraction method to obtain one or more center lines;

[0092] Perform straight line detection on the center line to obtain one or more first straight line segments.

[0093] In a possible embodiment, when the line completion module 33 determines the adjacent collinear line segment groups of each line segment group and connects the line segment group and the adjacent collinear line segment group to obtain the second straight line segment, the following steps are performed:

[0094] The first straight line segment at the first end of the line segment group is used as the current straight line segment, and the first straight line segments at the second ends of other line segment groups that meet the set collinearity condition are determined as candidate straight line segments;

[0095] Determine the distance information between the current straight line segment and each candidate straight line segment, and determine the target straight line segment from the candidate straight line segments according to the distance information;

[0096] The current straight line segment and the target straight line segment are connected to obtain a second straight line segment.

[0097] In a possible embodiment, when the line completion module 33 determines the first straight line segment at the second end of the other line segment group that satisfies the set collinearity condition as the candidate straight line segment, the following steps are performed:

[0098] Determine the perpendicular distance and angle information between the current straight line segment and the first straight line segment at the second end of each of the other line segment groups;

[0099] The first straight line segment whose vertical distance is within the set distance range and whose angle information is within the set angle range is taken as a candidate straight line segment.

[0100] In a possible embodiment, when the straight line completion module 33 determines the distance information between the current straight line segment and each candidate straight line segment and determines the target straight line segment from the candidate straight line segments according to the distance information, the following steps are performed:

[0101] The distance information between the first end of the current straight line segment and the second end of each candidate straight line segment is determined, and the candidate straight line segment corresponding to the minimum distance information is determined as the target straight line segment.

[0102] In a possible embodiment, when the result determination module 34 determines one or more target electric wire segments according to the first straight line segment and the second straight line segment, the result determination module 34 includes:

[0103] The second straight line segment is merged into the first straight line segment, and one or more target electric line segments are determined according to the first straight line segment, wherein the target electric line segments include the first straight line segment merged with the second straight line segment and the first straight line segment not merged with the second straight line segment.

[0104] In a possible embodiment, when the result determination module 34 merges the second straight line segment into the first straight line segment, the following steps are performed:

[0105] The similarity between the second straight line segment and the adjacent first straight line segment is determined, and the second straight line segment is merged into the first straight line segment corresponding to the maximum similarity.

[0106] In a possible embodiment, the wire detection device further includes a three-dimensional reconstruction module, which is used to perform line matching according to target wire segments corresponding to the multiple original images, and generate a three-dimensional wire according to the line matching result.

[0107] It is worth noting that in the above-mentioned embodiment of the wire detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0108] The embodiment of the present application further provides a wire detection device, which can be integrated with the wire detection device provided in the embodiment of the present application. Fig.13 Schematic diagram of a wire detection device provided by an embodiment of the present application. Fig.13 The wire detection device includes: an input device 43, an output device 44, a memory 42 and one or more processors 41; the memory 42 is used to store one or more programs; when one or more programs are executed by one or more processors 41, the one or more processors 41 implement the wire detection method provided in the above embodiment. The input device 43, the output device 44, the memory 42 and the processor 41 can be connected by a bus or other means. Fig.13The example of connecting through bus is taken in the following.

[0109] The memory 42 is a computing device readable storage medium, which can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the wire detection method provided in any embodiment of the present application (for example, the hole filling module 31, the line segment group determination module 32, the straight line completion module 33 and the result determination module 34 in the wire detection device). The memory 42 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 42 may further include a memory remotely arranged relative to the processor 41, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0110] The input device 43 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 44 may include a display device such as a display screen.

[0111] The processor 41 executes the software programs, instructions and modules stored in the memory 42 to perform various functional applications and data processing of the device, that is, to implement the above-mentioned wire detection method.

[0112] The wire detection device, equipment and computer provided above can be used to execute the wire detection method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0113] The embodiment of the present application also provides a storage medium storing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the wire detection method provided in the above embodiment. The wire detection method includes: performing hole filling processing on the first binary image obtained by wire segmentation according to the original image to obtain a second binary image; performing straight line detection on the second binary image to obtain one or more first straight line segments, and determining one or more line segment groups according to the first straight line segments, the line segment groups including one first straight line segment or multiple continuous first straight line segments; determining adjacent collinear line segment groups of each line segment group, and connecting the line segment groups and the adjacent collinear line segment groups to obtain second straight line segments; determining one or more target wire segments according to the first straight line segments and the second straight line segments.

[0114] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0115] Of course, the computer executable instructions of a storage medium storing computer executable instructions provided in an embodiment of the present application are not limited to the wire detection method provided above, and can also execute related operations in the wire detection method provided in any embodiment of the present application.

[0116] The wire detection devices, equipment and storage media provided in the above embodiments can execute the wire detection method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the wire detection method provided in any embodiment of the present application.

[0117] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A wire detection method, characterized in that: include: Performing hole filling processing on the first binary image obtained by performing wire segmentation according to the original image to obtain a second binary image; Performing straight line detection on the second binary image to obtain one or more first straight line segments, and determining one or more line segment groups based on the first straight line segments, wherein the line segment groups include one first straight line segment or multiple continuous first straight line segments; Determine adjacent collinear line segment groups of each of the line segment groups, and connect the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment; One or more target electric wire segments are determined according to the first straight line segment and the second straight line segment.

2. The wire detection method according to claim 1, characterized in that: The performing straight line detection on the second binary image to obtain one or more first straight line segments includes: Extracting the center line of the second binary image according to a set skeleton extraction method to obtain one or more center lines; Perform straight line detection on the center line to obtain one or more first straight line segments.

3. The wire detection method according to claim 1, characterized in that: The step of determining adjacent collinear line segment groups of each of the line segment groups and connecting the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment includes: The first straight line segment at the first end of the line segment group is used as the current straight line segment, and the first straight line segments at the second ends of the other line segment groups that meet the set collinearity condition are determined as candidate straight line segments; Determining distance information between the current straight line segment and each of the candidate straight line segments, and determining a target straight line segment from the candidate straight line segments according to the distance information; The current straight line segment and the target straight line segment are connected to obtain a second straight line segment.

4. The wire detection method according to claim 3, characterized in that: The step of determining the first straight line segment at the second end of the other line segment groups that satisfies the set collinearity condition as a candidate straight line segment comprises: Determine the perpendicular distance and angle information between the current straight line segment and the first straight line segment at the second end of each of the other line segment groups; The first straight line segment whose vertical distance is within a set distance range and whose angle information is within a set angle range is taken as a candidate straight line segment.

5. The wire detection method according to claim 3, characterized in that: The determining of distance information between the current straight line segment and each of the candidate straight line segments, and determining a target straight line segment from the candidate straight line segments according to the distance information, comprises: The distance information between the first end of the current straight line segment and the second end of each of the candidate straight line segments is determined, and the candidate straight line segment corresponding to the minimum distance information is determined as the target straight line segment.

6. The wire detection method according to claim 1, characterized in that: The step of determining one or more target electric wire segments according to the first straight line segment and the second straight line segment comprises: The second straight line segment is merged into the first straight line segment, and one or more target electric line segments are determined according to the first straight line segment, wherein the target electric line segments include the first straight line segment merged with the second straight line segment and the first straight line segment not merged with the second straight line segment.

7. The wire detection method according to claim 6, characterized in that: The step of merging the second straight line segment into the first straight line segment comprises: The similarity between the second straight line segment and the adjacent first straight line segment is determined, and the second straight line segment is merged into the first straight line segment corresponding to the maximum similarity.

8. The wire detection method according to claim 1, characterized in that: After determining one or more target electric wire segments according to the first straight line segment and the second straight line segment, the method further includes: Line matching is performed according to target wire segments corresponding to a plurality of original images, and a three-dimensional wire is generated according to the line matching result.

9. A wire detection device, characterized in that: It includes a hole filling module, a line segment group determination module, a straight line completion module and a result determination module, wherein: The hole filling module is configured to perform hole filling processing on the first binary image obtained by performing wire segmentation on the original image to obtain a second binary image; The line segment group determination module is configured to perform straight line detection on the second binary image to obtain one or more first straight line segments, and determine one or more line segment groups based on the first straight line segments, wherein the line segment groups include one first straight line segment or multiple continuous first straight line segments; The line completion module is configured to determine adjacent collinear line segment groups of each of the line segment groups, and connect the line segment groups and the adjacent collinear line segment groups to obtain a second straight line segment; The result determination module is configured to determine one or more target electric wire segments according to the first straight line segment and the second straight line segment.

10. A wire detection device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the wire detection method according to any one of claims 1 to 8.

11. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to perform the wire detection method according to any one of claims 1 to 8 when executed by a computer processor.

Images