Telegraph pole generation method and device based on line matching, equipment and storage medium

Abstract

The embodiment of the invention discloses a telegraph pole generation method and device based on line matching, equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, the target matching line segment of the first telegraph pole line segment in each original image in other original images is determined according to the telegraph pole head image position and the telegraph pole tail image position in the plurality of original images; the three-dimensional space line segment is determined according to the first telegraph pole line segment and the target matching line segment, the target telegraph pole is reconstructed in the electronic map according to the three-dimensional space line segment, matching of the target matching line segment is not affected by the inclination of the telegraph pole, and the target matching line segment can be accurately matched in other adjacent original images; accurate three-dimensional reconstruction of the telegraph pole can be realized without depending on high-precision and complete three-dimensional point cloud data, the accuracy of generating the telegraph pole is effectively improved, and the generation effect of the telegraph pole in the map is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating utility poles based on line matching. Background Art

[0002] The generation of utility poles in existing three-dimensional maps generally obtains point cloud data of the target area through radar surveying, extracts utility pole point cloud data from the point cloud data through a segmentation algorithm, and instantiates the utility pole point cloud data through a clustering algorithm. The point cloud bounding box of the utility pole is then calculated to obtain the length, width and height attributes of the utility pole, and then vectorized utility poles are generated based on the length, width and height attributes.

[0003] However, this method of generating electric poles in 3D maps requires high accuracy and integrity in the restoration of electric pole point cloud data. When the point cloud data at the bottom of the electric pole is largely incomplete, the point cloud data may be judged as wires or noise. When the point cloud data at the top of the electric pole is largely incomplete, the accurate height of the electric pole cannot be obtained, the accuracy of generating the electric pole is low, and the effect of generating electric poles in the map is poor. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for generating utility poles based on line matching, so as to solve the technical problems in the related art that the generation of utility poles in maps relies on high-precision and complete point cloud data, the accuracy of generated utility poles is low, and the generation effect of utility poles in maps is poor, so as to effectively improve the accuracy of generated utility poles and improve the generation effect of utility poles in maps.

[0005] In a first aspect, an embodiment of the present application provides a method for generating electric poles based on line matching, comprising:

[0006] Acquire multiple original images of the target area, and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each of the original images;

[0007] Determine the target matching line segment of the first electric pole line segment in each of the original images in other original images according to the image position of the electric pole head and the image position of the electric pole tail;

[0008] Determine a three-dimensional space line segment based on the first utility pole line segment and the target matching line segment;

[0009] The target utility pole is reconstructed in the electronic map according to the three-dimensional space line segment.

[0010] In a second aspect, an embodiment of the present application provides a utility pole generation device based on line matching, comprising a position determination module, a line segment matching module, a line segment determination module, and a target reconstruction module, wherein:

[0011] The position determination module is used to obtain multiple original images of the target area and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each of the original images;

[0012] The line segment matching module is used to determine the target matching line segment of the first electric pole line segment in each of the original images in other original images according to the image position of the electric pole head and the image position of the electric pole tail;

[0013] The line segment determination module is used to determine a three-dimensional space line segment based on the first electric pole line segment and the target matching line segment;

[0014] The target reconstruction module is used to reconstruct the target utility pole in the electronic map according to the three-dimensional space line segment.

[0015] In a third aspect, an embodiment of the present application provides a utility pole generation device based on line matching, comprising: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the utility pole generation method based on line matching as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the line matching-based utility pole generation method as described in the first aspect.

[0019] The embodiment of the present application determines the target matching line segment of the first utility pole segment in each original image in other original images according to the image position of the utility pole head and the image position of the utility pole tail in multiple original images, determines the three-dimensional space line segment according to the first utility pole segment and the target matching line segment, and reconstructs the target utility pole in the electronic map according to the three-dimensional space line segment. The matching of the target matching line segment is not affected by the inclination of the utility pole, and the target matching line segment can be accurately matched in other adjacent original images, and the three-dimensional space line segment corresponding to the utility pole can be accurately generated. Each utility pole can be accurately reconstructed in the electronic map, and accurate three-dimensional reconstruction of the utility pole can be achieved without relying on high-precision and complete three-dimensional point cloud data, thereby effectively improving the accuracy of generating the utility poles and improving the utility pole generation effect in the map. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a method for generating electric poles based on line matching provided in an embodiment of the present application;

[0021] Figure 2 is a flow chart of another method for generating electric poles based on line matching provided in an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of a candidate matching line segment determination process provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram for determining a candidate matching line segment provided in an embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of a target matching line segment determination process provided by an embodiment of the present application;

[0025] Figure 6 This is a schematic diagram of determining a target matching line segment provided in an embodiment of the present application;

[0026] Figure 7 It is a structural schematic diagram of a utility pole generation device based on line matching provided in an embodiment of the present application;

[0027] Figure 8 It is a structural schematic diagram of a utility pole generation device based on line matching provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0029] The line matching-based utility pole generation method provided in this scheme can be applied to obstacle recognition in farmland. For example, a common obstacle in farmland is utility poles. The utility poles in farmland can be identified according to the line matching-based utility pole generation method provided in this scheme, so as to identify the utility poles in farmland and reconstruct the utility poles in electronic maps, so as to facilitate obstacle avoidance based on utility poles during drone path planning or flight.

[0030] Figure 1 A flow chart of a method for generating utility poles based on line matching provided in an embodiment of the present application is given. The method for generating utility poles based on line matching provided in an embodiment of the present application can be executed by a utility pole generating device based on line matching. The utility pole generating device based on line matching can be implemented in hardware and / or software and integrated in a utility pole generating device based on line matching.

[0031] The following description is made by taking the line matching-based utility pole generation device executing the line matching-based utility pole generation method as an example. Figure 1 , the line matching-based utility pole generation method includes:

[0032] S110: Acquire multiple original images of the target area, and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each original image.

[0033] The target area provided by this solution can be understood as the area where the three-dimensional reconstruction of the electric poles is required, and the original image provided by this solution can be obtained by shooting images of the target area. For example, a drone equipped with a camera flies over the target area and continuously shoots multiple images, and multiple images of the complete flight are used as the original image. The original image provided by this solution can display a picture of a part of the target area, and the picture corresponding to the entire target area can be displayed by splicing all the original images. Among them, one or more electric poles can be photographed in part or all of the original images, and the electric poles can include the head of the electric pole (for example, the top of the electric pole that is higher than the ground and connected to the electric wire) and the tail of the electric pole (for example, the end of the electric pole that contacts the ground).

[0034] Exemplarily, multiple original images of the target area are obtained, and the electric poles in each original image, as well as the image positions of the electric pole heads and the electric pole tails are determined. The image position of the electric pole heads can be understood as the position of the electric pole heads displayed in the original image (e.g., the pixel position of the center point of the top of the electric pole in the original image), and the image position of the electric pole tails can be understood as the position of the electric pole tails displayed in the original image (e.g., the pixel position of the center point of the bottom of the electric pole in the original image).

[0035] In a possible embodiment, the determination of the electric poles, the image positions of the electric pole heads, and the image positions of the electric pole tails in each original image provided by the present solution can be obtained by analyzing a trained key point detection model. For example, the key point detection model is trained by a large number of sample images with the electric poles, the image positions of the electric pole heads, and the image positions of the electric pole tails marked, and after the training is completed, the key point detection model is configured in the electric pole generation device in the map. After obtaining multiple original images of the target area, these original images are sent to the key point detection model, and the electric poles, the image positions of the electric pole heads, and the image positions of the electric pole tails of each original image are determined by the key point detection model. Optionally, the image positions of the electric pole heads and the image positions of the electric pole tails provided by the present solution may appear in pairs, that is, a pair of image positions of the electric pole heads and the image positions of the electric pole tails can be determined for each electric pole.

[0036] It needs to be explained that the telephone poles, the image positions of the telephone pole heads and the image positions of the telephone pole tails detected in the original image may be one or more. For example, when there is only one picture corresponding to a telephone pole in the original image, the image position of the telephone pole head and the image position of the telephone pole tail corresponding to the telephone pole can be detected. When there are multiple pictures corresponding to telephone poles in the original image, the image positions of the telephone pole heads and the image positions of the telephone pole tails corresponding to multiple telephone poles can be detected.

[0037] S120: Determine, according to the image position of the head of the electric pole and the image position of the tail of the electric pole, the target matching line segment of the first electric pole line segment in each original image in other original images.

[0038] Exemplarily, after determining the image positions of the pole heads and the pole tails of the poles in multiple original images, the target matching segments of the first pole segment in each original image in other original images are determined based on the image positions of the pole heads and the pole tails corresponding to each original image.

[0039] The first electric pole is the electric pole in the original image, and the first electric pole line segment corresponding to the first electric pole can be determined according to the electric pole head image position and the electric pole tail image position belonging to the same electric pole in the original image (i.e., the paired electric pole head image position and electric pole tail image position). The target matching line segment is the electric pole line segment in other original images that matches the first electric pole line segment of the current original image, and the target matching line segment can be represented according to the electric pole head image position and the electric pole tail image position belonging to the same electric pole in the corresponding original image.

[0040] Generally, the matching first telephone pole line segment and the target matching line segment are the telephone pole line segments corresponding to the same telephone pole in different original images. For example, in different original images obtained by shooting at different angles, positions, postures, etc., the two telephone pole line segments corresponding to the imaging of the same telephone pole in different original images are the matching telephone pole line segments. The telephone pole line segment in one of the original images is the first telephone pole line segment, and the telephone pole line segments formed in other original images based on the shooting of the same telephone pole are the target matching line segments of the first telephone pole line segment in other original images.

[0041] When there are multiple scenes corresponding to electric poles in the original image, there are multiple first electric pole line segments. Correspondingly, the target matching line segments can be determined in other original images according to each first electric pole line segment in the original image. Among them, there can be one or more original images with target matching line segments matching the first electric pole line segment in the current original image, that is, some of the other original images have target matching line segments matching the first electric pole line segment, and some do not. When there are multiple first electric pole line segments in the current original image, the number of target matching line segments corresponding to different first electric pole line segments can be the same or different (for example, when different electric poles are photographed in different original images).

[0042] In one embodiment, when determining the target matching line segment of the first utility pole line segment in each original image in other original images, each original image may be traversed to determine the target matching line segment of the first utility pole line segment in each original image in other original images. And when there are multiple first utility pole line segments in the original image, for each first utility pole line segment in the original image, the target matching line segments in other original images are determined respectively.

[0043] Optionally, other adjacent original images corresponding to the first utility pole line segment may be one or more adjacent original images of the original image where the first utility pole line segment is located (the image serial numbers of other adjacent original images may be before and / or after the image serial number of the original image corresponding to the first utility pole line segment). When multiple original images are obtained through aerial photography by unmanned equipment, due to the large range of the aerial image, multiple utility poles may be identified in one original image, and each utility pole is a line segment in the original image. For the line segment corresponding to each utility pole in each original image (i.e., the first utility pole line segment), find the line segment of the utility pole in other adjacent original images (i.e., the target matching line segment). The first utility pole line segment and the target matching line segment are the line segments of the same utility pole in different original images.

[0044] In the related art, the recognition of electric poles is generally based on the inclination of the electric poles. If the inclination of the electric pole relative to the ground exceeds the set angle threshold, the recognition of the inclined electric pole will be ignored, resulting in the inability of the related art to recognize the inclined electric poles, and the recognition accuracy of the electric poles is low. This solution matches the target matching line segment in other original images with the first electric pole line segment in the current original image. The matching of the target matching line segment is not affected by the inclination of the electric pole. Even if the electric pole has a large inclination relative to the ground, it can still accurately match the target matching line segment in other adjacent original images.

[0045] S130: Determine a three-dimensional space line segment based on the first utility pole line segment and the target matching line segment.

[0046] Exemplarily, multiple three-dimensional space segments are determined in the three-dimensional space according to the matched first pole segments and the target matching segments, wherein the three-dimensional space can be determined based on the three-dimensional modeling of the target area. For example, according to the pose information corresponding to each original image, the pole head image position and the pole tail image position corresponding to the matched first pole segment and the target matching segment, the corresponding three-dimensional space segment is restored in the three-dimensional space.

[0047] In a possible embodiment, the line matching-based utility pole generation method provided by the present scheme, when determining a three-dimensional space segment based on a first utility pole segment and a target matching segment, may be: using a triangulation algorithm to determine a three-dimensional space head position and a three-dimensional space tail position based on the image positions corresponding to the first utility pole segment and the target matching segment, and determining the three-dimensional space segment based on the three-dimensional space head position and the three-dimensional space tail position.

[0048] Exemplarily, after determining the target matching line segments that each first pole line segment in each original image matches in other original images, based on the first pole line segments and target matching line segments that match between the two original images, and the image positions corresponding to the first pole line segments and the target matching line segments (including the image positions of the pole heads and the pole tails), a triangulation algorithm is used to calculate the corresponding three-dimensional space head positions and three-dimensional space tail positions of multiple poles in three-dimensional space. Further, a three-dimensional space line segment in three-dimensional space can be determined based on the corresponding three-dimensional space head position and three-dimensional space tail position of the pole, and the position and size of the three-dimensional space line segment in three-dimensional space correspond to the actual position of the corresponding pole in the target area.

[0049] Among them, the triangulation algorithm can triangulate two matching poles (i.e., the matching first pole line segment and the target matching line segment) on different original images (for example, triangulate the head position and the tail position of the pole), and realize the conversion from two-dimensional image coordinates to three-dimensional geographic coordinates. Exemplarily, for the first pole line segment and the target matching line segment that have a matching relationship between the original images, the triangulation algorithm is used to calculate the three-dimensional space head position according to the pole head image position corresponding to the first pole line segment and the target matching line segment in the two original images, and the three-dimensional space tail position is calculated according to the pole tail image position corresponding to the first pole line segment and the target matching line segment in the two original images. This solution uses the triangulation algorithm to determine the geographical location of each pole head and tail, accurately generates the three-dimensional space line segment corresponding to each pole in the three-dimensional space, accurately determines the geographical location of each pole, and improves the accuracy of generating poles in the electronic map.

[0050] In one possible embodiment, after determining the three-dimensional space segments based on the first utility pole segment and the target matching segment, clustering and fusion processing can be performed on these three-dimensional space segments to merge different three-dimensional space segments belonging to the same utility pole, reduce the existence of small three-dimensional space segments, and improve the accuracy of utility pole generation. It should be explained that after obtaining multiple three-dimensional space segments, since there may be a situation in which multiple three-dimensional space segments are generated for the same utility pole during the process of generating the three-dimensional space segments, it is necessary to integrate the three-dimensional space segments generated based on the same utility pole into one three-dimensional space segment to ensure the accuracy of utility pole generation.

[0051] Exemplarily, this solution clusters multiple 3D space line segments to merge 3D space line segments with similar positions into one 3D space line segment. When multiple 3D space line segments are generated based on the same electric pole in the original image, these 3D space line segments can be clustered into one 3D space line segment due to their close geographical locations. When one 3D space line segment is generated based on the same electric pole in the original image, the 3D space line segment is independently assigned to a cluster classification. This solution performs cluster fusion processing on 3D space line segments, accurately determines the 3D space line segments corresponding to each electric pole in the target area in the 3D scene, and improves the accuracy of electric pole generation.

[0052] In one embodiment, when clustering and fusion processing is performed on three-dimensional space segments, it can be: clustering processing is performed on multiple three-dimensional space segments based on the three-dimensional space head position and the three-dimensional space tail position; determining the mean head geographic location of the three-dimensional space head position in each cluster classification, and the mean tail geographic location of the three-dimensional space tail position, and based on the mean head geographic location and the mean tail geographic location, obtaining three-dimensional space segments corresponding to multiple cluster classifications respectively.

[0053] Exemplarily, according to the three-dimensional space head position and the three-dimensional space tail position of each three-dimensional space line segment, these three-dimensional space line segments are clustered, and the three-dimensional space line segments with similar three-dimensional space head position and three-dimensional space tail position (for example, the distance between the heads and the distance between the tails are within the set distance threshold) are clustered and merged to obtain multiple cluster classifications.

[0054] Furthermore, for each cluster classification, the mean head geographic location of the three-dimensional head position corresponding to all three-dimensional space segments in the cluster classification and the mean tail geographic location of the three-dimensional tail position are determined, and the three-dimensional space line segments after multiple cluster fusion are determined based on the mean head geographic location and the mean tail geographic location corresponding to each cluster classification.

[0055] It can be understood that the geographical location of each clustered fused 3D space segment can be determined based on the average of the head geographical location mean and the tail geographical location mean of the corresponding multiple 3D space segments, and the height of the electric pole of each clustered fused 3D space segment can be determined based on the difference between the head geographical location mean and the tail geographical location mean of the corresponding multiple 3D space segments. This solution effectively improves the accuracy of electric pole generation by clustering multiple 3D space segments.

[0056] S140: Reconstructing the target utility pole in the electronic map according to the three-dimensional space line segment.

[0057] For example, after determining multiple three-dimensional space segments, the target electric poles can be reconstructed in the electronic map according to these three-dimensional space segments, so that users can view the electric poles in the target area through the electronic map, or provide obstacle information for operating equipment to ensure the safety of the operation. Optionally, the electronic map provided by this solution can be determined based on the three-dimensional reconstruction of multiple original images, or can be determined by a set three-dimensional map, and the space corresponding to the electronic map is a three-dimensional space.

[0058] Optionally, the corresponding pole model can be obtained from a pre-built pole vector model library according to the pole type in the target area, and the pole model can be scaled according to the pole height of each three-dimensional space segment (which can be determined according to the three-dimensional space head position and the three-dimensional space tail position corresponding to the three-dimensional space segment), and the pole model is set according to the set width, and the pole model is placed at a position on the electronic map corresponding to the geographical location of the three-dimensional space segment, so as to reconstruct the target pole in the electronic map according to the three-dimensional space segment, and realize the vectorization and visualization of the pole on the electronic map. Optionally, the setting of the pole model width can be customized by the user, for example, the pole model width can be set according to the empirical value, or the pole model width can be set as the pole model width after the empirical width value of the pole is expanded based on safety considerations.

[0059] In the related art, the identification of electric poles is generally based on the inclination of the electric poles. If the inclination of the electric pole relative to the ground exceeds the set angle threshold, the identification of the inclined electric pole will be ignored, resulting in the related art being unable to identify the inclined electric poles and a low accuracy rate in identifying electric poles. This solution matches the target matching line segment in other original images with the first electric pole line segment in the current original image. The matching of the target matching line segment is not affected by the inclination of the electric pole. Even electric poles with a large inclination relative to the ground can still accurately match the target matching line segment in other adjacent original images, and accurately generate the three-dimensional space line segment corresponding to the electric pole, so that each electric pole can be accurately reconstructed and generated in the electronic map.

[0060] In the above, according to the image positions of the pole heads and the pole tails in multiple original images, the target matching segments of the first pole segment in each original image in other original images are determined, the three-dimensional space segments are determined according to the first pole segment and the target matching segment, and the target pole is reconstructed in the electronic map according to the three-dimensional space segments, and the matching of the target matching segment is not affected by the inclination of the pole, and the target matching segment can be accurately matched in other adjacent original images, and the three-dimensional space segment corresponding to the pole is accurately generated, and each pole can be accurately reconstructed in the electronic map, and accurate three-dimensional reconstruction of the pole can be achieved without relying on high-precision and complete three-dimensional point cloud data, which effectively improves the accuracy of generating poles and improves the pole generation effect in the map.

[0061] Based on the above embodiments, Figure 2 A flowchart of another method for generating electric poles based on line matching provided in an embodiment of the present application is given. The method for generating electric poles based on line matching is a specific embodiment of the above-mentioned method for generating electric poles based on line matching. Figure 2 , the line matching-based utility pole generation method includes:

[0062] S210: Acquire multiple original images of the target area, and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each original image.

[0063] S220: Determine the first utility pole line segment in each original image according to the image position of the utility pole head and the image position of the utility pole tail.

[0064] S230: Determine candidate matching line segments of the first electric pole segment in other original images according to the image position of the electric pole head and the image position of the electric pole tail.

[0065] Exemplarily, for each electric pole in each original image, the first electric pole line segment corresponding to each electric pole in each original image is determined according to the image position of the electric pole head and the image position of the electric pole tail.

[0066] Furthermore, for each first electric pole segment determined above, a candidate matching segment is determined in one or more other adjacent original images according to the electric pole head image position and the electric pole tail image position of the first electric pole segment.

[0067] Among them, each first electric pole line segment may have one or more candidate matching line segments in other original images, or there may be no candidate matching line segment (for example, there is no electric pole corresponding to the first electric pole line segment in other original images). Among them, the candidate matching line segment can be understood as a line segment that may match the first electric pole line segment in other original images (that is, a line segment corresponding to the same electric pole). When there are multiple candidate matching line segments, it is necessary to select the candidate matching line segment with the highest matching degree from the multiple candidate matching line segments as the target matching line segment of the first electric pole line segment in the other original image. When a candidate matching line segment is determined, the candidate matching line segment can be directly used as the target matching line segment of the first electric pole line segment in the other original image.

[0068] In one possible embodiment, Figure 3 As shown in a schematic diagram of a candidate matching line segment determination process, the line matching-based utility pole generation method provided by the present solution includes: determining the candidate matching line segments of the first utility pole line segment in other original images according to the image position of the utility pole head and the image position of the utility pole tail, including:

[0069] S231: Determine the first epipolar line and the second epipolar line of the first telegraph pole segment in other original images according to the image position of the telegraph pole head and the image position of the telegraph pole tail.

[0070] S232: Determine the similarity between the second electric pole line segment in the other original image and the first electric pole line segment according to the first epipolar line and the second epipolar line.

[0071] S233: Determine candidate matching line segments of the first electric pole line segment in other original images from the second electric pole line segment according to the similarity.

[0072] Exemplarily, for each first electric pole segment and each other adjacent original image, the first polar line of the first electric pole segment in the other original image (i.e., the polar line of the head of the first electric pole segment in the other original image) is determined according to the image position of the electric pole head of the first electric pole segment, and the second polar line of the first electric pole segment in the other original image (i.e., the polar line of the tail of the first electric pole segment in the other original image) is determined according to the image position of the electric pole tail of the first electric pole segment. The polar line can be understood as a straight line formed by projecting a point on the original image onto other adjacent original images.

[0073] For each other adjacent original image, the similarity between each second electric pole line segment in the other original image and the first electric pole line segment is determined based on the first epipolar line and the second epipolar line on the other original image. The second electric pole is the electric pole in the other original image, and the second electric pole line segment corresponding to the second electric pole can be determined based on the electric pole head image position and the electric pole tail image position (i.e., the paired electric pole head image position and electric pole tail image position) belonging to the same electric pole in the other original image. That is, the matching degree of each second electric pole line segment in the other original image with the first electric pole line segment is determined.

[0074] Furthermore, for each other adjacent original image, according to the similarity corresponding to each second electric pole line segment, the candidate matching line segment of the first electric pole line segment in the other original image is determined from the second electric pole line segment. For example, a matching degree threshold for screening the second electric pole line segment is set, and when the matching degree corresponding to the second electric pole line segment reaches the matching degree threshold, the second electric pole line segment is determined as a candidate matching line segment.

[0075] This scheme determines the similarity between the second utility pole segment in the other original image and the first utility pole segment based on the first epipolar line and the second epipolar line of the first utility pole segment in the other original image, and determines the candidate matching segments of the first utility pole segment in the other original image from the second utility pole segment based on the similarity, thereby effectively determining the candidate matching segments that may match the first utility pole segment, and improving the accuracy of utility pole generation.

[0076] In one embodiment, the line matching-based utility pole generation method provided by the present solution includes: determining a first length of the second utility pole segment in the other original image and a second length of the second utility pole segment within the range of the first and second utility pole segments, and determining the similarity between the second utility pole segment and the first utility pole segment based on the first and second epipolar lines.

[0077] Exemplarily, for each second telegraph pole segment in other original images, the first length of the second telegraph pole segment (i.e., the length of the second telegraph pole segment) and the second length of the second telegraph pole segment within the epipolar range formed by the first epipolar line and the second epipolar line are determined, and the similarity between the second telegraph pole segment and the first telegraph pole segment is determined based on the first length and the second length. In one embodiment, the ratio of the second length to the first length can be determined as the similarity between the second telegraph pole segment and the first telegraph pole segment, that is, the more the second telegraph pole segment is located within the epipolar range formed by the first epipolar line and the second epipolar line, the greater the similarity. This scheme determines the similarity between the second telegraph pole segment and the first telegraph pole segment based on the first length of the second telegraph pole segment and the second length within the epipolar range, accurately screens out the second telegraph pole segment that is outside the epipolar range or is mostly outside the epipolar range, retains the second telegraph pole segment that is within the epipolar range or is partially within the epipolar range as a candidate matching segment, and improves the efficiency of determining the target matching segment while ensuring the accuracy of determining the target matching segment.

[0078] like Figure 4 A schematic diagram for determining a candidate matching line segment is provided, wherein S1 is a current original image, S2 is an adjacent other original image corresponding to the current original image, L11 is a first electric pole line segment in the original image S1, and P1 and P2 are the positions of the electric pole head image position and the electric pole tail image position of the first electric pole line segment L11 in the original image S1.

[0079] In the original image S2, J1 and J2 are the first pole line and the second pole line corresponding to the image position P1 of the pole head and the image position P2 of the pole tail, respectively. Assume that six second pole line segments are identified in the original image S2: L21-L26, where the second pole line segment L24 is the pole line segment of the pole corresponding to the first pole line segment L11 in the original image S2, that is, the first pole line segment L11 and the second pole line segment L24 are pole line segments detected from the same pole, and the second pole line segments L21-L23, L25, and L26 are pole line segments detected from other poles.

[0080] After determining the first polar line J1 and the second polar line J2, the similarity between the second pole line segments in the original image S2 and the first pole line segments is determined based on the first polar line J1 and the second polar line J2. For example, the similarity between each second pole line segment and the first pole line segment is determined based on the ratio of the first length of the second pole line segment and the second length within the polar line range. Among them, the second pole segment L21 is entirely outside the epipolar range, and the second pole segment L21 is not used as a candidate matching segment; the second pole segments L22, L23 and L25 are partially within the epipolar range, and the second pole segments L22 and L23 are mostly within the epipolar range and the corresponding similarity reaches the matching threshold, so the second pole segments L22 and L23 can be used as candidate matching segments, and the second pole segment L25 is mostly outside the epipolar range and the corresponding similarity does not reach the matching threshold, so the second pole segment L25 is not used as a candidate matching segment; the second pole segments L24 and L26 are both within the epipolar range and the corresponding similarity reaches the matching threshold, so the second pole segments L24 and L26 can be used as candidate matching segments. At this time, the second pole segments L22, L23, L24 and L26 in the original image S2 are all used as candidate matching segments of the first pole segment L11 in the original image S2.

[0081] S240: Determine a target matching line segment of the first electric pole line segment in other original images from the candidate matching line segments according to the camera pose of the original image.

[0082] Exemplarily, for other original images in which multiple candidate matching line segments are determined, the target matching line segment of the first electric pole line segment in other original images is determined from multiple candidate matching line segments according to the current original image corresponding to the first electric pole line segment and the camera pose corresponding to the other original images in which multiple candidate matching line segments are determined (the camera pose can be determined according to the pose information of the camera when the original image is taken). Based on this, for each first electric pole line segment in each original image, the target matching line segment of the first electric pole line segment in other original images is determined. This scheme determines the candidate matching line segment of the first electric pole line segment in other original images according to the image position of the electric pole head and the image position of the electric pole tail, and determines the target matching line segment of the first electric pole line segment in other original images from the candidate matching line segments according to the camera pose of the original image, so as to accurately determine the target matching line segment of the first electric pole line segment in other original images and improve the accuracy of electric pole generation.

[0083] In one possible embodiment, Figure 5 As shown in a schematic diagram of a target matching line segment determination process, the line matching-based utility pole generation method provided by the present solution includes: determining the target matching line segment of the first utility pole line segment in other original images from the candidate matching line segments according to the camera pose of the original image;

[0084] S241: Determine a first projection plane corresponding to the first utility pole segment according to the camera pose of the original image.

[0085] S242: Determine a three-dimensional straight line according to the first utility pole line segment and the candidate matching line segments.

[0086] S243: Determine, based on the first projection plane and the three-dimensional straight line, a target matching line segment of the first electric pole line segment in other original images from the candidate matching line segments.

[0087] Exemplarily, for each first telephone pole segment and each corresponding candidate matching segment, the first projection plane corresponding to the first telephone pole segment is determined according to the camera pose of the original image in which it is located, and based on the image position of the telephone pole head and the image position of the telephone pole tail corresponding to the first telephone pole segment, as well as the image position of the telephone pole head and the image position of the telephone pole tail corresponding to the candidate matching segment, the three-dimensional straight lines of the first telephone pole segment and the candidate matching segment in the three-dimensional space are restored through a set three-dimensional reconstruction algorithm.

[0088] Furthermore, according to the first projection plane and the three-dimensional straight line corresponding to each candidate matching line segment, the target matching line segment of the first electric pole line segment in other original images is determined from multiple candidate matching line segments. This solution improves the accuracy of electric pole generation by accurately determining the target matching line segment of the first electric pole line segment in other original images from multiple candidate matching line segments according to the first projection plane corresponding to the first electric pole line segment and the three-dimensional straight line determined based on the first electric pole line segment and the candidate matching line segments.

[0089] In one embodiment, the line matching-based utility pole generation method provided by the present scheme, when determining the target matching line segment of the first utility pole line segment in other original images from candidate matching line segments according to the first projection plane and the three-dimensional straight line, includes: determining the angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane; determining the target matching line segment of the first utility pole line segment in other original images from the candidate matching line segments according to the angle information and the distance information.

[0090] Exemplarily, for each candidate matching line segment, the angle information (the angle formed by the three-dimensional straight line and the first projection plane) and the distance information (the distance between the three-dimensional straight line and the first projection plane) between the three-dimensional straight line corresponding to the candidate matching line segment and the first projection plane are determined, and the target matching line segment of the first electric pole line segment in other original images is determined from multiple candidate matching line segments based on the angle information and the distance information. For example, the candidate matching line segment with the smallest angle information and / or distance information is determined as the target matching line segment, or the angle information and the distance information are weighted summed, and the candidate matching line segment with the lowest weighted summation result is determined as the target matching line segment. This solution improves the accuracy of electric pole generation by accurately determining the target matching line segment of each first electric pole line segment in other original images from the candidate matching line segments based on the angle information and the distance information between the three-dimensional straight line corresponding to the candidate matching line segment and the first projection plane.

[0091] like Figure 6 A schematic diagram for determining a target matching line segment is provided, in which S1 is the current original image, S2 is an adjacent original image corresponding to the current original image, C1 and C2 are cameras of the original images S1 and S2 respectively, L11 is a first electric pole line segment in the original image S1, and L24 is a candidate matching line segment corresponding to L11 in the original image S2. When judging whether the candidate matching line segment L24 is the target matching line segment of the first electric pole line segment L11 in the original image S2, the first projection plane plane1 corresponding to the first electric pole line segment L11 can be determined according to the camera pose of the original image S1, and the three-dimensional straight line L34 of the first electric pole line segment L11 and the candidate matching line segment L24 in the three-dimensional space is restored through the set three-dimensional reconstruction algorithm, and the angle information and distance information of the three-dimensional straight line L34 and the first projection plane plane1 are determined. Similarly, the angle information and distance information of the three-dimensional straight lines L32, L33, L36 (not shown) corresponding to other candidate matching line segments (L22, L23, L26, not shown in the figure) in the original image S2 and the first projection plane plane1 are determined, and the candidate matching line segment with the smallest angle information and distance information, or the smallest numerical weighted sum of the angle information and distance information, is determined as the target matching line segment. Among them, the angle information and distance information of the three-dimensional straight line L34 on the first projection plane plane1 are both the smallest, and the candidate matching line segment L24 can be determined as the target matching line segment of the first electric pole line segment L11 in the original image S2.

[0092] S250: Determine a three-dimensional space line segment based on the first utility pole line segment and the target matching line segment.

[0093] S260: Reconstructing the target electric pole in the electronic map according to the three-dimensional space line segment.

[0094] In the above, according to the image positions of the pole heads and the pole tails in multiple original images, the target matching segments of the first pole segment in each original image in other original images are determined, the three-dimensional space segments are determined according to the first pole segment and the target matching segment, and the target pole is reconstructed in the electronic map according to the three-dimensional space segments, and the matching of the target matching segment is not affected by the inclination of the pole, and the target matching segment can be accurately matched in other adjacent original images, and the three-dimensional space segment corresponding to the pole is accurately generated, and each pole can be accurately reconstructed in the electronic map, and accurate three-dimensional reconstruction of the pole can be achieved without relying on high-precision and complete three-dimensional point cloud data, which effectively improves the accuracy of generating poles and improves the pole generation effect in the map. At the same time, by determining the candidate matching line segments of each first pole line segment in other original images according to the image position of the pole head and the image position of the pole tail, and determining the target matching line segments of each first pole line segment in other original images from the candidate matching line segments according to the camera posture of each original image, the target matching line segments of each first pole line segment in other original images are accurately determined, thereby improving the accuracy of pole generation.

[0095] Figure 7 A schematic diagram of the structure of a utility pole generation device based on line matching provided in an embodiment of the present application is given. Figure 7 The utility pole generation device based on line matching includes a position determination module 51, a line segment matching module 52, a line segment determination module 53 and a target reconstruction module 54.

[0096] Among them, the position determination module 51 is used to obtain multiple original images of the target area and determine the image position of the pole head and the image position of the pole tail in each original image; the line segment matching module 52 is used to determine the target matching line segment of the first pole line segment in each original image in other original images according to the image position of the pole head and the image position of the pole tail; the line segment determination module 53 is used to determine the three-dimensional space line segment based on the first pole line segment and the target matching line segment; the target reconstruction module 54 is used to reconstruct the target pole in the electronic map according to the three-dimensional space line segment.

[0097] In the above, according to the image positions of the pole heads and the pole tails in multiple original images, the target matching segments of the first pole segment in each original image in other original images are determined, the three-dimensional space segments are determined according to the first pole segment and the target matching segment, and the target pole is reconstructed in the electronic map according to the three-dimensional space segments, and the matching of the target matching segment is not affected by the inclination of the pole, and the target matching segment can be accurately matched in other adjacent original images, and the three-dimensional space segment corresponding to the pole is accurately generated, and each pole can be accurately reconstructed in the electronic map, and accurate three-dimensional reconstruction of the pole can be achieved without relying on high-precision and complete three-dimensional point cloud data, which effectively improves the accuracy of generating poles and improves the pole generation effect in the map.

[0098] In a possible embodiment, when the line segment matching module 52 determines the target matching line segment of the first electric pole line segment in each original image in other original images according to the electric pole head image position and the electric pole tail image position, it includes:

[0099] Determine the first utility pole line segment in each original image according to the image position of the utility pole head and the image position of the utility pole tail;

[0100] Determine candidate matching line segments of the first electric pole segment in other original images according to the image position of the electric pole head and the image position of the electric pole tail;

[0101] According to the camera pose of the original image, a target matching line segment of the first electric pole line segment in other original images is determined from the candidate matching line segments.

[0102] In a possible embodiment, when the line segment matching module 52 determines the candidate matching line segments of the first electric pole line segment in other original images according to the electric pole head image position and the electric pole tail image position, it includes:

[0103] Determine the first pole line and the second pole line of the first pole segment in other original images according to the pole head image position and the pole tail image position;

[0104] Determine the similarity between the second electric pole line segment and the first electric pole line segment in the other original image according to the first epipolar line and the second epipolar line;

[0105] Determine candidate matching line segments of the first electric pole line segment in other original images from the second electric pole line segment according to the similarity.

[0106] In a possible embodiment, when the line segment matching module 52 determines the similarity between the second utility pole line segment and the first utility pole line segment in other original images according to the first epipolar line and the second epipolar line, the line segment matching module 52 includes:

[0107] Determine a first length of a second utility pole segment in the other original image, and a second length of the second utility pole segment within an epipolar range formed by the first epipolar line and the second epipolar line;

[0108] The similarity between the second utility pole segment and the first utility pole segment is determined according to the first length and the second length.

[0109] In a possible embodiment, when the line segment matching module 52 determines the target matching line segment of the first electric pole line segment in other original images from the candidate matching line segments according to the camera pose of the original image, it includes:

[0110] Determine a first projection plane corresponding to the first electric pole segment according to the camera pose of the original image;

[0111] Determine a three-dimensional straight line according to the first electric pole line segment and the candidate matching line segments;

[0112] According to the first projection plane and the three-dimensional straight line, a target matching line segment of the first electric pole line segment in other original images is determined from the candidate matching line segments.

[0113] In a possible embodiment, when the line segment matching module 52 determines the target matching line segment of the first electric pole line segment in other original images from the candidate matching line segments according to the first projection plane and the three-dimensional straight line, it includes:

[0114] Determine the angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane;

[0115] According to the angle information and the distance information, a target matching line segment of the first electric pole line segment in other original images is determined from the candidate matching line segments.

[0116] In a possible embodiment, when the line segment determination module 53 determines the three-dimensional space line segment based on the first utility pole line segment and the target matching line segment, it includes: using a triangulation algorithm to determine the three-dimensional space head position and the three-dimensional space tail position based on the image positions corresponding to the first utility pole line segment and the target matching line segment, and determining the three-dimensional space line segment based on the three-dimensional space head position and the three-dimensional space tail position.

[0117] It is worth noting that in the above-mentioned embodiment of the utility pole generation device based on line matching, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0118] The embodiment of the present application also provides a line-matching-based utility pole generation device, which can integrate the line-matching-based utility pole generation apparatus provided in the embodiment of the present application. Figure 8 Schematic diagram of a utility pole generation device based on line matching provided by an embodiment of the present application. Figure 8 The utility pole generation device based on line matching includes: an input device 63, an output device 64, a memory 62 and one or more processors 61; the memory 62 is used to store one or more programs; when one or more programs are executed by one or more processors 61, the one or more processors 61 implement the utility pole generation method based on line matching as provided in the above embodiment. The input device 63, the output device 64, the memory 62 and the processor 61 can be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0119] The memory 62 is a computing device readable storage medium, which can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the line matching-based utility pole generation method provided in any embodiment of the present application (for example, the position determination module 51, the line segment matching module 52, the line segment determination module 53 and the target reconstruction module 54 in the line matching-based utility pole generation device). The memory 62 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 62 may further include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0120] The input device 63 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 64 may include a display device such as a display screen.

[0121] The processor 61 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 62, that is, realizes the above-mentioned utility pole generation method based on line matching.

[0122] The above-mentioned line-matching-based utility pole generation device, equipment and computer can be used to execute the line-matching-based utility pole generation method provided in any of the above-mentioned embodiments, and have corresponding functions and beneficial effects.

[0123] The embodiment of the present application also provides a storage medium storing computer executable instructions. When the above computer executable instructions are executed by a computer processor, they are used to execute the line matching-based utility pole generation method provided in the above embodiment. The line matching-based utility pole generation method includes: acquiring multiple original images of the target area, and determining the image position of the utility pole head and the image position of the utility pole tail in each original image; determining the target matching line segment of the first utility pole segment in each original image in other original images according to the image position of the utility pole head and the image position of the utility pole tail; determining a three-dimensional space line segment based on the first utility pole segment and the target matching line segment; and reconstructing the target utility pole in the electronic map according to the three-dimensional space line segment.

[0124] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0125] Of course, the storage medium storing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the line matching-based utility pole generation method provided above, can also execute the relevant operations in the line matching-based utility pole generation method provided in any embodiment of the present application.

[0126] The line matching-based utility pole generation apparatus, device and storage medium provided in the above embodiments can execute the line matching-based utility pole generation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the line matching-based utility pole generation method provided in any embodiment of the present application.

[0127] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for generating electric poles based on line matching, characterized in that: include: Acquire multiple original images of the target area, and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each of the original images; Determine the target matching line segment of the first electric pole line segment in each of the original images in other original images according to the image position of the electric pole head and the image position of the electric pole tail; Determine a three-dimensional space line segment based on the first utility pole line segment and the target matching line segment; The target utility pole is reconstructed in the electronic map according to the three-dimensional space line segment.

2. The method for generating electric poles based on line matching according to claim 1, characterized in that: The step of determining target matching line segments of the first electric pole line segments in each of the original images in other original images according to the electric pole head image position and the electric pole tail image position comprises: Determine the first utility pole line segment in each of the original images according to the utility pole head image position and the utility pole tail image position; Determine candidate matching line segments of the first electric pole segment in other original images according to the image position of the electric pole head and the image position of the electric pole tail; According to the camera pose of the original image, target matching line segments of each of the first utility pole line segments in other original images are determined from the candidate matching line segments.

3. The method for generating electric poles based on line matching according to claim 2, characterized in that: The step of determining candidate matching line segments of the first electric pole line segment in other original images according to the electric pole head image position and the electric pole tail image position includes: Determine a first epipolar line and a second epipolar line of the first telegraph pole segment in other original images according to the image position of the telegraph pole head and the image position of the telegraph pole tail; Determine the similarity between a second electric pole line segment in another original image and the first electric pole line segment according to the first epipolar line and the second epipolar line; Determine candidate matching line segments of the first utility pole line segment in other original images from the second utility pole line segment according to the similarity.

4. The method for generating electric poles based on line matching according to claim 3, characterized in that: The determining, based on the first epipolar line and the second epipolar line, the similarity between the second telegraph pole line segment in the other original image and the first telegraph pole line segment comprises: Determine a first length of a second utility pole segment in the other original image, and a second length of the second utility pole segment within an epipolar range formed by the first epipolar line and the second epipolar line; The similarity between the second utility pole segment and the first utility pole segment is determined according to the first length and the second length.

5. The method for generating electric poles based on line matching according to claim 2, characterized in that: The step of determining target matching line segments of each of the first utility pole line segments in other original images from the candidate matching line segments according to the camera pose of the original image includes: Determine a first projection plane corresponding to the first utility pole segment according to the camera pose of the original image; Determine a three-dimensional straight line according to the first utility pole line segment and the candidate matching line segment; According to the first projection plane and the three-dimensional straight line, target matching line segments of each of the first utility pole line segments in other original images are determined from the candidate matching line segments.

6. The method for generating electric poles based on line matching according to claim 5, characterized in that: The step of determining target matching line segments of each of the first utility pole line segments in other original images from the candidate matching line segments according to the first projection plane and the three-dimensional straight line includes: Determine the angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane; According to the angle information and the distance information, target matching line segments of each of the first utility pole line segments in other original images are determined from the candidate matching line segments.

7. The method for generating electric poles based on line matching according to claim 1, characterized in that: The determining of the three-dimensional space line segment based on the first utility pole line segment and the target matching line segment comprises: A triangulation algorithm is used to determine a three-dimensional space head position and a three-dimensional space tail position based on the image positions corresponding to the first utility pole line segment and the target matching line segment, and a three-dimensional space line segment is determined according to the three-dimensional space head position and the three-dimensional space tail position.

8. A utility pole generation device based on line matching, characterized in that: It includes a position determination module, a line segment matching module, a line segment determination module and a target reconstruction module, wherein: The position determination module is used to obtain multiple original images of the target area and determine the image position of the head of the electric pole and the image position of the tail of the electric pole in each of the original images; The line segment matching module is used to determine the target matching line segment of the first electric pole line segment in each of the original images in other original images according to the image position of the electric pole head and the image position of the electric pole tail; The line segment determination module is used to determine a three-dimensional space line segment based on the first electric pole line segment and the target matching line segment; The target reconstruction module is used to reconstruct the target utility pole in the electronic map according to the three-dimensional space line segment.

9. A utility pole generation device based on line matching, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the utility pole generation method based on line matching as described in any one of claims 1 to 7.

10. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the utility pole generation method based on line matching according to any one of claims 1 to 7.

Images