Wire generation method and device based on wire matching, equipment and storage medium

Abstract

The embodiment of the invention discloses a wire generation method and device based on wire matching, equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, line matching is performed according to the wire pixels in each original image to obtain the plurality of three-dimensional wire scatter points in the three-dimensional space, the three-dimensional fitting line segments are determined according to the three-dimensional wire scatter points, and the dense three-dimensional wire point cloud is generated according to the three-dimensional fitting line segments. According to the method, the three-dimensional electric wire can be generated in the electronic map according to the three-dimensional electric wire point cloud, the dense three-dimensional electric wire point cloud can still be generated for accurate electric wire three-dimensional reconstruction even if the electric wire point cloud in the original image is incomplete, the electric wire generation accuracy is effectively improved, and the electric wire generation effect is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating electric wires based on line matching. Background Art

[0002] The existing three-dimensional wire generation generally obtains the point cloud data of the target area through radar surveying, extracts the wire point cloud data from the point cloud data through a classification algorithm, segments all the wires according to the height extreme points of the wire point cloud data, and then performs wire fitting on the segmented wire point cloud data to achieve wire vectorization.

[0003] However, this 3D wire generation method has high requirements on the recovery accuracy and integrity of wire point cloud data. When the wire point cloud data is incomplete, it is impossible to accurately segment the wire point cloud data. When there is a lot of noise in the wire point cloud data, the wire fitting accuracy is low, the accuracy of the generated wire is low, and the wire generation effect is poor. Summary of the invention

[0004] The embodiments of the present application provide a wire generation method, device, equipment and storage medium based on line matching to solve the technical problems in the related art that wire generation relies on high-precision and complete point cloud data, the accuracy of generated wires is low, and the wire generation effect is poor, so as to effectively improve the accuracy of generated wires and improve the wire generation effect.

[0005] In a first aspect, an embodiment of the present application provides a method for generating electric wires based on wire matching, comprising:

[0006] Acquire multiple original images of the target area, and determine the wire pixels in each of the original images;

[0007] Performing straight line fitting according to the electric wire pixels to obtain a plurality of first two-dimensional fitting line segments, and determining a plurality of three-dimensional electric wire scatter points of each of the first two-dimensional fitting lines in the three-dimensional space;

[0008] Determine a three-dimensional fitting line segment according to the three-dimensional wire scattered points, and generate a three-dimensional wire point cloud according to the three-dimensional fitting line segment;

[0009] A three-dimensional electric wire is generated in an electronic map according to the three-dimensional electric wire point cloud.

[0010] In a second aspect, an embodiment of the present application provides a wire generation device based on line matching, comprising an object determination module, a scattered point determination module, a point cloud determination module and a target generation module, wherein:

[0011] The object determination module is used to obtain multiple original images of the target area and determine the wire pixels in each of the original images;

[0012] The scattered point determination module is used to obtain a plurality of first two-dimensional fitting line segments by performing straight line fitting according to the electric wire pixels, and determine a plurality of three-dimensional electric wire scattered points of each of the first two-dimensional fitting lines in the three-dimensional space;

[0013] The point cloud determination module is used to determine a three-dimensional fitting line segment according to the three-dimensional wire scattered points, and generate a three-dimensional wire point cloud according to the three-dimensional fitting line segment;

[0014] The target generation module is used to generate three-dimensional electric wires in the electronic map according to the three-dimensional electric wire point cloud.

[0015] In a third aspect, an embodiment of the present application provides a wire generation device based on wire matching, comprising: a memory and one or more processors;

[0016] The memory is used to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the wire generation method based on wire matching as described in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the wire generation method based on line matching as described in the first aspect.

[0019] The embodiment of the present application obtains a plurality of three-dimensional wire scatter points in a three-dimensional space by performing line matching according to the wire pixels in each original image, determines a three-dimensional fitting line segment according to the three-dimensional wire scatter points, and generates a dense three-dimensional wire point cloud according to the three-dimensional fitting line segment. A three-dimensional wire can be generated in an electronic map according to the three-dimensional wire point cloud. Even if the wire point cloud in the original image is largely incomplete, a dense three-dimensional wire point cloud can still be generated for accurate three-dimensional reconstruction of the wire, thereby effectively improving the accuracy of the generated wire and improving the wire generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a wire generation method based on wire matching provided in an embodiment of the present application;

[0021] Figure 2 is a flow chart of another wire generation method based on wire matching provided in an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of an effective contour determination process provided by an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of effective contour determination provided by an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a bounding box display provided by an embodiment of the present application;

[0025] Figure 6 It is a straight line fitting display schematic diagram provided in an embodiment of the present application;

[0026] Figure 7 This is another straight line fitting display schematic diagram provided in an embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of a three-dimensional wire scattered point determination process provided by an embodiment of the present application;

[0028] Fig. 9 It is a structural schematic diagram of a wire generation device based on wire matching provided in an embodiment of the present application;

[0029] Fig.10 It is a structural schematic diagram of a wire generation device based on wire matching provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0031] Figure 1 A flow chart of a wire generation method based on line matching provided in an embodiment of the present application is given. The wire generation method based on line matching provided in an embodiment of the present application can be executed by a wire generation device based on line matching. The wire generation device based on line matching can be implemented in hardware and / or software and integrated in a wire generation device based on line matching.

[0032] The following description is made by taking the wire generation method based on wire matching performed by the wire generation device based on wire matching as an example. Figure 1 , the wire generation method based on line matching includes:

[0033] S110: Acquire multiple original images of the target area, and determine the wire pixels in each original image.

[0034] The target area provided by this solution can be understood as the area where the three-dimensional reconstruction of the wires is required, and the original image provided by this solution can be obtained by shooting the target area. For example, a drone equipped with a camera flies over the target area and continuously shoots multiple images, and multiple images of the complete flight are used as the original image. The original image provided by this solution can display the image of a part of the target area, and the image corresponding to the entire target area can be displayed by splicing all the original images.

[0035] Among them, one or more electric wires may be captured in part or all of the original image, wherein some of the electric wires may be connected to a pole (e.g., the head of a pole), and the pole may include a pole head (e.g., the top of the pole above the ground and connected to the electric wire) and a pole tail (e.g., the end of the pole in contact with the ground). In existing electric wire generation schemes, the reconstruction of electric wires in electronic maps needs to rely on precise point cloud data and terrain. For example, the generation of electric wires needs to rely on poles. When the electric wires are not connected to poles or the poles are blocked in the original image, the part of the electric wires cannot be generated on the corresponding electronic map. This scheme uses multiple original images taken to restore dense three-dimensional electric wire point clouds, thereby restoring covered or ignored points (e.g., electric wire points that are not restored due to occlusion and changes in viewing angles). Accurate three-dimensional reconstruction of electric wires can be performed based on the three-dimensional electric wire point clouds, reducing the dependence of electric wire generation on terrain.

[0036] Exemplarily, multiple original images of the target area are obtained, and the wire pixels in each original image are determined. The wire pixels can be understood as the pixel points corresponding to the wires in the original image. Optionally, the original images can be semantically segmented using a pre-trained wire semantic segmentation model to determine the wire pixels in each original image.

[0037] In a possible embodiment, the line matching-based wire generation method provided by the present solution may be as follows: obtaining multiple original images of the target area, dividing the original images into multiple sub-images, and determining the wire pixels in each sub-image.

[0038] Exemplarily, after acquiring multiple original images of the target area, each original image is divided into multiple sub-images by means of window sliding, and then the wire pixels in each sub-image are determined respectively, so as to reduce the computational pressure on image processing and improve the efficiency of determining the wire pixels. For example, the wire semantic segmentation model is used to determine the corresponding wire pixels in each sub-image respectively, so as to reduce the computational pressure on the wire semantic segmentation model and improve the efficiency of determining the wire pixels. Optionally, after the original image is divided into multiple sub-images, based on the position offset of each sub-image relative to the origin of the original image, the position of the wire pixel or subsequent related information on the original image can be restored according to the position offset. Optionally, when the wire pixel is not identified in the sub-image, the sub-image can be ignored to reduce the amount of data processing and improve the efficiency of wire generation.

[0039] S120: performing straight line fitting according to the electric wire pixels to obtain a plurality of first two-dimensional fitting line segments, and determining a plurality of three-dimensional electric wire scatter points of each first two-dimensional fitting line in the three-dimensional space.

[0040] Exemplarily, a plurality of first two-dimensional fitting line segments are obtained by performing straight line fitting according to the electric wire pixels of the plurality of original images determined above, and a plurality of three-dimensional electric wire scatter points in a three-dimensional space are determined according to the plurality of first two-dimensional fitting line segments corresponding to the plurality of original images. The three-dimensional space can be determined based on the three-dimensional modeling of the target area. The first two-dimensional fitting line segments can be obtained by performing straight line fitting according to continuous electric wire pixels on the original image, or by performing straight line fitting according to continuous electric wire pixels on each sub-image after the original image is divided into a plurality of sub-images.

[0041] Optionally, multiple 3D wire scatter points in the 3D space may be determined by performing line matching on multiple first 2D fitting line segments corresponding to multiple original images, or multiple 3D wire scatter points may be reconstructed in the 3D space based on multiple first 2D fitting line segments corresponding to multiple original images. The multiple 3D wire scatter points may be sparse point clouds determined by performing line matching or 3D reconstruction on wire pixels on multiple original images in the 3D space.

[0042] Among them, since the multiple three-dimensional wire scatter points in the three-dimensional space are relatively sparse, if the three-dimensional wires are directly generated in the electronic map based on the three-dimensional wire scatter points, there will be a large error between the generated three-dimensional wires and the actual wires (errors in length, position, direction, angle, etc.). After obtaining the multiple three-dimensional wire scatter points in the three-dimensional space, this scheme further generates a dense three-dimensional wire point cloud based on the three-dimensional wire scatter points, restores the wire points that were previously ignored or covered, and reduces the error between the generated three-dimensional wires and the actual wires.

[0043] In a possible embodiment, after the wire generation method based on line matching provided by the present solution performs straight line fitting according to the wire pixels to obtain multiple first two-dimensional fitting line segments, it can also perform segmentation processing on the multiple first two-dimensional fitting line segments, for example, each first two-dimensional fitting line segment is equally divided into segments, and each first two-dimensional fitting line segment is divided into multiple first two-dimensional fitting line segments. When the multiple three-dimensional wire scatter points of each first two-dimensional fitting line in the three-dimensional space are subsequently determined, the multiple three-dimensional wire scatter points of each first two-dimensional fitting line in the three-dimensional space after the segmentation processing are determined, and the first endpoint position and the second endpoint position corresponding to each first two-dimensional fitting line are the positions of the two endpoints of the corresponding segment after cutting. The present solution generates more first two-dimensional fitting line segments by dividing each first two-dimensional fitting line segment into multiple first two-dimensional fitting line segments, and correspondingly generates more three-dimensional wire scatter points, thereby improving the density of the generated three-dimensional wire point cloud and improving the wire generation accuracy.

[0044] S130: determining a three-dimensional fitting line segment according to the three-dimensional electric wire scattered points, and generating a three-dimensional electric wire point cloud according to the three-dimensional fitting line segment.

[0045] Exemplarily, after determining a plurality of three-dimensional wire scatter points in the three-dimensional space, a three-dimensional fitting line segment is determined according to the three-dimensional wire scatter points, wherein at least one three-dimensional fitting line segment is determined corresponding to each of the plurality of three-dimensional wire scatter points corresponding to the first two-dimensional fitting line segment. Optionally, the three-dimensional fitting line segment can be represented by the three-dimensional fitting line segment starting position and the three-dimensional fitting line segment key position corresponding to the three-dimensional fitting line segment.

[0046] Furthermore, a dense 3D wire point cloud is generated according to each 3D fitting line segment. For example, within the straight line range corresponding to the 3D fitting line segment, a plurality of 3D space points with a set density (which can be set according to the wire generation accuracy requirement) are randomly generated, and these 3D space points are the 3D wire point clouds corresponding to the 3D wire scatter points.

[0047] S140: Generate three-dimensional electric wires in the electronic map according to the three-dimensional electric wire point cloud.

[0048] Exemplarily, based on a set 3D reconstruction algorithm, 3D wires are generated in an electronic map according to the above determined dense 3D wire point cloud, so that users can view wires in a target area through the electronic map, or provide obstacle information for operating equipment to ensure operating safety. Optionally, the electronic map provided by this solution can be determined based on 3D reconstruction of multiple original images, or can be determined through a set 3D map, and the space corresponding to the electronic map is a 3D space.

[0049] As described above, by performing line matching according to the wire pixels in each original image, a plurality of three-dimensional wire scattered points in the three-dimensional space are obtained, a three-dimensional fitting line segment is determined according to the three-dimensional wire scattered points, and a dense three-dimensional wire point cloud is generated according to the three-dimensional fitting line segment. A three-dimensional wire can be generated in an electronic map according to the three-dimensional wire point cloud. Even if the wire point cloud in the original image is largely incomplete, a dense three-dimensional wire point cloud can still be generated for accurate three-dimensional reconstruction of the wire, thereby effectively improving the accuracy of the generated wire and improving the wire generation effect.

[0050] Based on the above embodiments, Figure 2 A flowchart of another wire generation method based on line matching provided in an embodiment of the present application is given. The wire generation method based on line matching is a specific embodiment of the wire generation method based on line matching. Figure 2 , the wire generation method based on line matching includes:

[0051] S210: Acquire multiple original images of the target area, divide the original images into multiple sub-images, and determine the wire pixels in each sub-image.

[0052] S220: Determine a valid contour in the sub-image according to the wire pixels, and determine a bounding box of the valid contour.

[0053] Exemplarily, after dividing each original image into multiple sub-images and determining the wire pixels on each sub-image, for the sub-images with wire pixels, the effective contours of the sub-images are determined based on the wire pixels on each sub-image, and the bounding box of each effective contour on the sub-image is further determined.

[0054] The effective contour can be understood as the contour formed by multiple wire pixels corresponding to the wire in the original image. One or more effective contours can be determined in a sub-image (for example, when there are one or more image ranges corresponding to the wires on the sub-image). Optionally, the bounding box can be a parallelogram frame surrounding the wire pixels on the sub-image, and the side of the bounding box in the direction where the wire passes through or extends is parallel to an edge on the sub-image. The bounding box can be determined by performing a minimum bounding box detection on the effective contour in the sub-image.

[0055] In one possible embodiment, Figure 3 As shown in a schematic diagram of an effective contour determination process, the wire generation method based on line matching provided by the present solution includes:

[0056] S221: Determine candidate connected areas in the sub-image based on the wire pixels, and determine the area perimeter of each candidate connected area.

[0057] S222: Determine a valid connected area from multiple candidate connected areas according to the area perimeter, and determine a valid contour in the sub-image based on the valid connected area.

[0058] Exemplarily, for each sub-image with electric wire pixels, a candidate connected area (i.e., an area formed by a plurality of continuous electric wire pixels) in each sub-image is determined according to the electric wire pixels, and the area perimeter of each candidate connected area is determined. For a sub-image with electric wire pixels, there are one or more candidate connected areas in the sub-image.

[0059] Further, according to the area perimeter and the set perimeter threshold range (for example, represented by the minimum perimeter threshold), the candidate connected areas whose area perimeters are within the perimeter threshold range are determined from multiple candidate connected areas as valid connected areas, and multiple valid contours in the sub-image are determined according to each valid connected area. The valid contour is the contour corresponding to the area surrounded by each wire pixel in the valid connected area. Among them, one valid connected area corresponds to one valid contour. When the area perimeters of all candidate connected areas are within the perimeter threshold range, these candidate connected areas are all valid connected areas.

[0060] This solution determines a valid connected area from multiple candidate connected areas according to the area perimeter of the candidate connected area, determines the valid contour in the sub-image based on the valid connected area, filters out candidate connected areas that do not meet the perimeter threshold range (for example, when the area perimeter corresponding to the candidate connected area formed by the wire pixels is less than the minimum perimeter threshold, these wire pixels can be considered as noise and discarded), thereby improving the accuracy of wire generation.

[0061] S230: Perform straight line fitting on the bounding box to obtain a first two-dimensional fitting line segment corresponding to the bounding box.

[0062] S240: Perform line matching according to the first two-dimensional fitting line segment to obtain a plurality of three-dimensional electric wire scattered points in the three-dimensional space.

[0063] Exemplarily, for each bounding box on each sub-image, a straight line fitting is performed on the bounding box to obtain the first two-dimensional fitting line segment corresponding to each bounding box. Further, line matching (epipolar matching) is performed based on the first two-dimensional fitting line segments corresponding to different original images and sub-images to obtain multiple three-dimensional wire segments in the three-dimensional space, and multiple three-dimensional wire scatter points in the three-dimensional space are obtained based on these three-dimensional wire segments.

[0064] For example, after performing straight line fitting on the bounding boxes to obtain the first two-dimensional fitting line segments corresponding to each bounding box, for the first two-dimensional fitting line segments corresponding to each bounding box, determine the first two-dimensional fitting line segments that match it in the sub-images on other original images (for example, the first two-dimensional fitting line segments corresponding to the same wire determined on different sub-images are multiple first two-dimensional fitting line segments with a matching relationship), and according to the camera pose of each sub-image (that is, the camera pose of the corresponding original image) and the matching relationship of each first two-dimensional fitting line segment, determine multiple three-dimensional wire segments in the three-dimensional space based on the epipolar matching algorithm, and determine multiple three-dimensional wire scatter points in the three-dimensional space based on these three-dimensional wire segments.

[0065] Optionally, the three-dimensional wire scatter points can be represented by the positions of the two endpoints of the three-dimensional wire segment in the three-dimensional space, that is, one three-dimensional wire segment can determine two pairs of three-dimensional wire scatter points. This solution obtains a first two-dimensional fitting line segment by linearly fitting the bounding box of the effective contour in the sub-image, and obtains multiple three-dimensional wire scatter points in the three-dimensional space by line matching based on the first two-dimensional fitting line segment, so as to provide accurate three-dimensional wire scatter points for the subsequent generation of three-dimensional wire point clouds and improve the accuracy of wire generation.

[0066] In a possible embodiment, the wire generation method based on line matching provided by the present solution includes at least one of the following methods when performing straight line fitting on the bounding box to obtain the first two-dimensional fitting line segment corresponding to the bounding box:

[0067] Method 1: A first fitting line segment is obtained by performing straight line fitting according to the pixel position information corresponding to each wire pixel in the bounding box, and a first two-dimensional fitting line segment corresponding to the bounding box is determined based on the extreme points of the first fitting line segment in the horizontal and vertical directions.

[0068] Method 2: Determine the first midpoint position information of the first short side and the second midpoint position information of the second short side in the bounding box, and determine the first two-dimensional fitting line segment corresponding to the bounding box according to the first midpoint position information and the second midpoint position information.

[0069] For the first method, for each bounding box, the pixel position information corresponding to each wire pixel in the bounding box is determined (for example, the position information of the wire pixel on the sub-image), and a straight line fitting is performed in each bounding box according to the pixel position information corresponding to the wire pixel to obtain the first fitting line segment corresponding to each bounding box. Optionally, based on the RANSAC algorithm, a straight line fitting can be performed in each bounding box according to the pixel position information corresponding to the wire pixel, so that as many wire pixels as possible fall on the first fitting line segment, and the first fitting line segment better fits the line segment corresponding to the actual wire on the sub-image, thereby improving the accuracy of wire generation.

[0070] Further, the first two-dimensional fitting line segment corresponding to each bounding box is determined according to the extreme points of the first fitting line segment corresponding to each bounding box in the horizontal and vertical directions. For example, for each bounding box, the maximum horizontal coordinate point and the minimum horizontal coordinate point of the first fitting line segment corresponding to the bounding box in the horizontal direction, as well as the maximum vertical coordinate point and the minimum vertical coordinate point in the vertical direction are determined, the first endpoint is determined according to the maximum horizontal coordinate point and the maximum vertical coordinate point, and the second endpoint is determined according to the minimum horizontal coordinate point and the minimum vertical coordinate point, or the first endpoint is determined according to the minimum horizontal coordinate point and the maximum vertical coordinate point, and the second endpoint is determined according to the maximum horizontal coordinate point and the minimum vertical coordinate point, wherein the first endpoint and the second endpoint are the first endpoint and the second endpoint corresponding to the first two-dimensional fitting line segment. In this scheme, the first two-dimensional fitting line segment corresponding to the bounding box is determined by performing straight line fitting according to the pixel position information corresponding to each wire pixel in the bounding box, and the extreme points in different coordinate axis directions are determined, so as to better fit the first two-dimensional fitting line segment corresponding to the actual wire on the sub-image, and improve the accuracy of wire generation.

[0071] For method 2, for each bounding box, the extension line of the wire generally passes through the two short sides of the bounding box. Based on this, the first midpoint position information of the first short side and the second midpoint position information of the second short side in the bounding box are determined, and the first two-dimensional fitting line segment corresponding to the bounding box is determined according to the first midpoint position information and the second midpoint position information, that is, the point corresponding to the first midpoint position information and the point corresponding to the second midpoint position information are the first endpoint and the second endpoint corresponding to the first two-dimensional fitting line segment. This solution effectively improves the efficiency of wire generation while ensuring the accuracy of wire generation by quickly and accurately determining the first two-dimensional fitting line segment corresponding to the bounding box according to the end points of the two short sides of the bounding box.

[0072] Optionally, after performing straight line fitting on the bounding box to obtain a first two-dimensional fitting line segment corresponding to the bounding box, the positions of the endpoints of the first two-dimensional fitting line segment on the original image can be determined according to the position offset of each sub-image relative to the origin of the original image.

[0073] like Figure 4As shown in the schematic diagram of determining an effective contour, S1 in the figure is a sub-image in the original image. It is assumed that four candidate connected areas Z1-Z5 are determined in the sub-image S1 based on the wire pixels identified in the sub-image S1. The area perimeter of each candidate connected area is determined, and the candidate connected area whose area perimeter reaches the perimeter threshold range is determined as a valid connected area. Assuming that the area perimeter corresponding to the candidate connected area Z5 does not reach the perimeter threshold range, the candidate connected area Z5 is considered to be an invalid contour and the candidate connected area Z5 is discarded; if the area perimeter corresponding to the candidate connected areas Z1-Z4 reaches the perimeter threshold range, the candidate connected areas Z1-Z4 are valid connected areas, and the candidate connected areas Z1-Z4 can be determined as valid contours in the sub-image S1.

[0074] like Figure 5 A schematic diagram of a bounding box display is provided. Figure 4 Based on the multiple valid contours provided, for each valid contour, the wire pixels within the valid contour are determined, and a parallelogram box surrounding these wire pixels is determined, and the upper and lower sides of the surrounding box are parallel to the upper and lower sides of the sub-image S1, and the parallelogram box is used as the surrounding box corresponding to the valid contour. Figure 5 Here K1-K4 are the bounding boxes corresponding to the valid contours Z1-Z4 respectively.

[0075] like Figure 6 A straight line fitting diagram is provided. Figure 5 On the basis of the provided multiple bounding boxes, for each bounding box, a straight line fitting is performed in the bounding box based on the RANSAC algorithm according to the pixel position information corresponding to each electric line pixel in the bounding box to obtain a first fitting line segment, and then the maximum horizontal coordinate point and the minimum horizontal coordinate point of the first fitting line segment in the horizontal axis direction, as well as the maximum vertical coordinate point and the minimum vertical coordinate point in the vertical axis direction are determined, the first endpoint is determined by the maximum horizontal coordinate point and the maximum vertical coordinate point, the second endpoint is determined according to the minimum horizontal coordinate point and the minimum vertical coordinate point, and the first endpoint and the second endpoint are connected to obtain the first two-dimensional fitting line segment corresponding to the bounding box. For example, A1 and A2 corresponding to the bounding box K1 are the first endpoint and the second endpoint of the first fitting line segment N1, then A1 and A2 are connected to obtain the first two-dimensional fitting line segment L11 corresponding to the bounding box K1, and similarly, the first two-dimensional fitting line segments L12-L14 corresponding to the bounding boxes K2-K4 are obtained.

[0076] like Figure 7 Another straight line fitting diagram is shown in the following figure. Figure 5Based on the provided multiple bounding boxes, for each bounding box, the first midpoint position information of the first short side (the upper side in the figure) and the second midpoint position information of the second short side (the lower side in the figure) in the bounding box are determined, and the first two-dimensional fitting line segment corresponding to the bounding box is obtained by connecting the first midpoint position information and the second midpoint position information. For example, B1 and B2 corresponding to the bounding box K1 are the determined first midpoint position information and the second midpoint position information, then B1 and B2 are connected to obtain the first two-dimensional fitting line segment L21 corresponding to the bounding box K1, and similarly, the first two-dimensional fitting line segments L22-L24 corresponding to the bounding boxes K2-K4 are obtained.

[0077] In one possible embodiment, Figure 8 As shown in a schematic diagram of a three-dimensional wire scattered point determination process, the wire generation method based on line matching provided by the present solution performs line matching according to the first two-dimensional fitting line segment to obtain multiple three-dimensional wire scattered points in the three-dimensional space, including:

[0078] S241: Determine, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, a target matching line segment of the first two-dimensional fitting line segment in each original image in other original images.

[0079] S242: Determine a plurality of three-dimensional electric wire scattered points in the three-dimensional space based on the first two-dimensional fitting line segment and the target matching line segment.

[0080] Exemplarily, after performing straight line fitting on the bounding boxes to obtain the first two-dimensional fitting line segments corresponding to each bounding box, the target matching line segments of each first two-dimensional fitting line segment in each original image in other original images are determined according to the first endpoint positions and the second endpoint positions of each first two-dimensional fitting line segment in the original image, wherein the first endpoint position and the second endpoint position are respectively the positions of the two endpoints of the first two-dimensional fitting line segment in the original image.

[0081] Generally, the matching first two-dimensional fitting line segments and target matching line segments are the corresponding wire segments of the same wire in different original images. For example, in different original images obtained by shooting at different angles, positions, postures, etc., the two wire segments corresponding to the imaging of the same wire in different original images are the matching wire segments. The wire segment in one of the original images is the first two-dimensional fitting line segment, and the wire segment formed in the other original images based on the shooting of the same wire is the target matching line segment of the first two-dimensional fitting line segment in the other original images.

[0082] When there are multiple images corresponding to wires in the original image, there are multiple first two-dimensional fitting line segments. Correspondingly, the target matching line segments can be determined in other original images according to each first two-dimensional fitting line segment in the original image. Among them, there may be one or more original images with target matching line segments matching the first two-dimensional fitting line segment in the current original image, that is, some of the other original images have target matching line segments matching the first two-dimensional fitting line segment, and some do not. When there are multiple first two-dimensional fitting line segments in the current original image, the number of target matching line segments corresponding to different first two-dimensional fitting line segments may be the same or different (for example, when different wires are photographed in different original images).

[0083] In one embodiment, when determining the target matching line segment of the first two-dimensional fitting line segment in each original image in other original images, the other original images may be traversed to determine the target matching line segment of the first two-dimensional fitting line segment in each original image in other original images. And when there are multiple first two-dimensional fitting line segments in the original image, the target matching line segments in other original images are determined respectively for each first two-dimensional fitting line segment in the original image.

[0084] Optionally, the other adjacent original images corresponding to the first two-dimensional fitting line segment may be original images in one or more adjacent original images of the original image where the first two-dimensional fitting line segment is located (the image sequence numbers of the other adjacent original images may be before and / or after the image sequence number of the original image corresponding to the first two-dimensional fitting line segment). When multiple original images are obtained by aerial photography with unmanned equipment, due to the large range of the aerial photography image, multiple wires may be identified in one original image, and there may also be a situation where there are multiple line segment objects in one original image. Each wire is a line segment in the original image. For the line segment corresponding to each wire in each original image (i.e., the first two-dimensional fitting line segment), find the line segment of the wire in the original image of other adjacent original images (i.e., the target matching line segment). The first two-dimensional fitting line segment and the target matching line segment are the line segments of the same wire in different original images.

[0085] After determining the target matching line segments of the first two-dimensional fitting line segments in each original image in other original images, multiple three-dimensional space line segments in the three-dimensional space are determined according to each matched first two-dimensional fitting line segment and the target matching line segment, and multiple three-dimensional wire scatter points in the three-dimensional space are determined according to the endpoints of each three-dimensional space line segment, wherein the three-dimensional space can be determined based on the three-dimensional modeling of the target area. For example, according to the pose information of each original image, the endpoint positions corresponding to the matched first two-dimensional fitting line segment and the target matching line segment, the corresponding three-dimensional space line segment is restored in the three-dimensional space, and the two three-dimensional wire scatter points corresponding to the two endpoints of the same three-dimensional space line segment are paired three-dimensional wire scatter points.

[0086] In one embodiment, the line matching-based wire generation method provided by the present solution includes: using a triangulation algorithm to determine multiple first three-dimensional space endpoint positions and second three-dimensional space endpoint positions based on the image positions corresponding to the first two-dimensional fitting line segments and the target matching line segments, and determining multiple three-dimensional wire scatter points in the three-dimensional space based on the first three-dimensional space endpoint positions and the second three-dimensional space endpoint positions.

[0087] Exemplarily, after determining the target matching line segments that match each first two-dimensional fitting line segment in each original image in other original images, based on the first two-dimensional fitting line segments and target matching line segments that match each other between the original images, and the image positions corresponding to the first two-dimensional fitting line segments and the target matching line segments (the positions of the two endpoints of the line segments on the original images), a triangulation algorithm is used to calculate the first three-dimensional space endpoint positions and the second three-dimensional space endpoint positions corresponding to multiple wires in the three-dimensional space. These first three-dimensional space endpoint positions and the second three-dimensional space endpoint positions are multiple three-dimensional wire scatter points in the three-dimensional space.

[0088] Among them, the triangulation algorithm can triangulate two matching wires (i.e., the matching first two-dimensional fitting line segment and target matching line segment) on different images (for example, triangulate the two end points of the wires), and realize the conversion from two-dimensional image coordinates to three-dimensional geographic coordinates. Exemplarily, for the first two-dimensional fitting line segment and the target matching line segment that have a matching relationship between the original images, the triangulation algorithm is used to calculate the first three-dimensional space end point position and the second three-dimensional space end point position corresponding to the first two-dimensional fitting line segment and the target matching line segment in the two original images, thereby obtaining multiple three-dimensional wire scattered points in the three-dimensional space. This solution uses the triangulation algorithm to determine the geographic location of the two end points of each wire, accurately generates the first three-dimensional space end point position and the second three-dimensional space end point position corresponding to each wire in the three-dimensional space, and improves the accuracy of generating wires in the electronic map.

[0089] In one possible embodiment, after determining a plurality of three-dimensional wire scatter points in a three-dimensional space, clustering and fusion processing may be performed on these three-dimensional wire scatter points to merge different three-dimensional wire scatter points belonging to the same wire, thereby improving the accuracy of wire generation. It should be explained that after obtaining a plurality of three-dimensional wire scatter points, since there may be a situation in which a plurality of pairs of three-dimensional wire scatter points are generated for the same wire in the process of generating a three-dimensional space line segment, it is necessary to integrate the plurality of pairs of three-dimensional wire scatter points generated based on the same wire into a pair of three-dimensional wire scatter points to ensure the accuracy of wire generation.

[0090] Exemplarily, this solution clusters multiple 3D wire scatter points to merge 3D wire scatter points with similar positions into one 3D wire scatter point. When multiple 3D wire scatter points are generated based on the same wire in the original image, these 3D wire scatter points can be clustered into one 3D wire scatter point because of their close geographical locations. When one 3D wire scatter point is generated based on the same wire in the original image, the 3D wire scatter point is independently assigned to a cluster classification. This solution performs cluster fusion processing on the 3D wire scatter points to accurately determine the 3D wire scatter points corresponding to each wire in the target area in the 3D scene, thereby improving the accuracy of wire generation.

[0091] In a possible embodiment, the wire generation method based on line matching provided by the present solution includes: determining the target matching line segment of the first two-dimensional fitting line segment in each original image in other original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image;

[0092] S2411: Determine candidate matching line segments of each first two-dimensional fitting line segment in other original images according to the first endpoint positions and the second endpoint positions of the first two-dimensional fitting line segments in the original image.

[0093] S2422: Determine, from the candidate matching line segments, the target matching line segments of the first two-dimensional fitting line segments in each original image in other original images according to the camera pose of each original image.

[0094] Exemplarily, for each first two-dimensional fitting line segment in each original image, a candidate matching line segment is determined in one or more other adjacent original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image.

[0095] Among them, each first two-dimensional fitting line segment may have one or more candidate matching line segments in the other original image, or there may be no candidate matching line segment (for example, there is no electric wire corresponding to the first two-dimensional fitting line segment in the other original image). Among them, the candidate matching line segment can be understood as a line segment that may match the first two-dimensional fitting line segment in the other original image (that is, a line segment corresponding to the same electric wire). When there are multiple candidate matching line segments, it is necessary to select the candidate matching line segment with the highest matching degree from the multiple candidate matching line segments as the target matching line segment of the first two-dimensional fitting line segment in the other original image. When a candidate matching line segment is determined, the candidate matching line segment can be directly used as the target matching line segment of the first two-dimensional fitting line segment in the other original image.

[0096] Furthermore, for other original images from which multiple candidate matching line segments are determined, the target matching line segment of the first two-dimensional fitting line segment in other original images is determined from multiple candidate matching line segments according to the current original image corresponding to the first two-dimensional fitting line segment and the camera pose corresponding to the other original images from which multiple candidate matching line segments are determined (the camera pose can be determined according to the pose information of the camera when shooting the original image). Based on this, for each first two-dimensional fitting line segment in each original image, the target matching line segment of the first two-dimensional fitting line segment in other original images is determined. This solution determines the candidate matching line segments of each first two-dimensional fitting line segment in other original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, and determines the target matching line segment of each first two-dimensional fitting line segment in other original images from the candidate matching line segments according to the camera pose of each original image, thereby accurately determining the target matching line segment of each first two-dimensional fitting line segment in other original images and improving the accuracy of wire generation.

[0097] In a possible embodiment, the wire generation method based on line matching provided by the present solution includes: determining candidate matching line segments of each first two-dimensional fitting line segment in other original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image;

[0098] S24111: Determine first epipolar lines and second epipolar lines of each first two-dimensional fitting line segment in other original images according to first endpoint positions and second endpoint positions of the first two-dimensional fitting line segments in the original images.

[0099] S24112: Determine the similarity between the second two-dimensional fitting line segment in the other original image and the first two-dimensional fitting line segment according to the first epipolar line and the second epipolar line.

[0100] S24113: Determine candidate matching line segments of the first two-dimensional fitting line segment in other original images from the second two-dimensional fitting line segment according to the similarity.

[0101] Exemplarily, for each first two-dimensional fitting line segment and each other adjacent original image, the first epipolar line of the first two-dimensional fitting line segment in the other original image (i.e., the epipolar line of the first endpoint of the first two-dimensional fitting line segment in the other original image) is determined according to the first endpoint position of the first two-dimensional fitting line segment in the original image, and the second epipolar line of each first two-dimensional fitting line segment in the other original image (i.e., the epipolar line of the second endpoint of the first two-dimensional fitting line segment in the other original image) is determined according to the second endpoint position of the first two-dimensional fitting line segment in the original image. The epipolar line can be understood as a straight line formed by projecting a point on the original image onto other adjacent original images.

[0102] For each other adjacent original image, the similarity between each second two-dimensional fitting line segment in the other original image and the first two-dimensional fitting line segment is determined based on the first epipolar line and the second epipolar line on the other original image. The second two-dimensional fitting line segment is determined based on the determination method of the first two-dimensional fitting line segment on the original image, that is, the second two-dimensional fitting line segment can be obtained by linear fitting based on continuous linear pixels on the original image, or by dividing the original image into multiple sub-images and then performing linear fitting based on continuous linear pixels on each sub-image.

[0103] Furthermore, for each other adjacent original image, according to the similarity corresponding to each second two-dimensional fitting line segment, the candidate matching line segment of the first two-dimensional fitting line segment in the other original image is determined from the second two-dimensional fitting line segments. For example, a matching degree threshold for screening the second two-dimensional fitting line segment is set, and when the matching degree corresponding to the second two-dimensional fitting line segment reaches the matching degree threshold, the second two-dimensional fitting line segment is determined as a candidate matching line segment.

[0104] This solution determines the similarity between the second two-dimensional fitting line segment in the other original image and the first two-dimensional fitting line segment according to the first epipolar line and the second epipolar line of the first two-dimensional fitting line segment in the other original image, and determines the candidate matching line segments of the first two-dimensional fitting line segment in the other original image from the second two-dimensional fitting line segment according to the similarity, effectively determines the candidate matching line segments that may match the first two-dimensional fitting line segment, and improves the accuracy of wire generation.

[0105] In one embodiment, the line matching-based wire generation method provided by the present solution determines the similarity between the second two-dimensional fitting line segment in other original images and the first two-dimensional fitting line segment based on the first epipolar line and the second epipolar line. The method may be as follows: determining a first length of the second two-dimensional fitting line segment in the other original image, and a second length of the second two-dimensional fitting line segment within the epipolar range formed by the first epipolar line and the second epipolar line; and determining the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment based on the first length and the second length.

[0106] Exemplarily, for each second two-dimensional fitting line segment in other original images, the first length of the second two-dimensional fitting line segment (i.e., the length of the second two-dimensional fitting line segment) and the second length of the second two-dimensional fitting line segment within the epipolar range formed by the first epipolar line and the second epipolar line are determined, and the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment is determined based on the first length and the second length. In one embodiment, the ratio of the second length to the first length can be determined as the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment, that is, the more the second two-dimensional fitting line segment is located within the epipolar range formed by the first epipolar line and the second epipolar line, the greater the similarity. This solution determines the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment based on the first length of the second two-dimensional fitting line segment and the second length within the epipolar range, accurately screens out the second two-dimensional fitting line segment that is outside the epipolar range or is mostly outside the epipolar range, retains the second two-dimensional fitting line segment that is within the epipolar range or is partially within the epipolar range as a candidate matching line segment, and improves the efficiency of determining the target matching line segment while ensuring the accuracy of determining the target matching line segment.

[0107] In a possible embodiment, the line matching-based wire generation method provided by the present solution includes: determining the target matching line segments of the first two-dimensional fitting line segments in each original image from the candidate matching line segments according to the camera pose of each original image in other original images.

[0108] S24221: Determine a first projection plane corresponding to a first two-dimensional fitting line segment according to the camera pose of each original image.

[0109] S24222: Determine a three-dimensional straight line based on the first two-dimensional fitting line segment and the candidate matching line segments.

[0110] S24223: Determine, according to the first projection plane and the three-dimensional straight line, target matching line segments of each first two-dimensional fitting line segment in other original images from the candidate matching line segments.

[0111] Exemplarily, for each first two-dimensional fitting line segment and the corresponding candidate matching line segment, the first projection plane corresponding to the first two-dimensional fitting line segment is determined according to the camera pose of the original image in which it is located, and based on the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the current original image, as well as the first endpoint position and the second endpoint position of the second two-dimensional fitting line segment in the corresponding original image, the three-dimensional straight line of the first two-dimensional fitting line segment and the candidate matching line segment in the three-dimensional space is restored through a set three-dimensional reconstruction algorithm.

[0112] Furthermore, according to the first projection plane and the three-dimensional straight line corresponding to each candidate matching line segment, the target matching line segment of each first two-dimensional fitting line segment in other original images is determined from multiple candidate matching line segments. This solution improves the accuracy of wire generation by accurately determining the target matching line segment of the first two-dimensional fitting line segment in other original images from multiple candidate matching line segments according to the first projection plane corresponding to the first two-dimensional fitting line segment and the three-dimensional straight line determined based on the first two-dimensional fitting line segment and the candidate matching line segment.

[0113] In a possible embodiment, the line matching-based wire generation method provided by the present scheme, when determining the target matching line segments of each first two-dimensional fitting line segment in other original images from the candidate matching line segments according to the first projection plane and the three-dimensional straight line, can be: determining the angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane; and determining the target matching line segment of each first two-dimensional fitting line segment in other original images from the candidate matching line segments according to the angle information and the distance information.

[0114] Exemplarily, for each candidate matching line segment, the angle information (the angle formed by the three-dimensional straight line and the first projection plane) and the distance information (the distance between the three-dimensional straight line and the first projection plane) between the three-dimensional straight line corresponding to the candidate matching line segment and the first projection plane are determined, and the target matching line segment of the first two-dimensional fitting line segment in other original images is determined from multiple candidate matching line segments based on the angle information and the distance information. For example, the candidate matching line segment with the smallest angle information and / or distance information is determined as the target matching line segment, or the angle information and the distance information are weighted summed, and the candidate matching line segment with the lowest weighted summation result is determined as the target matching line segment. This solution improves the accuracy of wire generation by accurately determining the target matching line segment of each first two-dimensional fitting line segment in other original images from the candidate matching line segments based on the angle information and the distance information between the three-dimensional straight line corresponding to the candidate matching line segment and the first projection plane.

[0115] In one embodiment, the wire generation method based on line matching provided by the present solution can further perform segmentation processing on the multiple first two-dimensional fitting line segments after performing straight line fitting according to the wire pixels to obtain multiple first two-dimensional fitting line segments. At this time, each first two-dimensional fitting line segment is segmented into multiple first two-dimensional fitting line segments. At the same time, the second two-dimensional fitting line segment is segmented. At this time, each second two-dimensional fitting line segment is segmented into multiple second two-dimensional fitting line segments, that is, after performing straight line fitting on the corresponding two-dimensional fitting line segments of the bounding box determined according to the effective contour corresponding to the wire pixels in each original image, these two-dimensional fitting line segments are segmented into multiple two-dimensional fitting line segments. When performing line matching subsequently, it is performed based on each two-dimensional fitting line segment after the segmentation processing, and more three-dimensional wire scatter points can be generated accordingly, thereby improving the density of the generated three-dimensional wire point cloud and the wire generation accuracy.

[0116] S250: determining a three-dimensional fitting line segment according to the three-dimensional electric wire scattered points, and generating a three-dimensional electric wire point cloud according to the three-dimensional fitting line segment.

[0117] S260: Generate three-dimensional electric wires in the electronic map according to the three-dimensional electric wire point cloud.

[0118] As described above, by performing line matching based on the wire pixels in each original image to obtain multiple three-dimensional wire scattered points in three-dimensional space, determining three-dimensional fitting line segments based on the three-dimensional wire scattered points, and generating dense three-dimensional wire point clouds based on the three-dimensional fitting line segments, three-dimensional wires can be generated in the electronic map based on the three-dimensional wire point clouds. Even when the wire point clouds in the original image are relatively incomplete, dense three-dimensional wire point clouds can still be generated for accurate three-dimensional reconstruction of wires, effectively improving the accuracy of generated wires and improving wire generation effects. At the same time, by performing linear fitting on the bounding box of the effective contour in the sub-image to obtain the first two-dimensional fitting line segment, and performing line matching based on the first two-dimensional fitting line segment to obtain multiple three-dimensional wire scattered points in three-dimensional space, accurate three-dimensional wire scattered points are provided for the subsequent generation of three-dimensional wire point clouds, thereby improving the accuracy of wire generation.

[0119] Fig. 9 A schematic diagram of the structure of a wire generation device based on wire matching provided in an embodiment of the present application is given. Fig. 9 The line matching-based electric line generation device includes an object determination module 51 , a scattered point determination module 52 , a point cloud determination module 53 and a target generation module 54 .

[0120] Among them, the object determination module 51 is used to obtain multiple original images of the target area and determine the wire pixels in each original image; the scattered point determination module 52 is used to perform straight line fitting based on the wire pixels to obtain multiple first two-dimensional fitting line segments, and determine multiple three-dimensional wire scattered points of each first two-dimensional fitting line in the three-dimensional space; the point cloud determination module 53 is used to determine the three-dimensional fitting line segments based on the three-dimensional wire scattered points, and generate a three-dimensional wire point cloud based on the three-dimensional fitting line segments; the target generation module 54 is used to generate a three-dimensional wire in the electronic map based on the three-dimensional wire point cloud.

[0121] As described above, by performing line matching according to the wire pixels in each original image, a plurality of three-dimensional wire scattered points in the three-dimensional space are obtained, a three-dimensional fitting line segment is determined according to the three-dimensional wire scattered points, and a dense three-dimensional wire point cloud is generated according to the three-dimensional fitting line segment. A three-dimensional wire can be generated in an electronic map according to the three-dimensional wire point cloud. Even if the wire point cloud in the original image is largely incomplete, a dense three-dimensional wire point cloud can still be generated for accurate three-dimensional reconstruction of the wire, thereby effectively improving the accuracy of the generated wire and improving the wire generation effect.

[0122] In a possible embodiment, when the object determination module 51 acquires multiple original images of the target area and determines the wire pixels in each original image, it includes:

[0123] A plurality of original images of the target area are obtained, the original images are divided into a plurality of sub-images, and the wire pixels in each sub-image are determined.

[0124] In a possible embodiment, when the scattered point determination module 52 performs straight line fitting according to the electric line pixels to obtain a plurality of first two-dimensional fitting line segments and determines a plurality of three-dimensional electric line scattered points of each first two-dimensional fitting line in the three-dimensional space, the scattered point determination module 52 includes:

[0125] Determine a valid contour in the sub-image according to the wire pixels, and determine a bounding box of the valid contour;

[0126] Performing straight line fitting on the bounding box to obtain a first two-dimensional fitting line segment corresponding to the bounding box;

[0127] Line matching is performed according to the first two-dimensional fitting line segment to obtain a plurality of three-dimensional electric wire scattered points in the three-dimensional space.

[0128] In a possible embodiment, when the scattered point determination module 52 determines the effective contour in the sub-image according to the wire pixels, it includes:

[0129] Determine candidate connected areas in the sub-image based on the wire pixels, and determine the area perimeter of each candidate connected area;

[0130] An effective connected area is determined from a plurality of candidate connected areas according to the area perimeter, and an effective contour in the sub-image is determined based on the effective connected area.

[0131] In a possible embodiment, when the scattered point determination module 52 performs straight line fitting on the bounding box to obtain the first two-dimensional fitting line segment corresponding to the bounding box, at least one of the following methods is included:

[0132] Method 1: performing straight line fitting according to the pixel position information corresponding to each wire pixel in the bounding box to obtain a first fitting line segment, and determining a first two-dimensional fitting line segment corresponding to the bounding box based on the extreme points of the first fitting line segment in the horizontal and vertical directions;

[0133] Method 2: Determine the first midpoint position information of the first short side and the second midpoint position information of the second short side in the bounding box, and determine the first two-dimensional fitting line segment corresponding to the bounding box according to the first midpoint position information and the second midpoint position information.

[0134] In a possible embodiment, when the scattered point determination module 52 performs line matching according to the first two-dimensional fitting line segment to obtain a plurality of three-dimensional electric wire scattered points in the three-dimensional space, the scattered point determination module 52 includes:

[0135] Determine, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, a target matching line segment of the first two-dimensional fitting line segment in each original image in other original images;

[0136] A plurality of three-dimensional electric wire scattered points in the three-dimensional space are determined based on the first two-dimensional fitting line segment and the target matching line segment.

[0137] In a possible embodiment, when the scattered point determination module 52 determines the target matching line segment of the first two-dimensional fitting line segment in each original image in other original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, the scattered point determination module 52 includes:

[0138] Determine candidate matching line segments of each first two-dimensional fitting line segment in other original images according to first endpoint positions and second endpoint positions of the first two-dimensional fitting line segments in the original image;

[0139] According to the camera pose of each original image, a target matching line segment of the first two-dimensional fitting line segment in each original image in other original images is determined from the candidate matching line segments.

[0140] In a possible embodiment, when the scattered point determination module 52 determines the candidate matching line segments of each first two-dimensional fitting line segment in other original images according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, the scattered point determination module 52 includes:

[0141] Determine, according to the first endpoint positions and the second endpoint positions of the first two-dimensional fitting line segments in the original image, the first epipolar lines and the second epipolar lines of each first two-dimensional fitting line segment in the other original image;

[0142] Determine the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment in other original images according to the first epipolar line and the second epipolar line;

[0143] Determine candidate matching line segments of the first two-dimensional fitting line segment in other original images from the second two-dimensional fitting line segment according to the similarity.

[0144] In a possible embodiment, when the scattered point determination module 52 determines the similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment in other original images according to the first epipolar line and the second epipolar line, the following steps are performed:

[0145] Determine a first length of a second two-dimensional fitting line segment in the other original image, and a second length of the second two-dimensional fitting line segment within an epipolar range formed by the first epipolar line and the second epipolar line;

[0146] The similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment is determined according to the first length and the second length.

[0147] In a possible embodiment, when the scattered point determination module 52 determines the target matching line segments of the first two-dimensional fitting line segments in each original image from the candidate matching line segments according to the camera poses of each original image, the scattered point determination module 52 includes:

[0148] Determine a first projection plane corresponding to a first two-dimensional fitting line segment according to the camera pose of each original image;

[0149] Determine a three-dimensional straight line according to the first two-dimensional fitting line segment and the candidate matching line segment;

[0150] According to the first projection plane and the three-dimensional straight line, target matching line segments of each first two-dimensional fitting line segment in other original images are determined from the candidate matching line segments.

[0151] In a possible embodiment, when the scattered point determination module 52 determines the target matching line segments of each first two-dimensional fitting line segment in other original images from the candidate matching line segments according to the first projection plane and the three-dimensional straight line, the following steps are included:

[0152] Determine the angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane;

[0153] According to the angle information and the distance information, target matching line segments of each first two-dimensional fitting line segment in other original images are determined from the candidate matching line segments.

[0154] In a possible embodiment, when the scattered point determination module 52 determines a plurality of three-dimensional electric wire scattered points in the three-dimensional space based on the first two-dimensional fitting line segment and the target matching line segment, the scattered point determination module 52 includes:

[0155] A triangulation algorithm is used to determine multiple first three-dimensional space endpoint positions and second three-dimensional space endpoint positions based on the image positions corresponding to the first two-dimensional fitting line segment and the target matching line segment, and multiple three-dimensional wire scatter points in the three-dimensional space are determined according to the first three-dimensional space endpoint positions and the second three-dimensional space endpoint positions.

[0156] In a possible embodiment, after performing straight line fitting according to the electric line pixels to obtain a plurality of first two-dimensional fitting line segments, the scattered point determination module 52 further performs segmentation processing on the plurality of first two-dimensional fitting line segments.

[0157] It is worth noting that in the above-mentioned embodiment of the wire generation device based on line matching, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0158] The embodiment of the present application further provides a wire generation device based on line matching, and the wire generation device based on line matching can be integrated with the wire generation apparatus based on line matching provided in the embodiment of the present application. Fig.10 Schematic diagram of a wire generation device based on wire matching provided by an embodiment of the present application. Fig.10 The wire generation device based on line matching includes: an input device 63, an output device 64, a memory 62 and one or more processors 61; the memory 62 is used to store one or more programs; when the one or more programs are executed by the one or more processors 61, the one or more processors 61 implement the wire generation method based on line matching as provided in the above embodiment. The input device 63, the output device 64, the memory 62 and the processor 61 can be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0159] The memory 62, as a computing device readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the line matching-based wire generation method provided in any embodiment of the present application (for example, the object determination module 51, the scattered point determination module 52, the point cloud determination module 53 and the target generation module 54 in the line matching-based wire generation device). The memory 62 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 62 may further include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0160] The input device 63 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 64 may include a display device such as a display screen.

[0161] The processor 61 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 62, that is, realizes the above-mentioned wire generation method based on wire matching.

[0162] The above-mentioned wire generation device, equipment and computer based on wire matching can be used to execute the wire generation method based on wire matching provided in any of the above-mentioned embodiments, and have corresponding functions and beneficial effects.

[0163] The embodiment of the present application also provides a storage medium storing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the line matching-based wire generation method provided in the above embodiment. The line matching-based wire generation method includes: acquiring multiple original images of the target area and determining the wire pixels in each original image; performing straight line fitting according to the wire pixels to obtain multiple first two-dimensional fitting line segments, and determining multiple three-dimensional wire scatter points of each first two-dimensional fitting line in three-dimensional space; determining three-dimensional fitting line segments according to the three-dimensional wire scatter points, and generating a three-dimensional wire point cloud according to the three-dimensional fitting line segments; generating a three-dimensional wire in an electronic map according to the three-dimensional wire point cloud.

[0164] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0165] Of course, the computer executable instructions of a storage medium storing computer executable instructions provided in an embodiment of the present application are not limited to the wire generation method based on line matching provided above, and can also execute related operations in the wire generation method based on line matching provided in any embodiment of the present application.

[0166] The line matching-based wire generation apparatus, device and storage medium provided in the above embodiments can execute the line matching-based wire generation method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the line matching-based wire generation method provided in any embodiment of the present application.

[0167] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A wire generation method based on wire matching, characterized in that: include: Acquire multiple original images of the target area, and determine the wire pixels in each of the original images; Performing straight line fitting according to the electric wire pixels to obtain a plurality of first two-dimensional fitting line segments, and determining a plurality of three-dimensional electric wire scatter points of each of the first two-dimensional fitting lines in the three-dimensional space; Determine a three-dimensional fitting line segment according to the three-dimensional wire scattered points, and generate a three-dimensional wire point cloud according to the three-dimensional fitting line segment; A three-dimensional electric wire is generated in an electronic map according to the three-dimensional electric wire point cloud.

2. The wire generation method based on wire matching according to claim 1, characterized in that: The step of acquiring multiple original images of the target area and determining the wire pixels in each of the original images includes: A plurality of original images of the target area are acquired, the original images are divided into a plurality of sub-images, and the wire pixels in each of the sub-images are determined.

3. The wire generation method based on wire matching according to claim 2, characterized in that: The step of performing straight line fitting according to the electric wire pixels to obtain a plurality of first two-dimensional fitting line segments, and determining a plurality of three-dimensional electric wire scatter points of each of the first two-dimensional fitting lines in the three-dimensional space includes: Determine a valid contour in the sub-image according to the wire pixels, and determine a bounding box of the valid contour; Performing straight line fitting on the bounding box to obtain a first two-dimensional fitting line segment corresponding to the bounding box; Line matching is performed according to the first two-dimensional fitting line segment to obtain a plurality of three-dimensional electric wire scattered points in the three-dimensional space.

4. The wire generation method based on wire matching according to claim 3, characterized in that: The determining of the effective contour in the sub-image according to the wire pixels comprises: Determine candidate connected areas in the sub-image based on the wire pixels, and determine the area perimeter of each of the candidate connected areas; An effective connected area is determined from the plurality of candidate connected areas according to the area perimeter, and an effective contour in the sub-image is determined based on the effective connected area.

5. The wire generation method based on wire matching according to claim 3, characterized in that: The performing straight line fitting on the bounding box to obtain a first two-dimensional fitting line segment corresponding to the bounding box includes at least one of the following methods: Method 1: performing straight line fitting according to pixel position information corresponding to each wire pixel in the bounding box to obtain a first fitting line segment, and determining a first two-dimensional fitting line segment corresponding to the bounding box based on extreme points of the first fitting line segment in the horizontal and vertical directions; Method 2: Determine the first midpoint position information of the first short side and the second midpoint position information of the second short side in the bounding box, and determine the first two-dimensional fitting line segment corresponding to the bounding box according to the first midpoint position information and the second midpoint position information.

6. The wire generation method based on wire matching according to claim 3, characterized in that: The performing line matching according to the first two-dimensional fitting line segment to obtain a plurality of three-dimensional electric wire scattered points in the three-dimensional space includes: Determine, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, a target matching line segment of the first two-dimensional fitting line segment in each of the original images in other original images; A plurality of three-dimensional electric wire scattered points in a three-dimensional space are determined based on the first two-dimensional fitting line segment and the target matching line segment.

7. The wire generation method based on wire matching according to claim 6, characterized in that: The determining, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, a target matching line segment of the first two-dimensional fitting line segment in each of the original images in other original images includes: Determine candidate matching line segments of each of the first two-dimensional fitting line segments in other original images according to first endpoint positions and second endpoint positions of the first two-dimensional fitting line segments in the original image; According to the camera pose of each of the original images, the target matching line segments of the first two-dimensional fitting line segments in each of the original images in other original images are determined from the candidate matching line segments.

8. The wire generation method based on wire matching according to claim 7, characterized in that: The determining, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, candidate matching line segments of each of the first two-dimensional fitting line segments in other original images includes: Determine, according to the first endpoint position and the second endpoint position of the first two-dimensional fitting line segment in the original image, the first epipolar line and the second epipolar line of each of the first two-dimensional fitting line segments in other original images; Determine, according to the first epipolar line and the second epipolar line, a similarity between a second two-dimensional fitting line segment in another original image and the first two-dimensional fitting line segment; Determine candidate matching line segments of the first two-dimensional fitting line segment in other original images from the second two-dimensional fitting line segments according to the similarity.

9. The wire generation method based on wire matching according to claim 8, characterized in that: The determining, based on the first epipolar line and the second epipolar line, the similarity between the second two-dimensional fitting line segment in the other original image and the first two-dimensional fitting line segment comprises: Determine a first length of a second two-dimensional fitting line segment in the other original image, and a second length of the second two-dimensional fitting line segment within an epipolar line range formed by the first epipolar line and the second epipolar line; The similarity between the second two-dimensional fitting line segment and the first two-dimensional fitting line segment is determined according to the first length and the second length.

10. The wire generation method based on wire matching according to claim 7, characterized in that: The step of determining, from the candidate matching line segments according to the camera poses of the respective original images, target matching line segments of the first two-dimensional fitting line segments in the respective original images in other original images comprises: Determine a first projection plane corresponding to the first two-dimensional fitting line segment according to the camera pose of each of the original images; Determine a three-dimensional straight line according to the first two-dimensional fitting line segment and the candidate matching line segment; According to the first projection plane and the three-dimensional straight line, target matching line segments of each of the first two-dimensional fitting line segments in other original images are determined from the candidate matching line segments.

11. The wire generation method based on wire matching according to claim 10, characterized in that: The step of determining, from the candidate matching line segments, target matching line segments of each of the first two-dimensional fitting line segments in other original images according to the first projection plane and the three-dimensional straight line includes: Determine angle information and distance information between the three-dimensional straight line corresponding to each candidate matching line segment and the first projection plane; According to the angle information and the distance information, target matching line segments of each of the first two-dimensional fitting line segments in other original images are determined from the candidate matching line segments.

12. The wire generation method based on wire matching according to claim 6, characterized in that: The determining of a plurality of three-dimensional electric wire scattered points in a three-dimensional space based on the first two-dimensional fitting line segment and the target matching line segment comprises: A triangulation algorithm is used to determine a plurality of first three-dimensional space endpoint positions and a second three-dimensional space endpoint positions based on the image positions corresponding to the first two-dimensional fitting line segment and the target matching line segment, and a plurality of three-dimensional wire scatter points in the three-dimensional space are determined according to the first three-dimensional space endpoint positions and the second three-dimensional space endpoint positions.

13. The wire generation method based on wire matching according to any one of claims 1 to 12, characterized in that: After performing straight line fitting according to the electric line pixels to obtain a plurality of first two-dimensional fitting line segments, the method further includes: The plurality of first two-dimensional fitting line segments are segmented and divided.

14. A wire generation device based on wire matching, characterized in that: It includes an object determination module, a scattered point determination module, a point cloud determination module and a target generation module, wherein: The object determination module is used to obtain multiple original images of the target area and determine the wire pixels in each of the original images; The scattered point determination module is used to obtain a plurality of first two-dimensional fitting line segments by performing straight line fitting according to the electric wire pixels, and determine a plurality of three-dimensional electric wire scattered points of each of the first two-dimensional fitting lines in the three-dimensional space; The point cloud determination module is used to determine a three-dimensional fitting line segment according to the three-dimensional wire scattered points, and generate a three-dimensional wire point cloud according to the three-dimensional fitting line segment; The target generation module is used to generate three-dimensional electric wires in the electronic map according to the three-dimensional electric wire point cloud.

15. A wire generation device based on wire matching, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the wire generation method based on wire matching as described in any one of claims 1 to 13.

16. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the wire matching-based wire generation method according to any one of claims 1 to 13.

Images