Distribution network field three-dimensional reconstruction method, system and device and readable storage medium

Abstract

A distribution network site three-dimensional reconstruction method, system and device and a readable storage medium, and relates to the technical field of three-dimensional reconstruction, the method comprises the steps of obtaining a target unmanned aerial vehicle aerial image sequence, the target unmanned aerial vehicle aerial image sequence comprising multi-view images obtained by performing multi-angle shooting on a tower in a target distribution network site; extracting key points of each image in the aerial image sequence of the target unmanned aerial vehicle based on a key point detection algorithm; based on the extracted key points, cross-image matching is carried out through distance measurement and context information of the key points, so that corresponding points of the same object in different images are determined; based on each matched point pair and a preset camera parameter, generating a sparse point cloud by using a three-dimensional reconstruction algorithm; and generating a dense point cloud based on the sparse point cloud and the depth information of the image so as to realize three-dimensional reconstruction of the target distribution network site. According to the invention, details and features in a complex scene can be effectively captured, so that high-quality three-dimensional reconstruction is realized.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a distribution network site three-dimensional reconstruction method, system, device and readable storage medium. Background Art

[0002] Three-dimensional reconstruction is an important research field in computer vision. This technology uses two-dimensional images to restore the three-dimensional structure and texture information of real scenes. It is usually used in digital twins, smart cities, and smart manufacturing. With the help of three-dimensional reconstruction technology, people can express the objective world in virtual space and use computers to visualize it, so as to analyze and manage the real world. Considering that the three-dimensional model can intuitively display the structure and layout of the distribution network, the distribution network management personnel can have a clearer understanding of the operation status of the distribution network, so as to facilitate fault analysis and processing; therefore, in the field of power inspection and distribution network management, three-dimensional reconstruction of the distribution network site is of great significance.

[0003] However, due to the complexity of power equipment and the diversity of the environment, the accuracy and robustness of 3D reconstruction are limited to a certain extent, so that it may be difficult to obtain accurate 3D information in complex environments. Therefore, it is urgent to find an effective technical solution to capture the details and features in complex scenes and achieve high-quality 3D reconstruction. Summary of the invention

[0004] The present application provides a distribution network site three-dimensional reconstruction method, system, device and readable storage medium, which can effectively capture the details and features in complex scenes to achieve high-quality three-dimensional reconstruction.

[0005] In a first aspect, an embodiment of the present application provides a method for three-dimensional reconstruction of a distribution network site, comprising the following steps:

[0006] Acquire a target UAV aerial image sequence, wherein the target UAV aerial image sequence includes multi-view images obtained by shooting a pole tower in a target distribution network site at multiple angles;

[0007] Extract the key points of each image in the target UAV aerial image sequence based on the key point detection algorithm;

[0008] Based on the extracted key points, cross-image matching is performed through distance measurement and the context information of the key points to determine the corresponding points of the same object in different images;

[0009] Based on the matched point pairs and preset camera parameters, a sparse point cloud is generated using a 3D reconstruction algorithm;

[0010] A dense point cloud is generated based on the sparse point cloud and the depth information of the image to achieve three-dimensional reconstruction of the target distribution network site.

[0011] In combination with the first aspect, in one embodiment, before the step of acquiring the target drone aerial image sequence, the method further includes:

[0012] Obtain an initial UAV aerial image sequence;

[0013] Performing a multi-scale noise evaluation on each image in the initial UAV aerial image sequence, and determining a noise level of each image based on the obtained evaluation results;

[0014] Each image is denoised based on the determined noise level to form a target UAV aerial image sequence.

[0015] In combination with the first aspect, in one implementation, the formula for multi-scale noise evaluation is:

[0016]

[0017] n s =(w1×LN+w2 / (1+SNR));

[0018] Where, noise avg represents the multi-scale noise evaluation result, S represents the total number of scales considered, w1, w2 and w s represents the weight factor, n s It represents the comprehensive noise value of the sth scale, LN represents the noise level, and SNR represents the signal-to-noise ratio.

[0019] In combination with the first aspect, in one embodiment, before the step of acquiring the target drone aerial image sequence, the method further includes:

[0020] Obtain an initial UAV aerial image sequence;

[0021] Perform edge detection processing on each image in the initial UAV aerial image sequence based on the edge detection algorithm;

[0022] When it is determined according to the edge detection processing result that the image has an edge blur problem, the image is subjected to image edge enhancement processing to form a target UAV aerial image sequence.

[0023] In combination with the first aspect, in one embodiment, after the step of performing edge detection processing on each image in the initial drone aerial image sequence based on the edge detection algorithm, the method further includes:

[0024] When it is determined according to the edge detection processing result that there are multiple adjacent edge regions in the image, the overlap between the multiple adjacent edge regions is calculated;

[0025] If the overlap degree is greater than a preset overlap degree threshold, a plurality of adjacent edge regions are merged.

[0026] In combination with the first aspect, in one implementation, extracting key points of each image in the target drone aerial image sequence based on a key point detection algorithm includes:

[0027] For each image in the target UAV aerial image sequence, a multi-scale image pyramid is constructed through Gaussian filtering and downsampling;

[0028] Key point detection is performed on each scale image in the multi-scale image pyramid to obtain a series of key points and their corresponding feature descriptors by analyzing the image features of each pixel in the image and its area.

[0029] In combination with the first aspect, in one implementation, the cross-image matching is performed based on the extracted key points by measuring the distance and the context information of the key points to determine the corresponding points of the same object in different images, including:

[0030] In the cross-image matching process, all key points in different images are matched pairwise;

[0031] For each pair of matching points, the similarity between the feature descriptors is determined based on the distance measurement and the context information of the key points, and when the similarity is determined to be greater than a preset similarity threshold, the two currently matched key points are regarded as corresponding points of the same object in different images.

[0032] In a second aspect, an embodiment of the present application provides a distribution network site 3D reconstruction system, including:

[0033] A data acquisition module, which is used to acquire a target drone aerial image sequence, wherein the target drone aerial image sequence includes multi-view images obtained by shooting a pole tower in a target distribution network site at multiple angles;

[0034] The feature matching module is used to extract the key points of each image in the target drone aerial image sequence based on the key point detection algorithm; based on the extracted key points, cross-image matching is performed through distance measurement and the context information of the key points to determine the corresponding points of the same object in different images;

[0035] The 3D reconstruction module is used to generate a sparse point cloud based on the matched point pairs and preset camera parameters using a 3D reconstruction algorithm; and to generate a dense point cloud based on the sparse point cloud and the depth information of the image to achieve 3D reconstruction of the target distribution network site.

[0036] In conjunction with the second aspect, in one implementation, the data acquisition module is further used to:

[0037] Obtain an initial UAV aerial image sequence;

[0038] Performing a multi-scale noise evaluation on each image in the initial UAV aerial image sequence, and determining a noise level of each image based on the obtained evaluation results;

[0039] Each image is denoised based on the determined noise level to form a target UAV aerial image sequence.

[0040] In conjunction with the second aspect, in one implementation, the formula for multi-scale noise evaluation is:

[0041]

[0042] n s =(w1×LN+w2 / (1+SNR));

[0043] Where, noise avg represents the multi-scale noise evaluation result, S represents the total number of scales considered, w1, w2 and w s represents the weight factor, n s It represents the comprehensive noise value of the sth scale, LN represents the noise level, and SNR represents the signal-to-noise ratio.

[0044] In conjunction with the second aspect, in one implementation, the data acquisition module is further used to:

[0045] Obtain an initial UAV aerial image sequence;

[0046] Perform edge detection processing on each image in the initial UAV aerial image sequence based on the edge detection algorithm;

[0047] When it is determined according to the edge detection processing result that the image has an edge blur problem, the image is subjected to image edge enhancement processing to form a target UAV aerial image sequence.

[0048] In conjunction with the second aspect, in one implementation, the data acquisition module is specifically used to:

[0049] When it is determined according to the edge detection processing result that there are multiple adjacent edge regions in the image, the overlap between the multiple adjacent edge regions is calculated;

[0050] If the overlap degree is greater than a preset overlap degree threshold, a plurality of adjacent edge regions are merged.

[0051] In conjunction with the second aspect, in one implementation, the feature matching module is specifically used to:

[0052] For each image in the target UAV aerial image sequence, a multi-scale image pyramid is constructed through Gaussian filtering and downsampling;

[0053] Key point detection is performed on each scale image in the multi-scale image pyramid to obtain a series of key points and their corresponding feature descriptors by analyzing the image features of each pixel in the image and its area.

[0054] In conjunction with the second aspect, in one implementation, the feature matching module is further configured to:

[0055] In the cross-image matching process, all key points in different images are matched pairwise;

[0056] For each pair of matching points, the similarity between the feature descriptors is determined based on the distance measurement and the context information of the key points, and when the similarity is determined to be greater than a preset similarity threshold, the two currently matched key points are regarded as corresponding points of the same object in different images.

[0057] In the third aspect, an embodiment of the present application provides a distribution network site three-dimensional reconstruction device, which includes a processor, a memory, and a distribution network site three-dimensional reconstruction program stored in the memory and executable by the processor, wherein when the distribution network site three-dimensional reconstruction program is executed by the processor, the steps of the distribution network site three-dimensional reconstruction method as described above are implemented.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a distribution network site three-dimensional reconstruction program is stored. When the distribution network site three-dimensional reconstruction program is executed by a processor, the steps of the distribution network site three-dimensional reconstruction method as described above are implemented.

[0059] The beneficial effects brought by the technical solution provided by the embodiments of the present application include:

[0060] The multi-view images obtained by drone aerial photography can fully capture all angles and details of the pole towers in the distribution network site to avoid dead angles and blind spots that may exist in traditional inspections; at the same time, cross-image matching is performed by combining distance measurement and contextual information of key points to accurately find corresponding points of the same object in different images, providing accurate data support for subsequent three-dimensional reconstruction; then the three-dimensional reconstruction algorithm is used to generate sparse point clouds and dense point clouds, which can provide rich three-dimensional information to build a high-precision three-dimensional model, so as to more accurately reflect the actual situation of the distribution network site and provide strong support for subsequent fault analysis, maintenance decisions, etc. It can be seen that the present application can effectively capture the details and features in complex scenes to achieve high-quality three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for three-dimensional reconstruction of a distribution network site provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of functional modules of an embodiment of a distribution network site 3D reconstruction system provided in an embodiment of the present application;

[0063] Figure 3 This is a schematic diagram of the hardware structure of the distribution network site three-dimensional reconstruction equipment involved in the embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0066] In a first aspect, an embodiment of the present application provides a method for three-dimensional reconstruction of a distribution network site.

[0067] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the method for three-dimensional reconstruction of a distribution network site in this application. Figure 1 As shown, the distribution network site 3D reconstruction method includes:

[0068] Step S10: Acquire a target UAV aerial image sequence, wherein the target UAV aerial image sequence includes multi-view images obtained by photographing the poles and towers in the target distribution network site from multiple angles.

[0069] For example, it should be noted that the target distribution network site in this embodiment can be a 10kV distribution network site, or a 35kV distribution network site, or of course other types of distribution network sites, which can be determined according to actual needs and are not limited here. For the sake of simplicity of description, the subsequent embodiments will take the target distribution network site as a 10kV distribution network site as an example to explain the three-dimensional reconstruction process and principle.

[0070] In this embodiment, a drone is used to shoot the pole towers in the 10kV distribution network site from multiple angles to obtain multi-visual images, so as to generate a target drone aerial image sequence. Among them, the multi-angle aerial photography by drone can fully capture all angles and details of the pole towers in the 10kV distribution network site, so as to avoid dead angles and blind spots that may exist in traditional inspections.

[0071] Step S20: extracting key points of each image in the target UAV aerial image sequence based on a key point detection algorithm.

[0072] Exemplarily, in this embodiment, for each image in the target drone aerial image sequence, key points will be extracted by a key point detection algorithm to extract the key points of each image. It should be noted that the key point detection algorithm can be a gradient-based key point detection algorithm (such as Moravec operator, Harris operator, etc.), or a deep learning-based key point detection algorithm (such as Regression method, Heatmap method, etc.), or other key point detection algorithms (such as scale-invariant feature transformation algorithm, etc.). Which key point detection algorithm is used to achieve key point extraction of the image can be determined according to actual needs and is not limited here; in addition, how to achieve key point extraction in an image by a key point detection algorithm is common knowledge in the art, and for the sake of brevity of description, it will not be repeated here.

[0073] Step S30: Based on the extracted key points, cross-image matching is performed by measuring the distance and the context information of the key points to determine the corresponding points of the same object in different images.

[0074] Exemplarily, in this embodiment, the key points are matched across images by combining the distance measurement and the context information of the key points, so as to accurately find the corresponding points of the same object in different images, and provide accurate data support for subsequent three-dimensional reconstruction. It should be noted that, when performing distance measurement, at least one of the Euclidean distance, Manhattan distance, etc. can be preferably used as a distance measurement indicator, and the calculation methods and principles of the Euclidean distance, Manhattan distance, etc. are common knowledge in the field, and for the sake of brevity of description, they will not be repeated here; in addition, in order to improve the accuracy of the similarity measurement, the context information of the key points is introduced in this embodiment, and the context information of the key points refers to the image area around the key points.

[0075] Step S40: Based on the matched point pairs and preset camera parameters, a sparse point cloud is generated using a three-dimensional reconstruction algorithm.

[0076] Exemplarily, in this embodiment, for all the point pairs that have been successfully matched, the camera parameters and each point pair can be 3D reconstructed based on the 3D reconstruction algorithm to obtain a sparse point cloud. It should be noted that the 3D reconstruction algorithm can be a structure from motion SfM algorithm, a multi-view stereo vision MVS algorithm, or other deep learning algorithms, which can be determined according to actual needs and are not limited here. The method and principle of generating a sparse point cloud by a 3D reconstruction algorithm are common knowledge in the art and are not limited here for the sake of brevity of description.

[0077] In addition, the camera parameters include camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters include parameters such as focal length and principal point offset, while the camera extrinsic parameters include parameters such as rotation matrix and pose estimation. It can be understood that the camera intrinsic parameters mainly affect the distance relationship between the pixels in the image and the actual ground and the accurate position of the pixels, which in turn affects the accuracy of the generation of sparse point clouds; the camera extrinsic parameters describe the relative relationship between the camera coordinate system and the world coordinate system. By matching the feature points in different images and estimating the camera's extrinsic matrix, the three-dimensional coordinate relationship between the feature points can be calculated to generate a sparse point cloud.

[0078] Step S50: Generate a dense point cloud based on the sparse point cloud and the depth information of the image to achieve three-dimensional reconstruction of the target distribution network site.

[0079] Exemplarily, in this embodiment, the depth information of the image includes but is not limited to image depth, pixel depth and bit depth; the depth of the sparse point cloud is completed through the depth information of the image, that is, the missing depth information in the sparse point cloud is filled based on the depth information of the image to obtain complete dense point cloud data, thereby providing rich three-dimensional information, thereby achieving high-quality three-dimensional reconstruction of the 10kV distribution network site.

[0080] In summary, the multi-view images obtained by drone aerial photography in this embodiment can fully capture all angles and details of the pole towers in the distribution network site to avoid dead angles and blind spots that may exist in traditional inspections; at the same time, cross-image matching is performed by combining the distance measurement and the contextual information of key points to accurately find the corresponding points of the same object in different images, providing accurate data support for subsequent three-dimensional reconstruction; then the three-dimensional reconstruction algorithm is used to generate sparse point clouds and dense point clouds, which can provide rich three-dimensional information to build a high-precision three-dimensional model, so as to more accurately reflect the actual situation of the distribution network site and provide strong support for subsequent fault analysis, maintenance decisions, etc. It can be seen that the present application can effectively capture the details and features in complex scenes to achieve high-quality three-dimensional reconstruction.

[0081] In one embodiment, before step S10, that is, before the step of acquiring a target drone aerial image sequence, the method further includes:

[0082] Obtain an initial UAV aerial image sequence;

[0083] Performing a multi-scale noise evaluation on each image in the initial UAV aerial image sequence, and determining a noise level of each image based on the obtained evaluation results;

[0084] Each image is denoised based on the determined noise level to form a target UAV aerial image sequence.

[0085] Exemplarily, it is understandable that the performance and quality of aerial photography equipment such as drones may cause noise in the image, thereby affecting the matching accuracy of image feature points, etc. Therefore, in order to obtain better quality images as the basic images for three-dimensional modeling, after the drone is used to shoot the poles and towers in the 10kV distribution network site from multiple angles to obtain the initial drone aerial image sequence, each image in the initial drone aerial image sequence is traversed, and a multi-scale noise evaluation is performed on each image during the traversal process to obtain the evaluation results of each image respectively; then, for each image, the noise level of the image is determined based on the corresponding evaluation result, and the image is denoised according to the determined noise level; after all images are denoised, the target drone aerial image sequence can be generated.

[0086] In one embodiment, the multi-scale noise evaluation of each image can be implemented by the following formula:

[0087]

[0088] n s =(w1×LN+w2 / (1+SNR));

[0089] Where, noise avg represents the multi-scale noise evaluation result; S represents the total number of scales considered; w1, w2 and w s Represents the weight factor, and the specific values ​​of the three can be determined according to actual needs; n s It represents the comprehensive noise value of the sth scale; LN represents the noise level; SNR represents the signal-to-noise ratio.

[0090] It should be noted that the scale in this embodiment includes but is not limited to half, one quarter, one eighth, etc. of the original image size; the signal-to-noise ratio SNR is the ratio of signal power to noise power, and the weight factor w sIt is used to adjust the relative importance of the sth scale to the image noise assessment. The weight factors w1 and w2 are respectively used to adjust the relative importance of the noise level LN and the signal-to-noise ratio SNR in the comprehensive assessment. Their values ​​can be determined based on statistical processing, for example, the principal component analysis method can be used to help identify patterns and relationships in the data, thereby determining the value of the weight factor.

[0091] In one embodiment, before step S10, that is, before the step of acquiring a target drone aerial image sequence, the method further includes:

[0092] Obtain an initial UAV aerial image sequence;

[0093] Perform edge detection processing on each image in the initial UAV aerial image sequence based on the edge detection algorithm;

[0094] When it is determined according to the edge detection processing result that the image has an edge blur problem, the image is subjected to image edge enhancement processing to form a target UAV aerial image sequence.

[0095] Exemplarily, in this embodiment, in addition to performing image denoising on the image, it is also possible to consider using an edge detection algorithm to perform edge detection processing on the image, and based on the obtained processing results, when it is determined that the current image has edge blur, enhance the image edge to improve the image quality.

[0096] Specifically, after using a drone to take multi-angle photos of the pole towers in the 10kV distribution network site to obtain an initial drone aerial image sequence, each image in the initial drone aerial image sequence is traversed, and during the traversal process, edge detection processing is performed on each image using an edge detection algorithm; for each detected edge area, the width of each edge area can be calculated, and when it is determined that the width of the edge area is greater than a preset width threshold, it is considered that the current image has blurred edges, and the image edge enhancement processing is performed to form a target drone aerial image sequence.

[0097] It should be noted that the edge detection algorithm can be a gradient-based edge detection algorithm (such as Sobel operator, Canny operator, etc.), or an edge model-based algorithm (such as Hough transform, etc.), or other edge detection algorithms (such as Roberts operator, etc.), which can be determined according to actual needs and are not limited here; in addition, the method and principle of edge detection of an image by an edge detection algorithm are common knowledge in the art, and for the sake of brevity of description, they will not be repeated here. In addition, the specific value setting of the width threshold can be determined according to actual needs and is not limited here.

[0098] In one embodiment, after the step of performing edge detection processing on each image in the initial drone aerial image sequence based on the edge detection algorithm, the method further includes:

[0099] When it is determined according to the edge detection processing result that there are multiple adjacent edge regions in the image, the overlap between the multiple adjacent edge regions is calculated;

[0100] If the overlap degree is greater than a preset overlap degree threshold, a plurality of adjacent edge regions are merged.

[0101] Exemplarily, in this embodiment, in order to further improve the image quality, while achieving image enhancement, the overlap between multiple adjacent edge regions detected in the image (adjacent edge regions can be considered as regions close to each other in the image) can be calculated, and when it is determined that the overlap is greater than a preset overlap threshold, these regions are merged so that the connected edge regions are visually closer to a complete object to avoid edge breaks.

[0102] It should be noted that when preprocessing an image, one of the image denoising, image edge enhancement and image merging can be selected as a preprocessing method, or the three can be combined to preprocess the image, which can be determined according to actual needs and is not limited here. In addition, the specific value setting of the overlap threshold can be determined according to actual needs and is not limited here.

[0103] In one embodiment, in step S20, extracting key points of each image in the target drone aerial image sequence based on a key point detection algorithm includes:

[0104] For each image in the target UAV aerial image sequence, a multi-scale image pyramid is constructed through Gaussian filtering and downsampling;

[0105] Key point detection is performed on each scale image in the multi-scale image pyramid to obtain a series of key points and their corresponding feature descriptors by analyzing the image features of each pixel in the image and its area.

[0106] Exemplarily, in this embodiment, key point extraction is performed for each image in the target drone aerial image sequence. Specifically, a multi-scale image pyramid is first constructed by Gaussian filtering and downsampling: each image is used as the initial image, and the initial image is used as the bottom layer of the pyramid; and the original image is Gaussian filtered to reduce the high-frequency noise in the image; then the filtered image is downsampled, specifically reducing the size of the image, and retaining the main structure of the image while reducing the image resolution; after repeating the above Gaussian filtering and downsampling steps for multiple times, different scale images corresponding to each image can be generated, wherein each scale image is a layer of the pyramid. Then, based on each scale image, key point detection is performed separately to obtain a series of key points and related feature descriptors by analyzing the image features of each pixel in the image and the area where it is located; wherein the feature descriptor is used to describe the image characteristics of the area where the key point is located.

[0107] In one embodiment, in step S30, the cross-image matching is performed based on the extracted key points by measuring the distance and the context information of the key points to determine the corresponding points of the same object in different images, including:

[0108] In the cross-image matching process, all key points in different images are matched pairwise;

[0109] For each pair of matching points, the similarity between the feature descriptors is determined based on the distance measurement and the context information of the key points, and when the similarity is determined to be greater than a preset similarity threshold, the two currently matched key points are regarded as corresponding points of the same object in different images.

[0110] Exemplarily, in this embodiment, in the cross-image matching process, all key points in different images are matched in pairs; it should be noted that the key points between the same-scale images corresponding to different images are matched in pairs. For example, assuming that the number of pyramid layers is 5, the scale images include the 1st scale image to the 5th scale image. Then, for image A and image B, the key points in the 1st scale image corresponding to image A are matched in pairs with the key points in the 1st scale image corresponding to image B.

[0111] For each pair of matching points, the similarity between the feature descriptors will be determined based on the distance metric and the contextual information of the key points, and when it is determined that the similarity is greater than the preset similarity threshold, the two key points will be regarded as corresponding points of the same object in different images. Specifically, the distance between the two key point descriptors can be calculated first to obtain a preliminary similarity x; then, for each key point, multiple image blocks around it are extracted, and the similarity y between these image blocks is calculated; finally, the similarity x is fused with the similarity y to obtain the final similarity z. Subsequently, this similarity z will be compared with the preset similarity threshold to determine whether the two currently matched key points are corresponding points of the same object in different images.

[0112] It can be understood that, when determining corresponding points, this embodiment will compare the key point descriptors in one image with all the key point descriptors in another image, and calculate the similarity between them; in addition, before performing similarity matching, it is also necessary to set a similarity threshold, which is used to determine which key point pairs are matched, that is, when it is determined that the similarity between two key point descriptors is greater than the threshold, they will be regarded as matching key point pairs.

[0113] In a second aspect, an embodiment of the present application also provides a distribution network site three-dimensional reconstruction system.

[0114] In one embodiment, referring to Figure 2 , Figure 2 This is a functional module diagram of an embodiment of the network distribution site 3D reconstruction system of this application. Figure 2 As shown, the distribution network site 3D reconstruction system includes:

[0115] A data acquisition module, which is used to acquire a target drone aerial image sequence, wherein the target drone aerial image sequence includes multi-view images obtained by shooting a pole tower in a target distribution network site at multiple angles;

[0116] The feature matching module is used to extract the key points of each image in the target drone aerial image sequence based on the key point detection algorithm; based on the extracted key points, cross-image matching is performed through distance measurement and the context information of the key points to determine the corresponding points of the same object in different images;

[0117] The 3D reconstruction module is used to generate a sparse point cloud based on the matched point pairs and preset camera parameters using a 3D reconstruction algorithm; and to generate a dense point cloud based on the sparse point cloud and the depth information of the image to achieve 3D reconstruction of the target distribution network site.

[0118] Furthermore, in one embodiment, the data acquisition module is also used for:

[0119] Obtain an initial UAV aerial image sequence;

[0120] Performing a multi-scale noise evaluation on each image in the initial UAV aerial image sequence, and determining a noise level of each image based on the obtained evaluation results;

[0121] Each image is denoised based on the determined noise level to form a target UAV aerial image sequence.

[0122] Furthermore, in one embodiment, the formula for multi-scale noise evaluation is:

[0123]

[0124] n s =(w1×LN+w2 / (1+SNR));

[0125] Where, noise avg represents the multi-scale noise evaluation result, S represents the total number of scales considered, w1, w2 and w s represents the weight factor, n s It represents the comprehensive noise value of the sth scale, LN represents the noise level, and SNR represents the signal-to-noise ratio.

[0126] Furthermore, in one embodiment, the data acquisition module is also used for:

[0127] Obtain an initial UAV aerial image sequence;

[0128] Perform edge detection processing on each image in the initial UAV aerial image sequence based on the edge detection algorithm;

[0129] When it is determined according to the edge detection processing result that the image has an edge blur problem, the image is subjected to image edge enhancement processing to form a target UAV aerial image sequence.

[0130] Furthermore, in one embodiment, the data acquisition module is specifically used for:

[0131] When it is determined according to the edge detection processing result that there are multiple adjacent edge regions in the image, the overlap between the multiple adjacent edge regions is calculated;

[0132] If the overlap degree is greater than a preset overlap degree threshold, a plurality of adjacent edge regions are merged.

[0133] Furthermore, in one embodiment, the feature matching module is specifically used for:

[0134] For each image in the target UAV aerial image sequence, a multi-scale image pyramid is constructed through Gaussian filtering and downsampling;

[0135] Key point detection is performed on each scale image in the multi-scale image pyramid to obtain a series of key points and their corresponding feature descriptors by analyzing the image features of each pixel in the image and its area.

[0136] Furthermore, in one embodiment, the feature matching module is further configured to:

[0137] In the cross-image matching process, all key points in different images are matched pairwise;

[0138] For each pair of matching points, the similarity between the feature descriptors is determined based on the distance measurement and the context information of the key points, and when the similarity is determined to be greater than a preset similarity threshold, the two currently matched key points are regarded as corresponding points of the same object in different images.

[0139] Among them, the functional implementation of each module in the above-mentioned distribution network site 3D reconstruction system corresponds to the steps in the above-mentioned distribution network site 3D reconstruction method embodiment, and its functions and implementation processes are no longer repeated here.

[0140] In a third aspect, an embodiment of the present application provides a three-dimensional reconstruction device for a distribution network site. The three-dimensional reconstruction device for a distribution network site may be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0141] Reference Figure 3 , Figure 3 The hardware structure diagram of the distribution network site 3D reconstruction device involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the distribution network site 3D reconstruction device may include a processor, a memory, a communication interface and a communication bus.

[0142] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.

[0143] The communication interface includes input / output (I / O) interface, physical interface and logical interface, etc., which are used to realize the interconnection of devices inside the distribution network site 3D reconstruction device, and the interface used to realize the interconnection between the distribution network site 3D reconstruction device and other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.; the user device can be a display (Display), a keyboard (Keyboard), etc.

[0144] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0145] The processor may be a general-purpose processor, and the general-purpose processor may call the distribution network site 3D reconstruction program stored in the memory, and execute the distribution network site 3D reconstruction method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the distribution network site 3D reconstruction program is called may refer to the various embodiments of the distribution network site 3D reconstruction method of the present application, and will not be repeated here.

[0146] Those skilled in the art will understand that Figure 3 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0147] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0148] The readable storage medium of the present application stores a three-dimensional reconstruction program for a distribution network site, wherein when the three-dimensional reconstruction program for a distribution network site is executed by a processor, the steps of the three-dimensional reconstruction method for a distribution network site as described above are implemented.

[0149] Among them, the method implemented when the distribution network site three-dimensional reconstruction program is executed can refer to the various embodiments of the distribution network site three-dimensional reconstruction method of the present application, and will not be repeated here.

[0150] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0151] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0152] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0153] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0154] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0155] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0156] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for three-dimensional reconstruction of a distribution network site, characterized in that: The following steps are involved: Acquire a target UAV aerial image sequence, wherein the target UAV aerial image sequence includes multi-view images obtained by shooting a pole tower in a target distribution network site at multiple angles; Extract the key points of each image in the target UAV aerial image sequence based on the key point detection algorithm; Based on the extracted key points, cross-image matching is performed through distance measurement and the context information of the key points to determine the corresponding points of the same object in different images; Based on the matched point pairs and preset camera parameters, a sparse point cloud is generated using a 3D reconstruction algorithm; A dense point cloud is generated based on the sparse point cloud and the depth information of the image to achieve three-dimensional reconstruction of the target distribution network site.

2. The method for three-dimensional reconstruction of a distribution network site according to claim 1, characterized in that: Before the step of obtaining the target drone aerial image sequence, the method further includes: Obtain an initial UAV aerial image sequence; Performing a multi-scale noise evaluation on each image in the initial UAV aerial image sequence, and determining a noise level of each image based on the obtained evaluation results; Each image is denoised based on the determined noise level to form a target UAV aerial image sequence.

3. The method for three-dimensional reconstruction of a distribution network site according to claim 2, characterized in that: The formula for multi-scale noise evaluation is: n s =(w1×LN+w2 / (1+SNR)); Where noise avg represents the multi-scale noise evaluation result, S represents the total number of scales considered, w1, w2 and w s represents the weight factor, n s It represents the comprehensive noise value of the sth scale, LN represents the noise level, and SNR represents the signal-to-noise ratio.

4. The method for three-dimensional reconstruction of a distribution network site according to claim 1, characterized in that: Before the step of obtaining the target drone aerial image sequence, the method further includes: Obtain an initial UAV aerial image sequence; Perform edge detection processing on each image in the initial UAV aerial image sequence based on the edge detection algorithm; When it is determined that the image has edge blur according to the edge detection processing result, the image is subjected to image edge enhancement processing to form a target UAV aerial image sequence.

5. The method for three-dimensional reconstruction of a distribution network site according to claim 4, characterized in that: After the step of performing edge detection processing on each image in the initial drone aerial image sequence based on the edge detection algorithm, the method further includes: When it is determined according to the edge detection processing result that there are multiple adjacent edge regions in the image, the overlap between the multiple adjacent edge regions is calculated; If the overlap degree is greater than a preset overlap degree threshold, a plurality of adjacent edge regions are merged.

6. The method for three-dimensional reconstruction of a distribution network site according to claim 1, characterized in that: The key point detection algorithm is used to extract the key points of each image in the target drone aerial image sequence, including: For each image in the target UAV aerial image sequence, a multi-scale image pyramid is constructed through Gaussian filtering and downsampling; Key point detection is performed on each scale image in the multi-scale image pyramid to obtain a series of key points and their corresponding feature descriptors by analyzing the image features of each pixel in the image and its area.

7. The method for three-dimensional reconstruction of a distribution network site according to claim 1, characterized in that: The method of performing cross-image matching based on the extracted key points by measuring the distance and the context information of the key points to determine the corresponding points of the same object in different images includes: In the cross-image matching process, all key points in different images are matched pairwise; For each pair of matching points, the similarity between the feature descriptors is determined based on the distance measurement and the context information of the key points, and when the similarity is determined to be greater than a preset similarity threshold, the two currently matched key points are regarded as corresponding points of the same object in different images.

8. A distribution network site 3D reconstruction system, characterized in that: include: A data acquisition module, which is used to acquire a target drone aerial image sequence, wherein the target drone aerial image sequence includes multi-view images obtained by shooting a pole tower in a target distribution network site at multiple angles; The feature matching module is used to extract the key points of each image in the target drone aerial image sequence based on the key point detection algorithm; based on the extracted key points, cross-image matching is performed through distance measurement and the context information of the key points to determine the corresponding points of the same object in different images; A 3D reconstruction module, which is used to generate a sparse point cloud using a 3D reconstruction algorithm based on the matched point pairs and preset camera parameters; A dense point cloud is generated based on the sparse point cloud and the depth information of the image to achieve three-dimensional reconstruction of the target distribution network site.

9. A distribution network site 3D reconstruction device, characterized in that: The distribution network site three-dimensional reconstruction device includes a processor, a memory, and a distribution network site three-dimensional reconstruction program stored in the memory and executable by the processor. When the distribution network site three-dimensional reconstruction program is executed by the processor, the steps of the distribution network site three-dimensional reconstruction method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a three-dimensional reconstruction program for a distribution network site, wherein when the three-dimensional reconstruction program for a distribution network site is executed by a processor, the steps of the three-dimensional reconstruction method for a distribution network site as described in any one of claims 1 to 7 are implemented.

Images