Transmission tower three-dimensional reconstruction method and device based on depth feature recovery, terminal equipment and storage medium

Abstract

The invention discloses a transmission tower three-dimensional reconstruction method and device based on depth feature recovery, terminal equipment and a storage medium. The method comprises the following steps: acquiring internal parameters of a camera and to-be-reconstructed transmission tower images of different visual angles; inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map according to the shape, texture and edge of the transmission tower; calculating an initial point cloud coordinate of the transmission tower image according to the transmission tower image and the camera internal reference; performing multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image; and according to the depth feature map and the initial point cloud coordinates, performing point cloud coordinate densification on the initial point cloud coordinates corresponding to the depth feature map to obtain a three-dimensional model of the power transmission tower. According to the invention, the three-dimensional reconstruction precision of the power transmission tower is improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, device, terminal equipment and storage medium for three-dimensional reconstruction of a transmission tower based on deep feature recovery. Background Art

[0002] With the rapid development of the power system, higher requirements are placed on the safety monitoring and maintenance of transmission towers. Traditional transmission tower detection and monitoring mainly rely on manual inspections, which are inefficient and pose safety risks. In recent years, with the continuous development of computer vision technology and 3D reconstruction technology, new approaches have been provided for the automated detection and 3D reconstruction of transmission towers.

[0003] Currently, the commonly used three-dimensional reconstruction method for transmission towers can be based on technologies such as stereo vision. However, the stereo vision method is sensitive to lighting and texture conditions. In addition, the structure of transmission towers is complex, with a large number of slender poles and complex connectors, which leads to easy matching errors in complex environments, making it difficult to restore the detailed features of transmission towers and resulting in low reconstruction accuracy. Summary of the invention

[0004] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for three-dimensional reconstruction of a transmission tower based on deep feature recovery, which can effectively solve the problem that the prior art is difficult to restore the detailed features of the transmission tower and the three-dimensional reconstruction accuracy of the transmission tower is low.

[0005] An embodiment of the present invention provides a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery, comprising:

[0006] Obtain camera internal parameters and transmission tower images to be reconstructed at different viewing angles;

[0007] Inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map according to the shape, texture and edge of the transmission tower;

[0008] Calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters;

[0009] Perform multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image;

[0010] According to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower.

[0011] Furthermore, the transmission tower image is input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and an initial feature map is generated according to the shape, texture and edge of the transmission tower, including:

[0012] Input the transmission tower image into the preset neural network model to detect the transmission tower target and generate the transmission tower area image;

[0013] Performing convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower;

[0014] An initial feature map is generated based on the shape, texture and edge of the transmission tower.

[0015] Further, according to the transmission tower image and the camera internal parameters, the initial point cloud coordinates of the transmission tower image are calculated, including:

[0016] Perform convolution based on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points;

[0017] Find the local extreme value points in the preset Gaussian difference scale space according to a number of pixel points as the feature points of the transmission tower image;

[0018] Calculate the similarity measure between the feature points according to the feature points of the transmission tower image, filter the feature points corresponding to the similarity measure less than the preset similarity threshold, and obtain the feature point matching pairs;

[0019] Constructing a projection matrix of a transmission tower image according to the camera intrinsic parameters;

[0020] The initial point cloud coordinates are calculated based on the feature point matching pairs and the projection matrix.

[0021] Further, performing multi-layer convolution according to the initial feature map to generate a deep feature map of the transmission tower image includes:

[0022] Initializing and converting the initial feature map to obtain a multi-scale depth feature map;

[0023] The multi-scale depth feature map is subjected to multi-layer convolution to generate a depth feature map of the convolved transmission tower image.

[0024] Further, according to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of a transmission tower, including:

[0025] According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters;

[0026] According to the optimal registration parameters, determine the corresponding relationship between the depth feature map and the initial point cloud coordinates;

[0027] According to the corresponding relationship between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates;

[0028] Reconstruction is performed according to the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

[0029] As an improvement of the above solution, another embodiment of the present invention provides a three-dimensional reconstruction device for a transmission tower based on deep feature recovery, comprising:

[0030] A data acquisition module, used to obtain camera internal parameters and transmission tower images to be reconstructed at different viewing angles;

[0031] An initial feature map generation module is used to input the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generate an initial feature map according to the shape, texture and edge of the transmission tower;

[0032] An initial point cloud coordinate calculation module, used to calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters;

[0033] A depth feature map generation module, used for performing multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image;

[0034] The tower model construction module is used to densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

[0035] Furthermore, the initial feature map generation module includes:

[0036] A tower area generating unit, used for inputting the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image;

[0037] A tower feature extraction unit, used to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower;

[0038] The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

[0039] Furthermore, the initial point cloud coordinate calculation module includes:

[0040] A tower image convolution unit, used for performing convolution according to the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points;

[0041] A tower feature point determination unit is used to find local extreme value points in a preset Gaussian difference scale space according to a number of pixel points as feature points of the transmission tower image;

[0042] A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs;

[0043] A projection matrix construction unit, used to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters;

[0044] The point cloud coordinate calculation unit is used to calculate the initial point cloud coordinates according to the feature point matching pair and the projection matrix.

[0045] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a three-dimensional reconstruction method of a transmission tower based on deep feature recovery as described in the above embodiment.

[0046] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method of a transmission tower based on deep feature recovery described in the above embodiment.

[0047] By implementing the present invention, at least the following beneficial effects are achieved:

[0048] The present invention provides a method, device, terminal equipment and storage medium for three-dimensional reconstruction of a transmission tower based on deep feature recovery. The method can obtain camera intrinsic parameters and images of transmission towers to be reconstructed at different viewing angles; input the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generate an initial feature map according to the shape, texture and edge of the transmission tower; calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera intrinsic parameters; perform multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image; and perform point cloud coordinate densification on the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower. By obtaining the camera's intrinsic parameters and transmission tower images from different perspectives, and combining the preset neural network to extract shape, texture and edge information, an initial feature map is generated, which provides high-precision basic data for subsequent 3D reconstruction. The initial feature maps from different perspectives can obtain depth feature maps from different perspectives. In scenes that are sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the structure of the transmission tower is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, so that the detailed features of the transmission tower can be restored and the accuracy of 3D reconstruction of the transmission tower can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention;

[0050] Figure 2 It is a structural schematic diagram of a transmission tower three-dimensional reconstruction device based on deep feature recovery provided by an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of generating an initial feature map of a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of generating a depth feature map of a method for three-dimensional reconstruction of a transmission tower based on depth feature recovery provided by an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of point cloud coordinate densification of a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] See also Figure 1 , is a flow chart of a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention, comprising:

[0056] S1, obtaining camera internal parameters and transmission tower images to be reconstructed at different viewing angles;

[0057] S2, inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map according to the shape, texture and edge of the transmission tower;

[0058] S3. Calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters;

[0059] S4, performing multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image;

[0060] S5. Densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

[0061] Specifically, the camera intrinsic parameters include: focal length parameters and center point parameters. The camera intrinsic parameters are obtained from the transmission tower image file. Generally, the camera projection matrix of the images collected by the same acquisition device is the same; the transmission tower image to be reconstructed is an image from different perspectives; the preset neural network model includes: target detection network and convolutional neural network. The target detection network is such as the YOLOV8 target detection network, and the convolutional neural network is such as the ResNet152 neural network.

[0062] The training process of the YOLOV8 target detection network includes: 1) Data labeling: Use the target detection frame to label the tower area in the multi-view transmission tower training image. During the labeling process, pay attention to the fact that the target detection frame should fit the tower as closely as possible and contain as few background pixels as possible; 2) Network training: Use the labeled multi-view transmission tower training images to train the YOLOV8 network, monitor the loss changes during the training process, so that the trained target detection network can detect the tower area in the input image. The ResNet152 neural network uses a network with pre-trained weights that has been pre-trained on a general image dataset. It does not need to be trained again and is only used for feature extraction of the tower area image.

[0063] In a preferred embodiment of the present invention, the camera internal parameters and the transmission tower images to be reconstructed at different viewing angles are first obtained; then the transmission tower images are input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and an initial feature map is generated according to the shape, texture and edge of the transmission tower. For example, Figure 3 As shown in the figure, the input transmission tower image to be reconstructed first passes through the YOLOV8 target detection network, and after the transmission tower target detection, the tower area image is generated, and the background redundant area is removed; then the generated tower area image is input into the pre-trained ResNet152 neural network, and after multi-layer convolution and pooling operations, the initial feature maps at different levels are obtained. These initial feature maps contain the shape, texture, edge and other information of the transmission tower. The resolution of the final generated initial feature map is 1 / 4 of the tower area image resolution, and the number of feature map channels is 128. Then, according to the transmission tower image and the camera internal parameters, the initial point cloud coordinates of the transmission tower image are calculated. Then, according to the initial feature map, multi-layer convolution is performed to generate the depth feature map of the transmission tower image. The initial point cloud coordinates are sparse point cloud coordinates, which can only roughly describe the shape and position of the object and lack detailed information. The initial feature map guides the initial point cloud coordinates, and according to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower.

[0064] Preferably, the transmission tower image is input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and an initial feature map is generated according to the shape, texture and edge of the transmission tower, including:

[0065] Input the transmission tower image into the preset neural network model to detect the transmission tower target and generate the transmission tower area image;

[0066] Performing convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower;

[0067] An initial feature map is generated based on the shape, texture and edge of the transmission tower.

[0068] Specifically, the initial point cloud coordinates of the transmission tower image are calculated according to the transmission tower image and the camera internal parameters, including:

[0069] Perform convolution based on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points;

[0070] Find the local extreme value points in the preset Gaussian difference scale space according to a number of pixel points as the feature points of the transmission tower image;

[0071] Calculate the similarity measure between the feature points according to the feature points of the transmission tower image, filter the feature points corresponding to the similarity measure less than the preset similarity threshold, and obtain the feature point matching pairs;

[0072] Constructing a projection matrix of a transmission tower image according to the camera intrinsic parameters;

[0073] The initial point cloud coordinates are calculated based on the feature point matching pairs and the projection matrix.

[0074] In a preferred embodiment of the present invention, convolution is first performed based on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points; then, local extreme points in a preset Gaussian difference scale space are found based on the number of pixel points as feature points of the transmission tower image; then, a similarity measure between feature points is calculated based on the feature points of the transmission tower image, and feature points corresponding to similarity measures less than a preset similarity threshold are filtered to obtain feature point matching pairs. For example, the SIFT feature point detection and matching algorithm is used to extract and match feature points from transmission tower images of different perspectives to obtain feature point matching pairs. SIFT features are usually extracted using a scale-invariant feature transform (Scale-Invariant Feature Transform). In feature point detection, the Difference of Gaussian (DoG) in the scale space is used to find extreme points:

[0075] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)

[0076] Where G(x,y,σ) is a Gaussian function, I(x,y) is the input transmission tower image, and k is It is used to determine the scale ratio relationship between two adjacent Gaussian kernels in the scale space. By using Gaussian kernels of different scales for smoothing operations and then calculating the differences between them, the image features of transmission towers at different scales, such as edges and corners, can be detected. Different scales are reflected by k, so that the Gaussian function has different filtering degrees on the image. The transmission tower images after different degrees of Gaussian filtering will show different characteristics. The value of σ is 5 to ensure a stronger smoothing effect.

[0077] The projection matrix P of the transmission tower image is constructed according to the camera intrinsic parameters:

[0078]

[0079] Among them, f x 、f y is the focal length parameter in the camera intrinsic parameter, c x 、c y It is the center point parameter in the camera intrinsic parameter.

[0080] The point cloud data is recovered from the feature point matching pairs through triangulation. Assuming that the feature point matching pairs in the two views are p1 = (x1, y1) and p2 = (x2, y2), and the corresponding camera projection matrices are P, the formula for triangulating the initial point cloud coordinates X is:

[0081]

[0082] The subscripts represent the rows and columns of P, for example, P 23 is the value of the 2nd row and 3rd column of the P matrix. By solving the above linear equations, the initialization point cloud coordinate X can be calculated.

[0083] Schematically, performing multi-layer convolution according to the initial feature map to generate a deep feature map of the transmission tower image includes:

[0084] Initializing and converting the initial feature map to obtain a multi-scale depth feature map;

[0085] The multi-scale depth feature map is subjected to multi-layer convolution to generate a depth feature map of the convolved transmission tower image.

[0086] In a preferred embodiment of the present invention, a tower depth estimation module is constructed, such as Figure 4 As shown, it mainly includes the initialization of the depth feature map and the construction of four 3*3 convolutional layers. First, the initial feature map is initialized and converted to obtain a multi-scale depth feature map; then the multi-scale depth feature map is multi-layered convolutionally constructed to generate a depth feature map of the transmission tower image after convolution.

[0087] Specifically, according to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of a transmission tower, including:

[0088] According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters;

[0089] According to the optimal registration parameters, determine the corresponding relationship between the depth feature map and the initial point cloud coordinates;

[0090] According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates;

[0091] Reconstruction is performed according to the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

[0092] In a preferred embodiment of the present invention, through the correspondence between the depth feature map and the initial point cloud coordinate X, such as Figure 5 As shown, without changing the sparse initial point cloud coordinates X, the initial point cloud coordinates are densified to obtain dense point cloud coordinates Y, and then reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

[0093] By implementing this embodiment, the camera intrinsic parameters and the transmission tower images to be reconstructed at different viewing angles are obtained; the transmission tower images are input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower images, and an initial feature map is generated according to the shape, texture and edge of the transmission tower; the initial point cloud coordinates of the transmission tower image are calculated according to the transmission tower image and the camera intrinsic parameters; multi-layer convolution is performed according to the initial feature map to generate a depth feature map of the transmission tower image; according to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower. By obtaining the camera's intrinsic parameters and transmission tower images from different perspectives, and combining the preset neural network to extract shape, texture and edge information, an initial feature map is generated, which provides high-precision basic data for subsequent 3D reconstruction. The initial feature maps from different perspectives can obtain depth feature maps from different perspectives. In scenes that are sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the structure of the transmission tower is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, so that the detailed features of the transmission tower can be restored and the accuracy of 3D reconstruction of the transmission tower can be improved.

[0094] Through deep feature extraction and guided point cloud refinement, the detailed features of the transmission tower can be restored more accurately, matching errors in complex environments can be reduced, and reconstruction accuracy can be improved; the problem that the prior art is difficult to accurately restore the detailed features of the transmission tower can be solved, and the reconstruction results can be more refined to meet the needs of practical applications; compared with traditional laser scanning and other methods, the present invention does not require expensive equipment, the operation is relatively simple, and the data processing volume is also small, which reduces the cost and difficulty of reconstruction.

[0095] See also Figure 2 , is a schematic structural diagram of a transmission tower 3D reconstruction device based on deep feature recovery provided by an embodiment of the present invention, comprising:

[0096] A data acquisition module, used to obtain camera internal parameters and transmission tower images to be reconstructed at different viewing angles;

[0097] An initial feature map generation module is used to input the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generate an initial feature map according to the shape, texture and edge of the transmission tower;

[0098] An initial point cloud coordinate calculation module, used to calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters;

[0099] A depth feature map generation module, used for performing multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image;

[0100] The tower model construction module is used to densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

[0101] Specifically, the initial feature map generation module includes:

[0102] A tower area generating unit, used for inputting the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image;

[0103] A tower feature extraction unit, used to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower;

[0104] The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

[0105] Preferably, the initial point cloud coordinate calculation module includes:

[0106] A tower image convolution unit, used for performing convolution according to the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points;

[0107] A tower feature point determination unit is used to find local extreme value points in a preset Gaussian difference scale space according to a number of pixel points as feature points of the transmission tower image;

[0108] A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs;

[0109] A projection matrix construction unit, used to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters;

[0110] The point cloud coordinate calculation unit is used to calculate the initial point cloud coordinates according to the feature point matching pair and the projection matrix.

[0111] The present invention provides a three-dimensional reconstruction device for a transmission tower based on deep feature recovery. The device obtains camera intrinsic parameters and transmission tower images to be reconstructed at different viewing angles according to a data acquisition module. According to an initial feature map generation module, the transmission tower image is input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and an initial feature map is generated according to the shape, texture and edge of the transmission tower. In an initial point cloud coordinate calculation module, the initial point cloud coordinates of the transmission tower image are calculated according to the transmission tower image and the camera intrinsic parameters. In a depth feature map generation module, multi-layer convolution is performed according to the initial feature map to generate a depth feature map of the transmission tower image. Finally, through a tower model construction module, the initial point cloud coordinates corresponding to the depth feature map are densified according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower. By obtaining the camera's intrinsic parameters and transmission tower images from different perspectives, and combining the preset neural network to extract shape, texture and edge information, an initial feature map is generated, which provides high-precision basic data for subsequent 3D reconstruction. The initial feature maps from different perspectives can obtain depth feature maps from different perspectives. In scenes that are sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the structure of the transmission tower is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, so that the detailed features of the transmission tower can be restored and the accuracy of 3D reconstruction of the transmission tower can be improved.

[0112] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0113] Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] Another embodiment of the present invention further provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery as described in the above embodiment is implemented. The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0115] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0116] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0117] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method of a transmission tower based on deep feature recovery described in the above embodiment.

[0118] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0119] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A 3D reconstruction method for transmission towers based on deep feature recovery, characterized in that: include: Obtain camera internal parameters and transmission tower images to be reconstructed at different viewing angles; Inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map according to the shape, texture and edge of the transmission tower; Calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters; Perform multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image; According to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower.

2. The method for three-dimensional reconstruction of a transmission tower based on deep feature recovery according to claim 1, characterized in that: Inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map according to the shape, texture and edge of the transmission tower, including: Input the transmission tower image into the preset neural network model to detect the transmission tower target and generate the transmission tower area image; Performing convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower; An initial feature map is generated based on the shape, texture and edge of the transmission tower.

3. The method for three-dimensional reconstruction of a transmission tower based on deep feature recovery according to claim 1, characterized in that: The initial point cloud coordinates of the transmission tower image are calculated according to the transmission tower image and the camera internal parameters, including: Perform convolution based on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points; Find the local extreme value points in the preset Gaussian difference scale space according to a number of pixel points as the feature points of the transmission tower image; Calculate the similarity measure between the feature points according to the feature points of the transmission tower image, filter the feature points corresponding to the similarity measure less than the preset similarity threshold, and obtain the feature point matching pairs; Constructing a projection matrix of a transmission tower image according to the camera intrinsic parameters; The initial point cloud coordinates are calculated based on the feature point matching pairs and the projection matrix.

4. The method for three-dimensional reconstruction of a transmission tower based on deep feature recovery according to claim 1, characterized in that: Performing multi-layer convolution according to the initial feature map to generate a deep feature map of the transmission tower image, including: Initializing and transforming the initial feature map to obtain a multi-scale depth feature map; The multi-scale depth feature map is subjected to multi-layer convolution to generate a depth feature map of the convolved transmission tower image.

5. The method for three-dimensional reconstruction of a transmission tower based on deep feature recovery according to claim 1, characterized in that: According to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of a transmission tower, including: According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters; According to the optimal registration parameters, determine the corresponding relationship between the depth feature map and the initial point cloud coordinates; According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates; Reconstruction is performed according to the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

6. A 3D reconstruction device for transmission towers based on deep feature recovery, characterized in that: include: A data acquisition module, used to obtain camera internal parameters and transmission tower images to be reconstructed at different viewing angles; An initial feature map generation module is used to input the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generate an initial feature map according to the shape, texture and edge of the transmission tower; An initial point cloud coordinate calculation module, used to calculate the initial point cloud coordinates of the transmission tower image according to the transmission tower image and the camera internal parameters; A depth feature map generation module, used for performing multi-layer convolution according to the initial feature map to generate a depth feature map of the transmission tower image; The tower model construction module is used to densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

7. The device for 3D reconstruction of a transmission tower based on deep feature recovery according to claim 6, characterized in that: The initial feature map generation module comprises: A tower area generation unit is used to input the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image; A tower feature extraction unit, used to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture and edge of the transmission tower; The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

8. The device for 3D reconstruction of a transmission tower based on deep feature recovery according to claim 6, characterized in that: The initial point cloud coordinate calculation module includes: A tower image convolution unit, used for performing convolution according to the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points; A tower feature point determination unit is used to find local extreme value points in a preset Gaussian difference scale space according to a number of pixel points as feature points of the transmission tower image; A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs; A projection matrix construction unit, used to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters; The point cloud coordinate calculation unit is used to calculate the initial point cloud coordinates according to the feature point matching pair and the projection matrix.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a three-dimensional reconstruction method of a transmission tower based on deep feature recovery as described in any one of claims 1 to 5.

Images