Power transmission tower three-dimensional reconstruction method and device based on depth feature recovery, terminal equipment and storage medium

Abstract

The application discloses a power transmission tower three-dimensional reconstruction method and device based on deep feature recovery, a terminal equipment and a storage medium, and the method comprises the following steps: acquiring camera internal parameters and a power transmission tower image to be reconstructed at different angles; inputting the power transmission tower image into a preset neural network model to extract the shape, texture and edge of the power transmission tower in the power transmission tower image, and generating an initial feature map according to the shape, texture and edge of the power transmission tower; calculating the initial point cloud coordinates of the power transmission tower image according to the power transmission tower image and the camera internal parameters; performing multi-layer convolution according to the initial feature map to generate a deep feature map of the power transmission tower image; and performing point cloud coordinate densification on the initial point cloud coordinates corresponding to the deep feature map according to the deep feature map and the initial point cloud coordinates to obtain a power transmission tower three-dimensional model. Through the implementation of the application, the three-dimensional reconstruction accuracy of the power transmission tower is improved.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, device, terminal equipment and storage medium for three-dimensional reconstruction of a transmission tower based on depth feature recovery. Background Art

[0002] The rapid development of power systems has placed higher demands on the safety monitoring and maintenance of transmission towers. Traditional transmission tower inspection and monitoring relies primarily on manual inspections, which is inefficient and poses safety risks. In recent years, the continuous development of computer vision and 3D reconstruction technologies has provided new approaches for automated inspection and 3D reconstruction of transmission towers.

[0003] Currently, commonly used methods for 3D reconstruction of transmission towers can be based on technologies such as stereo vision. However, stereo vision methods are sensitive to lighting and texture conditions. In addition, transmission towers have complex structures, with a large number of slender poles and complex connectors. This leads to easy matching errors in complex environments, making it difficult to restore the detailed features of the transmission towers and resulting in low reconstruction accuracy. Summary of the Invention

[0004] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for three-dimensional reconstruction of transmission towers based on deep feature recovery, which can effectively solve the problem that the existing technology is difficult to restore the detailed features of transmission towers and the three-dimensional reconstruction accuracy of transmission towers is low.

[0005] An embodiment of the present invention provides a method for three-dimensional reconstruction of a transmission tower based on depth feature recovery, comprising:

[0006] Obtain camera intrinsic parameters and transmission tower images to be reconstructed from different perspectives;

[0007] Inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map based on the shape, texture and edge of the transmission tower;

[0008] Calculating the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters;

[0009] Perform multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image;

[0010] According to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower.

[0011] Further, the power transmission tower image is input into a preset neural network model to extract a shape, a texture and an edge of the power transmission tower in the power transmission tower image, generate an initial feature map according to the shape, the texture and the edge of the power transmission tower, and the initial feature map comprises:

[0012] The power transmission tower image is input into a preset neural network model to perform power transmission tower target detection and generate a power transmission tower region image;

[0013] According to the power transmission tower region image, a convolution operation and a pooling operation are performed to extract a shape, a texture and an edge of the power transmission tower;

[0014] According to the shape, the texture and the edge of the power transmission tower, an initial feature map is generated.

[0015] Further, according to the power transmission tower image and the camera internal parameter, an initial point cloud coordinate of the power transmission tower image is calculated, and the initial point cloud coordinate comprises:

[0016] According to the power transmission tower image and a preset scale Gaussian kernel, a plurality of pixel points are obtained through convolution;

[0017] According to the plurality of pixel points, a local extreme point in a preset Gaussian difference scale space is found out as a feature point of the power transmission tower image;

[0018] According to the feature point of the power transmission tower image, a similarity measure between the feature points is calculated, the feature points corresponding to a similarity measure less than a preset similarity threshold are filtered, and a feature point matching pair is obtained;

[0019] According to the camera internal parameter, a projection matrix of the power transmission tower image is constructed;

[0020] According to the feature point matching pair and the projection matrix, an initial point cloud coordinate is calculated.

[0021] Further, according to the initial feature map, a multi-layer convolution is performed to generate a deep feature map of the power transmission tower image, and the deep feature map comprises:

[0022] The initial feature map is initialized and converted to obtain a multi-scale deep feature map;

[0023] The multi-scale deep feature map is constructed through multi-layer convolution to generate a deep feature map of the power transmission tower image after convolution.

[0024] Further, according to the deep feature map and the initial point cloud coordinate, the initial point cloud coordinate corresponding to the deep feature map is densified to obtain a power transmission tower three-dimensional model, and the power transmission tower three-dimensional model comprises:

[0025] According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters;

[0026] According to the optimal registration parameters, determine the correspondence between the depth feature map and the initial point cloud coordinates;

[0027] According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates;

[0028] Reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

[0029] As an improvement to the above solution, another embodiment of the present invention provides a three-dimensional reconstruction device for a transmission tower based on depth feature recovery, comprising:

[0030] A data acquisition module is used to obtain camera internal parameters and images of the transmission tower to be reconstructed from different perspectives;

[0031] an initial feature map generating module, configured to input a transmission tower image into a preset neural network model to extract the shape, texture, and edge of the transmission tower in the transmission tower image, and generate an initial feature map based on the shape, texture, and edge of the transmission tower;

[0032] An initial point cloud coordinate calculation module is used to calculate the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters;

[0033] A depth feature map generation module, configured to perform multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image;

[0034] The tower model construction module is used to densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

[0035] Furthermore, the initial feature map generation module includes:

[0036] A tower area generation unit is used to input the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image;

[0037] A tower feature extraction unit, configured to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture, and edge of the transmission tower;

[0038] The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

[0039] Furthermore, the initial point cloud coordinate calculation module includes:

[0040] a tower image convolution unit, configured to perform convolution on the transmission tower image and a Gaussian kernel of a preset scale to obtain a plurality of pixel points;

[0041] A tower feature point determination unit is used to find local extreme points in a preset Gaussian difference scale space based on a number of pixel points as feature points of the transmission tower image;

[0042] A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs;

[0043] A projection matrix construction unit, configured to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters;

[0044] The point cloud coordinate calculation unit is used to calculate the initial point cloud coordinates according to the feature point matching pairs and the projection matrix.

[0045] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a three-dimensional reconstruction method of a transmission tower based on deep feature recovery as described in the above embodiment.

[0046] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method of a transmission tower based on deep feature recovery described in the above embodiment.

[0047] By implementing the present invention, at least the following beneficial effects are achieved:

[0048] The present invention provides a method, apparatus, terminal device and storage medium for three-dimensional reconstruction of a transmission tower based on depth feature recovery. The method can obtain camera intrinsic parameters and images of a transmission tower to be reconstructed from different perspectives; input the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generate an initial feature map based on the shape, texture and edge of the transmission tower; calculate the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters; perform multi-layer convolution on the initial feature map to generate a depth feature map of the transmission tower image; and perform point cloud coordinate densification on the initial point cloud coordinates corresponding to the depth feature map based on the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower. By obtaining camera intrinsic parameters and transmission tower images from different perspectives, and combining them with a preset neural network to extract shape, texture, and edge information, an initial feature map is generated, providing high-precision basic data for subsequent 3D reconstruction. Initial feature maps from different perspectives are used to obtain depth feature maps from different perspectives. In scenarios sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the transmission tower structure is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, making it possible to restore the detailed features of the transmission tower and improve the accuracy of 3D reconstruction of the transmission tower. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of a method for 3D reconstruction of a transmission tower based on deep feature recovery provided by one embodiment of the present invention;

[0050] Figure 2 1 is a schematic structural diagram of a transmission tower 3D reconstruction device based on deep feature recovery provided by an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of generating an initial feature map of a method for 3D reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of depth feature map generation in a method for 3D reconstruction of a transmission tower based on depth feature recovery provided by one embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of point cloud coordinate densification of a method for three-dimensional reconstruction of a transmission tower based on deep feature recovery provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0055] Referring to Figure 1 is a flowchart of a power transmission tower three-dimensional reconstruction method based on deep feature recovery provided by an embodiment of the present application, comprising:

[0056] S1, obtaining camera intrinsic parameters and power transmission tower images to be reconstructed at different viewing angles;

[0057] S2, inputting the power transmission tower images into a preset neural network model to extract the shape, texture and edge of the power transmission tower in the power transmission tower images, and generating an initial feature map according to the shape, texture and edge of the power transmission tower;

[0058] S3, calculating the initial point cloud coordinates of the power transmission tower images according to the power transmission tower images and the camera intrinsic parameters;

[0059] S4, generating a deep feature map of the power transmission tower images by multi-layer convolution according to the initial feature map;

[0060] S5, performing point cloud coordinate densification on the initial point cloud coordinates corresponding to the deep feature map according to the deep feature map and the initial point cloud coordinates, to obtain a power transmission tower three-dimensional model.

[0061] Specifically, the camera intrinsic parameters include focal length parameters and center point parameters, and the camera intrinsic parameters are obtained from a power transmission tower image file. Generally, the camera projection matrix of images collected by the same collection device is the same. The power transmission tower images to be reconstructed are images at different viewing angles. The preset neural network model includes a target detection network and a convolutional neural network. The target detection network is, for example, a YOLOV8 target detection network, and the convolutional neural network is, for example, a ResNet152 neural network.

[0062] The YOLOV8 object detection network training process includes: 1) Data annotation: Using target detection boxes to annotate the tower areas in multi-view transmission tower training images. During the annotation process, ensure that the target detection boxes fit the towers as closely as possible and include as few background pixels as possible. 2) Network training: Using the annotated multi-view transmission tower training images to train the YOLOV8 network, monitoring the loss changes during the training process to ensure that the trained target detection network can detect the tower areas in the input images. The ResNet152 neural network uses a network with pre-trained weights that was pre-trained on a general image dataset and does not require further training. It is only used for feature extraction of the tower area images.

[0063] In a preferred embodiment of the present invention, the camera intrinsic parameters and the transmission tower images to be reconstructed at different viewing angles are first obtained; the transmission tower images are then input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower images, and an initial feature map is generated based on the shape, texture and edge of the transmission tower. For example, Figure 3 As shown, the input transmission tower image to be reconstructed is first passed through the YOLOV8 object detection network. After detecting the transmission tower object, an image of the tower region is generated, and redundant background areas are removed. The generated tower region image is then input into a pre-trained ResNet152 neural network. After multiple layers of convolution and pooling operations, initial feature maps at different levels are generated. These initial feature maps contain information such as the shape, texture, and edges of the transmission tower. The resulting initial feature map has a resolution of 1 / 4 that of the tower region image and 128 channels. The initial point cloud coordinates of the transmission tower image are then calculated based on the transmission tower image and the camera intrinsic parameters. Multiple layers of convolution are then performed on this initial feature map to generate a depth feature map of the transmission tower image. The initial point cloud coordinates are sparse and can only roughly describe the object shape and position, lacking detailed information. The initial feature map guides the initial point cloud coordinates, and based on the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower.

[0064] Preferably, the transmission tower image is input into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and an initial feature map is generated based on the shape, texture and edge of the transmission tower, including:

[0065] Input the transmission tower image into the preset neural network model to detect the transmission tower target and generate the transmission tower area image;

[0066] Performing convolution and pooling operations on the transmission tower area image to extract the shape, texture, and edge of the transmission tower;

[0067] Generate an initial feature map based on the shape, texture, and edge of the transmission tower.

[0068] Specifically, calculating the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters includes:

[0069] Perform convolution on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points;

[0070] Find the local extreme points in the preset Gaussian difference scale space based on a number of pixel points as the feature points of the transmission tower image;

[0071] Calculate the similarity measure between the feature points based on the feature points of the transmission tower image, filter the feature points corresponding to the similarity measure less than the preset similarity threshold, and obtain the feature point matching pairs;

[0072] Constructing a projection matrix of the transmission tower image according to the camera intrinsic parameters;

[0073] Initial point cloud coordinates are calculated based on the feature point matching pairs and the projection matrix.

[0074] In a preferred embodiment of the present invention, convolution is first performed based on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points; then, based on the number of pixel points, local extreme points in a preset Gaussian difference scale space are found as feature points of the transmission tower image; then, based on the feature points of the transmission tower image, a similarity measure between feature points is calculated, and feature points corresponding to similarity measures less than a preset similarity threshold are filtered to obtain feature point matching pairs. For example, the SIFT feature point detection and matching algorithm is used to extract and match feature points from transmission tower images of different perspectives to obtain feature point matching pairs. SIFT features are usually extracted using a scale-invariant feature transform (Scale-Invariant Feature Transform). In feature point detection, the Difference of Gaussian (DoG) in the scale space is used to find extreme points:

[0075] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)

[0076] Where G(x,y,σ) is a Gaussian function, I(x,y) is the input transmission tower image, and k is This function determines the scale ratio between two adjacent Gaussian kernels in scale space. By smoothing with Gaussian kernels of different scales and calculating their differences, it detects transmission tower image features at different scales, such as edges and corners. Different scales are represented by k, which results in different degrees of filtering applied by the Gaussian function. Transmission tower images filtered with different degrees of Gaussian filtering exhibit distinct characteristics. A value of 5 for σ ensures a stronger smoothing effect.

[0077] The projection matrix P of the transmission tower image is constructed according to the camera intrinsic parameters:

[0078]

[0079] Among them, f x 、f y is the focal length parameter in the camera intrinsic parameter, c x 、c y is the center point parameter in the camera intrinsic parameter.

[0080] The point cloud data is recovered from the feature point matching pairs through triangulation. Assuming that the feature point matching pairs in the two views are p1 = (x1, y1) and p2 = (x2, y2), and the corresponding camera projection matrices are P, the formula for triangulating the initial point cloud coordinate X is:

[0081]

[0082] The subscripts represent the rows and columns of P, for example, P 23 is the value of the 2nd row and 3rd column of the P matrix. By solving the above linear equations, the initialization point cloud coordinate X can be calculated.

[0083] Schematically, multi-layer convolution is performed based on the initial feature map to generate a depth feature map of the transmission tower image, including:

[0084] Initializing and transforming the initial feature map to obtain a multi-scale depth feature map;

[0085] The multi-scale depth feature map is subjected to multi-layer convolution to generate the depth feature map of the convolved transmission tower image.

[0086] In a preferred embodiment of the present invention, a tower depth estimation module is constructed, such as Figure 4 As shown in the figure, it mainly includes the initialization of the depth feature map and the construction of four 3*3 convolutional layers. The initial feature map is first initialized and converted to obtain a multi-scale depth feature map. The multi-scale depth feature map is then subjected to multi-layer convolution to generate a depth feature map of the transmission tower image after convolution.

[0087] Specifically, based on the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower, including:

[0088] According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters;

[0089] According to the optimal registration parameters, determine the correspondence between the depth feature map and the initial point cloud coordinates;

[0090] According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates;

[0091] Reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

[0092] In a preferred embodiment of the present invention, through the correspondence between the depth feature map and the initial point cloud coordinate X, such as Figure 5 As shown, without changing the sparse initial point cloud coordinates X, the initial point cloud coordinates are densified to obtain the dense point cloud coordinates Y, and then reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional transmission tower model corresponding to the transmission tower image to be reconstructed.

[0093] By implementing this embodiment, camera intrinsic parameters and images of transmission towers to be reconstructed from different perspectives are obtained; the transmission tower images are input into a preset neural network model to extract the shape, texture, and edges of the transmission towers in the transmission tower images, and an initial feature map is generated based on the shape, texture, and edges of the transmission towers; initial point cloud coordinates of the transmission tower image are calculated based on the transmission tower image and the camera intrinsic parameters; multi-layer convolution is performed based on the initial feature map to generate a depth feature map of the transmission tower image; and based on the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower. By obtaining camera intrinsic parameters and transmission tower images from different perspectives, and combining them with a preset neural network to extract shape, texture, and edge information, an initial feature map is generated, providing high-precision basic data for subsequent 3D reconstruction. Initial feature maps from different perspectives are used to obtain depth feature maps from different perspectives. In scenarios sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the transmission tower structure is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, making it possible to restore the detailed features of the transmission tower and improve the accuracy of 3D reconstruction of the transmission tower.

[0094] Through deep feature extraction and guided point cloud refinement, the detailed features of transmission towers can be restored more accurately, matching errors in complex environments can be reduced, and reconstruction accuracy can be improved. This solves the problem that existing technologies are difficult to accurately restore the detailed features of transmission towers, making the reconstruction results more refined and meeting the needs of practical applications. Compared with traditional methods such as laser scanning, the present invention does not require expensive equipment, is relatively simple to operate, and has a small amount of data processing, reducing the cost and difficulty of reconstruction.

[0095] See also Figure 2 , is a schematic structural diagram of a transmission tower 3D reconstruction device based on deep feature recovery provided by one embodiment of the present invention, comprising:

[0096] A data acquisition module is used to obtain camera internal parameters and images of the transmission tower to be reconstructed from different perspectives;

[0097] an initial feature map generating module, configured to input a transmission tower image into a preset neural network model to extract the shape, texture, and edge of the transmission tower in the transmission tower image, and generate an initial feature map based on the shape, texture, and edge of the transmission tower;

[0098] An initial point cloud coordinate calculation module is used to calculate the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters;

[0099] A depth feature map generation module, configured to perform multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image;

[0100] The tower model construction module is used to densify the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower.

[0101] Specifically, the initial feature map generation module includes:

[0102] A tower area generation unit is used to input the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image;

[0103] A tower feature extraction unit, configured to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture, and edge of the transmission tower;

[0104] The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

[0105] Preferably, the initial point cloud coordinate calculation module includes:

[0106] a tower image convolution unit, configured to perform convolution on the transmission tower image and a Gaussian kernel of a preset scale to obtain a plurality of pixel points;

[0107] A tower feature point determination unit is used to find local extreme points in a preset Gaussian difference scale space based on a number of pixel points as feature points of the transmission tower image;

[0108] A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs;

[0109] A projection matrix construction unit, configured to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters;

[0110] The point cloud coordinate calculation unit is used to calculate the initial point cloud coordinates according to the feature point matching pairs and the projection matrix.

[0111] The present invention provides a three-dimensional reconstruction device for a transmission tower based on depth feature recovery. The device comprises the following steps: a data acquisition module acquires camera intrinsic parameters and images of the transmission tower to be reconstructed from different perspectives; an initial feature map generation module inputs the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generates an initial feature map based on the shape, texture and edge of the transmission tower; an initial point cloud coordinate calculation module calculates the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters; a depth feature map generation module performs multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image; and finally, a tower model construction module densifies the initial point cloud coordinates corresponding to the depth feature map based on the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower. By obtaining camera intrinsic parameters and transmission tower images from different perspectives, and combining them with a preset neural network to extract shape, texture, and edge information, an initial feature map is generated, providing high-precision basic data for subsequent 3D reconstruction. Initial feature maps from different perspectives are used to obtain depth feature maps from different perspectives. In scenarios sensitive to lighting or texture conditions, the depth feature map and point cloud coordinate densification can increase the density of point cloud data and the details of the transmission tower. Even if the transmission tower structure is complex, the complex structure and geometric spatial position of the transmission tower can be fully captured, making it possible to restore the detailed features of the transmission tower and improve the accuracy of 3D reconstruction of the transmission tower.

[0112] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0113] Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] Another embodiment of the present invention provides a terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method for three-dimensional reconstruction of a transmission tower based on deep feature recovery as described in the above embodiment. The terminal device may be a computing device such as a desktop computer, a laptop, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0115] The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0116] The memory can be used to store the computer program, and the processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0117] Another embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program. When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method of a transmission tower based on deep feature recovery described in the above embodiment.

[0118] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. The computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0119] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A 3D reconstruction method for transmission towers based on deep feature recovery, characterized in that: include: Obtain camera intrinsic parameters and transmission tower images to be reconstructed from different perspectives; Inputting the transmission tower image into a preset neural network model to extract the shape, texture and edge of the transmission tower in the transmission tower image, and generating an initial feature map based on the shape, texture and edge of the transmission tower; Calculating the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters; Perform multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image; According to the depth feature map and the initial point cloud coordinates, densifying the initial point cloud coordinates corresponding to the depth feature map to obtain a three-dimensional model of the transmission tower; Calculating the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters includes: Perform convolution on the transmission tower image and a Gaussian kernel of a preset scale to obtain a number of pixel points; Find the local extreme points in the preset Gaussian difference scale space based on a number of pixel points as the feature points of the transmission tower image; Calculate the similarity measure between the feature points based on the feature points of the transmission tower image, filter the feature points corresponding to the similarity measure less than the preset similarity threshold, and obtain the feature point matching pairs; Constructing a projection matrix of the transmission tower image according to the camera intrinsic parameters; Calculating initial point cloud coordinates based on the feature point matching pairs and the projection matrix; Among them, the projection matrix is ​​P: 、 is the focal length parameter in the camera intrinsic parameter, 、 is the center point parameter in the camera intrinsic parameter; The point cloud data is recovered from the feature point matching pairs through triangulation. The formula for triangulating the initial point cloud coordinate X is: The feature point matching pairs are and ; The subscripts represent the rows and columns of P, P ij is the value of the i-th row and j-th column of the P matrix; According to the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain a three-dimensional model of the transmission tower, including: According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters; According to the optimal registration parameters, determine the correspondence between the depth feature map and the initial point cloud coordinates; According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates; Reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

2. The method for 3D reconstruction of a transmission tower based on deep feature recovery according to claim 1, wherein: Inputting the transmission tower image into a preset neural network model to extract the shape, texture, and edge of the transmission tower in the transmission tower image, and generating an initial feature map based on the shape, texture, and edge of the transmission tower, including: Input the transmission tower image into the preset neural network model to detect the transmission tower target and generate the transmission tower area image; Performing convolution and pooling operations on the transmission tower area image to extract the shape, texture, and edge of the transmission tower; Generate an initial feature map based on the shape, texture, and edge of the transmission tower.

3. The method for 3D reconstruction of a transmission tower based on deep feature recovery according to claim 1, wherein: Performing multi-layer convolution on the initial feature map to generate a depth feature map of the transmission tower image, including: Initializing and transforming the initial feature map to obtain a multi-scale depth feature map; The multi-scale depth feature map is subjected to multi-layer convolution to generate the depth feature map of the convolved transmission tower image.

4. A 3D reconstruction device for transmission towers based on deep feature recovery, characterized in that: include: A data acquisition module is used to obtain camera internal parameters and images of the transmission tower to be reconstructed from different perspectives; an initial feature map generating module, configured to input a transmission tower image into a preset neural network model to extract the shape, texture, and edge of the transmission tower in the transmission tower image, and generate an initial feature map based on the shape, texture, and edge of the transmission tower; An initial point cloud coordinate calculation module is used to calculate the initial point cloud coordinates of the transmission tower image based on the transmission tower image and the camera intrinsic parameters; A depth feature map generation module, configured to perform multi-layer convolution based on the initial feature map to generate a depth feature map of the transmission tower image; A tower model construction module is used to perform point cloud coordinate densification on the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower; The initial point cloud coordinate calculation module includes: a tower image convolution unit, configured to perform convolution on the transmission tower image and a Gaussian kernel of a preset scale to obtain a plurality of pixel points; A tower feature point determination unit is used to find local extreme points in a preset Gaussian difference scale space based on a number of pixel points as feature points of the transmission tower image; A feature point matching unit is used to calculate a similarity measure between feature points based on the feature points of the transmission tower image, filter feature points corresponding to similarity measures less than a preset similarity threshold, and obtain feature point matching pairs; A projection matrix construction unit, configured to construct a projection matrix of the transmission tower image according to the camera intrinsic parameters; A point cloud coordinate calculation unit, configured to calculate initial point cloud coordinates based on the feature point matching pairs and the projection matrix; Among them, the projection matrix is ​​P: 、 is the focal length parameter in the camera intrinsic parameter, 、 is the center point parameter in the camera intrinsic parameter; The point cloud data is recovered from the feature point matching pairs through triangulation. The formula for triangulating the initial point cloud coordinate X is: The feature point matching pairs are and ; The subscripts represent the rows and columns of P, P ij is the value of the i-th row and j-th column of the P matrix; The tower model construction module is used to perform point cloud coordinate densification on the initial point cloud coordinates corresponding to the depth feature map according to the depth feature map and the initial point cloud coordinates to obtain a three-dimensional model of the transmission tower, including: According to the depth feature map and the initial point cloud coordinates, the distance between each pixel in the depth feature map and the initial point cloud coordinates is calculated to obtain the optimal registration parameters; According to the optimal registration parameters, determine the correspondence between the depth feature map and the initial point cloud coordinates; According to the correspondence between the depth feature map and the initial point cloud coordinates, the initial point cloud coordinates corresponding to the depth feature map are densified to obtain dense point cloud coordinates; Reconstruction is performed based on the dense point cloud coordinates to generate a three-dimensional model of the transmission tower corresponding to the transmission tower image to be reconstructed.

5. The apparatus for 3D reconstruction of a transmission tower based on deep feature recovery according to claim 4, wherein: The initial feature map generation module includes: A tower area generation unit is used to input the transmission tower image into a preset neural network model to perform transmission tower target detection and generate a transmission tower area image; A tower feature extraction unit, configured to perform convolution and pooling operations on the transmission tower area image to extract the shape, texture, and edge of the transmission tower; The feature map generating unit is used to generate an initial feature map according to the shape, texture and edge of the transmission tower.

6. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for three-dimensional reconstruction of a transmission tower based on deep feature recovery according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the three-dimensional reconstruction method of a transmission tower based on depth feature recovery according to any one of claims 1 to 3.

Images