Camera pose estimation method and device, storage medium and electronic device

Abstract

The application discloses a camera pose estimation method and device, a storage medium and an electronic device, relates to the field of computer vision, and comprises the following steps: acquiring a first target image and first point cloud data; preprocessing the first target image to obtain a second target image, and preprocessing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and performing pose estimation on a target camera according to geographical position information of the target camera, the third target image and the third point cloud data by using a target estimation model to obtain pose information of the target camera. Through the application, the problem that the accuracy of pose estimation on the camera is relatively low due to the fact that the pose estimation on the camera is performed according to the brightness information of the image in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a camera pose estimation method and device, a storage medium and an electronic device. BACKGROUND

[0002] In a power transmission system, it is usually necessary to use a camera to monitor the power transmission channel, so as to prevent potential risks and ensure the continuity and safety of power supply. In order to realize the control of the orientation of the camera in the power transmission channel and accurately patrol the running state of the electrical elements in the power transmission channel, the external parameter information of the camera needs to be determined, so as to realize the pose estimation of the camera. In the related art, the pose of the camera is estimated according to the pixel brightness information of the video image captured by the camera in the power transmission channel. However, due to different exposures of different cameras, there is a certain difference in the image brightness captured by different cameras, thereby causing the problem of low accuracy of pose estimation of the camera.

[0003] In view of the problem of low accuracy of pose estimation of the camera due to the difference in image brightness captured by different cameras in the related art, an effective solution has not been proposed yet. SUMMARY

[0004] The main purpose of the present application is to provide a camera pose estimation method and device, a storage medium and an electronic device, so as to solve the problem of low accuracy of pose estimation of the camera due to the difference in image brightness captured by different cameras in the related art.

[0005] In order to achieve the above purpose, according to one aspect of the present application, a camera pose estimation method is provided. The method comprises: acquiring a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; preprocessing the first target image to obtain a second target image, and preprocessing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image by a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data by a target point cloud semantic segmentation model to obtain third point cloud data; and estimating the pose of the target camera according to the geographic location information of the target camera, the third target image and the third point cloud data by a target estimation model, to obtain the pose information of the target camera.

[0006] Further, the first target image is preprocessed to obtain a second target image, including: performing downsizing processing on the first target image to obtain a target size first target image; performing filtering processing on the target size first target image to obtain a filtered first target image; and labeling a target object in the filtered first target image to obtain the second target image.

[0007] Further, the target camera is pose estimated by a target estimation model according to the geographic position information of the target camera, the third target image and the third point cloud data to obtain pose information of the target camera, including: cropping the third point cloud data according to the first target region to obtain fourth point cloud data; projecting the fourth point cloud data according to the geographic position information of the target camera to obtain a point cloud data projection map; and pose estimating the target camera by the target estimation model according to the point cloud data projection map and the third target image to obtain the pose information of the target camera.

[0008] Further, the target camera is pose estimated by the target estimation model according to the point cloud data projection map and the third target image to obtain the pose information of the target camera, including: establishing a correspondence between points in the point cloud data projection map and pixel points in the third target image; mapping the points in the point cloud data projection map to the corresponding pixel points in the third target image by the target estimation model according to the correspondence to obtain a target matrix; and determining the pose information of the target camera according to the target matrix.

[0009] Further, before the second target image is semantically segmented by the target image semantic segmentation model to obtain a segmented third target image, the method further includes: obtaining a training sample set, wherein the training sample set at least includes a first training image, first training point cloud data and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; preprocessing the first training image to obtain a second training image, and preprocessing the first training point cloud data to obtain second training point cloud data; training an initial image semantic segmentation model according to the second training image to obtain the target image semantic segmentation model, and training an initial point cloud semantic segmentation model according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

[0010] Further, after the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model, the method further comprises: segmenting the second training image by using the target image semantic segmentation model to obtain third training image; segmenting the second training point cloud data by using the target point cloud semantic segmentation model to obtain third training point cloud data; and training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

[0011] Further, training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model comprises: performing pose estimation on the training camera by using the initial estimation model according to the third training image and the third training point cloud data to obtain predicted pose information of the training camera; determining a target loss function of the initial estimation model according to the real pose information and the predicted pose information; and training the initial estimation model according to the target loss function to obtain the target estimation model.

[0012] To achieve the above object, according to another aspect of the present application, a pose estimation device for a camera is provided. The device comprises: a first acquisition unit configured to acquire a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; a first preprocessing unit configured to preprocess the first target image to obtain a second target image, and preprocess the first point cloud data to obtain second point cloud data; a first segmentation unit configured to perform semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and perform semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and an estimation unit configured to perform pose estimation on the target camera by using a target estimation model according to geographical position information of the target camera, the third target image and the third point cloud data to obtain pose information of the target camera.

[0013] Further, the first preprocessing unit comprises: a processing module configured to perform size reduction processing on the first target image to obtain a target-size first target image; a filtering module configured to perform filtering processing on the target-size first target image to obtain a filtered first target image; and a labeling module configured to label target objects in the filtered first target image to obtain the second target image.

[0014] Further, the estimation unit comprises: a clipping module configured to clip the third point cloud data according to the first target region to obtain fourth point cloud data; a projection module configured to project the fourth point cloud data according to geographical position information of the target camera to obtain a point cloud data projection image; and a first estimation module configured to perform pose estimation on the target camera according to the point cloud data projection image and the third target image by using the target estimation model to obtain pose information of the target camera.

[0015] Further, the first estimation module comprises: an establishing submodule configured to establish a correspondence between a point in the point cloud data projection image and a pixel in the third target image; a mapping submodule configured to map the point in the point cloud data projection image to the corresponding pixel in the third target image by using the target estimation model according to the correspondence to obtain a target matrix; and a determining submodule configured to determine the pose information of the target camera according to the target matrix.

[0016] Further, the apparatus further comprises: a second acquisition unit configured to acquire a training sample set, wherein the training sample set at least comprises a first training image, first training point cloud data, and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting region of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; a second preprocessing unit configured to pre-process the first training image to obtain a second training image, and pre-process the first training point cloud data to obtain second training point cloud data; and a first training unit configured to train an initial image semantic segmentation model according to the second training image to obtain the target image semantic segmentation model, and train an initial point cloud semantic segmentation model according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

[0017] Further, the apparatus further comprises: a second segmentation unit configured to segment the second training image by using the target image semantic segmentation model to obtain a third training image; a third segmentation unit configured to segment the second training point cloud data by using the target point cloud semantic segmentation model to obtain third training point cloud data; and a second training unit configured to train an initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

[0018] Further, the second training unit comprises: a second estimation module, configured to perform pose estimation on the training camera according to the third training image and the third training point cloud data by using the initial estimation model, to obtain predicted pose information of the training camera; a determination module, configured to determine a target loss function of the initial estimation model according to the real pose information and the predicted pose information; and a training module, configured to train the initial estimation model according to the target loss function, to obtain a target estimation model.

[0019] In order to achieve the above object, according to an aspect of the present application, a computer readable storage medium is provided, the storage medium stores a program, wherein the program controls the device where the storage medium is located to execute the pose estimation method of the camera according to any one of the above aspects when the program is running.

[0020] In order to achieve the above object, according to another aspect of the present application, an electronic device is also provided, the electronic device comprises one or more processors and a memory, the memory is used to store the pose estimation method of the camera according to any one of the above aspects implemented by the one or more processors.

[0021] By the present application, the following steps are adopted: obtaining a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; preprocessing the first target image to obtain a second target image, and preprocessing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and performing pose estimation on the target camera according to geographical position information of the target camera, the third target image and the third point cloud data by using a target estimation model to obtain pose information of the target camera, thereby solving the problem that in the related art, when performing pose estimation on a camera according to pixel brightness information of an image shot by the camera in a power transmission channel, the accuracy of the pose estimation on the camera is relatively low due to the difference in image brightness shot by different cameras. In the present application, the problem that the accuracy of the pose estimation on the target camera is relatively low due to the pose estimation on the target camera according to the pixel brightness information of the image shot by the camera in the power transmission channel is avoided, the purpose of performing pose estimation on the target camera by using the target estimation model is achieved, and the effect of improving the accuracy of the pose estimation on the target camera is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are incorporated herein for a purpose of explanation and as an auxiliary to explain the present application, and are not for a restrictive purpose to the present application. In the drawings:

[0023] Figure 1 is a flow of a camera pose estimation method provided according to an embodiment of the present application Figure One ;

[0024] Figure 2 is a training flowchart of a target image semantic segmentation model according to an embodiment of the present application

[0025] Figure 3 is a training flowchart of a target point cloud semantic segmentation model according to an embodiment of the present application

[0026] Figure 4 is a training flowchart of a target estimation model according to an embodiment of the present application

[0027] Figure 5 is a flow of a camera pose estimation method provided according to an embodiment of the present application Figure Two ;

[0028] Figure 6 is a schematic diagram of a camera pose estimation device provided according to an embodiment of the present application

[0029] Figure 7 is a schematic diagram of an electronic device provided according to an embodiment of the present application DETAILED DESCRIPTION

[0030] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should belong to the protection scope of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] For ease of description, the following describes some nouns or terms related to the embodiments of the present application:

[0034] Median function: The median function is a statistical function for calculating the middle value of a set of numerical values.

[0035] HENET: Hierarchical Encoder-decoder Network is a deep learning model for semantic segmentation. It is based on the encoder-decoder structure, which can segment the input image at the pixel level into different semantic categories.

[0036] PointNet++: PointNet++ is a deep learning method for point cloud data processing, which can be used for point cloud classification, segmentation and semantic segmentation tasks. In the semantic segmentation task, PointNet++ can classify the points in each point cloud into different semantic categories.

[0037] It should be noted that the relevant information (including but not limited to camera geographic location information, power transmission channel information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties. For example, the system and related users or institutions have an interface, before obtaining the relevant information, the interface needs to send a request to the aforementioned user or institution, and after receiving the consent information feedback from the aforementioned user or institution, the relevant information is obtained.

[0038] The present application will be described below in conjunction with the preferred implementation steps, Figure 1 is the flow of the camera pose estimation method provided by the embodiments of the present application Figure One As shown in Figure 1 , the method comprises the following steps:

[0039] In step S101, a first target image corresponding to a first target region and first point cloud data corresponding to a second target region are obtained, wherein the first target region is a shooting region of the target camera, and the second target region is a region of the power transmission channel where the target camera is located.

[0040] For example, the first target image currently shot by the target camera and the point cloud data of the region of the power transmission channel where the target camera is located are obtained, and the point cloud data of the region of the power transmission channel can be collected by using a drone or a backpack radar and the like.

[0041] In step S102, the first target image is preprocessed to obtain a second target image, and the first point cloud data is preprocessed to obtain second point cloud data.

[0042] For example, the first target image is preprocessed by denoising, filtering, color correction, scale adjustment and information labeling to obtain a processed second target image, and the point cloud data is preprocessed by denoising, filtering, sampling and information labeling to obtain processed second point cloud data.

[0043] In step S103, the second target image is subjected to semantic segmentation by using a target image semantic segmentation model to obtain a third target image, and the second point cloud data is subjected to semantic segmentation by using a target point cloud semantic segmentation model to obtain third point cloud data.

[0044] For example, the target objects (such as trees, power transmission towers, houses, roads and the like) in the second target image are subjected to semantic segmentation by using the target semantic segmentation model to obtain the third target image, and the target objects in the second point cloud data are subjected to semantic segmentation by using the semantic segmentation model to obtain the third point cloud data.

[0045] In step S104, the target camera is subjected to pose estimation by using a target estimation model according to the geographic position information of the target camera, the third target image and the third point cloud data to obtain pose information of the target camera.

[0046] For example, the target estimation model is used to predict the external parameters of the target camera according to the geographic position information of the target camera, the third target image and the third point cloud data, so as to perform pose estimation on the target camera to obtain the pose information of the target camera, i.e., the external parameter matrix of the target camera.

[0047] In summary, in the present application, the problem that the pose estimation accuracy of the target camera is relatively low due to the pose estimation of the camera according to the pixel brightness information of the image shot by the camera in the power transmission channel is avoided, the purpose of performing pose estimation on the target camera by using the target estimation model is achieved, and the effect of improving the pose estimation accuracy of the target camera is achieved.

[0048] Optionally, in the camera pose estimation method provided by the embodiment of the present application, the pre-processing of the first target image to obtain the second target image comprises: performing a reduction processing on the first target image to obtain a target size first target image; performing a filtering processing on the target size first target image to obtain a filtered first target image; and labeling a target object in the filtered first target image to obtain the second target image.

[0049] For example, the first target image is first reduced in size to obtain a target size first target image, and then a median filtering processing is performed on the target size first target image to reduce noise in the target size first target image to obtain a filtered first target image, and then a semantic label is added to a target object in the filtered first target image to obtain the second target image, wherein the target object can include typical objects such as power transmission towers, roads, trees, buildings and ground.

[0050] In an optional embodiment, the first target image can be median filtered using the method shown in formula (1) as follows:

[0051] I s (x,y)=median(I(x-k,y-k),I(x-k,y-k+1),…I(x+k,y+k)) (1)

[0052] Wherein, I s (x,y) is the pixel value in the filtered first target image, (x,y) is the position of the pixel point in the first target image, x is the row coordinate, y is the column coordinate, the median function is used to calculate the median, I(x-k,y-k) represents the pixel value with a distance of k in row coordinate and k in column coordinate from I(x,y), and k represents the pixel value with a distance of k from x (or y).

[0053] By pre-processing the first target image, the accuracy of semantic segmentation of the target image is improved.

[0054] It should be noted that the first point cloud data also needs to be filtered and denoised, and the first point cloud data can be filtered using Gaussian filtering as shown in formula (2) as follows:

[0055]

[0056] Wherein, P ′ is the position of the point in the filtered first point cloud data, i is used to represent the i th point in the point cloud data, w(i) is the weight of the i th point in the point cloud data, and P(i) is the position of the i th point in the point cloud data before filtering.

[0057] Optionally, in the camera pose estimation method provided in the embodiments of the present application, the target estimation model is used to perform pose estimation on the target camera according to the geographic position information of the target camera, the third target image and the third point cloud data, to obtain the pose information of the target camera, including: performing cropping on the third point cloud data according to the first target region to obtain fourth point cloud data; projecting the fourth point cloud data according to the geographic position information of the target camera to obtain a point cloud data projection image; and performing pose estimation on the target camera according to the point cloud data projection image and the third target image by the target estimation model to obtain the pose information of the target camera.

[0058] For example, first, the third point cloud data is cropped according to the first target region, and unnecessary point cloud data in the third point cloud data other than the first target region is cropped to improve the efficiency and accuracy of subsequent processing, to obtain fourth point cloud data. Then, the installation information of the camera is obtained by querying the tower station account according to the geographic position information of the target camera, to determine the rough orientation of the target camera, so that the fourth point cloud data is projected to project the three-dimensional point cloud data into a two-dimensional point cloud data projection image, and ensure that the points in the two-dimensional point cloud data projection image can be accurately mapped to the corresponding image position. Then, pose estimation is performed on the target camera according to the point cloud data projection image and the third target image by the target estimation model to obtain the pose information of the target camera.

[0059] By cropping the third point cloud data to obtain the fourth point cloud data, the fourth point cloud data and the third target image are accurately corresponded, so that the points in the fourth point cloud data can be corresponded to the pixel points in the third target image, and the accuracy of estimating the pose information of the target camera is improved.

[0060] Optionally, in the camera pose estimation method provided in the embodiments of the present application, the target estimation model is used to perform pose estimation on the target camera according to the point cloud data projection image and the third target image to obtain the pose information of the target camera, including: establishing a correspondence between the points in the point cloud data projection image and the pixel points in the third target image; mapping the points in the point cloud data projection image to the corresponding pixel points in the third target image according to the correspondence by the target estimation model to obtain a target matrix; and determining the pose information of the target camera according to the target matrix.

[0061] For example, the correspondence between the points in the point cloud data projection image and the pixel points in the third target image is established according to the semantic labels in the point cloud data projection image and the semantic labels in the third target image. For example, the points in the point cloud data projection image with the semantic label of the power transmission tower are associated with the pixel points in the third target image with the same semantic label of the power transmission tower, to establish the above correspondence.

[0062] Then, the points in the point cloud data projection image are mapped into corresponding pixel points in the third target image according to the correspondence through the target estimation model, a transformation matrix (i.e., the target matrix described above) is output, and finally the pose information of the target camera, i.e., the extrinsic parameters of the target camera, is determined according to the transformation matrix.

[0063] In an optional embodiment, the target matrix can be calculated according to the least square method, and specifically, the points in the point cloud data projection image can be mapped by using the method shown in formula (3) to obtain the target matrix, and the specific process is as follows:

[0064] P ing =T·P pc (3)

[0065] wherein P ing represents a point in the point cloud data, P pc represents a point in the third target image, and T represents the target matrix.

[0066] By mapping the points in the point cloud data projection image into corresponding pixel points in the third target image through the target estimation model, the target matrix is output, and the pose information of the target camera is determined according to the target matrix, thereby avoiding estimating the pose of the target camera through the brightness information of the pixel points in the third target image, and the problem of low accuracy of the pose estimation of the target camera is solved, and the accuracy of the pose estimation of the target camera is improved.

[0067] Optionally, in the camera pose estimation method provided in the embodiments of the present application, before the second target image is subjected to semantic segmentation through the target image semantic segmentation model to obtain the segmented third target image, the method further includes: obtaining a training sample set, wherein the training sample set at least includes a first training image, first training point cloud data, and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; the first training image is preprocessed to obtain a second training image, and the first training point cloud data is preprocessed to obtain second training point cloud data; the initial image semantic segmentation model is trained according to the second training image to obtain the target image semantic segmentation model, and the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

[0068] For example, the first training point cloud data, the first training image, and the real pose information of the training camera corresponding to the first training image are obtained first, wherein the first training image is an image shot by the training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located.

[0069] The first training image is further preprocessed by median filtering, color correction, scale adjustment, information labeling, etc. to obtain a second training image, and the first training point cloud data is preprocessed by noise reduction, filtering, sampling, information labeling, etc. to obtain second training point cloud data. Then, the initial image semantic segmentation model is trained according to the second training image to obtain a target image semantic segmentation model, and the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain a target semantic segmentation model.

[0070] In an optional embodiment, the target image semantic segmentation model can be trained by using the flowchart as shown in Figure 2 The first training image with real pose information is obtained, and the target object in the first training image is preprocessed to obtain a second training image. The target object in the second training image has corresponding labeling information. The second training image is subjected to semantic segmentation by using the initial image semantic segmentation model. The image semantic segmentation model is iteratively trained for multiple times, and whether a termination condition is reached is determined during the training. When the termination condition is not reached, the parameters of the image semantic segmentation model are adjusted. Finally, the optimal weight parameters are obtained, and the target image semantic segmentation model is obtained.

[0071] In an optional embodiment, the initial point cloud semantic segmentation model can be trained by using the flowchart as shown in Figure 3 The first training point cloud data is obtained, and the target object in the first training point cloud data is preprocessed to obtain second training point cloud data. The target object in the second training point cloud data has corresponding labeling information. The second training point cloud data is subjected to semantic segmentation by using the initial point cloud semantic segmentation model. The point cloud semantic segmentation model is iteratively trained for multiple times, and whether a termination condition is reached is determined during the training. When the termination condition is not reached, the parameters of the point cloud semantic segmentation model are adjusted. Finally, the optimal weight parameters are obtained, and the target point cloud semantic segmentation model is obtained.

[0072] The initial image semantic segmentation model is trained by using the second training image to obtain the target image semantic segmentation model, which improves the accuracy of the target image semantic segmentation model in performing semantic segmentation on images. The initial point cloud semantic segmentation model is trained by using the second training point cloud data to obtain the target point cloud semantic segmentation model, which improves the accuracy of the target point cloud semantic segmentation model in performing semantic segmentation on point cloud data.

[0073] Optionally, in the camera pose estimation method provided in the embodiments of the present application, after the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model, the method further comprises: segmenting the second training image by the target image semantic segmentation model to obtain third training image; segmenting the second training point cloud data by the target point cloud semantic segmentation model to obtain third training point cloud data; training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

[0074] For example, the second training image is first segmented by the target image semantic segmentation model to segment the target object in the second training image to obtain the third training image, and the second training point cloud data is segmented by the target point cloud semantic segmentation model to segment the target object in the second training point cloud data to obtain the third training point cloud data, and then the initial estimation model is trained according to the third training image and the third training point cloud data to obtain the target estimation model.

[0075] By segmenting the second training image and the second training point cloud data to obtain the third training image and the third training point cloud data, and training the initial estimation model to obtain the target estimation model, the accuracy of the target estimation model in pose estimation of the camera is improved.

[0076] Optionally, in the camera pose estimation method provided in the embodiments of the present application, training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model comprises: estimating the pose of the training camera by the initial estimation model according to the third training image and the third training point cloud data to obtain predicted pose information of the training camera; determining a target loss function of the initial estimation model according to the real pose information and the predicted pose information; training the initial estimation model according to the target loss function to obtain the target estimation model.

[0077] For example, the predicted pose information of the training camera is first output by the initial estimation model according to the third training image and the third training point cloud data, and then the target loss function of the initial estimation model is calculated according to the real pose information corresponding to the first training image in the training sample set and the predicted pose information output by the initial estimation model, and then the predicted pose information is approximated to the real pose information by minimizing the target loss function, so as to realize the training of the initial estimation model to obtain the target estimation model.

[0078] In an optional embodiment, as shown in FIG. 6, the camera pose estimation method provided in the embodiments of the present application can further comprise the following steps. Figure 4The flowchart shown implements the training of the target estimation model as follows: First, based on the semantic segmentation result label of the third training image and the semantic segmentation result projection label of the third training point cloud data, the correspondence between the pixels in the third training image and the points in the two-dimensional projection data of the third training point cloud data is matched. Then, based on the correspondence, the predicted pose information is calculated. Then, through multiple iterations, the target loss function is calculated until the model converges, and the target estimation model is obtained.

[0079] In an optional embodiment, the target loss function can be calculated using the method shown in Equation (4), as follows:

[0080]

[0081] Where L represents the target loss function, i represents the i-th third training image, N represents the number of third training images, and p i For true pose information, q i To predict pose information.

[0082] The initial estimation model is trained by the target loss function to obtain the target estimation model, which improves the accuracy of the target pose estimation model in estimating the pose of the camera.

[0083] In an alternative embodiment, the following can be employed: Figure 5 The flowchart shown illustrates the implementation of camera pose estimation, specifically as follows: First, 3D point cloud data of the power transmission channel and 2D images captured by the target camera are acquired. Then, the 3D point cloud data and 2D images are preprocessed, and their formats are converted to tensor formats. Next, the 2D image is semantically segmented using a semantic segmentation model based on HRNET to obtain the segmented 2D image. Similarly, the 3D point cloud data is semantically segmented using a semantic segmentation model based on PointNet++ to obtain the segmented 3D point cloud data. Subsequently, the segmented 3D point cloud data is cropped and projected to obtain a 2D point cloud data projection map. Finally, a target estimation model based on a cross-domain matching algorithm estimates the pose of the target camera based on the segmented 2D point cloud data projection map, the segmented 3D point cloud data, and the target camera's installation location information to obtain the target camera's pose information.

[0084] The method for estimating the pose of a camera provided in the embodiments of the present application comprises the following steps: obtaining a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; pre-processing the first target image to obtain a second target image, and pre-processing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and estimating the pose of the target camera by using a target estimation model according to geographical position information of the target camera, the third target image and the third point cloud data to obtain pose information of the target camera. The method solves the problem that, in the related art, when the pose of a camera is estimated according to pixel brightness information of an image shot by the camera in a power transmission channel, the accuracy of the pose estimation of the camera is relatively low due to the difference in image brightness shot by different cameras. In the present application, the problem that the accuracy of the pose estimation of the target camera is relatively low due to the estimation of the pose of the target camera according to the pixel brightness information of the image shot by the camera in the power transmission channel is avoided, the purpose of estimating the pose of the target camera by using the target estimation model is achieved, and the effect of improving the accuracy of the pose estimation of the target camera is achieved.

[0085] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0086] The embodiments of the present application also provide a camera pose estimation device. It should be noted that the camera pose estimation device of the embodiments of the present application can be used to execute the method for estimating the pose of a camera provided in the embodiments of the present application. The camera pose estimation device provided in the embodiments of the present application is introduced as follows.

[0087] Figure 6 is a schematic diagram of the camera pose estimation device according to the embodiments of the present application. As shown in Figure 6 the device comprises a first acquisition unit 601, a first preprocessing unit 602, a first segmentation unit 603 and an estimation unit 604.

[0088] The first acquisition unit 601 is configured to acquire a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located;

[0089] The first preprocessing unit 602 is configured to preprocess the first target image to obtain a second target image, and preprocess the first point cloud data to obtain second point cloud data.

[0090] The first segmentation unit 603 is configured to perform semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and perform semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data.

[0091] The estimation unit 604 is configured to perform pose estimation on the target camera according to geographical position information of the target camera, the third target image and the third point cloud data by using a target estimation model to obtain pose information of the target camera.

[0092] The pose estimation device for the camera provided in the embodiments of the present application comprises a first acquisition unit 601 configured to acquire a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; a first preprocessing unit 602 configured to preprocess the first target image to obtain a second target image, and preprocess the first point cloud data to obtain second point cloud data; a first segmentation unit 603 configured to perform semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and perform semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and an estimation unit 604 configured to perform pose estimation on the target camera according to geographical position information of the target camera, the third target image and the third point cloud data by using a target estimation model to obtain pose information of the target camera. The above technical solution solves the problem in the related art that the pose estimation of the camera is performed according to the pixel brightness information of the image shot by the camera in the power transmission channel, and the accuracy of the pose estimation of the camera is relatively low due to the difference in the brightness of the images shot by different cameras. In the present application, the pose estimation of the target camera is not performed according to the pixel brightness information of the image shot by the camera in the power transmission channel, and the problem of low accuracy of the pose estimation of the target camera is avoided. The pose estimation of the target camera is performed by using the target estimation model, and the accuracy of the pose estimation of the target camera is improved.

[0093] Optionally, in the camera pose estimation apparatus provided by the embodiment of the present application, the first preprocessing unit 602 comprises: a processing module, configured to perform reduction processing on the first target image to obtain a target-size first target image; a filtering module, configured to perform filtering processing on the target-size first target image to obtain a filtered first target image; and a labeling module, configured to label the target object in the filtered first target image to obtain a second target image.

[0094] Optionally, in the camera pose estimation apparatus provided by the embodiment of the present application, the estimation unit 604 comprises: a clipping module, configured to clip the third point cloud data according to the first target region to obtain fourth point cloud data; a projection module, configured to project the fourth point cloud data according to the geographic position information of the target camera to obtain a point cloud data projection image; and a first estimation module, configured to perform pose estimation on the target camera according to the point cloud data projection image and the third target image by using a target estimation model to obtain the pose information of the target camera.

[0095] Optionally, in the camera pose estimation apparatus provided by the embodiment of the present application, the first estimation module comprises: an establishing sub-module, configured to establish a correspondence between the points in the point cloud data projection image and the pixel points in the third target image; a mapping sub-module, configured to map the points in the point cloud data projection image to the corresponding pixel points in the third target image by using the target estimation model according to the correspondence to obtain a target matrix; and a determining sub-module, configured to determine the pose information of the target camera according to the target matrix.

[0096] Optionally, in the camera pose estimation apparatus provided by the embodiment of the present application, the apparatus further comprises: a second acquisition unit, configured to acquire a training sample set, wherein the training sample set at least comprises a first training image, first training point cloud data and real pose information corresponding to the first training image, the first training image is an image corresponding to the shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; a second preprocessing unit, configured to pre-process the first training image to obtain a second training image, and pre-process the first training point cloud data to obtain second training point cloud data; and a first training unit, configured to train an initial image semantic segmentation model according to the second training image to obtain a target image semantic segmentation model, and train an initial point cloud semantic segmentation model according to the second training point cloud data to obtain a target point cloud semantic segmentation model.

[0097] Optionally, in the camera pose estimation device provided by the embodiment of the present application, the device further comprises: a second segmentation unit configured to segment the second training image by using a target image semantic segmentation model to obtain a third training image; a third segmentation unit configured to segment the second training point cloud data by using a target point cloud semantic segmentation model to obtain third training point cloud data; and a second training unit configured to train the initial estimation model according to the third training image and the third training point cloud data to obtain a target estimation model.

[0098] Optionally, in the camera pose estimation device provided by the embodiment of the present application, the second training unit comprises: a second estimation module configured to perform pose estimation on the training camera by using the initial estimation model according to the third training image and the third training point cloud data to obtain predicted pose information of the training camera; a determination module configured to determine a target loss function of the initial estimation model according to the real pose information and the predicted pose information; and a training module configured to train the initial estimation model according to the target loss function to obtain the target estimation model.

[0099] The camera pose estimation device comprises a processor and a memory, and the first acquisition unit 601, the first preprocessing unit 602, the first segmentation unit 603 and the estimation unit 604 are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0100] The processor comprises a core, and the core retrieves the corresponding program units from the memory. The core can be set to one or more, and the accuracy of pose estimation of the camera can be improved by adjusting the core parameters.

[0101] The memory can comprise a non-persistent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory comprises at least one memory chip.

[0102] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the camera pose estimation method.

[0103] The embodiment of the present application provides a processor, which is used to run a program, and the program is executed to realize the camera pose estimation method.

[0104] As Figure 7As shown, the embodiment of the present application provides an electronic device, the device comprising a processor, a memory, and a program stored on the memory and executable on the processor, and the processor implements the following steps when executing the program: obtaining a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; preprocessing the first target image to obtain a second target image, and preprocessing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image through a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data through a target point cloud semantic segmentation model to obtain third point cloud data; and performing pose estimation on the target camera according to geographical position information of the target camera, the third target image, and the third point cloud data through a target estimation model to obtain pose information of the target camera.

[0105] Optionally, in the method for estimating the pose of the camera provided in the embodiment of the present application, the preprocessing of the first target image to obtain the second target image comprises: performing reduction processing on the first target image to obtain a target-size first target image; performing filtering processing on the target-size first target image to obtain a filtered first target image; and labeling a target object in the filtered first target image to obtain the second target image.

[0106] Optionally, in the method for estimating the pose of the camera provided in the embodiment of the present application, the performing of the pose estimation on the target camera according to the geographical position information of the target camera, the third target image, and the third point cloud data through the target estimation model to obtain the pose information of the target camera comprises: performing cropping on the third point cloud data according to the first target region to obtain fourth point cloud data; performing projection on the fourth point cloud data according to the geographical position information of the target camera to obtain a point cloud data projection image; and performing the pose estimation on the target camera according to the point cloud data projection image and the third target image through the target estimation model to obtain the pose information of the target camera.

[0107] Optionally, in the method for estimating the pose of the camera provided in the embodiment of the present application, the performing of the pose estimation on the target camera according to the point cloud data projection image and the third target image through the target estimation model to obtain the pose information of the target camera comprises: establishing a corresponding relationship between points in the point cloud data projection image and pixel points in the third target image; mapping the points in the point cloud data projection image to corresponding pixel points in the third target image through the target estimation model according to the corresponding relationship to obtain a target matrix; and determining the pose information of the target camera according to the target matrix.

[0108] Optionally, in the camera pose estimation method provided in the embodiments of the present application, before the second target image is segmented by the target image semantic segmentation model to obtain the segmented third target image, the method further comprises: obtaining a training sample set, wherein the training sample set at least includes a first training image, first training point cloud data and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; the first training image is preprocessed to obtain a second training image, and the first training point cloud data is preprocessed to obtain second training point cloud data; the initial image semantic segmentation model is trained according to the second training image to obtain the target image semantic segmentation model, and the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

[0109] Optionally, in the camera pose estimation method provided in the embodiments of the present application, after the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model, the method further comprises: segmenting the second training image by the target image semantic segmentation model to obtain a third training image; segmenting the second training point cloud data by the target point cloud semantic segmentation model to obtain third training point cloud data; and training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

[0110] Optionally, in the camera pose estimation method provided in the embodiments of the present application, training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model comprises: estimating the pose of the training camera according to the third training image and the third training point cloud data by the initial estimation model to obtain predicted pose information of the training camera; determining a target loss function of the initial estimation model according to the real pose information and the predicted pose information; and training the initial estimation model according to the target loss function to obtain the target estimation model. The device in this paper can be a server, a PC, a PAD, a mobile phone, etc.

[0111] The application further provides a computer program product, which is adapted to execute the program of the following method steps when executed on a data processing device: obtaining a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; performing preprocessing on the first target image to obtain a second target image, and performing preprocessing on the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image by using a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data by using a target point cloud semantic segmentation model to obtain third point cloud data; and performing pose estimation on the target camera by using a target estimation model according to geographical position information of the target camera, the third target image and the third point cloud data to obtain pose information of the target camera.

[0112] Optionally, in the method for estimating the pose of the camera provided in the embodiments of the application, the preprocessing of the first target image to obtain the second target image comprises: performing reduction processing on the first target image to obtain a target-size first target image; performing filtering processing on the target-size first target image to obtain a filtered first target image; and labeling a target object in the filtered first target image to obtain the second target image.

[0113] Optionally, in the method for estimating the pose of the camera provided in the embodiments of the application, the pose estimation on the target camera by using the target estimation model according to the geographical position information of the target camera, the third target image and the third point cloud data to obtain the pose information of the target camera comprises: performing cropping on the third point cloud data according to the first target region to obtain fourth point cloud data; performing projection on the fourth point cloud data according to the geographical position information of the target camera to obtain a point cloud data projection image; and performing pose estimation on the target camera by using the target estimation model according to the point cloud data projection image and the third target image to obtain the pose information of the target camera.

[0114] Optionally, in the method for estimating the pose of the camera provided in the embodiments of the application, the pose estimation on the target camera by using the target estimation model according to the point cloud data projection image and the third target image to obtain the pose information of the target camera comprises: establishing a corresponding relationship between points in the point cloud data projection image and pixel points in the third target image; mapping the points in the point cloud data projection image to corresponding pixel points in the third target image by using the target estimation model according to the corresponding relationship to obtain a target matrix; and determining the pose information of the target camera according to the target matrix.

[0115] Optionally, in the camera pose estimation method provided by the embodiment of the application, before the second target image is segmented by the target image semantic segmentation model to obtain the segmented third target image, the method further comprises: obtaining a training sample set, wherein the training sample set at least comprises a first training image, first training point cloud data, and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; the first training image is preprocessed to obtain a second training image, and the first training point cloud data is preprocessed to obtain second training point cloud data; the initial image semantic segmentation model is trained according to the second training image to obtain the target image semantic segmentation model, and the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

[0116] Optionally, in the camera pose estimation method provided by the embodiment of the application, after the initial point cloud semantic segmentation model is trained according to the second training point cloud data to obtain the target point cloud semantic segmentation model, the method further comprises: segmenting the second training image by the target image semantic segmentation model to obtain a third training image; segmenting the second training point cloud data by the target point cloud semantic segmentation model to obtain third training point cloud data; and training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

[0117] Optionally, in the camera pose estimation method provided by the embodiment of the application, training the initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model comprises: performing pose estimation on the training camera according to the third training image and the third training point cloud data by the initial estimation model to obtain predicted pose information of the training camera; determining a target loss function of the initial estimation model according to the real pose information and the predicted pose information; and training the initial estimation model according to the target loss function to obtain the target estimation model.

[0118] Those skilled in the art will understand that the embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0119] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0120] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0121] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0122] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0123] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, for storing instructions and data used and / or generated by the computing device. The memory can also include non-volatile memory, such as read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other non-volatile memory.

[0124] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0125] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0126] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0127] The above merely provides embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for pose estimation of a camera, the method comprising: The method comprises: obtaining a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; preprocessing the first target image to obtain a second target image, and preprocessing the first point cloud data to obtain second point cloud data; performing semantic segmentation on the second target image through a target image semantic segmentation model to obtain a third target image, and performing semantic segmentation on the second point cloud data through a target point cloud semantic segmentation model to obtain third point cloud data; estimating the pose of the target camera through a target estimation model based on geographical position information of the target camera, the third target image, and the third point cloud data to obtain pose information of the target camera; wherein estimating the pose of the target camera through the target estimation model based on the geographical position information of the target camera, the third target image, and the third point cloud data to obtain the pose information of the target camera comprises: cropping the third point cloud data based on the first target region to obtain fourth point cloud data; projecting the fourth point cloud data based on the geographical position information of the target camera to obtain a point cloud data projection image; estimating the pose of the target camera through the target estimation model based on the point cloud data projection image and the third target image to obtain the pose information of the target camera; wherein projecting the fourth point cloud data based on the geographical position information of the target camera to obtain the point cloud data projection image comprises: querying a tower station account based on the geographical position information of the target camera to obtain installation information of the target camera, determining the orientation of the target camera, projecting the three-dimensional point cloud data into a two-dimensional point cloud data projection image.

2. The method of claim 1, wherein, The preprocessing of the first target image to obtain the second target image comprises: performing downsizing processing on the first target image to obtain a target-sized first target image; performing filtering processing on the target-sized first target image to obtain a filtered first target image; annotating a target object in the filtered first target image to obtain the second target image.

3. The method of claim 1, wherein, The estimating of the pose of the target camera through the target estimation model based on the point cloud data projection image and the third target image to obtain the pose information of the target camera comprises: establishing a correspondence between points in the point cloud data projection image and pixel points in the third target image; mapping the points in the point cloud data projection image to corresponding pixel points in the third target image through the target estimation model based on the correspondence to obtain a target matrix; determining the pose information of the target camera based on the target matrix.

4. The method of claim 1, wherein, Before performing semantic segmentation on the second target image through the target image semantic segmentation model to obtain a segmented third target image, the method further comprises: Obtaining a training sample set, wherein the training sample set at least includes a first training image, first training point cloud data, and real pose information corresponding to the first training image, the first training image is an image corresponding to a shooting area of a training camera, and the first training point cloud data is point cloud data of a power transmission channel region where the training camera is located; Preprocessing the first training image to obtain a second training image, and preprocessing the first training point cloud data to obtain second training point cloud data; Training an initial image semantic segmentation model according to the second training image to obtain the target image semantic segmentation model, and training an initial point cloud semantic segmentation model according to the second training point cloud data to obtain the target point cloud semantic segmentation model.

5. The method of claim 4, wherein, After training the initial point cloud semantic segmentation model according to the second training point cloud data to obtain the target point cloud semantic segmentation model, the method further includes: Segmenting the second training image through the target image semantic segmentation model to obtain a third training image; Segmenting the second training point cloud data through the target point cloud semantic segmentation model to obtain third training point cloud data; Training an initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model.

6. The method of claim 5, wherein, Training an initial estimation model according to the third training image and the third training point cloud data to obtain the target estimation model includes: Estimating the pose of the training camera according to the third training image and the third training point cloud data through the initial estimation model to obtain predicted pose information of the training camera; Determining a target loss function of the initial estimation model according to the real pose information and the predicted pose information; Training the initial estimation model according to the target loss function to obtain the target estimation model.

7. An apparatus for pose estimation of a camera, the apparatus comprising: includes: A first obtaining unit is configured to obtain a first target image corresponding to a first target region and first point cloud data corresponding to a second target region, wherein the first target region is a shooting region of a target camera, and the second target region is a region of a power transmission channel where the target camera is located; A first preprocessing unit is configured to preprocess the first target image to obtain a second target image, and preprocess the first point cloud data to obtain second point cloud data; A first segmentation unit is configured to perform semantic segmentation on the second target image through a target image semantic segmentation model to obtain a third target image, and perform semantic segmentation on the second point cloud data through a target point cloud semantic segmentation model to obtain third point cloud data; An estimation unit is configured to estimate the pose of the target camera according to geographical position information of the target camera, the third target image, and the third point cloud data through a target estimation model to obtain pose information of the target camera; The estimation unit comprises: a clipping module configured to clip the third point cloud data according to the first target region to obtain fourth point cloud data; a projection module configured to project the fourth point cloud data according to geographical position information of the target camera to obtain a point cloud data projection image; and a first estimation module configured to estimate a pose of the target camera according to the point cloud data projection image and the third target image by using a target estimation model to obtain pose information of the target camera. The projection module is configured to project the fourth point cloud data according to the geographical position information of the target camera to obtain a point cloud data projection image, including: querying a tower station account according to the geographical position information of the target camera to obtain installation information of the target camera, determining a direction of the target camera, and projecting three-dimensional point cloud data into a two-dimensional point cloud data projection image.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program controls the storage medium to execute the camera pose estimation method of any one of claims 1 to 6 when the program is executed.

9. An electronic device, comprising: The device comprises one or more processors and a memory, wherein the memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the camera pose estimation method of any one of claims 1 to 6.

Images