Laser obstacle clearance tracking aiming method and device

Abstract

The invention discloses a laser obstacle clearance tracking aiming method and device. The method comprises the following steps: acquiring current image data and current point cloud data of a target obstacle in a power transmission line captured by at least two sensors at the same timestamp; mapping a target feature point in the current image data to a three-dimensional space where the current point cloud data is located to obtain mapped target point cloud data; taking each coordinate point in the target point cloud data as a central point, and determining a local feature point set and a local geometric feature in a neighborhood range corresponding to each central point; and performing feature tracking on the local geometric features based on a preset dynamic feedback control mechanism so as to adjust the focus position of the laser to perform tracking aiming of the target obstacle to obtain a corresponding aiming trajectory. According to the technical scheme, through a real-time feedback control mechanism, the laser focus position can be dynamically adjusted according to target changes, the tracking precision can be improved, the automation degree is high, the method can adapt to a complex environment, and the obstacle removing efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of transmission line maintenance, and particularly to a laser obstacle removal tracking and aiming method and device. Background Art

[0002] The ranging function of existing equipment usually relies on external sensors, which are separated from the laser main axis, resulting in the optical axis being prone to deviation in a complex vibration environment, thereby affecting the ranging accuracy and obstacle removal effect. In addition, the rangefinder is highly sensitive to obstacles (such as leaves, branches, etc.), and may not be able to accurately obtain the true distance of the target obstacle in the presence of interference, causing the laser focus to deviate from the main obstacle removal object and reducing the operation efficiency. The dynamic focusing system has insufficient response ability in multi-target processing scenarios and limited adjustment rate, making it difficult to meet the rapid processing requirements of complex working conditions. In addition, the functional modules of the existing system are relatively independent, and the overall design is complex, which not only increases the equipment manufacturing cost but also improves the maintenance difficulty. Summary of the Invention

[0003] In view of this, the present invention provides a laser obstacle removal tracking and aiming method and device, which can improve the tracking accuracy, enhance the robustness, have a high degree of automation, and can adapt to complex environments, significantly improving the obstacle removal efficiency.

[0004] According to one aspect of the present invention, an embodiment of the present invention provides a laser obstacle removal tracking and aiming method, which is applied to a laser obstacle removal tracking and aiming device; the laser obstacle removal tracking and aiming method includes:

[0005] Obtain the current image data and current point cloud data of the target obstacle in the transmission line captured by at least two sensors in the focusing lens at the same timestamp;

[0006] Map the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain the mapped target point cloud data;

[0007] Taking each coordinate point in the target point cloud data as the center point, determine the local feature point set and corresponding local geometric features within the neighborhood range corresponding to each center point;

[0008] Based on a preset dynamic feedback control mechanism, perform feature tracking on the local geometric features to adjust the focus position of the laser to perform tracking and aiming on the target obstacle to obtain the corresponding aiming trajectory.

[0009] According to another aspect of the present invention, an embodiment of the present invention further provides a laser obstacle removal tracking and aiming device, the device includes: a pan-tilt, a focusing lens, a data processing center, a laser, and a control terminal; wherein, the focusing lens is installed on the pan-tilt;

[0010] Among them, the focusing lens includes a control main board, a zoom drive module, a lidar, a visible light camera, and a focusing lens group;

[0011] Among them, the lidar and the visible light camera are respectively used to capture the current image data and the current point cloud data of the target obstacle in the transmission line at the same timestamp, and send them to the data processing center for processing;

[0012] The data processing center executes the laser obstacle clearing tracking and aiming method according to any embodiment of the present invention;

[0013] The control terminal is used to receive the target position data processed by the data processing center, and according to the target position data and a preset PID control algorithm, control the control main board in the focusing lens to feedback a zoom signal to the laser;

[0014] The laser is used to track and aim at the target obstacle, and output a corresponding modulated laser for focused emission.

[0015] According to another aspect of the present invention, an embodiment of the present invention further provides an electronic device, and the electronic device includes:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the laser obstacle clearing tracking and aiming method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, and the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the laser obstacle clearing tracking and aiming method according to any embodiment of the present invention when executed.

[0020] According to another aspect of the present invention, an embodiment of the present invention further provides a computer program product, characterized in that the computer program product includes a computer program, and the computer program realizes the laser obstacle clearing tracking and aiming method according to any embodiment of the present invention when executed by a processor.

[0021] The technical effect of the present invention is that by obtaining the current image data of the target obstacle in the transmission line captured by at least two sensors in the focusing lens at the same time stamp, as well as the current point cloud data, mapping the target feature points in the current image data to the three-dimensional space where the current point cloud data is located, obtaining the mapped target point cloud data, and adopting the multi-sensor fusion method, it is possible to utilize the detailed information of the image and the spatial information of the point cloud; by taking each coordinate point in the target point cloud data as the center point, determining the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to each center point, and adopting three-dimensional feature extraction, the limitations of traditional image features on targets with less texture are overcome, and the tracking ability for targets with less obvious features is enhanced; by presetting a dynamic feedback control mechanism to perform feature tracking on the local geometric features, adjusting the focal position of the laser to perform tracking and aiming at the target obstacle to obtain the corresponding aiming trajectory, the tracking accuracy can be improved, the robustness can be enhanced, precise positioning and tracking of dynamic targets can be achieved, and the degree of automation is high, and it can adapt to complex environments.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 It is a flowchart of a laser obstacle clearing tracking and aiming method provided by an embodiment of the present invention;

[0025] Figure 2 It is a flowchart of another laser obstacle clearing tracking and aiming method provided by an embodiment of the present invention;

[0026] Figure 3 It is a schematic flowchart of yet another laser obstacle clearing tracking and aiming method provided by an embodiment of the present invention;

[0027] Figure 4 It is a structural block diagram of a laser obstacle clearing tracking and aiming device provided by an embodiment of the present invention;

[0028] Figure 5 It is a structural block diagram of another laser obstacle clearing tracking and aiming device provided by an embodiment of the present invention;

[0029] Figure 6Schematic diagram of a focusing lens provided by an embodiment of the present invention;

[0030] Figure 7 Schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0031] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0033] In one embodiment, Figure 1 Flowchart of a laser obstacle clearing, tracking and aiming method provided by an embodiment of the present invention. This embodiment is applicable to the situation of using a laser to track and aim at a target object for obstacle clearing, and this method can be executed by a laser obstacle clearing, tracking and aiming device.

[0034] As Figure 1 shown, the specific steps of the laser obstacle clearing, tracking and aiming method in this embodiment include:

[0035] S110. Obtain the current image data and the current point cloud data of the target obstacle in the transmission line captured by at least two sensors in the focusing lens at the same timestamp.

[0036] Among them, the sensors may include: a visible light camera and a lidar, which are respectively used to capture the image data and the point cloud data of the target obstacle at the same timestamp. The target obstacle may include, but is not limited to, trees (including tree branches and trunks), ice and snow, frost, and fallen kites in the transmission line.

[0037] In this embodiment, the current image data is the image data of the target obstacle in the transmission line captured by using the first type of sensor (for example, it can be a visible light camera), and the image data may include the feature points of the target obstacle. The current point cloud data is the point cloud data of the target obstacle in the transmission line captured by using the second type of sensor (for example, it can be a lidar), and the point cloud data can be used to represent the three-dimensional data set of the target obstacle. Each point in the three-dimensional data set represents a position in space, and the position can be characterized in the form of three-dimensional coordinates.

[0038] In this embodiment, at least two sensors (for example, a visible light camera and a lidar) are in the focusing lens of the laser obstacle removal tracking and aiming device. At the same timestamp, the visible light camera is used to obtain the visible light image including the target obstacle in the transmission line, and the three-dimensional lidar is used to synchronously obtain the three-dimensional point cloud data of the target obstacle in the transmission line. Then, the obtained visible light image and three-dimensional point cloud data are respectively sent to the data processing center, and the data processing center receives the current image data of the target obstacle in the transmission line captured at the same timestamp, as well as the current point cloud data. In this embodiment, since it is at the same timestamp, it can ensure that the timestamps of the visible light image and the three-dimensional point cloud data are consistent, that is, to ensure the synchronization of the visible light image and the three-dimensional point cloud data in the time dimension for accurate registration and fusion.

[0039] S120. Map the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain the mapped target point cloud data.

[0040] Among them, the target feature points can be understood as the feature points of the obstacle, and the feature points can characterize the significant features of the obstacle, which can be used for subsequent identification, matching, or tracking of the target, etc. The target point cloud data refers to the point cloud data with fused features obtained after mapping, which can be understood as fusing the target feature points in the current image data with the current point cloud data in the three-dimensional space.

[0041] In this embodiment, the external parameter matrix between the visible light camera and the lidar can be calibrated by using a calibration board. The calibration result of the external parameter matrix can characterize the coordinate transformation relationship between the two. Thus, through the coordinate transformation relationship between the visible light camera and the lidar, the target feature points in the current image data can be mapped to the three-dimensional space where the current point cloud data is located; it can be understood that the transformation matrix from the camera coordinate system to the lidar coordinate system is found, including the rotation matrix and the translation vector. Through calibration, the lidar point cloud can be projected onto the image coordinate system, or the image feature points can be mapped to the point cloud coordinate system.

[0042] S130. Taking each coordinate point in the target point cloud data as the center point, determine the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to each center point.

[0043] Among them, the local feature point set includes multiple feature points, and these multiple feature points can form local geometric features. The local geometric features can also be called topological features, which can be understood as the geometric properties of the center point and its neighborhood.

[0044] In this embodiment, since the target point cloud data obtained by the mapping includes multiple feature points after fusion, and the coordinate points corresponding to each feature point, for each coordinate point in the target point cloud data, respectively taking the coordinate point as the center point, search for the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to the center point. In some embodiments, a preset neighborhood range corresponding to the center point can be defined, and the three-dimensional feature descriptor method is used to determine the neighborhood feature points within the neighborhood range corresponding to the center point, and the corresponding local geometric features; in this embodiment, the three-dimensional feature descriptor method can describe the mathematical representation of the geometric and topological characteristics of a certain point and its neighborhood in the three-dimensional point cloud, and can distinguish different local structures. In some embodiments, the local geometric features within the neighborhood range corresponding to the center point can be determined through various methods such as adaptive neighborhood, K-nearest neighbor, and adaptive neighborhood, and then the local geometric features are extracted through distribution-based features, statistics-based features, and deep learning-based features. In some embodiments, the local geometric features within the neighborhood range corresponding to the center point can also be determined by other methods, and this embodiment does not limit this here.

[0045] S140. Based on a preset dynamic feedback control mechanism, perform feature tracking on the local geometric features to adjust the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory.

[0046] Among them, the preset dynamic feedback control mechanism can be understood as a control method that makes the output reach the expected value by adjusting the input in real time, that is, dynamically adjusting the control input in real time through a feedback loop.

[0047] In this embodiment, the feature tracking strategy is adjusted in real time through a dynamic feedback mechanism to cope with changes in the position, shape, or environment of feature points. The local geometric features can be tracked according to the Iterative Closest Point (ICP) algorithm to obtain target position data, and then the target position data is transmitted to the control terminal, so that the control terminal controls the control main board in the focusing lens to feedback the zoom signal to the laser according to the target position data and the preset PID control algorithm, thereby adjusting the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory, so as to achieve the realization of feedback control; in other embodiments, the PointNet++ model based on deep learning can also be used to track the changes of target features in real time, and the target position data obtained by real-time tracking is transmitted to the control terminal, so that the control terminal controls the control main board in the focusing lens to feedback the zoom signal to the laser according to the target position data and the preset PID control algorithm, thereby adjusting the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory. This embodiment is not limited here.

[0048] The technical solution of the embodiment of the present invention obtains the current image data and the current point cloud data of the target obstacle in the transmission line captured by at least two sensors at the same timestamp respectively, maps the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain the mapped target point cloud data, and adopts the multi-sensor fusion method to be able to utilize the detailed information of the image and the spatial information of the point cloud; by taking each coordinate point in the target point cloud data as the center point, the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to each center point are determined, and three-dimensional feature extraction is adopted, which overcomes the limitations of traditional image features on targets with less texture and enhances the tracking ability of targets with unclear features; by presetting a dynamic feedback control mechanism to perform feature tracking on local geometric features to adjust the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory, the tracking accuracy can be improved, the robustness can be enhanced, the precise positioning and tracking of dynamic targets can be realized, and the degree of automation is high, and it can adapt to complex environments.

[0049] In one embodiment, after adjusting the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory, it further includes:

[0050] The Kalman filtering method is used to smooth the aiming trajectory to obtain the processed target trajectory, so that the laser clears the obstacle according to the target trajectory.

[0051] Among them, the Kalman filtering method is a filtering method in the prior art, which can adjust data with large fluctuations into smooth data, and will not be specifically described in this embodiment.

[0052] In this embodiment, after adjusting the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory, the Kalman filtering method is used to smooth the aiming trajectory to obtain the processed target trajectory, which is a smoother and more accurate trajectory, so that the laser can clear obstacles according to the target trajectory. Thus, noise interference can be reduced and various environmental changes can be coped with.

[0053] In one embodiment, Figure 2 FIG. 6 is a flowchart of another laser obstacle clearing tracking and aiming method provided by an embodiment of the present invention. On the basis of the above embodiments, this embodiment further refines mapping the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain the mapped target point cloud data; determining the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to each center point with each coordinate point in the target point cloud data as the center point; and performing feature tracking on the local geometric features based on a preset dynamic feedback control mechanism to adjust the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory.

[0054] As Figure 2 shown, the laser obstacle clearing tracking and aiming method in this embodiment may specifically include the following steps:

[0055] S210. Obtain the current image data and the current point cloud data of the target obstacle in the transmission line captured by at least two sensors in the focusing lens at the same time stamp.

[0056] S220. Determine the coordinate mapping relationship between the visible light camera and the lidar.

[0057] In this embodiment, determining the coordinate mapping relationship between the visible light camera and the lidar can be understood as performing external parameter calibration to find the transformation matrix between the camera coordinate system and the lidar coordinate system, including the rotation matrix and the translation vector. Among them, the rotation matrix describes the rotation of the camera coordinate system relative to the lidar coordinate system. The translation vector describes the translation of the camera coordinate system relative to the lidar coordinate system. Through calibration, the image feature points can be mapped to the point cloud coordinate system.

[0058] In one embodiment, the determination of the coordinate mapping relationship between the visible light camera and the lidar includes: respectively using the visible light camera and the lidar to capture the image and the three-dimensional point cloud of the calibration board at different perspectives; for the image and the three-dimensional point cloud obtained at each different perspective, respectively matching the corner points in the image at different perspectives with the corresponding corner points in the three-dimensional point cloud to obtain the corresponding matching results; and performing external parameter calibration of the visible light camera and the lidar according to the matching results to obtain the calibration results.

[0059] Among them, the calibration result characterizes the coordinate mapping relationship between the camera and the radar; among them, the external parameters include: a rotation matrix and a translation vector.

[0060] In this embodiment, the calibration board is a calibration board with a known geometric shape. The size and corner positions of the calibration board are known. When calibrating the external parameters of the visible light camera and the lidar, the calibration board is used for multi-view shooting, and the external parameter matrix is calculated to obtain the corresponding calibration result, which is used to accurately map the image coordinates and the point cloud coordinates.

[0061] In this embodiment, the visible light camera and the lidar are respectively used to capture images and 3D point clouds of the calibration board from different perspectives. For the images and 3D point clouds obtained from each different perspective, the corner points in the images from different perspectives are respectively matched with the corresponding corner points in the 3D point clouds to obtain the corresponding matching results. Then, based on the matching results, the external parameters of the visible light camera and the lidar are calibrated to obtain the calibration result, and further, the image feature points are mapped to the point cloud coordinate system. Specifically, the calibration is performed by minimizing the error of the corner points. In addition, a non-linear optimization method (such as Levenberg-Marquardt) can be used to further optimize the external parameter matrix.

[0062] S230. Select the target feature points corresponding to the target obstacle in the current image data.

[0063] In this embodiment, for the current image data captured by the visible light camera, the target feature points corresponding to the target obstacle are selected therefrom. It can be understood that the user can select the target feature points of the target obstacle in the current image through the interface. Exemplarily, the target feature points can be certain specific positions on the tree trunk.

[0064] S240. Map the target feature points to the point cloud space where the current point cloud data is located according to the coordinate mapping relationship to obtain the mapped target point cloud data.

[0065] In this embodiment, through the coordinate mapping relationship between the camera and the radar, the target feature points are mapped to the point cloud space where the current point cloud data is located to obtain the mapped target point cloud data. It can be understood that the target feature points are selected in the visible light image, and the calibrated external parameters are used to convert the selected image coordinates into the spatial coordinates in the 3D point cloud.

[0066] S250. Take each coordinate point in the target point cloud data as the center point, and for each center point, define a preset neighborhood range corresponding to the center point; where the neighborhood range of the center point includes neighborhood feature points.

[0067] In this embodiment, for each coordinate point in the target point cloud data, each coordinate point in the target point cloud data is respectively used as the center point. Thus, for each center point, a preset neighborhood range corresponding to the center point is defined for extracting local features or performing local analysis. The preset neighborhood range is within a certain range corresponding to the center point, that is, taking a certain center point as the center, the points within a certain range around it are extracted. Specifically, a fixed-radius neighborhood can be used: taking the center point as the center of the sphere, all points within the fixed radius are extracted as the preset neighborhood range corresponding to the center point; alternatively, the neighborhood can be defined by the K-nearest neighbor method: the K points closest to the center point are extracted.

[0068] S260. Use the three-dimensional feature descriptor method to determine the neighborhood feature points within the neighborhood range corresponding to the center point, form the neighborhood feature points into a local feature point set, and determine the local feature point set and the corresponding local geometric features.

[0069] Among them, the three-dimensional feature descriptor method includes one of the following: local feature descriptor (Signature of Histograms of OrienTations, SHOT), fast point feature histograms (Fast Point Feature Histograms, FPFH). Among them, FPFH is an efficient three-dimensional point cloud local feature descriptor. FPFH generates a feature descriptor by describing the geometric relationship of a certain point in the point cloud and its neighborhood. SHOT is a three-dimensional point cloud local feature descriptor based on the local reference frame (LRF). It generates a feature descriptor by describing the geometric distribution of a certain point in the point cloud and its neighborhood, and has rotational invariance and strong discriminability. In this embodiment, in addition to the above three-dimensional feature descriptor methods, descriptors such as point feature histograms (Point Feature Histograms, PFH) and viewpoint feature histogram VFH (Viewpoint Feature Histogram) can also be used, and this embodiment does not limit them here.

[0070] In this embodiment, a three-dimensional feature descriptor method can be adopted to determine the neighborhood feature points within the neighborhood range corresponding to the center point, form the local feature points into a local feature point set, and determine the local feature point set and the corresponding local geometric features. Specifically, when the three-dimensional feature descriptor method is FPFH, for the center point and its neighborhood points, geometric features such as the normal vector angle, projection distance, and angle feature are calculated, and the above features are quantified into a histogram to obtain the Simplified PFH (SPFH). Then, by weighting the PFH (SPFH) of the neighborhood points, the weighted SPFH values are quantified into the FPFH descriptor. When the three-dimensional feature descriptor method is SHOT, a local coordinate system is constructed for each point to ensure that the descriptor is invariant to rotation and translation. A local coordinate system is constructed for each point to ensure that the descriptor is invariant to rotation and translation.

[0071] S270. Track the local geometric features according to the Iterative Closest Point (ICP) algorithm to obtain the target position data.

[0072] Among them, the target position data includes: coordinate data and pose data.

[0073] In this embodiment, the Iterative Closest Point (ICP) algorithm can be iteratively optimized to gradually align the source point cloud to the target point cloud. It can be understood that by matching the current local geometric features corresponding to the current moment with the previous local geometric features corresponding to the previous moment, the coordinate data and pose data of the corresponding target feature points are obtained through tracking according to the matching result.

[0074] In one embodiment, tracking the local geometric features according to the Iterative Closest Point (ICP) algorithm to obtain the target position data includes:

[0075] Obtain the previous local geometric features corresponding to the target obstacle from the data repository;

[0076] Take the local geometric features as the current local geometric features, and match the current local geometric features with the previous local geometric features to obtain a feature matching result; among them, the feature matching result characterizes the change situation of the features;

[0077] When the feature matching result indicates that the change situation of the features does not exceed the change threshold, it is determined that the current local geometric features and the previous local geometric features have not changed;

[0078] When the feature matching result indicates that the change situation of the features exceeds the change threshold, it is determined that the current local geometric features and the previous local geometric features have changed. Update the previous local geometric features to the current local geometric features, and use the updated current local geometric features as the target position data.

[0079] Among them, the previous local geometric feature can be understood as the local geometric feature at the previous moment.

[0080] In this embodiment, the previous local geometric feature corresponding to the target obstacle is obtained from the data repository, the local geometric feature is used as the current local geometric feature, and the current local geometric feature and the previous local geometric feature are matched to obtain a feature matching result; among them, the feature matching result characterizes the change situation of the feature. When the change situation of the feature in the feature matching result does not exceed the change threshold, it is determined that the current local geometric feature and the previous local geometric feature have not changed. When the change situation of the feature in the feature matching result exceeds the change threshold, it is determined that the current local geometric feature and the previous local geometric feature have changed, the previous local geometric feature is updated to the current local geometric feature, and the updated current local geometric feature is used as the target position data.

[0081] S280. Transmit the target position data to the control terminal, so that the control terminal controls the control main board in the focusing lens to feedback a zoom signal to the laser according to the target position data and the preset PID control algorithm, thereby adjusting the focal position of the laser to perform tracking and aiming at the target obstacle to obtain a corresponding aiming trajectory.

[0082] Among them, the preset PID control algorithm can achieve precise control of the pan-tilt, focusing lens and laser through the combination of three links: proportional, integral and derivative.

[0083] In this embodiment, the target position data is transmitted to the control terminal, so that the control terminal controls the control main board in the focusing lens to feedback a zoom signal to the laser according to the target position data and the preset PID control algorithm, thereby adjusting the focal position of the laser to perform tracking and aiming at the target obstacle to obtain a corresponding aiming trajectory.

[0084] In this embodiment, the control terminal adjusts the focal position of the laser to perform tracking and aiming at the target obstacle according to the target position data and the preset PID control algorithm to obtain a corresponding aiming trajectory, including: determining the current position of the focusing lens on the pan-tilt; among them, the current position at least includes: the current angle; calculating the error between the target position data and the current position data, calculating the control output of the PID controller according to the error, and adjusting the pitch angle and yaw angle of the pan-tilt according to the control output, so that the focusing lens zooms, and driving the laser to output a corresponding modulation current for focused emission, and looping through the above steps to track the target in real time and adjust the pan-tilt angle.

[0085] The technical solution of the embodiment of the present invention determines the coordinate mapping relationship between the visible light camera and the lidar, maps the target feature points into the point cloud space where the current point cloud data is located according to the coordinate mapping relationship to obtain the mapped target point cloud data, and adopts a multi-sensor fusion method to utilize the detailed information of the image and the spatial information of the point cloud; taking each coordinate point in the target point cloud data as the center point, for each center point, a preset neighborhood range corresponding to the center point is defined, and the neighborhood feature points within the neighborhood range corresponding to the center point are determined by using the three-dimensional feature descriptor method, and the neighborhood feature points are combined into a local feature point set, and the local feature point set and the corresponding local geometric features are determined. By using three-dimensional feature extraction, it overcomes the limitations of traditional image features on targets with less texture and enhances the tracking ability for targets with unclear features; the local geometric features are tracked according to the Iterative Closest Point (ICP) algorithm to obtain the target position data, and the target position data is transmitted to the control terminal, so that the control terminal controls the control main board in the focusing lens to feedback the zoom signal to the laser according to the target position data and the preset PID control algorithm, thereby adjusting the focal position of the laser to track and aim at the target obstacle to obtain the corresponding aiming trajectory, which can improve the tracking accuracy, enhance the robustness, realize the precise positioning and tracking of dynamic targets, and has a high degree of automation and can adapt to complex environments.

[0086] Exemplarily, for better understanding of the laser obstacle clearing tracking and aiming method, Figure 3 FIG. is a schematic flow chart of another laser obstacle clearing tracking and aiming method provided by an embodiment of the present invention. As Figure 3 shown, the specific flow of the laser obstacle clearing tracking and aiming method is as follows:

[0087] a1. Data acquisition and synchronization.

[0088] Use the visible light camera to acquire visible light images, and use the 3D lidar to synchronously acquire the 3D point cloud data corresponding to the visible light images, ensure that the timestamps of the visible light images and the 3D point cloud data are consistent, and send the acquired visible light images and 3D point cloud data to the pan-tilt control system; by fusing the visible light images and the 3D point cloud data, the advantages of the two sensors are comprehensively utilized to improve the accuracy of target positioning and tracking.

[0089] a2. Sensor calibration and data registration.

[0090] Calibration content: Perform external parameter calibration of the camera and the LiDAR to obtain the coordinate conversion relationship between the two.

[0091] Method: Use a calibration board to take multi-view images and calculate the external parameter matrix. The calibration result is used to accurately map the image coordinates and the point cloud coordinates.

[0092] In this embodiment, the extrinsic calibration between the visible light camera and the 3D lidar is performed to obtain the coordinate transformation relationship between the visible light camera and the 3D lidar;

[0093] a3. Target selection and mapping

[0094] In the visible light image: The operator selects a target point (such as a specific position on the tree trunk) through the interface.

[0095] Coordinate transformation: Using the calibrated extrinsic parameters, the selected image coordinates are transformed into spatial coordinates in the 3D point cloud.

[0096] Target mapping and feature extraction: Select a target point in the visible light image, and using the sensor calibration data, map this point to the 3D point cloud and extract the surrounding 3D features.

[0097] In this embodiment, a target point is selected in the visible light image, and using the calibrated extrinsic parameters, the selected image coordinates are transformed into spatial coordinates in the 3D point cloud.

[0098] a4. Topological feature extraction within the neighborhood range of the center point

[0099] Region selection: Taking the mapped spatial coordinates as the center, define a certain neighborhood range.

[0100] Feature description: Use 3D feature descriptors (such as FPFH, SHOT) to extract the geometric and topological features within this neighborhood.

[0101] In this embodiment, taking the mapped spatial coordinates as the center, define a certain neighborhood range, and use 3D feature descriptors to extract the geometric and topological features within this neighborhood.

[0102] a5. Target tracking

[0103] Algorithm selection: Use the improved ICP algorithm or the deep learning-based PointNet++ model to track the changes of target features in real time.

[0104] Data update: The continuously acquired point cloud data is used to update the position and pose of the target.

[0105] In this embodiment, use the improved ICP algorithm or the deep learning-based PointNet++ model to track the changes of target features in real time, and the continuously acquired point cloud data is used to update the position and pose of the target;

[0106] a6. Pan-tilt control and laser aiming

[0107] Control strategy: Transmit the real-time tracked target position data to the pan-tilt control. Through the PID control algorithm, adjust the rotation of the pan-tilt to achieve the precise aiming of the laser cannon at the target.

[0108] Automatic control pan-tilt: Feed back the tracking result to the pan-tilt control to achieve automatic aiming of the laser cannon at dynamic targets.

[0109] In this embodiment, the target position data obtained from real-time tracking is transmitted to the pan-tilt control system. Through the PID control algorithm, the rotation of the pan-tilt is adjusted to achieve precise aiming of the laser cannon at the target.

[0110] a7. Smoothing processing of the trajectory

[0111] Filtering processing: Use Kalman filtering to smooth the target trajectory and reduce noise interference.

[0112] Robustness enhancement: In response to environmental changes, add an adaptive parameter adjustment mechanism to improve the stability of the system under different conditions.

[0113] Use Kalman filtering to smooth the target trajectory and reduce noise interference. In response to environmental changes, add an adaptive parameter adjustment mechanism to improve the stability under different conditions.

[0114] In one embodiment, Figure 4 This is the structural block diagram of a laser obstacle clearing, tracking and aiming device provided by an embodiment of the present invention. As Figure 4 shown, the device includes a pan-tilt 410, a focusing lens 420, a data processing center 430, a laser 440, and a control terminal 450; among them, the focusing lens 420 is installed on the pan-tilt 410;

[0115] Among them, the focusing lens 420 includes a control main board, a zoom drive module, a lidar, a visible light camera, and a focusing lens group;

[0116] Among them, the lidar and the visible light camera are respectively used to capture the current image data and the current point cloud data of the target obstacle in the transmission line at the same timestamp, and send them to the data processing center for processing;

[0117] The data processing center 430 executes the laser obstacle clearing, tracking and aiming method as described in any one of the embodiments of the present invention;

[0118] The control terminal 450 is used to receive the target position data processed by the data processing center, and according to the target position data and a preset PID control algorithm, control the control main board in the focusing lens to feedback a zoom signal to the laser;

[0119] The laser 440 is used to track and aim at the target obstacle and output corresponding modulated laser for focused emission.

[0120] In one embodiment, the laser obstacle clearing tracking and aiming system further includes: a power supply;

[0121] Wherein, the power supply is connected to the laser through a power cord and is used to supply power to the laser obstacle clearing tracking and aiming system.

[0122] In one embodiment, the optical axes of the focusing lens group, the lidar, and the visible light camera are parallel.

[0123] In one embodiment, the pan-tilt is a two-axis pan-tilt.

[0124] In one embodiment, the focusing lens is connected to the data processing center; the focusing lens is also connected to the laser; the laser is connected to the power supply and the control terminal.

[0125] The laser obstacle clearing tracking and aiming device provided by the embodiment of the present invention can execute the laser obstacle clearing tracking and aiming method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0126] For better understanding of the laser obstacle clearing tracking and aiming device, Figure 5 FIG. is a structural block diagram of another laser obstacle clearing tracking and aiming device provided by an embodiment of the present invention, Figure 6 FIG. is a schematic structural diagram of a focusing lens provided by an embodiment of the present invention.

[0127] As Figure 5 shown, the device includes: a focusing lens 2 mounted on a pan-tilt 1, a data processing center 3 connected to the focusing lens 2, a laser 4 connected to the focusing lens 2, a power supply 5 connected to the laser 4, and a control terminal 6. In addition, Figure 5 7 in represents the trunk in the power transmission line; 8 represents the focused light emitted.

[0128] As Figure 6 shown, the focusing lens 2 includes a control main board 21, a zoom drive module 22, a lidar 23, a camera 24, and a focusing lens group 25. The optical axes of the focusing lens group 25, the lidar 23, and the camera 24 are parallel; the image data obtained by the camera 24 and the three-dimensional point cloud data obtained by the lidar 23 are transmitted to the data processing center 3 for processing and sent to the control terminal 6 and the control main board 21 of the focusing lens 2.

[0129] In one embodiment, Figure 7A schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0130] As Figure 7 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0132] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the laser obstacle clearing tracking aiming method.

[0133] In some embodiments, the laser obstacle clearing tracking and aiming method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the laser obstacle clearing tracking and aiming method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the laser obstacle clearing tracking and aiming method by any other suitable means (e.g., by means of firmware).

[0134] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable laser obstacle clearing tracking and aiming device, such that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0138] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0140] It should be understood that various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0141] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0142] It should be understood that various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0143] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A laser obstacle removal tracking and aiming method, characterized in that: Applied to a laser obstacle removal tracking and aiming device, the method comprises: Acquire current image data of a target obstacle in the power transmission line captured by at least two sensors in the focusing lens at the same time stamp, and current point cloud data; Mapping the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain mapped target point cloud data; Taking each coordinate point in the target point cloud data as a center point, determining a local feature point set and a corresponding local geometric feature within a neighborhood range corresponding to each center point; The local geometric features are tracked based on a preset dynamic feedback control mechanism to adjust the focal position of the laser to track and aim at the target obstacle to obtain a corresponding aiming trajectory.

2. The method according to claim 1, characterized in that After adjusting the focal position of the laser to track and aim at the target obstacle to obtain a corresponding aiming trajectory, the method further includes: The aiming trajectory is smoothed by using a Kalman filter to obtain a processed target trajectory, so that the laser can clear obstacles according to the target trajectory.

3. The method according to claim 1, characterized in that The at least two sensors include: a visible light camera and a laser radar; accordingly, mapping the target feature points in the current image data to the three-dimensional space where the current point cloud data is located to obtain the mapped target point cloud data includes: Determine a coordinate mapping relationship between the visible light camera and the laser radar; Selecting a target feature point corresponding to the target obstacle in the current image data; The target feature points are mapped to the point cloud space where the current point cloud data is located according to the coordinate mapping relationship to obtain mapped target point cloud data.

4. The method according to claim 3, characterized in that: The determination of the coordinate mapping relationship between the visible light camera and the laser radar includes: Use visible light camera and lidar to capture images and 3D point clouds of the calibration plate at different viewing angles; For each image and three-dimensional point cloud obtained at different viewing angles, the corner points in the image at different viewing angles and the corresponding corner points in the three-dimensional point cloud are matched to obtain corresponding matching results; The visible light camera and the lidar are calibrated with external parameters according to the matching result to obtain a calibration result; wherein the calibration result represents the coordinate mapping relationship between the camera and the lidar; wherein the external parameters include: a rotation matrix and a translation vector.

5. The method according to claim 1, characterized in that The determining of the local feature point set and the corresponding local geometric features within the neighborhood range corresponding to each of the central points includes: For each center point, a preset neighborhood range corresponding to the center point is defined; wherein the neighborhood range of the center point includes neighborhood feature points; A three-dimensional feature descriptor method is used to determine neighborhood feature points within a neighborhood range corresponding to the center point, each of the neighborhood feature points is formed into a local feature point set, and the local feature point set is determined; and the corresponding local geometric features.

6. The method according to claim 5, characterized in that The three-dimensional feature descriptor method includes one of the following: a local feature descriptor SHOT, a fast point feature histogram FPFH.

7. The method according to claim 1, characterized in that The tracking of the local geometric features based on the preset dynamic feedback control mechanism to adjust the focal position of the laser to track and aim at the target obstacle to obtain a corresponding aiming trajectory includes: Tracking the local geometric features according to an iterative closest point IPC algorithm to obtain target position data; The target position data is transmitted to a control terminal, so that the control terminal controls the control mainboard in the focusing lens to feed back a zoom signal to the laser according to the target position data and a preset PID control algorithm, thereby adjusting the focal position of the laser to track and aim at the target obstacle to obtain a corresponding aiming trajectory; Wherein, the target position data includes: coordinate data and posture data.

8. The method according to claim 7, characterized in that The tracking of the local geometric features according to the iterative closest point IPC algorithm to obtain target position data includes: Acquire the last local geometric feature corresponding to the target obstacle from the data repository; The local geometric feature is used as the current local geometric feature, and the current local geometric feature is matched with the previous local geometric feature to obtain a feature matching result; wherein the feature matching result represents a change in the feature; When the feature matching result is that the change of the feature does not exceed the change threshold, determining that the current local geometric feature and the previous local geometric feature have not changed; When the feature matching result is that the change of the feature exceeds the change threshold, it is determined that the current local geometric feature and the previous local geometric feature have changed, the previous local geometric feature is updated to the current local geometric feature, and the updated current local geometric feature is used as the target position data.

9. A laser obstacle removal tracking and aiming device, characterized in that: The laser obstacle removal tracking and aiming device comprises: a pan-tilt platform, a focusing lens, a data processing center, a laser and a control terminal; wherein the focusing lens is mounted on the pan-tilt platform; Among them, the focusing lens includes a control mainboard, a zoom drive module, a laser radar, a visible light camera and a focusing lens group; The laser radar and the visible light camera are respectively used to capture the current image data of the target obstacle in the transmission line and the current point cloud data at the same timestamp, and send them to the data processing center for processing; The data processing center executes the laser obstacle removal tracking and aiming method as described in any one of claims 1 to 8; The control terminal is used to receive the target position data processed by the data processing center, and control the control mainboard in the focusing lens to feed back the zoom signal to the laser according to the target position data and a preset PID control algorithm; The laser is used to track and aim at the target obstacle, and output corresponding modulated laser for focused emission.

10. The device according to claim 9, characterized in that The optical axes of the focusing lens group, the laser radar and the visible light camera are parallel.

11. The device according to claim 9, characterized in that The gimbal is a two-axis gimbal.

12. The device according to claim 9, characterized in that The focusing lens is connected to the data processing center; the focusing lens is also connected to the laser; the laser is connected to a power source and the control terminal.

Images