A transmission line distance measurement method, device, equipment and storage medium
The binocular image is obtained by a binocular camera and the distance measurement disparity value is calculated, which solves the problem of difficult to balance real-time and accuracy in the distance measurement method of transmission line, and improves the distance measurement accuracy and real-timeness of machinery around the transmission line. Provides guarantees for the safety of transmission lines.
Patent Information
- Application Number
- CN202310766921.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-06-26
AI Technical Summary
The existing transmission line ranging method is difficult to balance the real-time and accuracy of ranging, resulting in the inability to effectively prevent safety accidents when construction machinery comes into contact with the transmission line.
A binocular camera is used to obtain binocular images, determine the characteristic point pair of the target object through the left eye image and the right eye image, calculate the distance of the target object, and combine the target parameters of the binocular camera to accurately determine the distance of the target object.
It improves the accuracy and real-timeness of large-scale mechanical distance measurement around the transmission line, and enhances the guarantee of safety of transmission line.
Smart Images

Figure CN116819229B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a distance measurement method, device, equipment and storage medium for a power transmission line. Background Art
[0002] Transmission lines play an important role in power transportation in power grids. In recent years, my country has accelerated the promotion of urban construction, and the area of transmission corridors has gradually been compressed by nearby construction projects, resulting in large construction machinery such as cranes or booms illegally touching the lines, which in turn leads to safety accidents such as line disconnection, tripping or short circuit of transmission lines, affecting the safe and stable operation of the power system. Therefore, the distance measurement of construction machinery near transmission lines has become the main content of preventing external force damage.
[0003] At present, the use of convolutional neural network-based target detection models and ranging algorithms for the ranging of large-scale machinery on power transmission lines can improve ranging accuracy and detection efficiency. However, the use of lightweight algorithm models cannot guarantee ranging accuracy, and high-precision large-scale algorithm models cannot meet real-time requirements, making it difficult to achieve a balance between real-time and accuracy in transmission line ranging. Summary of the invention
[0004] The present invention provides a distance measurement method, device, equipment and storage medium for a power transmission line, so as to improve the accuracy and real-time performance of distance detection of the power transmission line.
[0005] According to one aspect of the present invention, a method for measuring distance of a power transmission line is provided, the method comprising:
[0006] Obtain a set of binocular images taken by a binocular camera for a target power transmission line, and determine a target object in the binocular image according to a left-eye image and a right-eye image; wherein the binocular image includes a left-eye image and a right-eye image;
[0007] Determine a plurality of feature point pairs corresponding to the target object based on the left image and the right image, wherein two feature pixel points in the feature point pairs correspond to the same position of the target object;
[0008] The ranging disparity value corresponding to the target object is determined based on the plurality of feature point pairs, and the actual distance of the target object from the binocular camera is determined according to the feature point pairs, the target parameters of the binocular camera and the ranging disparity value.
[0009] According to another aspect of the present invention, there is provided a distance measuring device for a power transmission line, the device comprising:
[0010] A target object determination module is used to obtain a set of binocular images taken by a binocular camera for a target power transmission line, and determine the target object in the binocular image according to the left image and the right image; wherein the binocular image includes the left image and the right image;
[0011] A feature point pair determination module is used to determine a plurality of feature point pairs corresponding to the target object based on the left-eye image and the right-eye image, wherein two feature pixel points in the feature point pair correspond to the same position of the target object;
[0012] The ranging module is used to determine the ranging disparity value corresponding to the target object based on multiple feature point pairs, and determine the actual distance of the target object from the binocular camera according to the feature point pairs, the target parameters of the binocular camera and the ranging disparity value.
[0013] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:
[0014] at least one processor; and
[0015] a memory communicatively connected to at least one processor; wherein,
[0016] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the transmission line distance measurement method of any embodiment of the present invention.
[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transmission line distance measurement method of any embodiment of the present invention when executed.
[0018] The technical solution of the embodiment of the present invention obtains a group of binocular images taken by a binocular camera for a target transmission line, and determines the target object in the binocular image according to the left image and the right image. Since the binocular image includes the left image and the right image, it can simulate the situation of real eye observation and capture more information about the transmission line. Based on the left image and the right image, multiple feature point pairs corresponding to the target object are determined, wherein two feature pixel points in the feature point pair correspond to the same position of the target object. Based on the same or similar feature content in the binocular image, the pixel points for ranging calculation are reduced to achieve efficient ranging. Based on multiple feature point pairs, the ranging parallax value corresponding to the target object is determined, and the actual distance of the target object from the binocular camera is determined according to the feature point pair, the target parameters of the binocular camera and the ranging parallax value. By calculating the parallax value of the feature point pair of the target object, the actual distance from the target object to the camera can be simply, quickly and effectively determined. The problem that the existing transmission line ranging method is difficult to balance the real-time and accuracy of transmission line ranging is solved, the accuracy and real-time of ranging of large machinery around the transmission line are improved, and the safety of the transmission line is guaranteed.
[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 is a flow chart of a transmission line distance measurement method provided according to an embodiment of the present invention;
[0022] Figure 2 is a flow chart of another transmission line distance measurement method provided according to an embodiment of the present invention;
[0023] Figure 3 is a flowchart of a specific transmission line distance measurement method provided according to an embodiment of the present invention;
[0024] Figure 4 is a structural block diagram of a distance measuring device for a power transmission line provided according to an embodiment of the present invention;
[0025] Figure 5 It is a structural block diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION
[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0027] It should be noted that the terms "first" and "second" and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0028] Figure 1 It is a flow chart of a transmission line distance measurement method provided according to an embodiment of the present invention. This embodiment can be applied to the scenario of performing risk distance measurement around a transmission line based on binocular images, and can be executed by a transmission line distance measurement device. The transmission line distance measurement device can be implemented in the form of hardware and / or software and configured in a processor of an electronic device.
[0029] like Figure 1 As shown, the distance measurement method of the transmission line includes the following steps:
[0030] S110, obtaining a group of binocular images taken by a binocular camera for a target power transmission line, and determining a target object in the binocular image according to a left image and a right image, wherein the binocular image includes a left image and a right image.
[0031] The target transmission line is a transmission line for which risk distance measurement around the transmission line is to be performed, for example, it may be a transmission line with hidden dangers of damage by external forces.
[0032] The target objects include equipment or tools that may cause damage to the target transmission line, for example, large machinery such as tower cranes, cranes, excavators, pump trucks or bulldozers.
[0033] It is understandable that when large-scale machinery is under construction near the target transmission line, the target transmission line may be damaged. However, the construction range of large-scale machinery is limited, and it is believed that large-scale machinery will not cause external force damage to the target transmission line outside its construction range. Therefore, in order to determine the actual distance from the target object to the binocular camera, it is necessary to determine whether the target object exists in the binocular image.
[0034] Specifically, in order to detect large machinery near the target transmission line, a binocular camera can be set near the target transmission line. Use the binocular camera to shoot a video of the target transmission line. Each frame of the binocular image of the video includes a left image and a right image. The left image is an image taken by the left camera of the binocular camera, and the right image is an image taken by the right camera of the binocular camera. Determine whether there is a target object in the binocular image through image classification, semantic segmentation, object detection or instance segmentation.
[0035] Furthermore, determining the target object in the binocular image according to the left-eye image and the right-eye image includes: inputting one of the left-eye image or the right-eye image into a pre-trained target detection model to determine the target object in the binocular image.
[0036] Specifically, in order to determine whether there is a target object in the binocular image, it is necessary to select one of the binocular images, input the one of the binocular images into a pre-trained target detection model, and determine the target area image corresponding to the target object.
[0037] Optionally, a target detection model is used to determine the type, number and location of target objects in one of the images, for example, it can be Faster R-CNN, SSD (Single Shot MultiBox Detector), single-shot detector (you only look once, YOLO) series models or NanoDet model.
[0038] In a specific embodiment, the specific structure of the target detection model may include a backbone network and a detection head, wherein the backbone network is used to extract the feature data of one of the images, input the feature data of the image into the detection head, output the type, number and position of the target object, and realize the positioning and classification of the target object. Further, between the backbone network and the detection head of the target detection model, a neck module is added to extract the complex features of the images at different stages, which can better fuse / extract the features of the backbone network, thereby improving the performance of the target detection model. In addition, the target detection model can also be optimized by network architecture optimization, batch normalization, adjusting the learning rate and optimizer to improve the efficiency and accuracy of model training. Among them, network architecture optimization can include strategies such as weight sharing, skip connection, application of convolutional layers, balancing depth and width. Batch normalization refers to a method for standardizing the output data of the neural network layer, which helps to accelerate the training speed of the model, improve the generalization ability of the model, and solve the problem of gradient disappearance. The optimizer is gradient-based and is used to update trainable parameters, including stochastic gradient descent (SGD), SGD with momentum, and adaptive moment estimation (Adam).
[0039] Exemplarily, the target detection model is the NanoDet model, which is an ultra-fast and lightweight mobile anchor-free target detection model. The model mainly consists of three components: the backbone network, the neck module, and the detection head. After one of the images is input into the model, the backbone network is first used to extract features and generate feature maps; then, the feature maps are multi-scale feature fused through the neck module, and the multi-scale features are directly added to obtain features with rich semantic information; finally, the detection head recognizes the features in pixels. Furthermore, the generalized focal loss (GFL) is used as the loss function, which can optimize continuous value targets, reduce the computational overhead of the detection head, and can converge quickly on lightweight models, facilitating training and transplantation.
[0040] In a specific embodiment, first, images of various types of target objects are used as samples, and the types of target objects and their positions in the images are used as labels to train the target detection model to obtain a trained target detection model; a single image is input into the trained target detection model, and features of the image are extracted through a backbone network; the feature data of the image is input into a neck module to extract complex features of the image at different stages; finally, the feature data and complex features of the image are input into a detection head, and the type, number and position of the target object are output to achieve the positioning and classification of the target object.
[0041] Exemplarily, first, images of large-scale construction machinery in various scenarios are obtained by photographing, collecting, and searching the Internet, and the above images are preprocessed to remove images with abnormal brightness, noise, and image blur, and the preprocessed images of various large-scale machinery are used as samples; the large-scale machinery in the image is annotated using an annotation tool, and saved as an xml tag file in the VOC data format; all samples are divided into a training set and a validation set in a 4:1 ratio, and the NanoDet model is trained using the samples of the training set and the corresponding tag files, and the NanoDet model is verified using the validation set; the NanoDet model is optimized based on an adaptive moment estimation (Adam) optimizer, and a preset number of rounds (for example, 100 rounds) are trained in total, and the learning rate of the first round is set to a preset learning rate (for example, 0.001). After each round of update is completed, the learning rate is multiplied by a preset ratio (for example, 90%) as the learning rate for the next round. At the same time, observe the change of the loss value of the validation set. When the loss value does not decrease for a set number of consecutive rounds (for example, 5), stop training and use the NanoDet model with the lowest loss value as the pre-trained target detection model. Secondly, input the left-eye image into the pre-trained NanoDet model, extract the features of the left-eye image, and generate a feature map; then, the feature map is multi-scale feature fused through the neck module, and the multi-scale features are directly added to obtain the final feature; finally, the detection head performs target recognition on the final feature in pixels, and obtains the length, width and center point coordinates of the rectangular box of the large machinery in the left-eye image, so as to realize the positioning and classification of the target object.
[0042] Optionally, you can also use existing pre-trained weights to train the target detection model, which can speed up model convergence.
[0043] Optionally, an attention mechanism may be introduced into the target detection model to select information to be focused on based on the attention mechanism. Optionally, the attention mechanism may be one of a vanilla attention mechanism, a coordinate attention mechanism, a self-attention mechanism, a hierarchical attention mechanism, or a multi-head attention mechanism.
[0044] S120: Determine a plurality of feature point pairs corresponding to the target object based on the left-eye image and the right-eye image.
[0045] The two feature pixel points in the feature point pair correspond to the same position of the target object.
[0046] It can be understood that in order to determine the actual distance from the target object to the binocular camera based on the binocular image, it is necessary to determine the pixel points corresponding to the same position of the target object in the left image and the right image respectively to form a feature point pair; based on the feature point pair, the actual distance from the target object in the binocular image to the binocular camera is determined.
[0047] Specifically, in the left image, all the pixels of the target object are determined, and each pixel is used as the first feature pixel; in the right image, the pixels at the same position corresponding to each first feature pixel are determined as the second feature pixel. Each first feature pixel and the second feature pixel at the same position corresponding to it form a feature point pair. Optionally, in the right image, all the pixels of the target object are determined, and each pixel is used as the first feature pixel; in the left image, the pixels at the same position corresponding to each first feature pixel are determined as the second feature pixel to obtain a feature point pair.
[0048] The advantage of doing this is that the points for which the actual distance needs to be determined are limited to the feature point pairs of the target object, which can reduce the amount of calculation in the next step of calculating the distance based on the left and right images. The actual distance is only calculated for the pixel points of the target object, thereby improving the speed and accuracy of risk distance measurement around the transmission line.
[0049] S130, determining a ranging disparity value corresponding to the target object based on the plurality of feature point pairs, and determining an actual distance of the target object from the binocular camera according to the feature point pairs, target parameters of the binocular camera and the ranging disparity value.
[0050] This embodiment determines the actual distance from the target object to the binocular camera based on the binocular ranging principle, and the specific implementation method is not specifically limited. For example, the angle difference between the two cameras of the binocular camera is determined based on the left image and the right image, and the actual distance from the target object to the binocular camera is estimated based on the angle difference. The advantage of this is that the influence of spatial deformation is taken into account, and the accuracy of transmission line distance measurement is improved.
[0051] The target parameters include the distance (b) and focal length (f) between the first pixel point in the feature point pair and the two optical centers of the binocular camera, and the mapping relationship between the binocular images captured by the binocular camera.
[0052] Specifically, the pixel value of each first characteristic pixel point in the first image and the pixel value of each second characteristic pixel point in the second image are determined respectively according to the mapping relationship in the target parameters to obtain the first pixel and the second pixel; the pixel value difference between each first characteristic pixel point and the second characteristic pixel point is calculated to obtain the ranging disparity value. The advantage of this is that the second pixel can be directly determined according to the mapping relationship between the binocular images and the first pixel, and then the actual distance is determined according to their ranging disparity value, the distance between the optical centers and the focal length.
[0053] Exemplarily, the two cameras of the binocular camera are placed horizontally. First, according to a preset number of first feature pixel points of the target object in the left image, the coordinates (XL) of the first feature pixel points in the horizontal direction are determined; according to the mapping relationship between the binocular images and the first feature pixel points, a preset number of second feature pixel points are determined in the right image, and the coordinates (XR) of the second feature pixel points in the horizontal direction are obtained; the difference between the horizontal coordinates (XL) of each point of the target object in the left image and the horizontal coordinates (XR) of the corresponding point in the right image is determined to obtain the ranging parallax value (X), and the actual distance (Z) of the target object from the binocular camera is obtained according to formula (1):
[0054]
[0055] Wherein, X is the mean of the difference between a preset number of XLs and a preset number of XRs.
[0056] Further, determining the ranging disparity value corresponding to the target object based on the plurality of feature point pairs includes:
[0057] First, a target bounding box corresponding to the target object is determined, the diagonal intersection of the target bounding box is obtained as the measurement center, and the target reference area is determined based on the measurement center and the preset area window.
[0058] The target boundary box is a closed shape including the boundary of the target object, and can be any shape, such as a circle or a rectangle, etc., which is not specifically limited in this embodiment. Similarly, the preset area window can also be any shape.
[0059] In one embodiment, according to the position of the target object in one of the images, a rectangular bounding box corresponding to the regional image of the target object in the image is determined; the intersection of the diagonals of the rectangular bounding box is determined as the measurement center; within the rectangular bounding box, an area of a preset area window around the measurement center is selected to obtain a target reference area.
[0060] Exemplarily, the intersection of the diagonals of the detected rectangular boundary box of the construction machinery is selected as the measurement center; the rectangular area window size is selected to be 3×3, and the center of the rectangular area window is made to coincide with the measurement center to obtain the target reference area.
[0061] Then, the disparity value corresponding to each pair of feature points in the target reference area is determined, and the disparity value with the largest number of occurrences is taken as the ranging disparity value corresponding to the target object.
[0062] Specifically, the coordinates of all pixel points in the target reference area of one of the images are determined to obtain the coordinates of the first target pixel point; based on the feature point pair, the coordinates of the pixel point at the same position corresponding to the first target pixel point in the other image are determined to obtain the coordinates of the second target pixel point; the difference between the coordinates of the first target pixel point and the coordinates of the second target pixel point is calculated, and the disparity value that appears the most times is used as the ranging disparity value corresponding to the target object for further distance calculation.
[0063] Exemplarily, the two cameras of the binocular camera are placed horizontally, and the horizontal coordinates of all pixels in the rectangular area with a window size of 3×3 around the measurement center in the left image are determined to obtain the coordinates of the first target pixel; according to the feature point pair, the horizontal coordinates of the pixel points in the right image corresponding to the same position of the target object as all the pixels in the target reference area in the left image are determined to obtain the coordinates of the second target pixel; the difference between the coordinates of the first target pixel and the coordinates of the second target pixel is calculated, and all disparity values are counted, and the disparity value with the most occurrences is used as the ranging disparity value corresponding to the target object. It should be noted that if there is no disparity value with the most occurrences, it is considered that there is no valid disparity value in the target reference area, and the window size needs to be increased (for example, the window size is increased by 2 each time, and the window size is adjusted to 5×5), and the disparity value is continued to be calculated until a valid disparity value appears. The advantage of this is that the ranging disparity value is determined according to the statistical results, even if the selected measurement center is an invalid disparity or does not belong to a feature point pair, accurate distance information can be obtained, and the effectiveness and accuracy of ranging can be enhanced.
[0064] Optionally, since errors may exist during the processing and installation of the camera and lens, the imaging model of the camera is not a simple linear transformation model, and radial distortion and tangential distortion exist. Therefore, it is necessary to calibrate the binocular camera to obtain the target parameters and distortion coefficients to correct the binocular image. Specifically, the binocular camera can be calibrated based on the Zhang Zhengyou calibration method to obtain the internal parameters, external parameters and distortion coefficients of the binocular camera; the left and right images are corrected according to the internal parameters, external parameters and distortion coefficients of the binocular camera. The advantage of this is that the measurement error of the binocular image is taken into account, the calculation accuracy is improved, and the accuracy of risk ranging around the transmission line is further improved.
[0065] In a specific embodiment, the binocular camera is calibrated based on the Zhang Zhengyou calibration method. First, a binocular camera is used to shoot multiple reference images of different angles of a preset reference object; based on the multiple reference images, the left camera and the right camera of the binocular camera are calibrated respectively to obtain the distortion coefficients, internal parameters and external parameters of the left camera and the right camera, and the target parameters of the binocular camera are generated according to the internal parameters and the external parameters; the left image and the right image are distorted and stereo corrected according to the internal parameters and the external parameters of the binocular camera to eliminate the radial and tangential distortion errors of the binocular image.
[0066] Exemplarily, a chessboard (for example, the number of grids is 10×10, and the size of each grid is 25.8 mm×25.8 mm) is used as a calibration reference, the binocular camera is kept stationary, and the chessboard is translated and / or rotated to obtain multiple (for example, 30) corresponding chessboard images (size is 640×480) taken from different orientations; based on the multiple chessboard images, the left camera and the right camera of the binocular camera are calibrated respectively using the stereo camera calibrator toolbox in the MATLAB software to obtain the internal parameter matrix and the external parameter matrix of the left camera and the right camera; according to the internal parameter matrix and the external parameter matrix, the target parameters of the binocular camera are generated; using the Undistort Points() and StereoRectify() functions in OpenCV, the left eye image and the right eye image are distorted and stereo corrected according to the internal parameters and external parameters of the binocular camera to obtain the distortion parameter matrix, and the radial and tangential distortion errors of the binocular image are eliminated.
[0067] Furthermore, other calibration methods may be used to calibrate the binocular camera to obtain target parameters of the binocular camera, which is not specifically limited in this embodiment.
[0068] The technical solution of the embodiment of the present invention obtains a group of binocular images taken by a binocular camera for a target transmission line, determines the target object in the binocular image according to the left image and the right image, and because the binocular image includes the left image and the right image, it can simulate the situation of real eye observation, capture more information about the transmission line, and determine multiple feature point pairs corresponding to the target object based on the left image and the right image, wherein two feature pixel points in the feature point pair correspond to the same position of the target object, and based on the same or similar feature content in the binocular image, reduce the pixel points for ranging calculation, and achieve high-efficiency ranging. Based on multiple feature point pairs, the ranging parallax value corresponding to the target object is determined, and the actual distance of the target object from the binocular camera is determined according to the feature point pair, the target parameters of the binocular camera and the ranging parallax value. By calculating the parallax value of the feature point pair of the target object, the actual distance from the target object to the camera can be simply, quickly and effectively determined. The problem that the existing transmission line ranging method is difficult to balance the real-time and accuracy of transmission line ranging is solved, and the accuracy and real-time of ranging of large machinery around the transmission line are improved, providing protection for the safety of the transmission line.
[0069] Figure 2 It is a flowchart of another transmission line ranging method provided according to an embodiment of the present invention. This embodiment can be applied to the scenario of performing risk ranging around transmission lines based on binocular images. This embodiment and the transmission line ranging method in the above embodiment belong to the same inventive concept. On the basis of the above embodiment, it further describes the process of determining multiple feature point pairs corresponding to the target object based on the left-eye image and the right-eye image.
[0070] like Figure 2 As shown, the distance measurement method of the transmission line includes:
[0071] S210, obtaining a group of binocular images taken by a binocular camera for a target power transmission line, and determining a target object in the binocular image according to a left image and a right image, wherein the binocular image includes a left image and a right image.
[0072] S220 , performing feature point matching on the left image and the right image based on a preset feature matching algorithm to obtain a plurality of matching point pairs corresponding to the target object.
[0073] The feature matching algorithm may be a scale-invariant feature transform (SIFT) algorithm, a speeded up robust features (SURF) matching algorithm or an ORB (Oriented Fast and Rotated Brief) matching algorithm.
[0074] In one embodiment, the feature matching algorithm is an ORB matching algorithm, which is an extremely fast feature point extraction and description algorithm and has high application value in scenarios with high real-time requirements. Specifically, first, the feature points of the binocular image are detected using the fast feature detection (Features from accelerated segment test, FAST) algorithm to obtain the initial matching point pair; then, the descriptor is calculated using the BRIEF algorithm and represented in binary form. The advantage of this is that it not only saves storage space but also shortens the matching time.
[0075] Exemplarily, first, the feature points of the binocular image are detected using a fast feature detection FAST algorithm to obtain initial matching point pairs; then, the descriptors are calculated using the BRIEF algorithm; finally, the similarity between the descriptors of each initial matching point pair is compared with a preset similarity threshold, and the initial matching point pairs with a similarity greater than the preset similarity threshold (e.g., 80%) are taken as the final matching point pairs, thereby reducing the amount of calculation of the ranging disparity value and speeding up the ranging speed.
[0076] It is understandable that it is also possible to determine a region image including the target object, directly perform feature matching on the region image of the target object, obtain a feature matching point pair of the region image, obtain a first feature pixel point of the target object in the region image, and then determine a second feature pixel point based on the mapping relationship of the binocular image. The advantage of doing so is that it can further narrow the scope of feature matching, reduce the amount of feature matching data, and speed up feature matching.
[0077] S230 , screening the multiple matching point pairs based on a preset screening strategy to obtain multiple feature point pairs corresponding to the target object.
[0078] The preset screening strategy includes an epipolar line constraint strategy and / or a pixel position arrangement constraint strategy.
[0079] It is understandable that, according to the principle of the camera, the points on the target object are projected onto the left-eye image and the right-eye image, and the position order of the pixels does not change in different projection spaces, following the row order consistency and column order consistency. However, in a real binocular camera, there are no two camera image planes that are completely coplanar and aligned in rows, and it is necessary to adjust the pixel position arrangement so that the row order consistency and column order consistency of the pixels in the binocular image are consistent. Exemplarily, based on row order consistency, when point A is to the left of point B in the left-eye image, then in the right-eye image, the matching point of point A should also be to the left of the matching point of point B, and a pixel position arrangement constraint strategy is set to filter out matching points that do not meet the row order consistency and column order consistency.
[0080] In addition, according to the camera imaging principle, when the same point is projected onto two images with different perspectives, the constraint formed by the image point and the camera optical center under the projection model is called the epipolar constraint. Through the epipolar constraint principle, we can find the corresponding pixel points in the two images to achieve image matching. Therefore, the epipolar constraint strategy is set.
[0081] In summary, for the matching point pairs obtained by the feature matching algorithm, it is necessary to adopt the epipolar constraint strategy and / or the pixel position arrangement constraint strategy based on the imaging principle to eliminate the erroneous matching points, improve the matching accuracy of the feature matching algorithm, and reduce the false matching rate of the feature matching algorithm.
[0082] Furthermore, when the preset screening strategy includes a pixel position arrangement constraint strategy, multiple matching point pairs are screened based on the preset screening strategy to obtain multiple feature point pairs corresponding to the target object, including:
[0083] First, the feature pixel point in the left image of the matching point pair is taken as the first screening point, and the feature pixel point in the right image is taken as the second screening point.
[0084] Specifically, based on the matching point pair of the target object, the pixel points of the matching point pair of the target object on the left-eye image are determined to obtain the first screening point; based on the mapping relationship between the two cameras of the binocular camera, the pixel points of the target object at the same position corresponding to the first screening point on the right-eye image are determined to obtain the second screening point.
[0085] Secondly, for the first screening point, eight image pixel points in the neighborhood of the first screening point are obtained in the left-eye image as first neighborhood points, and a first position vector is constructed based on the obtained eight first neighborhood points.
[0086] Specifically, traverse each first filtering point in the left-eye image to determine the Euclidean distance from the remaining first filtering points to the current first filtering point; sort the pixels corresponding to the first eight first filtering points in ascending order of distance, and select them as the first domain points of the current first filtering point; calculate the sequential consistency description coefficient of the first domain point based on formula (2), form an eight-dimensional vector, and obtain the first position vector.
[0087]
[0088] Among them, x0 and y0 are the horizontal and vertical coordinates of the current first screening point, respectively. i and i are the horizontal and vertical coordinates of the i-th first field point of the current first screening point, and ρ is the sequential consistency description coefficient.
[0089] Then, for the second screening point, eight pixel points in the neighborhood of the second pixel point are acquired in the right-eye image as second neighborhood points, and a second position vector is constructed based on the acquired eight second neighborhood points.
[0090] Specifically, traverse each second filtering point in the right-eye image to determine the Euclidean distance from the remaining second filtering points to the current second filtering point; sort the pixels corresponding to the first eight second filtering points in ascending order of distance, and select them as the second domain points of the current second filtering point; calculate the order consistency description coefficient of the first domain point based on formula (2), form an eight-dimensional vector, and obtain the second position vector.
[0091] Finally, the vector difference between the first position vector and the second position vector is determined, and a plurality of matching point pairs are screened based on the vector difference to obtain a plurality of feature point pairs corresponding to the target object.
[0092] Specifically, for each first position vector and second position vector, an XOR operation is performed on the elements at corresponding positions in the first position vector and the second position vector to obtain an initial vector difference; the first position vector (T L ) and the second position vector (T R ) and the sum of the initial vector differences of all positions in the target vector difference (D diff ); if the target vector difference is 0, the first filter point corresponding to the first position vector and the second filter point corresponding to the second position vector are a feature point pair corresponding to the target object; if the target vector difference is not 0, it indicates that there are mismatched points in the neighborhood of the first filter point, that is, the first filter point corresponding to the first position vector and the second filter point corresponding to the second position vector are not a feature point pair corresponding to the target object, and the first filter point, the first field point, the second filter point and the second field point corresponding to the first position vector are deleted from the matching point pairs.
[0093]
[0094] Exemplarily, for the convenience of query, for the first screening point and the second screening point, k-dimensional trees are constructed respectively to obtain the first k-dimensional tree and the second k-dimensional tree; for each of the first screening point and the second screening point, the first k-dimensional tree and the second k-dimensional tree are queried respectively to determine the eight points closest to the first screening point in Euclidean distance and the eight points closest to the second screening point in Euclidean distance to obtain the first neighborhood point and the second neighborhood point; the order consistency description coefficients of the first neighborhood point and the second neighborhood point are calculated based on formula (2) to form an eight-dimensional vector to obtain the first position vector (T L ) and the second position vector (T R ); Calculate the first position vector (T based on formula (3) L ) and the second position vector (TR ) to obtain the target vector difference; if the target vector difference is equal to 0, the first filter point corresponding to the first position vector and the second filter point corresponding to the second position vector are a feature point pair corresponding to the target object; if the target vector difference is not equal to 0, the first filter point corresponding to the first position vector and the second filter point corresponding to the second position vector are not a feature point pair corresponding to the target object, and the first filter point, the first field point, the second filter point and the second field point corresponding to the first position vector are deleted from the matching point pairs.
[0095] In one embodiment, when the vector difference is greater than a first preset threshold, the feature point pair and its corresponding first neighborhood point and second neighborhood point are deleted from the multiple matching point pairs to obtain multiple feature point pairs corresponding to the target object.
[0096] Specifically, a first preset threshold value can also be set. If the target vector difference is less than / equal to the first preset threshold value, the first filtering point corresponding to the first position vector and the second filtering point corresponding to the second position vector are a feature point pair corresponding to the target object; if the target vector difference is greater than the first preset threshold value, the first filtering point corresponding to the first position vector and the second filtering point corresponding to the second position vector are not a feature point pair corresponding to the target object, and the feature point pairs and their corresponding first neighborhood points and second neighborhood points are deleted from multiple matching point pairs to obtain multiple feature point pairs corresponding to the target object.
[0097] In the case where the preset screening strategy includes an epipolar constraint strategy, a plurality of matching point pairs are screened based on the preset screening strategy to obtain a plurality of feature point pairs corresponding to the target object, including:
[0098] First, epipolar correction is performed on the left and right images.
[0099] Specifically, according to the target parameters of the binocular camera, an epipolar correction method is used to perform epipolar correction on the left eye image and the right eye image, so that the epipolar lines in the left eye image and the epipolar lines in the right eye image are on the same horizontal line, that is, the left eye image and the right eye image are parallel, and the obtained rows correspond to the left eye image and the right eye image.
[0100] Exemplarily, according to the target parameters of the binocular camera, the Bouguet method is used to perform epipolar correction on the left-eye image and the right-eye image. If the right-eye image is a projected image, for the first pixel point on the left-eye image, only a one-dimensional search is performed on the same row as the first pixel point in the right-eye image to determine the second pixel point on the right-eye image corresponding to the same position as the first pixel point, thereby obtaining the matching point pair to be processed.
[0101] Then, for each pair of matching points to be processed, the epipolar similarity corresponding to the pair of matching points is determined based on the left and right images after epipolar correction.
[0102] Specifically, in the left-eye image and the right-eye image after epipolar correction, the matching point pairs to be processed are determined; for each matching point pair to be processed, the epipolar similarity corresponding to the matching point pair is calculated, and it can be determined whether the matching point pair satisfies the epipolar constraint to determine the matching point pairs that satisfy the epipolar constraint.
[0103] Exemplarily, the epipolar similarity corresponding to the matching point pair (P1 and P2) is calculated based on formula (4):
[0104] C α =|P 1x -P2′ x |(4)
[0105] Among them, P 1x is the horizontal coordinate of the matching point P1 in the left image before epipolar correction, P2′ x It is the projection horizontal coordinate of the matching point P2 in the right eye image after epipolar correction.
[0106] Finally, if the epipolar similarity is greater than the set threshold, the matching point pairs are eliminated to obtain multiple feature point pairs corresponding to the target object.
[0107] Specifically, for each matching point pair, if the epipolar line similarity is greater than a set threshold, the matching point pair is regarded as an incorrect matching point pair and is removed; if the epipolar line similarity is less than / equal to the set threshold, the matching point pair is regarded as a feature point pair corresponding to the target object.
[0108] Exemplarily, for a matching point pair (P1 and P2), if the epipolar line similarity of the matching point pair is greater than a set threshold (for example, 10), P1 and P2 are incorrect matching point pairs and this matching point pair is deleted; when the epipolar line similarity of the matching point pair (P1 and P2) is less than / equal to 10, P1 and P2 are a feature point pair corresponding to the target object.
[0109] S240, determining a ranging disparity value corresponding to the target object based on the plurality of feature point pairs, and determining an actual distance of the target object from the binocular camera according to the feature point pairs, target parameters of the binocular camera, and the ranging disparity value.
[0110] Figure 3 is a flowchart of a specific transmission line distance measurement method provided according to an embodiment of the present invention. Figure 3 As shown, the method includes:
[0111] S1. Calibrate the binocular camera to obtain the internal parameters, external parameters and distortion parameter matrix of the binocular camera.
[0112] S2. Obtain a group of binocular images taken by a binocular camera for a target power transmission line, wherein the binocular images include a left-eye image and a right-eye image.
[0113] The calibrated left and right cameras are installed in parallel to take real-time photos of the target transmission line. By calling the callback function, the left and right images are obtained at the same time and sent to the processor in real time. The processor performs image preprocessing operations such as white balance and / or brightness-based grayscale transformation on the left and right images to obtain clear binocular images, which can improve the calculation accuracy of risk distance measurement around the transmission line.
[0114] S3. Correct the left eye image and the right eye image according to the internal parameters, external parameters and distortion parameter matrix of the binocular camera to obtain an aligned binocular image as the binocular image to be matched.
[0115] S4. Use the NanoDet model to detect the target object in the left image and determine the target area image of the target object.
[0116] The left-eye image is input into the NanoDet model, and the type and quantity of large machinery in the left-eye image, a rectangular frame including the edge of the target object, and the coordinates of the upper left and lower right corners of the rectangular frame are output. The image in the rectangular frame including the target object is used as the target area image.
[0117] S5. Perform feature matching on the target area image using an ORB matching algorithm to obtain a plurality of first matching point pairs corresponding to the target object.
[0118] S6. Perform epipolar correction on the target area image, and based on the epipolar constraint strategy, perform screening processing on the first matching point pairs corresponding to the target object to obtain a plurality of second matching point pairs corresponding to the target object.
[0119] S7. Based on the pixel position arrangement constraint strategy, the second matching point pairs are screened to obtain a plurality of feature point pairs corresponding to the target object.
[0120] S8. Determine a ranging disparity value corresponding to the target object based on a plurality of feature point pairs corresponding to the target object.
[0121] S9. Determine the actual distance between the target object and the binocular camera according to the feature point pair, the internal parameters, the external parameters and the ranging parallax value of the binocular camera.
[0122] The technical solution of this embodiment performs feature matching on the binocular image to obtain matching point pairs; screens the matching point pairs through an epipolar constraint strategy and / or a pixel position arrangement constraint strategy to obtain matching point pairs corresponding to the target object; and performs distance measurement based on the matching points corresponding to the target object, thereby solving the problem of low accuracy of the existing distance measurement method for the transmission line, and can perform real-time monitoring of risks around the transmission line, further accelerating the speed and accuracy of transmission line detection.
[0123] Figure 4 This is a structural block diagram of a transmission line distance measurement device provided according to an embodiment of the present invention. This embodiment can be applied to scenarios where risk distance measurement around transmission lines is performed based on binocular images. The device can be implemented in the form of hardware and / or software and integrated into a processor of an electronic device with application development capabilities.
[0124] like Figure 4 As shown, the distance measuring device of the transmission line includes:
[0125] The target object determination module 301 is used to obtain a group of binocular images taken by the binocular camera for the target transmission line, and determine the target object in the binocular image according to the left image and the right image; wherein the binocular image includes the left image and the right image; the feature point pair determination module 302 is used to determine a plurality of feature point pairs corresponding to the target object based on the left image and the right image, wherein two feature pixel points in the feature point pair correspond to the same position of the target object; the ranging module 303 is used to determine the ranging parallax value corresponding to the target object based on the plurality of feature point pairs, and determine the actual distance of the target object from the binocular camera according to the feature point pair, the target parameters of the binocular camera and the ranging parallax value.
[0126] Optionally, the distance measurement module 303 further includes a disparity value submodule, and the disparity value submodule is specifically used for:
[0127] Determine a target bounding box corresponding to the target object, obtain the diagonal intersection of the target bounding box as a measurement center, and determine a target reference area based on the measurement center and a preset area window;
[0128] The disparity value corresponding to each feature point pair in the target reference area is determined, and the disparity value with the largest number of occurrences is taken as the ranging disparity value corresponding to the target object.
[0129] Optionally, the target object determination module 301 further includes a target detection submodule, which is specifically used to:
[0130] One of the left eye image or the right eye image is input into a pre-trained target detection model to determine the target object in the binocular image, wherein the target detection model includes a NanoDet model.
[0131] Optionally, the feature point pair determination module 302 is further used for:
[0132] Matching feature points of the left image and the right image based on a preset feature matching algorithm to obtain a plurality of matching point pairs corresponding to the target object, wherein the feature matching algorithm includes an ORB matching algorithm;
[0133] The plurality of matching point pairs are screened based on a preset screening strategy to obtain a plurality of feature point pairs corresponding to the target object, wherein the preset screening strategy includes an epipolar line constraint strategy and / or a pixel position arrangement constraint strategy.
[0134] Optionally, the feature point pair determination module 302 further includes a pixel position arrangement constraint submodule, and the pixel position arrangement constraint submodule is specifically used for:
[0135] The feature pixel point in the left image of the matching point pair is used as the first screening point, and the feature pixel point in the right image is used as the second screening point;
[0136] For the first screening point, eight image pixel points in the neighborhood of the first screening point are obtained in the left-eye image as first neighborhood points, and a first position vector is constructed based on the obtained eight first neighborhood points;
[0137] For the second screening point, eight pixel points in the neighborhood of the second pixel point are obtained in the right-eye image as second neighborhood points, and a second position vector is constructed based on the obtained eight second neighborhood points;
[0138] The vector difference between the first position vector and the second position vector is determined, and a plurality of matching point pairs are screened based on the vector difference to obtain a plurality of feature point pairs corresponding to the target object.
[0139] Optionally, the pixel position arrangement constraint submodule is also used to: when the vector difference is greater than a first preset threshold, delete the feature point pair and its corresponding first neighborhood point and second neighborhood point from multiple matching point pairs to obtain multiple feature point pairs corresponding to the target object.
[0140] Optionally, the feature point pair determination module 302 further includes an epipolar line constraint submodule, and the epipolar line constraint submodule is specifically used for:
[0141] Perform epipolar correction on the left and right images;
[0142] For each pair of matching points to be processed, the epipolar similarity corresponding to the matching point pair is determined based on the left image and the right image after epipolar correction;
[0143] If the epipolar line similarity is greater than the set threshold, the matching point pairs are eliminated to obtain multiple feature point pairs corresponding to the target object.
[0144] The technical solution of the embodiment of the present invention obtains a group of binocular images taken by a binocular camera for a target transmission line, determines the target object in the binocular image according to the left image and the right image, and because the binocular image includes the left image and the right image, it can simulate the situation of real eye observation, capture more information about the transmission line, and determine multiple feature point pairs corresponding to the target object based on the left image and the right image, wherein two feature pixel points in the feature point pair correspond to the same position of the target object, and based on the same or similar feature content in the binocular image, reduce the pixel points for ranging calculation, and achieve high-efficiency ranging. Based on multiple feature point pairs, the ranging parallax value corresponding to the target object is determined, and the actual distance of the target object from the binocular camera is determined according to the feature point pair, the target parameters of the binocular camera and the ranging parallax value. By calculating the parallax value of the feature point pair of the target object, the actual distance from the target object to the camera can be simply, quickly and effectively determined. The problem that the existing transmission line ranging method is difficult to balance the real-time and accuracy of transmission line ranging is solved, and the accuracy and real-time of ranging of large machinery around the transmission line are improved, providing protection for the safety of the transmission line.
[0145] The transmission line distance measuring device provided in the embodiment of the present invention can execute the transmission line distance measuring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0146] Figure 5 It is a block diagram of an electronic device provided according to an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0147] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0148] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0149] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a distance measurement method for a transmission line.
[0150] In some embodiments, the distance measurement method of the power transmission line can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the distance measurement method of the power transmission line described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the distance measurement method of the power transmission line in any other appropriate manner (for example, by means of firmware).
[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0152] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0153] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0154] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0155] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0156] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0157] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0158] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for measuring the distance of a transmission line, characterized in that: include: Acquire a set of binocular images taken by a binocular camera for a target power transmission line, and determine a target object in the binocular images according to a left-eye image and a right-eye image, wherein the binocular images include the left-eye image and the right-eye image; Determine a plurality of feature point pairs corresponding to the target object based on the left-eye image and the right-eye image, wherein two feature pixel points in the feature point pairs correspond to the same position of the target object; Determine a ranging disparity value corresponding to the target object based on the plurality of feature point pairs, and determine an actual distance of the target object from the binocular camera according to the feature point pairs, target parameters of the binocular camera and the ranging disparity value; The target parameters include a mapping relationship between the first feature pixel point in the feature point pair, the distance and focal length between the two optical centers of the binocular camera, and the binocular image collected by the binocular camera; The step of determining a ranging disparity value corresponding to the target object based on the plurality of feature point pairs, and determining an actual distance of the target object from the binocular camera according to the feature point pairs, target parameters of the binocular camera, and the ranging disparity value, comprises: Determine a second pixel value of a corresponding second feature pixel in the right image according to the mapping relationship in the target parameter and a first pixel value of a first feature pixel in the left image; Calculate the pixel value difference between each first characteristic pixel point and the second characteristic pixel point to obtain the ranging disparity value; Determine the actual distance according to the ranging parallax value, the distance between the optical centers and the focal length; Wherein, determining the target object in the binocular image according to the left-eye image and the right-eye image includes: Inputting one of the left eye image or the right eye image into a pre-trained target detection model to determine the target object in the binocular image, wherein the target detection model includes a NanoDet model; Wherein, determining the ranging disparity value corresponding to the target object based on the plurality of feature point pairs includes: Determine a target bounding box corresponding to the target object, obtain a diagonal intersection of the target bounding box as a measurement center, and determine a target reference area based on the measurement center and a preset area window; Determine the disparity value corresponding to each pair of feature points in the target reference area, and use the disparity value with the largest number of occurrences as the ranging disparity value corresponding to the target object; If there is no disparity value with the highest number of occurrences, increase the window size; Wherein, determining a plurality of feature point pairs corresponding to the target object based on the left-eye image and the right-eye image includes: Performing feature point matching on the left image and the right image based on a preset feature matching algorithm to obtain a plurality of matching point pairs corresponding to the target object, wherein the feature matching algorithm includes an ORB matching algorithm; Screening the plurality of matching point pairs based on a preset screening strategy to obtain a plurality of feature point pairs corresponding to the target object, wherein the preset screening strategy includes an epipolar constraint strategy and a pixel position arrangement constraint strategy; In the case where the preset screening strategy includes a pixel position arrangement constraint strategy, the screening process is performed on the plurality of matching point pairs based on the preset screening strategy to obtain a plurality of feature point pairs corresponding to the target object, including: Taking the characteristic pixel point in the left image among the matching point pairs as the first screening point, and taking the characteristic pixel point in the right image as the second screening point; For the first screening point, eight image pixel points in the neighborhood of the first screening point are obtained in the left-eye image as first neighborhood points, and a first position vector is constructed based on the obtained eight first neighborhood points; For the second screening point, eight pixel points in the neighborhood of the second pixel point are acquired in the right-eye image as second neighborhood points, and a second position vector is constructed based on the acquired eight second neighborhood points; Determine the vector difference between the first position vector and the second position vector, and perform screening processing on the plurality of matching point pairs based on the vector difference to obtain a plurality of feature point pairs corresponding to the target object; The determining of the vector difference between the first position vector and the second position vector, and screening the plurality of matching point pairs based on the vector difference to obtain a plurality of feature point pairs corresponding to the target object, includes: Calculating the difference between the first position vector and the second position vector to obtain a target vector difference; If the target vector difference is equal to 0, determining that a first screening point corresponding to the first position vector and a second screening point corresponding to the second position vector are a feature point pair corresponding to the target object; If the target vector difference is not equal to 0, it is determined that the first filtering point corresponding to the first position vector and the second filtering point corresponding to the second position vector are not a feature point pair corresponding to the target object, and the first filtering point, the first neighborhood point, the second filtering point and the second neighborhood point corresponding to the first position vector are deleted from the multiple matching point pairs.
2. The method according to claim 1, characterized in that In the case where the preset screening strategy includes an epipolar constraint strategy, the screening process of the plurality of matching point pairs based on the preset screening strategy to obtain a plurality of feature point pairs corresponding to the target object includes: Performing epipolar correction on the left-eye image and the right-eye image; For each of the matching point pairs to be processed, determining the epipolar similarity corresponding to the matching point pair based on the left eye image and the right eye image after epipolar correction; If the epipolar line similarity is greater than a set threshold, the matching point pairs are eliminated to obtain a plurality of feature point pairs corresponding to the target object.
3. A distance measuring device for a power transmission line, characterized in that: include: A target object determination module, used to obtain a set of binocular images taken by a binocular camera for a target power transmission line, and determine the target object in the binocular image according to a left-eye image and a right-eye image, wherein the binocular image includes the left-eye image and the right-eye image; A feature point pair determination module, configured to determine a plurality of feature point pairs corresponding to the target object based on the left-eye image and the right-eye image, wherein two feature pixel points in the feature point pairs correspond to the same position of the target object; A ranging module, configured to determine a ranging disparity value corresponding to the target object based on the plurality of feature point pairs, and determine an actual distance between the target object and the binocular camera according to the feature point pairs, target parameters of the binocular camera and the ranging disparity value; The target parameters include a mapping relationship between the first feature pixel point in the feature point pair, the distance and focal length between the two optical centers of the binocular camera, and the binocular image collected by the binocular camera; The ranging module is specifically used for: Determine a second pixel value of a corresponding second feature pixel in the right image according to the mapping relationship in the target parameter and a first pixel value of a first feature pixel in the left image; Calculate the pixel value difference between each first characteristic pixel point and the second characteristic pixel point to obtain the ranging disparity value; Determine the actual distance according to the ranging parallax value, the distance between the optical centers and the focal length; The target object determination module further includes a target detection submodule, and the target detection submodule is specifically used to: Inputting one of the left eye image or the right eye image into a pre-trained target detection model to determine the target object in the binocular image, wherein the target detection model includes a NanoDet model; The distance measurement module further includes a disparity value submodule, and the disparity value submodule is used to: Determine a target bounding box corresponding to the target object, obtain a diagonal intersection of the target bounding box as a measurement center, and determine a target reference area based on the measurement center and a preset area window; Determine the disparity value corresponding to each pair of feature points in the target reference area, and use the disparity value with the largest number of occurrences as the ranging disparity value corresponding to the target object; If there is no disparity value with the highest number of occurrences, increase the window size; The feature point pair determination module is also used for: Matching feature points of the left image and the right image based on a preset feature matching algorithm to obtain a plurality of matching point pairs corresponding to the target object, wherein the feature matching algorithm includes an ORB matching algorithm; Screening multiple matching point pairs based on a preset screening strategy to obtain multiple feature point pairs corresponding to the target object, wherein the preset screening strategy includes an epipolar constraint strategy and a pixel position arrangement constraint strategy; The feature point pair determination module further includes a pixel position arrangement constraint submodule, and the pixel position arrangement constraint submodule is used to: The feature pixel point in the left image of the matching point pair is used as the first screening point, and the feature pixel point in the right image is used as the second screening point; For the first screening point, eight image pixel points in the neighborhood of the first screening point are obtained in the left-eye image as first neighborhood points, and a first position vector is constructed based on the obtained eight first neighborhood points; For the second screening point, eight pixel points in the neighborhood of the second pixel point are obtained in the right-eye image as second neighborhood points, and a second position vector is constructed based on the obtained eight second neighborhood points; Determine the vector difference between the first position vector and the second position vector, and perform screening processing on a plurality of matching point pairs based on the vector difference to obtain a plurality of feature point pairs corresponding to the target object; The pixel position arrangement constraint submodule is specifically used for: Calculating the difference between the first position vector and the second position vector to obtain a target vector difference; If the target vector difference is equal to 0, determining that a first screening point corresponding to the first position vector and a second screening point corresponding to the second position vector are a feature point pair corresponding to the target object; If the target vector difference is not equal to 0, it is determined that the first filtering point corresponding to the first position vector and the second filtering point corresponding to the second position vector are not a feature point pair corresponding to the target object, and the first filtering point, the first neighborhood point, the second filtering point and the second neighborhood point corresponding to the first position vector are deleted from the multiple matching point pairs.
4. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the transmission line distance measurement method according to any one of claims 1 to 2.
5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the distance measurement method for a power transmission line according to any one of claims 1 to 2 when executed.
Citation Information
Patent Citations
Power transmission line safety distance measurement method, computer equipment and storage medium
CN111798507A
Pipeline change detection method based on deep learning and unmanned aerial vehicle images
CN111967337A
Method and system for measuring sag of power transmission line
CN115294145A
Target positioning method and system, and related device
WO2022143237A1
