A binocular stereo vision method and system for measuring the safety distance in electric power operations

Abstract

The present invention discloses a binocular stereo vision method and system for measuring the safety distance in electric power operations, which relates to the technical field of image processing, and solves the technical problems that it is difficult to perform target recognition based on binocular images; secondly, it is difficult to extract features according to the target recognition results and perform feature matching; then, it is difficult to construct a disparity map based on the results of feature matching and then reconstruct a three-dimensional space model; finally, it is difficult to analyze and calculate the safety distances between electric power operation personnel, live equipment, and large high-altitude operation vehicles, and it is difficult to comprehensively analyze and evaluate the safety of the electric power operation site. The present invention performs feature extraction and feature matching after identifying electric power operation personnel, live equipment, and large high-altitude operation vehicles in binocular images; measures the safety distances between electric power operation personnel, live equipment, and large high-altitude operation vehicles after constructing a disparity map and reconstructing a three-dimensional space model.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and specifically relates to a binocular stereo vision method and system for measuring the safety distance in power operations. Background Art

[0002] Binocular stereo vision captures two images of the same scene from different positions by two cameras, and calculates the disparity between corresponding points in these two images to obtain the three-dimensional geometric information of the object. The binocular stereo vision technology can provide non-contact high-precision distance measurement and is applicable to various complex environments and scenarios. With the application of binocular stereo vision technology in the measurement of the safety distance in power operations, binocular cameras are used to collect binocular images and perform stereo matching to obtain the spatial information of the power operation site, and by measuring the safety distance in power operations, the safety status of the power operation site is monitored in real time to improve the safety of the power working environment.

[0003] The existing binocular stereo vision methods and systems for measuring the safety distance in power operations have the following problems: First, it is difficult to perform target recognition based on binocular images; second, it is difficult to perform feature extraction based on the target recognition result and difficult to perform feature matching; then, it is difficult to construct a disparity map based on the result of feature matching and difficult to reconstruct a three-dimensional space model; finally, it is difficult to analyze and calculate the safety distances between power operation personnel, energized equipment, and large aerial work vehicles, and it is difficult to comprehensively analyze and evaluate the safety of the power operation site. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; for this purpose, the present invention provides a binocular stereo vision method and system for measuring the safety distance in power operations to solve the following technical problems:

[0005] First, it is difficult to perform target recognition based on binocular images; second, it is difficult to perform feature extraction based on the target recognition result and difficult to perform feature matching; then, it is difficult to construct a disparity map based on the result of feature matching and difficult to reconstruct a three-dimensional space model; finally, it is difficult to analyze and calculate the safety distances between power operation personnel, energized equipment, and large aerial work vehicles, and it is difficult to comprehensively analyze and evaluate the safety of the power operation site.

[0006] To solve the above problems, the first aspect of the present invention provides a binocular stereo vision method for measuring the safety distance in power operations, including the following steps:

[0007] S1: Real-time collect binocular images and binocular parameters of the power operation site through a binocular camera, where the binocular images include: a left image and a right image; the binocular parameters include: a baseline distance and a focal length;

[0008] S2: Preprocess the collected binocular images, including: rectification, denoising, image enhancement, and grayscale conversion; preprocess the collected binocular parameters, including: data cleaning and standardization;

[0009] S3: Perform object recognition on the preprocessed binocular images using an object recognition algorithm; extract features from the binocular images based on the object recognition results, and perform feature matching based on the results of feature extraction;

[0010] S4: Construct a disparity map based on the results of feature matching; analyze and calculate the depth of each pixel point based on the disparity map and binocular parameters; analyze and calculate the three-dimensional coordinates of each pixel point based on the principle of triangulation and reconstruct a three-dimensional space model;

[0011] S5: Analyze and calculate the safety distances between power operation personnel, energized equipment, and large aerial work vehicles based on the three-dimensional space model; comprehensively analyze and evaluate the safety of the power operation site;

[0012] Among them, in step S4, according to the results of feature matching, a path set is constructed by using the semi-global matching algorithm, including: horizontal direction, vertical direction, and diagonal direction; according to each matched pixel point, a cost function is obtained based on the pixel values in the left image and the right image and by using a weighted average formula; the cumulative cost of each pixel point is obtained based on the cost function and the difference in pixel values; the disparity value is obtained through the objective function after using the cumulative cost; a disparity map is generated by matching the disparity values for each pixel; the depth of each pixel point is calculated based on the disparity map, and the three-dimensional coordinates of each pixel point are calculated based on the disparity map and binocular parameters and then converted to the three-dimensional coordinates in the world coordinate system; a point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud through a surface reconstruction algorithm.

[0013] As a further solution of the present invention: The object recognition performed on the preprocessed binocular images using an object recognition algorithm in step S3 includes the following steps:

[0014] Perform object recognition on the preprocessed binocular images using an object recognition algorithm to real-time recognize the key objects at the power operation site, where the key objects include: power operation personnel, energized equipment, and large power operation vehicles;

[0015] Generate a historical image dataset by collecting historical binocular images; annotate the key objects in the historical binocular images by using an image annotation tool, and the annotation includes: the category and location of the key objects; the categories of the key objects include: power operation personnel, energized equipment, and large power operation vehicles, and the location of the key objects is obtained through a bounding box;

[0016] Divide the historical image dataset containing annotations into a training set and a test set; perform object recognition by using the YOLO model of the object recognition algorithm and introduce an attention mechanism into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, input the training set into the YOLO model with the attention mechanism for model training;

[0017] According to the trained YOLO model, input the binocular images of the power operation site after real-time acquisition and preprocessing into the trained YOLO model, automatically identify the key objects in the binocular images and output the recognition results, where the recognition results include: the category and location of the key objects.

[0018] As a further solution of the present invention: in step S3, feature extraction is performed on the binocular images according to the object recognition results, and feature matching is performed according to the results of the feature extraction, including the following steps:

[0019] According to the category and location of the key objects recognized in the binocular images, by using the feature point detection algorithm, detect and extract the feature points of the key objects in the left image and the right image respectively, including: corner points and edge points;

[0020] Perform feature description according to the extracted feature points; generate feature vectors by quantifying the local features of the area around the feature points, including: the gradient direction and magnitude of the surrounding area;

[0021] By using the feature point matching algorithm, match the feature points of the left image with the feature points of the right image to obtain a set of matching point pairs, denoted as the first matching result; according to the RANSAC random sample consensus algorithm, remove the mismatched point pairs to obtain the second matching result; filter the mismatched point pairs by analyzing the score gap between the first matching result and the second matching result to obtain the result of feature matching;

[0022] According to the result of feature matching, draw the matching feature points on the left image and the right image respectively.

[0023] As a further solution of the present invention: in step S4, construct a disparity map according to the result of feature matching, including the following steps:

[0024] According to the result of feature matching, construct a path set by using the semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction;

[0025] According to each matching pixel point, calculate the cumulative cost by analyzing along each path, by analyzing the formula:

[0026]

[0027]

[0028] Obtain the cumulative cost , where represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, , and represent that the weight coefficients are 0.4, 0.3, and 0.3 respectively, is a constant ; and represent the penalty coefficients; and are indicator functions, which take the value of 1 when and are true, and 0 otherwise, represents the disparity value of the previous pixel or adjacent pixel on the path;

[0029] By analyzing the formula:

[0030] Obtain the disparity value ; where , , and represent the cumulative costs in the horizontal direction, vertical direction, left diagonal direction, and right diagonal direction respectively, , and are weight coefficients; Remove the false matches by checking that the disparity values of the corresponding positions of the pixels in the left image and the right image are different; Generate a disparity map by constructing the matching disparity values for each pixel.

[0031] As a further solution of the present invention: In step S4, analyzing and calculating the depth of each pixel point according to the disparity map and binocular parameters includes the following steps:

[0032] According to the constructed disparity map, by analyzing the depth formula of each pixel point:

[0033] Obtain the depth of each pixel point ; where represents the focal length, B represents the baseline distance, represents the disparity value of each pixel point.

[0034] As a further solution of the present invention: In step S4, analyzing and calculating the three-dimensional coordinates of each pixel point according to the triangulation principle and reconstructing a three-dimensional space model includes the following steps:

[0035] The depth of each pixel calculated based on the analysis of the disparity map , and calculate the three-dimensional coordinates of each pixel through the analysis of the disparity map and binocular parameters, and reconstruct the three-dimensional space model; for each pixel in the disparity map, let the pixel coordinates of the corresponding point in the left image be ( , ), then the pixel coordinates of the corresponding point in the right image are ( , ), where, ;

[0036] Using the principle of triangulation, through the analysis formula:

[0037] Get the three-dimensional coordinates (X, Y, Z); where B represents the baseline length, f represents the focal length of the camera, represents the disparity value of each pixel, represents the depth of each pixel;

[0038] Convert the three-dimensional coordinates from the three-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system by using the rotation matrix and translation vector of the binocular camera;

[0039] By analyzing the formula: , get the three-dimensional coordinates of the world coordinate system ( ), where ( ) are the three-dimensional coordinates of the camera coordinate system, R is the rotation matrix, and T is the translation vector;

[0040] Repeat the above steps for each pixel, calculate the three-dimensional coordinates of each pixel in the disparity map, and combine the three-dimensional coordinates of each pixel to form a point cloud, and reconstruct the three-dimensional space model from the point cloud by using the surface reconstruction algorithm.

[0041] As a further solution of the present invention: in step S5, according to the three-dimensional space model, analyze and calculate the safety distances between power operation personnel, live equipment and large high-altitude operation vehicles, including the following steps:

[0042] Obtain the three-dimensional coordinates of the power operation personnel ( ), the three-dimensional coordinates of the live equipment ( ) and the three-dimensional coordinates of the large high-altitude operation vehicle ( ) from the three-dimensional space model;

[0043] By analyzing the formula:

[0044] Get the distance between the power operation personnel and the live equipment ;

[0045] By analyzing the formula:

[0046] Obtain the distance between the electric power operation personnel and the large-scale aerial work vehicle ;

[0047] By analyzing the formula:

[0048] Obtain the distance between the large-scale aerial work vehicle and the energized equipment ;

[0049] Compare and analyze the calculated distances between the electric power operation personnel, the energized equipment and the large-scale aerial work vehicle with the safety thresholds between the electric power operation personnel and the energized equipment, between the electric power operation personnel and the large-scale aerial work vehicle, and between the large-scale aerial work vehicle and the energized equipment respectively to measure the safety distance:

[0050] When , the distance between the electric power operation personnel and the energized equipment is a safe distance, otherwise it is not a safe distance; when , the distance between the electric power operation personnel and the large-scale aerial work vehicle is a safe distance, otherwise it is not a safe distance; when , the distance between the large-scale aerial work vehicle and the energized equipment is a safe distance, otherwise it is not a safe distance; where is the safety threshold between the electric power operation personnel and the energized equipment, is the safety threshold between the electric power operation personnel and the large-scale aerial work vehicle, is the safety threshold between the large-scale aerial work vehicle and the energized equipment.

[0051] As a further solution of the present invention: The comprehensive analysis and evaluation of the safety of the electric power operation site in step S5 includes the following steps:

[0052] According to the calculated safe distances between the electric power operation personnel, the energized equipment and the large-scale aerial work vehicle, through the analysis formula:

[0053] Obtain the safety evaluation value T of the electric power operation site; when T 1, it means that the electric power operation site is safe, otherwise it means that there are potential safety hazards in the electric power operation site.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] By utilizing the target recognition algorithm and the attention mechanism, the present invention accurately and quickly identifies power operation personnel, energized equipment, and large aerial work vehicles in binocular images, reduces recognition errors, and improves operation efficiency; after extracting features from the binocular images based on the target recognition results and then completing feature matching, the interference of irrelevant information is reduced, and the accuracy and efficiency of matching are improved;

[0056] The present invention constructs a disparity map through the result of feature matching, and obtains the depth of each pixel point through binocular parameter analysis; after analyzing the three-dimensional coordinates of each pixel point based on the principle of triangulation, a three-dimensional space model is reconstructed, providing a reliable basis for subsequent safety distance analysis;

[0057] Through the three-dimensional space model, the present invention measures the safety distances between power operation personnel, energized equipment, and large aerial work vehicles; through comprehensive analysis of the measurement results, the safety of the power operation site is evaluated, providing a scientific safety evaluation means for the power operation site. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0059] Figure 1 is the flowchart of the method of the present invention;

[0060] Figure 2 is the system module diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0062] Please refer to Figure 1 As shown, the first aspect embodiment of the present invention provides a binocular stereo vision power operation safety distance measurement method and system, including the following steps:

[0063] S1: Through binocular cameras, binocular images and binocular parameters of the power operation site are collected in real time. The binocular images include: a left image and a right image; the binocular parameters include: a baseline distance and a focal length;

[0064] S2: Preprocess the collected binocular images, including: rectification, denoising, image enhancement, and grayscale conversion; preprocess the collected binocular parameters, including: data cleaning and standardization;

[0065] S3: Perform object recognition on the preprocessed binocular images using an object recognition algorithm; extract features from the binocular images based on the object recognition results, and perform feature matching based on the results of feature extraction;

[0066] S4: Construct a disparity map based on the results of feature matching; analyze and calculate the depth of each pixel point based on the disparity map and binocular parameters; analyze and calculate the three-dimensional coordinates of each pixel point according to the principle of triangulation and reconstruct a three-dimensional space model;

[0067] S5: Analyze and calculate the safety distances between power operation personnel, energized equipment, and large aerial work vehicles based on the three-dimensional space model; comprehensively analyze and evaluate the safety of the power operation site;

[0068] Among them, in step S4, according to the results of feature matching, a path set is constructed by using the semi-global matching algorithm, including: horizontal direction, vertical direction, and diagonal direction; according to each matched pixel point, a cost function is obtained based on the pixel values in the left image and the right image and using the weighted average formula; the cumulative cost of each pixel point is obtained based on the cost function and the difference in pixel values; the disparity value is obtained through the objective function after using the cumulative cost; a disparity map is generated by matching the disparity values for each pixel; the depth of each pixel point is analyzed and calculated based on the disparity map, and the three-dimensional coordinates of each pixel point are analyzed and calculated through the disparity map and binocular parameters and then converted to the three-dimensional coordinates in the world coordinate system; a point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud through a surface reconstruction algorithm.

[0069] Specifically, a binocular camera is installed at the power operation site to collect binocular images and binocular parameters of the operation site in real time. The binocular camera should have high precision and high resolution to ensure the image quality. The preprocessing of the binocular images includes: performing distortion correction on the binocular images to ensure that the images are not distorted; using a filtering algorithm to remove noise in the images; enhancing the contrast and brightness of the images to make them clearer; and converting the color images into grayscale images to simplify subsequent processing. The preprocessing of the binocular parameters includes: removing invalid or abnormal data in the binocular parameters; and converting the binocular parameters into a unified format and unit for subsequent calculations. By using an object recognition algorithm, power operation personnel, live equipment, and large aerial operation vehicles are recognized in the preprocessed binocular images, and features are extracted based on the object recognition results. Matching is performed between the features extracted from the left image and the right image respectively to find the corresponding feature points. According to the result of feature matching, a disparity map is constructed, where the disparity of each pixel point in the left and right images is represented. According to the disparity map and the binocular parameters, the depth of each pixel point is calculated using the principle of triangulation. Combining the depth information and the positional relationship of the binocular camera, the three-dimensional coordinates of each pixel point are calculated. According to the three-dimensional coordinates of all pixel points, a three-dimensional space model of the power operation site is reconstructed. In the three-dimensional space model, the actual distances between the power operation personnel, live equipment, and large aerial operation vehicles are calculated and compared with the safety threshold, and the safety condition of the power operation site is comprehensively evaluated.

[0070] In one embodiment of the present invention, the object recognition in step S3 using an object recognition algorithm based on the preprocessed binocular images includes the following steps:

[0071] Using an object recognition algorithm based on the preprocessed binocular images to perform object recognition, and real-time recognizing key objects at the power operation site, where the key objects include: power operation personnel, live equipment, and large power operation vehicles;

[0072] By collecting historical binocular images, a historical image dataset is generated; by using an image annotation tool to annotate the key objects in the historical binocular images, the annotation includes: the category and position of the key objects; the categories of the key objects include: power operation personnel, live equipment, and large power operation vehicles, and the position of the key objects is obtained through a bounding box;

[0073] Dividing the historical image dataset containing annotations into a training set and a test set; performing object recognition using the YOLO model of the object recognition algorithm and introducing an attention mechanism into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, inputting the training set into the YOLO model with an attention mechanism for model training;

[0074] According to the trained YOLO model, the binocular images of the power operation site after real-time acquisition and preprocessing are input into the trained YOLO model, and the key objects in the binocular images are automatically recognized and the recognition results are output. The recognition results include: the category and location of the key objects.

[0075] Specifically, a large number of historical binocular images are collected to generate a historical image dataset, including various scenes of the power operation site, especially key objects such as power operation personnel, energized equipment, and large power operation vehicles. The collected binocular images are labeled using an image annotation tool, and the annotation includes the category and location of the key objects. Among them, the category annotation is used to identify the type of key objects in the image, and the location annotation represents the location of the key objects in the image through bounding boxes. The labeled historical image dataset is randomly divided into a training set and a test set. The training set is used to train the YOLO model, and the test set is used to evaluate the performance of the YOLO model; when dividing the historical image dataset, it is ensured that the left image and the right image at the same time are in the same training set or test set. The YOLO model is selected as the target recognition algorithm, and on the basis of the YOLO model, an attention mechanism is introduced to improve the recognition performance of the model. The attention mechanism can help the YOLO model pay more attention to the key regions in the image, thereby improving the accuracy and robustness of recognition. An attention module is added to the feature extraction network of the YOLO model and the parameters of the YOLO model are adjusted according to the actual situation, including: learning rate, batch size, number of iterations, etc., to ensure that the YOLO model can fully learn the features in the image. The training set is input into the YOLO model with an attention mechanism for training. During the training process, the YOLO model will continuously optimize its parameters and improve the recognition performance of key objects. The binocular images of the power operation site are collected in real time and preprocessed. The preprocessed binocular images are input into the trained YOLO model, and the trained YOLO model automatically recognizes the key objects in the binocular images and outputs the category and location of the key objects.

[0076] In one embodiment of the present invention, in step S3, feature extraction is performed on the binocular image according to the target recognition result, and feature matching is performed according to the result of feature extraction, including the following steps:

[0077] According to the category and location of the key objects recognized in the binocular image, by using a feature point detection algorithm, the feature points of the key objects are detected and extracted in the left image and the right image respectively, including: corner points and edge points;

[0078] Feature description is performed according to the extracted feature points; a feature vector is generated by quantifying the local features of the area around the feature points, including: the gradient direction and magnitude of the surrounding area;

[0079] By using a feature point matching algorithm, the feature points of the left image are matched with the feature points of the right image to obtain a set of matching point pairs, which is represented as the first matching result; according to the RANSAC (Random Sample Consensus) algorithm, the mis-matched point pairs are removed to obtain the second matching result; by analyzing the score gap between the first matching result and the second matching result, the mis-matched point pairs are filtered to obtain the result of feature matching.

[0080] According to the result of feature matching, the matched feature points are respectively drawn on the left image and the right image.

[0081] Specifically, according to the target recognition result, in the left image and the right image, for the position area of each key object, a feature point detection algorithm is applied, including: Harris corner detection, Shi-Tomasi corner detection or SIFT (Scale-Invariant Feature Transform) etc. to detect corner points and edge points. For each detected feature point, the gradient direction and magnitude in its surrounding area are calculated to generate a feature vector. The feature vector should be able to describe the local texture and shape information around the feature point for effective comparison in the subsequent matching process. A feature point matching algorithm, including: brute-force matching, FLANN (Fast Library for Approximate Nearest Neighbors) matching etc., is used to preliminarily match the feature points in the left image with the feature points in the right image, where the result of the preliminary matching is a set of matching point pairs as the first matching result. The mis-matched point pairs in the first matching result are removed by applying the RANSAC algorithm. The RANSAC algorithm estimates a fundamental matrix or a homography matrix by randomly selecting a set of feature point pairs and evaluates the consistency of the matching point pairs based on these matrices. After being processed by the RANSAC algorithm, a more accurate set of matching point pairs is obtained, which is called the second matching result. By analyzing the score gap between the first matching result and the second matching result, where the score can be calculated based on the distance, similarity or other metric criteria between the matching point pairs. By setting a score threshold to filter out the matching point pairs with lower scores, thereby further reducing mis-matches; among them, the selection of the score threshold can be based on practical experience or experiments, and usually a relatively low quantile in the score distribution such as the 10% or 20% quantile is selected as the threshold. According to the result of feature matching, the matched feature points are respectively drawn on the left image and the right image by drawing dots, line segments or other marks at the feature point positions.

[0082] In one embodiment of the present invention, in step S4, constructing a disparity map according to the result of feature matching includes the following steps:

[0083] According to the result of feature matching, a path set is constructed by using the semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction;

[0084] According to each matched pixel point, the cumulative cost is calculated by analyzing along each path through the analysis formula:

[0085]

[0086]

[0087] Obtain the cumulative cost , where represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, 、 and represent that the weight coefficients are 0.4, 0.3, and 0.3 respectively, is a constant ; and represent the penalty coefficients; and are indicator functions, which take the value of 1 when and are true, and 0 otherwise, represents the disparity value of the previous pixel or adjacent pixel on the path;

[0088] By analyzing the formula:

[0089] Obtain the disparity value ; where 、 、 and represent the cumulative costs in the horizontal direction, vertical direction, left diagonal direction, and right diagonal direction respectively, 、 、 and are weight coefficients; Remove the incorrect matches by checking that the disparity values of the corresponding positions of the pixels in the left image and the right image are different; Generate a disparity map by constructing the matching disparity values for each pixel.

[0090] Specifically, according to the results of feature matching, use the semi - global matching algorithm to construct a set of paths, including: paths in the horizontal direction, vertical direction, and diagonal direction, and these paths will be used for the calculation of cumulative cost. In the calculation of cumulative cost of the semi - global matching algorithm and represent that the penalty coefficients are 10 and 120 respectively; where the penalty coefficient is a parameter for controlling the smoothness of disparity change, and the constant term is to avoid the situation of division by zero and is dynamically adjusted according to the actual situation. By multiplying the absolute value of the brightness difference between two pixels in the left image and the right image, the sum of squares of the brightness difference between two pixels, and the normalized squared difference by the weight coefficients 、 and Obtain the cost function, where the weight coefficients , and are dynamically adjusted according to the actual situation. For each matched pixel point, after analyzing and calculating the cumulative cost along each path and comparing the cumulative costs on different paths, the disparity value with the minimum cumulative cost can be selected as the optimal disparity value for the current pixel point. Among them, the weight coefficient of the cumulative cost in the horizontal direction is 0.4, the weight coefficient of the cumulative cost in the vertical direction is 0.3, the weight coefficient of the cumulative cost in the left diagonal direction is 0.2, and the weight coefficient of the cumulative cost in the right diagonal direction is 0.1. The weight coefficients are dynamically adjusted according to the actual content of the binocular images. After obtaining the initial disparity map, remove the incorrect matches by checking whether the disparity values at the corresponding positions of the pixels in the left image in the right image are the same. If they are different, it may indicate the existence of incorrect matches, and these points need to be removed from the disparity map. After the above steps, a disparity map can be obtained, which reflects the disparity relationship between each pixel point in the left image and the right image.

[0091] In one embodiment of the present invention, in step S4, analyzing and calculating the depth of each pixel point according to the disparity map and binocular parameters includes the following steps:

[0092] According to the constructed disparity map, by analyzing the depth formula of each pixel point:

[0093] Obtain the depth of each pixel point ; where represents the focal length, B represents the baseline distance, represents the disparity value of each pixel point.

[0094] Specifically, according to the generated disparity map combined with the binocular parameters, the binocular parameters include: the focal length and the baseline distance, and analyze the formula to calculate the depth of each pixel point.

[0095] In one embodiment of the present invention, in step S4, analyzing and calculating the three-dimensional coordinates of each pixel point according to the principle of triangulation and reconstructing the three-dimensional space model includes the following steps:

[0096] The depth of each pixel point calculated by analyzing the disparity map , and calculate the three-dimensional coordinates of each pixel point through the disparity map and binocular parameters, and reconstruct the three-dimensional space model; for each pixel point in the disparity map, set the pixel coordinates of the corresponding point in the left image as ( , ), then the pixel coordinates of the corresponding point in the right image are ( , ), wherein, ;

[0097] Using the principle of triangulation, by analyzing the formula:

[0098] The three-dimensional coordinates (X, Y, Z) are obtained; wherein, B represents the baseline length, f represents the focal length of the camera, represents the parallax value of each pixel point, represents the depth of each pixel point;

[0099] The three-dimensional coordinates are converted from the three-dimensional coordinates in the camera coordinate system to the three-dimensional coordinates in the world coordinate system by using the rotation matrix and translation vector of the binocular camera;

[0100] By analyzing the formula: , the three-dimensional coordinates in the world coordinate system ( ) are obtained, where ( ) are the three-dimensional coordinates in the camera coordinate system, R is the rotation matrix, and T is the translation vector;

[0101] The above steps are repeated for each pixel point. By calculating the three-dimensional coordinates of each pixel point in the disparity map and combining the three-dimensional coordinates of each pixel point, a point cloud is formed. A three-dimensional space model is reconstructed from the point cloud by using a surface reconstruction algorithm.

[0102] Specifically, according to the disparity map, the coordinates of each pixel point are obtained, and the three-dimensional coordinates are obtained by the principle of triangulation. By using the rotation matrix and translation vector of the binocular camera, the three-dimensional coordinates are converted from the camera coordinate system to the world coordinate system. The above steps are repeated for each pixel point in the disparity map, and the three-dimensional coordinates of each pixel point in the disparity map are analyzed and calculated, and these three-dimensional coordinates are combined to form a point cloud. Using surface reconstruction algorithms, including: Delaunay triangulation, BPA algorithm, Power Crust algorithm, etc., a continuous triangular mesh is generated according to the formed point cloud data, thereby constructing the three-dimensional surface of the object and reconstructing the three-dimensional space model.

[0103] In one embodiment of the present invention, in step S5, according to the three-dimensional space model, analyzing and calculating the safety distances between power operation personnel, live equipment, and large high-altitude operation vehicles includes the following steps:

[0104] Obtaining the three-dimensional coordinates of power operation personnel ( ), the three-dimensional coordinates of live equipment ( ), and the three-dimensional coordinates of large high-altitude operation vehicles ( ) according to the three-dimensional space model;

[0105] By analyzing the formula:

[0106] Obtain the distance between the power operation personnel and the energized equipment ;

[0107] By analyzing the formula:

[0108] Obtain the distance between the power operation personnel and the large high-altitude operation vehicle ;

[0109] By analyzing the formula:

[0110] Obtain the distance between the large high-altitude operation vehicle and the energized equipment ;

[0111] Compare and analyze the calculated distances between the power operation personnel, the energized equipment and the large high-altitude operation vehicle with the safety thresholds between the power operation personnel and the energized equipment, between the power operation personnel and the large high-altitude operation vehicle, and between the large high-altitude operation vehicle and the energized equipment respectively to measure the safety distance:

[0112] When , the distance between the power operation personnel and the energized equipment is a safe distance, otherwise it is not a safe distance; when , the distance between the power operation personnel and the large high-altitude operation vehicle is a safe distance, otherwise it is not a safe distance; when , the distance between the large high-altitude operation vehicle and the energized equipment is a safe distance, otherwise it is not a safe distance; where is the safety threshold between the power operation personnel and the energized equipment, is the safety threshold between the power operation personnel and the large high-altitude operation vehicle, is the safety threshold between the large high-altitude operation vehicle and the energized equipment.

[0113] Specifically, obtain the three-dimensional coordinates of the power operation personnel, the energized equipment and the large high-altitude operation vehicle according to the three-dimensional space model, and obtain the distances between the power operation personnel and the energized equipment, between the power operation personnel and the large high-altitude operation vehicle, and between the large high-altitude operation vehicle and the energized equipment by using the Euclidean distance formula. Compare the calculated distances with the safety thresholds respectively to determine whether it is a safe distance. The safety threshold is jointly determined according to national standards and actual situations and can be dynamically adjusted.

[0114] In one embodiment of the present invention, the comprehensive analysis and evaluation of the safety of the power operation site in step S5 includes the following steps:

[0115] According to the safety distances calculated through analysis among power operation personnel, live equipment, and large aerial work vehicles, the safety valuation T of the power operation site is obtained through the analysis formula:

[0116] When T ≥ 1, it indicates that the power operation site is safe; otherwise, it indicates that there are potential safety hazards at the power operation site.

[0117] Specifically, according to the calculated safety distances among power operation personnel, live equipment, and large aerial work vehicles, the safety valuation of the power operation site is obtained through a formula. A threshold of 1 is set, and by comparing and analyzing the safety valuation of the power operation site with the threshold, the safety of the power site is evaluated in real time; among them, the threshold can be dynamically adjusted according to actual situations or historical data.

[0118] Please refer to Figure 2 as shown. The present invention is a binocular stereo vision power operation safety distance measurement system, including the following modules:

[0119] Image acquisition module: Through binocular cameras, binocular images and binocular parameters of the power operation site are acquired in real time. The binocular images include: left image and right image; the binocular parameters include: baseline distance and focal length;

[0120] Image preprocessing module: Preprocess the acquired binocular images, including: correction, denoising, image enhancement, and grayscale conversion; preprocess the acquired binocular parameters, including: data cleaning and standardization;

[0121] Image processing module: Perform target recognition on the preprocessed binocular images using a target recognition algorithm; extract features from the binocular images according to the target recognition results, and perform feature matching according to the results of feature extraction;

[0122] Image analysis module: Construct a disparity map according to the results of feature matching; analyze and calculate the depth of each pixel point according to the disparity map and binocular parameters; analyze and calculate the three-dimensional coordinates of each pixel point according to the principle of triangulation and reconstruct a three-dimensional space model;

[0123] Measurement and evaluation module: Analyze and calculate the safety distances among power operation personnel, live equipment, and large aerial work vehicles according to the three-dimensional space model; comprehensively analyze and evaluate the safety of the power operation site.

[0124] The above embodiments are only used to illustrate the technical methods of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical methods of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A binocular stereo vision power operation safety distance measurement method, characterized in that: The following steps are involved: S1: Using a binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, wherein the binocular images include: a left image and a right image; and the binocular parameters include: a baseline distance and a focal length; S2: preprocessing the acquired binocular images, including: correction, denoising, image enhancement and grayscale; preprocessing the acquired binocular parameters, including: data cleaning and standardization; S3: performing target recognition using a target recognition algorithm based on the preprocessed binocular image; performing feature extraction on the binocular image based on the target recognition result, and performing feature matching based on the feature extraction result; S4: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; analyze and calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model; S5: Based on the three-dimensional spatial model, analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site; Among them, in step S4, according to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; according to each matched pixel point, a cost function is obtained according to the pixel values ​​in the left image and the right image and a weighted average formula; the cumulative cost of each pixel point is obtained according to the difference between the cost function and the pixel value; the disparity value is obtained by using the cumulative cost and the objective function; a disparity map is constructed by matching the disparity value of each pixel; the depth of each pixel point is calculated according to the disparity map analysis, and the three-dimensional coordinates of each pixel point are calculated by the disparity map and binocular parameter analysis and then converted to the three-dimensional coordinates of the world coordinate system; point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud by a surface reconstruction algorithm; Wherein, constructing a disparity map according to the result of feature matching in step S4 includes the following steps: According to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; According to each matching pixel point, the cumulative cost is calculated by analyzing along each path, through the analytical formula: ； ； Get the cumulative cost ,in, represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, , and It means the weight coefficients are 0.4, 0.3 and 0.3 respectively. is a constant ; and represents the penalty coefficient; and is the indicator function, when and If true, the value is 1, otherwise it is 0. Indicates the disparity value of the previous pixel or adjacent pixel on the path; By analyzing the formula: ; Get the disparity value ;in, , , and Represent the cumulative costs in the horizontal, vertical, left diagonal, and right diagonal directions, respectively. , and is the weight coefficient; remove false matches by checking that the disparity values ​​of the pixels in the left image at the corresponding positions in the right image are different; and construct a disparity map by matching the disparity values ​​for each pixel.

2. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S3, target recognition is performed using a target recognition algorithm according to the preprocessed binocular image, including the following steps: Target recognition is performed using a target recognition algorithm based on the preprocessed binocular image to identify key objects at the power operation site in real time, including power operation personnel, live equipment, and large power operation vehicles; Generate a historical image dataset by collecting historical binocular images; annotate key objects in the historical binocular images by using image annotation tools, the annotations include: the category and location of the key objects; the categories of the key objects include: power workers, live equipment and large power operation vehicles, and the location of the key objects is obtained through the bounding box; The historical image dataset containing annotations is divided into a training set and a test set; the target recognition is performed by using the YOLO model of the target recognition algorithm and an attention mechanism is introduced into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, the training set is input into the YOLO model with the attention mechanism for model training; According to the trained YOLO model, the binocular images of the power operation site that have been collected and preprocessed in real time are input into the trained YOLO model, and the key objects in the binocular images are automatically identified and the recognition results are output. The recognition results include: the category and position of the key objects.

3. A binocular stereo vision power operation safety distance measurement method according to claim 2, characterized in that: In step S3, feature extraction is performed on the binocular image according to the target recognition result, and feature matching is performed according to the feature extraction result, which includes the following steps: According to the category and position of the key object identified in the binocular image, by using a feature point detection algorithm, the feature points of the key object are detected and extracted in the left image and the right image respectively, including corner points and edge points; Describe the features based on the extracted feature points; generate feature vectors by quantifying the local features of the area around the feature points, including: the gradient direction and size of the surrounding area; By using a feature point matching algorithm, the feature points of the left image are matched with the feature points of the right image to obtain a set of matching point pairs, which are represented as a first matching result; according to the RANSAC random sample consensus algorithm, the second matching result of the mismatched point pairs is removed; by analyzing the score difference between the first matching result and the second matching result, the mismatched point pairs are filtered to obtain the feature matching result; According to the result of feature matching, the matched feature points are drawn on the left image and the right image respectively.

4. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: The step S4 calculates the depth of each pixel according to the disparity map and binocular parameter analysis, including the following steps: According to the constructed disparity map, by analyzing the depth formula of each pixel: ; Get the depth of each pixel ;in, represents focal length, B represents baseline distance, Indicates the disparity value of each pixel.

5. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S4, the three-dimensional coordinates of each pixel are analyzed and calculated according to the triangulation principle and the three-dimensional space model is reconstructed, including the following steps: The depth of each pixel calculated based on the disparity map analysis , and calculate the 3D coordinates of each pixel through disparity map and binocular parameter analysis to reconstruct the 3D space model; according to each pixel in the disparity map, the pixel coordinates of the same name point in the left image are set to , then the pixel coordinates of the same-name point in the right image are ,in, ; Using the principle of triangulation, by analyzing the formula: ; Get the three-dimensional coordinates (X, Y, Z); where B represents the baseline length and f represents the focal length of the camera. Represents the disparity value of each pixel, Indicates the depth of each pixel; The three-dimensional coordinates are converted from the three-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system by using the rotation matrix and translation vector of the stereo camera; By analyzing the formula: , get the three-dimensional coordinates of the world coordinate system ( ),in( ) is the three-dimensional coordinate of the camera coordinate system, R is the rotation matrix, and T is the translation vector; Repeat the above steps for each pixel point, calculate the 3D coordinates of each pixel point in the disparity map, and combine the 3D coordinates of each pixel point to form a point cloud. Then, reconstruct the 3D space model from the point cloud by using the surface reconstruction algorithm.

6. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S5, the safe distances between power workers, live equipment and large aerial work vehicles are analyzed and calculated based on the three-dimensional space model, including the following steps: The three-dimensional coordinates of the power workers are obtained according to the three-dimensional space model ( ), three-dimensional coordinates of live equipment ( ) and the three-dimensional coordinates of large aerial work vehicles ( ); By analyzing the formula: ; Get the distance between power workers and live equipment ; By analyzing the formula: ; Get the distance between power workers and large aerial work vehicles ; By analyzing the formula: ; Get the distance between large aerial work vehicles and live equipment ; The calculated distances between power workers, live equipment and large aerial work vehicles are compared with the safety thresholds between power workers and live equipment, between power workers and large aerial work vehicles, and between large aerial work vehicles and live equipment to measure the safety distances: when When the distance between the power workers and the live equipment is a safe distance, otherwise it is not a safe distance; when When the distance between the power workers and the large aerial work vehicles is a safe distance, otherwise it is not a safe distance; when When the distance between large aerial work vehicles and live equipment is a safe distance, otherwise it is not a safe distance; It is the safety threshold between power workers and live equipment. It is the safety threshold between power workers and large aerial work vehicles. It is the safety threshold between large aerial work vehicles and live equipment.

7. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: The comprehensive analysis and assessment of the safety of the power operation site in step S5 includes the following steps: According to the calculated safety distance between power workers, live equipment and large aerial work vehicles, the analytical formula is: ; Get the safety estimate T of the power operation site; when T If it is 1, it means the power operation site is safe, otherwise it means there are safety hazards at the power operation site.

8. A binocular stereo vision electric power operation safety distance measurement system, using a binocular stereo vision electric power operation safety distance measurement method as claimed in any one of claims 1 to 7, characterized in that: Includes the following modules: Image acquisition module: through the binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, the binocular images include: left image and right image; the binocular parameters include: baseline distance and focal length; Image preprocessing module: preprocess the acquired binocular images, including correction, denoising, image enhancement and grayscale conversion; preprocess the acquired binocular parameters, including data cleaning and standardization; Image processing module: Use target recognition algorithm to identify the target according to the preprocessed binocular image; extract features of the binocular image according to the target recognition result, and perform feature matching according to the feature extraction result; Image analysis module: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model; Measurement and evaluation module: Based on the three-dimensional space model, analyze and calculate the safe distance between power workers, energized equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site.

Images