Binocular stereoscopic vision electric power operation safety distance measurement method and system

Abstract

The invention discloses a binocular stereoscopic vision electric power operation safety distance measurement method and system, relates to the technical field of image processing, and solves the problem that target identification is difficult to carry out according to a binocular image; secondly, feature extraction is difficult to carry out according to a target recognition result, and feature matching is difficult to carry out; thirdly, a three-dimensional space model is difficult to reconstruct after a disparity map is difficult to construct according to a feature matching result; and finally, the technical problems of difficulty in analyzing and calculating the safety distance among the electric power operation personnel, the electrified equipment and the large-scale high-altitude operation vehicle and difficulty in comprehensively analyzing and evaluating the safety of the electric power operation site are solved. According to the method, feature extraction and feature matching are carried out after electric power operation personnel, live-line equipment and a large-scale high-altitude operation vehicle in a binocular image are identified; the safety distances among the electric power operating personnel, the live-line equipment and the large overhead working vehicle are measured by constructing a disparity map and reconstructing a three-dimensional space model.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a binocular stereoscopic vision electric power operation safety distance measurement method and system. Background Art

[0002] Binocular stereo vision is to capture two images of the same scene from different positions with two cameras and calculate the parallax between corresponding points in the two images to obtain the three-dimensional geometric information of the object. Binocular stereo vision technology can provide non-contact high-precision distance measurement and is suitable for various complex environments and scenes. With the application of binocular stereo vision technology in the measurement of safe distance for power operations, binocular cameras are used to collect binocular images and perform stereo matching to obtain spatial information of the power operation site. By measuring the safe distance for power operations, the safety status of the power operation site can be monitored in real time to improve the safety of the power working environment.

[0003] The existing binocular stereo vision safety distance measurement method and system for power operations have the following problems: first, it is difficult to identify the target based on the binocular image; second, it is difficult to extract features based on the target recognition results, and it is difficult to perform feature matching; then, it is difficult to construct a disparity map based on the result of feature matching and reconstruct the three-dimensional space model; finally, it is difficult to analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles, and it is difficult to comprehensively analyze and evaluate the safety of the power operation site. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a binocular stereoscopic vision power operation safety distance measurement method and system, which are used to solve the following technical problems:

[0005] First, it is difficult to identify the target based on the binocular image; second, it is difficult to extract features based on the target recognition results, and it is difficult to perform feature matching; then, it is difficult to reconstruct the three-dimensional space model after constructing the disparity map based on the result of feature matching; finally, it is difficult to analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles, and it is difficult to comprehensively analyze and evaluate the safety of the power operation site.

[0006] To solve the above problems, a first aspect of the present invention provides a binocular stereo vision power operation safety distance measurement method, comprising the following steps:

[0007] S1: Using a binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, wherein the binocular images include: a left image and a right image; and the binocular parameters include: a baseline distance and a focal length;

[0008] S2: preprocessing the acquired binocular images, including: correction, denoising, image enhancement and grayscale; preprocessing the acquired binocular parameters, including: data cleaning and standardization;

[0009] S3: performing target recognition using a target recognition algorithm based on the preprocessed binocular image; performing feature extraction on the binocular image based on the target recognition result, and performing feature matching based on the feature extraction result;

[0010] S4: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; analyze and calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model;

[0011] S5: Based on the three-dimensional spatial model, analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site; Among them, in step S4, according to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; according to each matched pixel point, a cost function is obtained according to the pixel values ​​in the left image and the right image and using a weighted average formula; the cumulative cost of each pixel point is obtained according to the difference between the cost function and the pixel value; the disparity value is obtained by using the cumulative cost and then using the objective function; a disparity map is constructed by matching the disparity value for each pixel; the depth of each pixel point is calculated according to the disparity map analysis, and the three-dimensional coordinates of each pixel point are calculated by the disparity map and binocular parameter analysis and then converted to the three-dimensional coordinates of the world coordinate system; point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud by a surface reconstruction algorithm.

[0012] As a further solution of the present invention: in step S3, target recognition is performed using a target recognition algorithm according to the preprocessed binocular image, comprising the following steps:

[0013] Target recognition is performed using a target recognition algorithm based on the preprocessed binocular image to identify key objects at the power operation site in real time, including power operation personnel, live equipment, and large power operation vehicles;

[0014] Generate a historical image dataset by collecting historical binocular images; annotate key objects in the historical binocular images by using image annotation tools, the annotations include: the category and location of the key objects; the categories of the key objects include: power workers, live equipment and large power operation vehicles, and the location of the key objects is obtained through the bounding box;

[0015] The historical image dataset containing annotations is divided into a training set and a test set; the target recognition is performed by using the YOLO model of the target recognition algorithm and an attention mechanism is introduced into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, the training set is input into the YOLO model with the attention mechanism for model training;

[0016] According to the trained YOLO model, the binocular images of the power operation site that have been collected and preprocessed in real time are input into the trained YOLO model, and the key objects in the binocular images are automatically identified and the recognition results are output. The recognition results include: the category and position of the key objects.

[0017] As a further solution of the present invention: in step S3, feature extraction is performed on the binocular image according to the target recognition result, and feature matching is performed according to the feature extraction result, including the following steps:

[0018] According to the category and position of the key object identified in the binocular image, by using a feature point detection algorithm, the feature points of the key object are detected and extracted in the left image and the right image respectively, including corner points and edge points;

[0019] Describe the features based on the extracted feature points; generate feature vectors by quantifying the local features of the area around the feature points, including: the gradient direction and size of the surrounding area;

[0020] By using a feature point matching algorithm, the feature points of the left image are matched with the feature points of the right image to obtain a set of matching point pairs, which are represented as a first matching result; according to the RANSAC random sample consensus algorithm, the second matching result of the mismatched point pairs is removed; by analyzing the score difference between the first matching result and the second matching result, the mismatched point pairs are filtered to obtain the feature matching result;

[0021] According to the result of feature matching, the matched feature points are drawn on the left image and the right image respectively.

[0022] As a further solution of the present invention: constructing a disparity map according to the result of feature matching in step S4 includes the following steps:

[0023] According to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction;

[0024] According to each matching pixel point, the cumulative cost is calculated by analyzing along each path, through the analytical formula:

[0025]

[0026]

[0027] Get the cumulative cost ,in, represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, , and It means the weight coefficients are 0.4, 0.3 and 0.3 respectively. is a constant ; and represents the penalty coefficient; and is the indicator function, when and If true, the value is 1, otherwise it is 0. Indicates the disparity value of the previous pixel or adjacent pixel on the path; By analyzing the formula:

[0028] Get the disparity value ;in, , , and Represent the cumulative costs in the horizontal, vertical, left diagonal, and right diagonal directions, respectively. , and is the weight coefficient; remove false matches by checking that the disparity values ​​of the pixels in the left image at the corresponding positions in the right image are different; and construct a disparity map by matching the disparity values ​​for each pixel.

[0029] As a further solution of the present invention: the depth of each pixel is calculated according to the disparity map and binocular parameter analysis in step S4, comprising the following steps:

[0030] According to the constructed disparity map, by analyzing the depth formula of each pixel:

[0031] Get the depth of each pixel ;in, represents focal length, B represents baseline distance, Indicates the disparity value of each pixel.

[0032] As a further solution of the present invention: the step S4 analyzes and calculates the three-dimensional coordinates of each pixel point and reconstructs the three-dimensional space model according to the triangulation principle, including the following steps:

[0033] The depth of each pixel calculated based on the disparity map analysis , and calculate the 3D coordinates of each pixel through disparity map and binocular parameter analysis to reconstruct the 3D space model; according to each pixel in the disparity map, the pixel coordinates of the same name point in the left image are set to ( , ), then the pixel coordinates of the same-name point in the right image are ( , ),in, ;

[0034] Using the principle of triangulation, by analyzing the formula:

[0035] Get the three-dimensional coordinates (X, Y, Z); where B represents the baseline length and f represents the focal length of the camera. Represents the disparity value of each pixel, Indicates the depth of each pixel;

[0036] The three-dimensional coordinates are converted from the three-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system by using the rotation matrix and translation vector of the stereo camera;

[0037] By analyzing the formula: , get the three-dimensional coordinates of the world coordinate system ( ),in( ) is the three-dimensional coordinate of the camera coordinate system, R is the rotation matrix, and T is the translation vector;

[0038] Repeat the above steps for each pixel point, calculate the 3D coordinates of each pixel point in the disparity map, and combine the 3D coordinates of each pixel point to form a point cloud. Then, reconstruct the 3D space model from the point cloud by using the surface reconstruction algorithm.

[0039] As a further solution of the present invention: in step S5, the safe distance between the power workers, the live equipment and the large aerial work vehicle is analyzed and calculated according to the three-dimensional space model, including the following steps:

[0040] The three-dimensional coordinates of the power workers are obtained according to the three-dimensional space model ( ), three-dimensional coordinates of live equipment ( ) and the three-dimensional coordinates of large aerial work vehicles ( );

[0041] By analyzing the formula:

[0042] Get the distance between power workers and live equipment ;

[0043] By analyzing the formula:

[0044] Get the distance between power workers and large aerial work vehicles ;

[0045] By analyzing the formula:

[0046] Get the distance between large aerial work vehicles and live equipment ;

[0047] The calculated distances between power workers, live equipment and large aerial work vehicles are compared with the safety thresholds between power workers and live equipment, between power workers and large aerial work vehicles, and between large aerial work vehicles and live equipment to measure the safety distances:

[0048] when When the distance between the power workers and the live equipment is a safe distance, otherwise it is not a safe distance; when When the distance between the power workers and the large aerial work vehicles is a safe distance, otherwise it is not a safe distance; when When the distance between large aerial work vehicles and live equipment is a safe distance, otherwise it is not a safe distance; It is the safety threshold between power workers and live equipment. It is the safety threshold between power workers and large aerial work vehicles. It is the safety threshold between large aerial work vehicles and live equipment.

[0049] As a further solution of the present invention: the comprehensive analysis and evaluation of the safety of the power operation site in step S5 includes the following steps:

[0050] According to the calculated safety distance between power workers, live equipment and large aerial work vehicles, the analytical formula is:

[0051] Get the safety estimate T of the power operation site; when T If it is 1, it means the power operation site is safe, otherwise it means there are safety hazards at the power operation site.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention uses a target recognition algorithm and an attention mechanism to accurately and quickly identify power workers, live equipment, and large aerial work vehicles in binocular images, thereby reducing recognition errors and improving work efficiency. The invention also performs feature matching after performing feature extraction on binocular images based on target recognition results, thereby reducing interference from irrelevant information and improving matching accuracy and efficiency.

[0054] The present invention constructs a disparity map through the result of feature matching, and obtains the depth of each pixel through binocular parameter analysis; after analyzing the three-dimensional coordinates of each pixel through the triangulation principle, a three-dimensional space model is reconstructed, which provides a reliable basis for subsequent safety distance analysis;

[0055] The present invention measures the safe distances between power workers, live equipment and large aerial work vehicles through a three-dimensional space model; comprehensively analyzes and evaluates the safety of the power operation site through the measurement results, providing a scientific safety assessment method for the power operation site. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a flow chart of the method of the present invention;

[0058] Figure 2 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 As shown, the first embodiment of the present invention provides a binocular stereo vision power operation safety distance measurement method and system, comprising the following steps:

[0061] S1: Using a binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, wherein the binocular images include: a left image and a right image; and the binocular parameters include: a baseline distance and a focal length;

[0062] S2: preprocessing the acquired binocular images, including: correction, denoising, image enhancement and grayscale; preprocessing the acquired binocular parameters, including: data cleaning and standardization;

[0063] S3: performing target recognition using a target recognition algorithm based on the preprocessed binocular image; performing feature extraction on the binocular image based on the target recognition result, and performing feature matching based on the feature extraction result;

[0064] S4: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; analyze and calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model;

[0065] S5: Based on the three-dimensional spatial model, analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site; Among them, in step S4, according to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; according to each matched pixel point, a cost function is obtained according to the pixel values ​​in the left image and the right image and using a weighted average formula; the cumulative cost of each pixel point is obtained according to the difference between the cost function and the pixel value; the disparity value is obtained by using the cumulative cost and then using the objective function; a disparity map is constructed by matching the disparity value for each pixel; the depth of each pixel point is calculated according to the disparity map analysis, and the three-dimensional coordinates of each pixel point are calculated by the disparity map and binocular parameter analysis and then converted to the three-dimensional coordinates of the world coordinate system; point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud by a surface reconstruction algorithm.

[0066] Specifically, a binocular camera is installed at the power operation site to collect binocular images and binocular parameters of the operation site in real time, wherein the binocular camera should have high precision and high resolution to ensure image quality. Preprocessing the binocular image includes: performing distortion correction on the binocular image to ensure that the image is not distorted; using a filtering algorithm to remove noise in the image; improving the contrast and brightness of the image to make the image clearer; converting the color image into a grayscale image to simplify subsequent processing. Preprocessing the binocular parameters includes: removing invalid or abnormal data in the binocular parameters; converting the binocular parameters into a unified format and unit for subsequent calculation. The power workers, live equipment and large aerial work vehicles are identified in the preprocessed binocular image by using a target recognition algorithm, and features are extracted based on the target recognition results. Matching is performed between the features extracted from the left image and the right image to find the corresponding feature points. Based on the result of feature matching, a disparity map is constructed, which represents the disparity of each pixel in the left and right images. Based on the disparity map and binocular parameters, the depth of each pixel is calculated using the triangulation principle. Combining the depth information and the positional relationship of the binocular camera, the 3D coordinates of each pixel are calculated. Based on the 3D coordinates of all pixels, the 3D spatial model of the power operation site is reconstructed. In the 3D spatial model, the actual distance between power workers, live equipment, and large aerial work vehicles is calculated and compared with the safety threshold, and a comprehensive assessment of the safety status of the power operation site is conducted.

[0067] In one embodiment of the present invention, the step S3 performs target recognition using a target recognition algorithm according to the preprocessed binocular image, including the following steps:

[0068] Target recognition is performed using a target recognition algorithm based on the preprocessed binocular image to identify key objects at the power operation site in real time, including power operation personnel, live equipment, and large power operation vehicles;

[0069] Generate a historical image dataset by collecting historical binocular images; annotate key objects in the historical binocular images by using image annotation tools, the annotations include: the category and location of the key objects; the categories of the key objects include: power workers, live equipment and large power operation vehicles, and the location of the key objects is obtained through the bounding box;

[0070] The historical image dataset containing annotations is divided into a training set and a test set; the target recognition is performed by using the YOLO model of the target recognition algorithm and an attention mechanism is introduced into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, the training set is input into the YOLO model with the attention mechanism for model training;

[0071] According to the trained YOLO model, the binocular images of the power operation site that have been collected and preprocessed in real time are input into the trained YOLO model, and the key objects in the binocular images are automatically identified and the recognition results are output. The recognition results include: the category and position of the key objects.

[0072] Specifically, a large number of historical binocular images are collected to generate a historical image dataset, which includes various scenes of power operation sites, especially key objects such as power operation personnel, live equipment and large power operation vehicles. The collected binocular images are annotated by using image annotation tools, and the annotations include the category and location of the key objects. Among them, the category annotation is used to identify the type of key objects in the image, and the position annotation uses a bounding box to represent the location of the key objects in the image. The annotated historical image dataset is randomly divided into a training set and a test set, wherein the training set is used to train the YOLO model, and the test set is used to evaluate the performance of the YOLO model; when dividing the historical image dataset, it is ensured that the left image and the right image at the same time are in the same training set or test set. The YOLO model is selected as the target recognition algorithm, and on the basis of the YOLO model, the attention mechanism is introduced to improve the recognition performance of the model. The attention mechanism can help the YOLO model pay more attention to the key areas in the image, thereby improving the accuracy and robustness of recognition. Add an attention module to the feature extraction network of the YOLO model and adjust the parameters of the YOLO model according to the actual situation, including learning rate, batch size, number of iterations, etc., to ensure that the YOLO model can fully learn the features in the image. Input the training set into the YOLO model with attention mechanism for training. During the training process, the YOLO model will continuously optimize its parameters to improve the recognition performance of key objects. Collect binocular images of the power operation site in real time and preprocess them. Input the preprocessed binocular images into the trained YOLO model. The trained YOLO model automatically identifies the key objects in the binocular images and outputs the categories and locations of the key objects.

[0073] In one embodiment of the present invention, the step S3 extracts features from the binocular image according to the target recognition result, and performs feature matching according to the feature extraction result, including the following steps:

[0074] According to the category and position of the key object identified in the binocular image, by using a feature point detection algorithm, the feature points of the key object are detected and extracted in the left image and the right image respectively, including corner points and edge points;

[0075] Describe the features based on the extracted feature points; generate feature vectors by quantifying the local features of the area around the feature points, including: the gradient direction and size of the surrounding area;

[0076] By using a feature point matching algorithm, the feature points of the left image are matched with the feature points of the right image to obtain a set of matching point pairs, which are represented as a first matching result; according to the RANSAC random sample consensus algorithm, the second matching result of the mismatched point pairs is removed; by analyzing the score difference between the first matching result and the second matching result, the mismatched point pairs are filtered to obtain the feature matching result;

[0077] According to the result of feature matching, the matched feature points are drawn on the left image and the right image respectively.

[0078] Specifically, according to the target recognition results, in the left image and the right image, for the location area of ​​each key object, a feature point detection algorithm is applied, including: Harris corner detection, Shi-Tomasi corner detection or SIFT, etc. to detect corner points and edge points. For each detected feature point, the gradient direction and size of the surrounding area are calculated to generate a feature vector. The feature vector should be able to describe the local texture and shape information around the feature point so as to make an effective comparison in the subsequent matching process. Using a feature point matching algorithm, including: brute force matching, FLANN matching, etc., the feature points in the left image are preliminarily matched with the feature points in the right image, where the result of the preliminary matching is a set of matching point pairs as the first matching result. The mismatched point pairs in the first matching result are removed by applying the RANSAC random sample consensus algorithm. The RANSAC algorithm estimates a basic matrix or homography matrix by randomly selecting a set of feature point pairs, and evaluates the consistency of the matching point pairs based on these matrices. After being processed by the RANSAC algorithm, a more accurate set of matching point pairs is obtained, which is called the second matching result. By analyzing the score difference between the first matching result and the second matching result, the score can be calculated based on the distance, similarity or other metrics between the matching point pairs. By setting a score threshold, the matching point pairs with lower scores are filtered out, thereby further reducing mismatches; wherein, the selection of the score threshold can be based on actual experience or experiments, and a lower quantile in the score distribution, such as the 10% or 20% quantile, is usually selected as the threshold. According to the result of feature matching, the matched feature points are drawn on the left image and the right image respectively by drawing dots, line segments or other marks at the position of the feature points.

[0079] In one embodiment of the present invention, constructing a disparity map according to the result of feature matching in step S4 includes the following steps:

[0080] According to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction;

[0081] According to each matching pixel point, the cumulative cost is calculated by analyzing along each path, through the analytical formula:

[0082]

[0083]

[0084] Get the cumulative cost ,in, represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, , and It means the weight coefficients are 0.4, 0.3 and 0.3 respectively. is a constant ; and represents the penalty coefficient; and is the indicator function, when and If true, the value is 1, otherwise it is 0. Indicates the disparity value of the previous pixel or adjacent pixel on the path; By analyzing the formula:

[0085] Get the disparity value ;in, , , and Represent the cumulative costs in the horizontal, vertical, left diagonal, and right diagonal directions, respectively. , , and is the weight coefficient; remove false matches by checking that the disparity values ​​of the pixels in the left image at the corresponding positions in the right image are different; and construct a disparity map by matching the disparity values ​​for each pixel.

[0086] Specifically, based on the result of feature matching, a path set is constructed using a semi-global matching algorithm, including horizontal, vertical and diagonal paths, which will be used to calculate the cumulative cost. and The penalty coefficients are 10 and 120 respectively; the penalty coefficient is used to control the smoothness of the parallax change, and the constant term is used to avoid division by zero, which is dynamically adjusted according to the actual situation. The absolute value of the brightness difference between the two pixels of the left image and the right image, the square of the brightness difference between the two pixels, and the normalized square difference are multiplied by the weight coefficient , and Get the cost function, where the weight coefficient , and Dynamically adjust according to actual conditions. For each matched pixel point, after analyzing and calculating the cumulative cost along each path, by comparing the cumulative costs on different paths, the disparity value with the minimum cumulative cost can be selected as the optimal disparity value for the current pixel point. Among them, the weight coefficient of the cumulative cost in the horizontal direction is The weight coefficient of the vertical accumulative cost is 0.4. The weight coefficient of the cumulative cost in the left diagonal direction is 0.3 The weight coefficient of the accumulative cost is 0.2 and the right diagonal direction The weight coefficient is 0.1, and the weight coefficient is dynamically adjusted according to the actual content of the binocular image. After obtaining the initial disparity map, mismatch removal is performed by checking whether the disparity values ​​of the pixels in the left image and the corresponding positions in the right image are the same. If they are not the same, it may indicate that there is a mismatch, and these points need to be removed from the disparity map. After the above steps, a disparity map can be obtained, which reflects the disparity relationship between each pixel in the left image and the right image.

[0087] In one embodiment of the present invention, the step S4 of calculating the depth of each pixel according to the disparity map and binocular parameter analysis includes the following steps:

[0088] According to the constructed disparity map, by analyzing the depth formula of each pixel:

[0089] Get the depth of each pixel ;in, represents focal length, B represents baseline distance, Indicates the disparity value of each pixel.

[0090] Specifically, the depth of each pixel is calculated by analyzing a formula based on the generated disparity map combined with binocular parameters, including focal length and baseline distance.

[0091] In one embodiment of the present invention, the step S4 analyzes and calculates the three-dimensional coordinates of each pixel point and reconstructs the three-dimensional space model according to the triangulation principle, including the following steps:

[0092] The depth of each pixel calculated based on the disparity map analysis , and calculate the 3D coordinates of each pixel through disparity map and binocular parameter analysis to reconstruct the 3D space model; according to each pixel in the disparity map, the pixel coordinates of the same name point in the left image are set to ( , ), then the pixel coordinates of the same-name point in the right image are ( , ),in, ;

[0093] Using the principle of triangulation, by analyzing the formula:

[0094] Get the three-dimensional coordinates (X, Y, Z); where B represents the baseline length and f represents the focal length of the camera. Represents the disparity value of each pixel, Indicates the depth of each pixel;

[0095] The three-dimensional coordinates are converted from the three-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system by using the rotation matrix and translation vector of the stereo camera;

[0096] By analyzing the formula: , get the three-dimensional coordinates of the world coordinate system ( ),in( ) is the three-dimensional coordinate of the camera coordinate system, R is the rotation matrix, and T is the translation vector;

[0097] Repeat the above steps for each pixel point, calculate the 3D coordinates of each pixel point in the disparity map, and combine the 3D coordinates of each pixel point to form a point cloud. Then, reconstruct the 3D space model from the point cloud by using the surface reconstruction algorithm.

[0098] Specifically, the coordinates of each pixel are obtained according to the disparity map, and the three-dimensional coordinates are obtained by the triangulation principle. The three-dimensional coordinates are converted from the camera coordinate system to the world coordinate system by using the rotation matrix and translation vector of the binocular camera. The above steps are repeated for each pixel in the disparity map, the three-dimensional coordinates of each pixel in the disparity map are analyzed and calculated, and these three-dimensional coordinates are combined to form a point cloud. Surface reconstruction algorithms, including Delaunay triangulation, BPA algorithm, Power Crust algorithm, etc., are used to generate continuous triangular meshes based on the formed point cloud data, thereby constructing the three-dimensional surface of the object and reconstructing the three-dimensional space model.

[0099] In one embodiment of the present invention, the step S5 analyzes and calculates the safe distance between the power workers, the live equipment and the large aerial work vehicle according to the three-dimensional space model, including the following steps:

[0100] The three-dimensional coordinates of the power workers are obtained according to the three-dimensional space model ( ), three-dimensional coordinates of live equipment ( ) and the three-dimensional coordinates of large aerial work vehicles ( );

[0101] By analyzing the formula:

[0102] Get the distance between power workers and live equipment ;

[0103] By analyzing the formula:

[0104] Get the distance between power workers and large aerial work vehicles ;

[0105] By analyzing the formula:

[0106] Get the distance between large aerial work vehicles and live equipment ;

[0107] The calculated distances between power workers, live equipment and large aerial work vehicles are compared with the safety thresholds between power workers and live equipment, between power workers and large aerial work vehicles, and between large aerial work vehicles and live equipment to measure the safety distances:

[0108] when When the distance between the power workers and the live equipment is a safe distance, otherwise it is not a safe distance; when When the distance between the power workers and the large aerial work vehicles is a safe distance, otherwise it is not a safe distance; when When the distance between large aerial work vehicles and live equipment is a safe distance, otherwise it is not a safe distance; It is the safety threshold between power workers and live equipment. It is the safety threshold between power workers and large aerial work vehicles. It is the safety threshold between large aerial work vehicles and live equipment.

[0109] Specifically, the three-dimensional coordinates of the power workers, live equipment and large aerial work vehicles are obtained according to the three-dimensional space model, and the distance between the power workers and live equipment, the distance between the power workers and large aerial work vehicles, and the distance between large aerial work vehicles and live equipment are obtained by using the Euclidean distance formula. The calculated distances are compared with the safety thresholds to determine whether they are safe distances. The safety thresholds are determined based on national standards and actual conditions and can be adjusted dynamically.

[0110] In one embodiment of the present invention, the comprehensive analysis and evaluation of the safety of the power operation site in step S5 includes the following steps:

[0111] According to the calculated safety distance between power workers, live equipment and large aerial work vehicles, the analytical formula is:

[0112] Get the safety estimate T of the power operation site; when T If it is 1, it means the power operation site is safe, otherwise it means there are safety hazards at the power operation site.

[0113] Specifically, based on the calculated safe distances between power workers, live equipment, and large aerial work vehicles, the safety estimate of the power operation site is obtained through a formula. The threshold is set to 1, and the safety estimate of the power operation site is compared and analyzed with the threshold to evaluate the safety of the power site in real time; the threshold can be dynamically adjusted according to actual conditions or historical data.

[0114] See also Figure 2 As shown, the present invention is a binocular stereo vision power operation safety distance measurement system, comprising the following modules:

[0115] Image acquisition module: through the binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, the binocular images include: left image and right image; the binocular parameters include: baseline distance and focal length;

[0116] Image preprocessing module: preprocess the acquired binocular images, including correction, denoising, image enhancement and grayscale conversion; preprocess the acquired binocular parameters, including data cleaning and standardization;

[0117] Image processing module: Use target recognition algorithm to identify the target according to the preprocessed binocular image; extract features of the binocular image according to the target recognition result, and perform feature matching according to the feature extraction result;

[0118] Image analysis module: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model;

[0119] Measurement and evaluation module: Based on the three-dimensional space model, analyze and calculate the safe distance between power workers, energized equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site.

[0120] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A binocular stereo vision power operation safety distance measurement method, characterized in that: The following steps are involved: S1: Using a binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, wherein the binocular images include: a left image and a right image; and the binocular parameters include: a baseline distance and a focal length; S2: preprocessing the acquired binocular images, including: correction, denoising, image enhancement and grayscale; preprocessing the acquired binocular parameters, including: data cleaning and standardization; S3: performing target recognition using a target recognition algorithm based on the preprocessed binocular image; performing feature extraction on the binocular image based on the target recognition result, and performing feature matching based on the feature extraction result; S4: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; analyze and calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model; S5: Based on the three-dimensional spatial model, analyze and calculate the safe distance between power workers, live equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site; Among them, in step S4, according to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; according to each matched pixel point, a cost function is obtained according to the pixel values ​​in the left image and the right image and using a weighted average formula; the cumulative cost of each pixel point is obtained according to the difference between the cost function and the pixel value; the disparity value is obtained by using the cumulative cost and then using the objective function; a disparity map is constructed by matching the disparity value for each pixel; the depth of each pixel point is calculated according to the disparity map analysis, and the three-dimensional coordinates of each pixel point are calculated by the disparity map and binocular parameter analysis and then converted to the three-dimensional coordinates of the world coordinate system; point cloud data is generated, and a three-dimensional space model is reconstructed from the point cloud by a surface reconstruction algorithm.

2. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S3, target recognition is performed using a target recognition algorithm according to the preprocessed binocular image, including the following steps: Target recognition is performed using a target recognition algorithm based on the preprocessed binocular image to identify key objects at the power operation site in real time, including power operation personnel, live equipment, and large power operation vehicles; Generate a historical image dataset by collecting historical binocular images; annotate key objects in the historical binocular images by using image annotation tools, the annotations include: the category and location of the key objects; the categories of the key objects include: power workers, live equipment and large power operation vehicles, and the location of the key objects is obtained through the bounding box; The historical image dataset containing annotations is divided into a training set and a test set; the target recognition is performed by using the YOLO model of the target recognition algorithm and an attention mechanism is introduced into the YOLO model; after adding an attention module to the feature extraction network of the YOLO model and adjusting the YOLO model parameters, the training set is input into the YOLO model with the attention mechanism for model training; According to the trained YOLO model, the binocular images of the power operation site that have been collected and preprocessed in real time are input into the trained YOLO model, and the key objects in the binocular images are automatically identified and the recognition results are output. The recognition results include: the category and position of the key objects.

3. A binocular stereo vision power operation safety distance measurement method according to claim 2, characterized in that: In step S3, feature extraction is performed on the binocular image according to the target recognition result, and feature matching is performed according to the feature extraction result, which includes the following steps: According to the category and position of the key object identified in the binocular image, by using a feature point detection algorithm, the feature points of the key object are detected and extracted in the left image and the right image respectively, including corner points and edge points; Describe the features based on the extracted feature points; generate feature vectors by quantifying the local features of the area around the feature points, including: the gradient direction and size of the surrounding area; By using a feature point matching algorithm, the feature points of the left image are matched with the feature points of the right image to obtain a set of matching point pairs, which are represented as a first matching result; according to the RANSAC random sample consensus algorithm, the second matching result of the mismatched point pairs is removed; by analyzing the score difference between the first matching result and the second matching result, the mismatched point pairs are filtered to obtain the feature matching result; According to the result of feature matching, the matched feature points are drawn on the left image and the right image respectively.

4. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: The step S4 constructs a disparity map according to the result of feature matching, including the following steps: According to the result of feature matching, a path set is constructed by using a semi-global matching algorithm, including: horizontal direction, vertical direction and diagonal direction; According to each matching pixel point, the cumulative cost is calculated by analyzing along each path, through the analytical formula: ； ； Get the cumulative cost ,in, represents the cost function, represents the pixel value in the left image, represents the pixel value in the right image, , and It means the weight coefficients are 0.4, 0.3 and 0.3 respectively. is a constant ; and represents the penalty coefficient; and is the indicator function, when and If true, the value is 1, otherwise it is 0. Indicates the disparity value of the previous pixel or adjacent pixel on the path; By analyzing the formula: ; Get the disparity value ;in, , , and Represent the cumulative costs in the horizontal, vertical, left diagonal, and right diagonal directions, respectively. , and is the weight coefficient; remove false matches by checking that the disparity values ​​of the pixels in the left image at the corresponding positions in the right image are different; and construct a disparity map by matching the disparity values ​​for each pixel.

5. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: The step S4 calculates the depth of each pixel according to the disparity map and binocular parameter analysis, including the following steps: According to the constructed disparity map, by analyzing the depth formula of each pixel: ; Get the depth of each pixel ;in, represents focal length, B represents baseline distance, Indicates the disparity value of each pixel.

6. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S4, the three-dimensional coordinates of each pixel are analyzed and calculated according to the triangulation principle and the three-dimensional space model is reconstructed, including the following steps: The depth of each pixel calculated based on the disparity map analysis , and calculate the 3D coordinates of each pixel through disparity map and binocular parameter analysis to reconstruct the 3D space model; according to each pixel in the disparity map, the pixel coordinates of the same name point in the left image are set to ( , ), then the pixel coordinates of the same-name point in the right image are ( , ),in, ; Using the principle of triangulation, by analyzing the formula: ; Get the three-dimensional coordinates (X, Y, Z); where B represents the baseline length and f represents the focal length of the camera. Represents the disparity value of each pixel, Indicates the depth of each pixel; The three-dimensional coordinates are converted from the three-dimensional coordinates of the camera coordinate system to the three-dimensional coordinates of the world coordinate system by using the rotation matrix and translation vector of the stereo camera; By analyzing the formula: , get the three-dimensional coordinates of the world coordinate system ( ),in( ) is the three-dimensional coordinate of the camera coordinate system, R is the rotation matrix, and T is the translation vector; Repeat the above steps for each pixel point, calculate the 3D coordinates of each pixel point in the disparity map, and combine the 3D coordinates of each pixel point to form a point cloud. Then, reconstruct the 3D space model from the point cloud by using the surface reconstruction algorithm.

7. A binocular stereo vision power operation safety distance measurement method according to claim 1, characterized in that: In step S5, the safe distances between power workers, live equipment and large aerial work vehicles are analyzed and calculated based on the three-dimensional space model, including the following steps: The three-dimensional coordinates of the power workers are obtained according to the three-dimensional space model ( ), three-dimensional coordinates of live equipment ( ) and the three-dimensional coordinates of large aerial work vehicles ( ); By analyzing the formula: ; Get the distance between power workers and live equipment ; By analyzing the formula: ; Get the distance between power workers and large aerial work vehicles ; By analyzing the formula: ; Get the distance between large aerial work vehicles and live equipment ; The calculated distances between power workers, live equipment and large aerial work vehicles are compared with the safety thresholds between power workers and live equipment, between power workers and large aerial work vehicles, and between large aerial work vehicles and live equipment to measure the safety distances: when When the distance between the power workers and the live equipment is a safe distance, otherwise it is not a safe distance; when When the distance between the power workers and the large aerial work vehicles is a safe distance, otherwise it is not a safe distance; when When the distance between large aerial work vehicles and live equipment is a safe distance, otherwise it is not a safe distance; It is the safety threshold between power workers and live equipment. It is the safety threshold between power workers and large aerial work vehicles. It is the safety threshold between large aerial work vehicles and live equipment.

8. The binocular stereo vision safety distance measurement method for electric power operation according to claim 1 is characterized in that: The comprehensive analysis and assessment of the safety of the power operation site in step S5 includes the following steps: According to the calculated safety distance between power workers, live equipment and large aerial work vehicles, the analytical formula is: ; Get the safety estimate T of the power operation site; when T If it is 1, it means the power operation site is safe, otherwise it means there are safety hazards at the power operation site.

9. A binocular stereo vision electric power operation safety distance measurement system, using a binocular stereo vision electric power operation safety distance measurement method as claimed in any one of claims 1 to 8, characterized in that: Includes the following modules: Image acquisition module: through the binocular camera, real-time acquisition of binocular images and binocular parameters of the power operation site, the binocular images include: left image and right image; the binocular parameters include: baseline distance and focal length; Image preprocessing module: preprocess the acquired binocular images, including correction, denoising, image enhancement and grayscale conversion; preprocess the acquired binocular parameters, including data cleaning and standardization; Image processing module: Use target recognition algorithm to identify the target according to the preprocessed binocular image; extract features of the binocular image according to the target recognition result, and perform feature matching according to the feature extraction result; Image analysis module: construct a disparity map based on the result of feature matching; calculate the depth of each pixel based on the disparity map and binocular parameter analysis; calculate the three-dimensional coordinates of each pixel based on the triangulation principle and reconstruct the three-dimensional space model; Measurement and evaluation module: Based on the three-dimensional space model, analyze and calculate the safe distance between power workers, energized equipment and large aerial work vehicles; comprehensively analyze and evaluate the safety of the power operation site.

Images