Method and system for recognizing safe distance of near electric work machine based on binocular vision

Abstract

The application provides a kind of near electric operation machine safety distance recognition method and system based on binocular vision, uses binocular camera to collect original image containing transmission line;The collected original image is filtered and denoised;The processed image is input into neural network model one, and transmission line image segmentation processing is carried out;The segmentation image containing transmission line position information after segmentation is superimposed with the original image, and then input into neural network model two, and binocular stereo matching processing is carried out, and depth map is output;According to the depth map, the estimated distance is obtained;The estimated distance is compared with the preset threshold value, if the estimated distance is within the preset threshold value, the distance between the working machine and the transmission line is safe, otherwise, the distance between the working machine and the transmission line is unsafe, and the alarm function is triggered.The beneficial effects of the application are: manpower cost is saved, work safety, reliability and recognition accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of near-electricity operation safety, and in particular to a method and system for recognizing safe distance of near-electricity operation machinery based on binocular vision. BACKGROUND

[0002] In recent years, with the acceleration of national urbanization construction, various types of engineering construction operations are increasing. Due to the large-scale of construction machinery, if the safe distance from the power transmission line is not paid attention to, the power transmission line will be touched by mistake, and a safety accident will occur. Due to the weak safety and legal awareness of individual construction units and personnel, the safety risk of engineering construction projects is increasingly prominent. When construction encounters power facilities, there are many safety hazards. The "Electric Power Facilities Protection Regulations" clearly stipulates that only qualified units can operate near the power transmission line, and the operating personnel must wear safety helmets and wear insulating shoes. The most critical thing is that a certain safe distance must be maintained from the power transmission line during operation. City power transmission lines are often laid along streets, and power transmission lines are easily damaged by external forces, making construction more difficult. If the safe distance from the power transmission line is not paid attention to during construction, a safety accident will occur, and the safe and reliable operation of the power transmission network will be affected.

[0003] The power transmission line is generally in the form of a long and thin shape, which is not easy to observe, and the color is generally black and gray, which is easily confused with the background color on cloudy days. In rainy and snowy weather and in the case of failure, the failure characteristics are not obvious and difficult to detect, increasing the safety risk during construction operations. At present, due to conditions and cost constraints, in the past engineering construction operations, when the construction machinery involved in the power transmission line moves and operates, a special person needs to be dispatched to command, regardless of day or night, to ensure a safe distance, and the construction machinery is strictly prohibited from touching the power transmission line to prevent electric shock accidents. Artificial observation is often carried out by the naked eye or a telescope. Due to the complex construction environment, workers often need to walk or follow the construction vehicle, which is very labor-intensive. However, the fatigue of the human eye and the subjectivity of manual judgment seriously affect the accuracy and automation level of the construction operation process, increasing the workload and safety hazards of personnel. At present, there are also methods such as ultrasonic waves and millimeter wave radars to monitor the safe distance from the power transmission line during construction operations to achieve the purpose of construction operation safety. However, in harsh working environments (such as low-visibility rainy, snowy, and foggy days and mud attached to the surface of the sensor), the stability is reduced and the precision is not high. In addition, in the construction environment with power transmission lines, due to the shape characteristics of the power transmission lines, the sensor distance measurement method is not effective, and problems such as the inability to accurately detect the target, the loss and confusion of the front and back echoes, and the like are easily caused, which can easily lead to safety accidents.

[0004] When the operator carries out field construction, mainly relies on manpower, sensor and other means to estimate distance, but the safety risk is big, the reliability is low. Lack of means to judge distance in the first time for the operator, therefore, it is imminent to study an overall solution to solve the problem. SUMMARY

[0005] In order to solve the above problems, the application provides a kind of near electric operation machinery safety distance recognition method and system based on binocular vision, binocular image segmentation based on coding and decoding network can separate target transmission line from background in the binocular image pair of transmission line collected by binocular camera, achieve the purpose of transmission line positioning and identification;Using the binocular camera built in the construction robot arm to collect the image containing the transmission line, the binocular image stereo matching based on twin residual network can correspond the feature points in the binocular image, output the depth map, and achieve the purpose of distance measurement. Use image information to get estimated distance, and design alarm function, and apply it to the safety problem of construction operation process. The binocular camera is used to interpret the image, realizes the intelligent visual monitoring of construction operation equipment, improves the safety, reliability and economy of construction operation, and has wide market application prospect.

[0006] The recognition method mainly includes the following steps:

[0007] S1: using binocular camera to collect original image containing transmission line;

[0008] S2: filtering and denoising the original image collected in step S1;

[0009] S3: input the image processed in step S2 to neural network model one, and carry out transmission line image segmentation processing;

[0010] S4: superimpose the segmentation image containing transmission line position information after step S3 with the original image, and then input to neural network model two, and carry out binocular stereo matching processing, output depth map;

[0011] S5: get estimated distance according to the depth map;

[0012] S6: compare the estimated distance with the standard safety distance, if the estimated distance is within the safety distance, the distance between the construction machinery and the transmission line is safe, otherwise, the distance between the construction machinery and the transmission line is unsafe, trigger alarm function;

[0013] S7: place the estimated distance and the safety distance on the liquid crystal screen to display, so as to carry out real-time monitoring.

[0014] Further, the same target is synchronously photographed by using the binocular stereo camera after completion of calibration, left eye image and right eye image of different angles are obtained, and original image is formed.

[0015] Further, in step S2, bilateral filtering algorithm is adopted to filter and denoise the spatial proximity and pixel similarity in the original image.

[0016] Further, the first neural network model in step S3 is an encoding-decoding network.

[0017] Further, the second neural network model in step S4 is a twin residual network, which uses the global information of the superimposed image to perform stereo matching on the power line.

[0018] Further, in step S5, the specific steps of obtaining distance information from the depth map include: determining the camera optical center, baseline and focal length, and using the pixel triangle formed by the imaging plane and the object, and the optical center and the object to obtain the distance information.

[0019] A kind of near electric operation mechanical safety distance recognition system based on binocular vision is developed by FPGA programmable logic array platform, the system includes: hardware part and software part, hardware part includes: binocular image acquisition module, image processing module, distance display module, software part includes: target image segmentation module, binocular stereo matching module, distance measurement module, distance alarm module;

[0020] Binocular image acquisition module is the input module of the whole system, the input end of binocular image acquisition module is collected to the original image containing power line by binocular camera image sensor, and image data is acquired;Binocular image acquisition module output end is used to be connected with image processing module;

[0021] Image processing module is used to filter and denoise target power line in image according to the image data input by binocular image acquisition module, the input end of image processing module is connected with the output end of binocular image acquisition module, and the image generated by the output end of image processing module is sent to target image segmentation module;

[0022] Distance display module is the output module of the whole system, for the visualized distance of operation machine and power line, the distance is directly displayed quantitatively;Distance display module input end is connected with distance measurement module and distance alarm module output end;

[0023] Target image segmentation module is used to identify and locate the position of power line in image according to the image filtered and denoised by image processing module, and carries out segmentation processing;The input end of target image segmentation module is connected with the output end of image processing module, and the output end of target image segmentation module is connected with the input end of binocular stereo matching module;

[0024] The binocular stereo matching module is used for binocular stereo matching on the image segmented by the target image segmentation module, so as to obtain a binocular image depth map, and the depth map is used for distance estimation.

[0025] The distance measurement module is used for measuring the safe distance between the binocular camera embedded in the working operation machine and the power transmission line by using the different depths represented by the pixel chroma in the depth map and the trigonometric function and similar triangle principle.

[0026] The distance alarm module is used for alarming the safe distance, and is convenient for setting a threshold to compare the estimated distance obtained by the distance measurement module with the preset standard safe distance, and alarming the distance exceeding the standard safe distance range.

[0027] Further, the target image segmentation module adopts a neural network segmentation algorithm, wherein the neural network image segmentation algorithm includes a threshold-based image segmentation algorithm, an edge detection-based image segmentation algorithm, a genetic algorithm-based image segmentation algorithm, a feature coding-based image segmentation algorithm and a region selection-based image segmentation algorithm.

[0028] Further, the binocular stereo matching module adopts a neural network stereo matching algorithm, wherein the neural network stereo matching includes a feature matching algorithm, a region matching algorithm, a dynamic programming matching algorithm and an image segmentation matching algorithm.

[0029] The technical scheme provided by the present application has the beneficial effects of saving labor cost, improving operation safety, reliability and recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0030] The present application will be further described below in combination with the drawings and examples, wherein:

[0031] Figure 1 is a flow chart of a near-electric operation machine safe distance recognition method based on binocular vision in an embodiment of the present application.

[0032] Figure 2 is an alarm function implementation flow chart in an embodiment of the present application.

[0033] Figure 3 is the schematic diagram of the alarm interface in the embodiment of the present application taking 220kV as an example.

[0034] Figure 4 is the principle diagram of the similar triangle distance conversion in the embodiment of the present application.

[0035] Figure 5 is the framework diagram of a near electric work machine safety distance recognition system based on binocular vision in the embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to have a clearer understanding of the technical features, objectives and effects of the present application, the specific embodiments of the present application will be described in detail with reference to the drawings.

[0037] The embodiment of the present application provides a near electric work machine safety distance recognition method based on binocular vision, and the FPGA can be programmed on site according to specific conditions, and the FPGA can be very conveniently written multiple times, so that on-site debugging and upgrading become very convenient, and the programmable logic device FPGA adopts a parallel processing technology to realize high-speed processing of images, and is widely applied in the field of image processing. Because the parallel processing meets the real-time requirements of tracking, the high integration degree and small volume meet the demand of system portability, the algorithm upgrade is flexible and the cost is relatively low, so we use the FPGA as the processing platform of the electric work machine and the safety distance measuring device of the power transmission line.

[0038] Please refer to Figure 1 , Figure 1 is the flowchart of a near electric work machine safety distance recognition method based on binocular vision in the embodiment of the present application, and specifically includes the following steps:

[0039] S1: using a binocular camera to collect images containing a power transmission line;

[0040] In this embodiment, the area where the power transmission line is located is collected by using a binocular camera. The same target is synchronously photographed by using a binocular stereo camera which is completed with calibration, and left eye images and right eye images of different angles are obtained;

[0041] Further, a multi-view stereo system can also be used, and three cameras arranged in a triangle on the same vertical plane are used for simultaneous shooting; a splicing algorithm in the image processing algorithm is used to obtain a larger area of target display.

[0042] S2: filtering and denoising the collected images containing the power transmission line; the algorithm used for filtering and denoising here includes least square filtering algorithm, bilateral filtering algorithm, median filtering algorithm, mean filtering algorithm, etc.

[0043] In this embodiment, bilateral filtering is used to filter and denoise the image.

[0044] In this step, the bilateral filter is a nonlinear bilateral filter used to smooth the image and reduce noise while preserving edges. The kernel function of the bilateral filter is the result of the combination of the spatial domain kernel and the pixel range domain kernel.

[0045] In flat areas, the difference between the pixel values of adjacent pixels is small, and the corresponding value domain weight is close to 1. At this time, the spatial domain weight plays a major role, which is equivalent to directly performing Gaussian blur on this area. Therefore, the flat area is equivalent to Gaussian blur. In the edge area, the difference between the pixel values of adjacent pixels is large, and the corresponding value domain weight is close to 0, which causes the kernel function to drop, and the current pixel is less affected, thereby preserving the edge detail information of the original image. That is, in areas where the pixel intensity changes little, the bilateral filter has a similar effect to Gaussian filtering, while in areas with large intensity gradients such as image edges, the gradient can be maintained. The formula of bilateral filtering is as follows:

[0046]

[0047] wherein, represents the image data after bilateral filtering; W p represents the weight sum of each pixel value in the filter window, used for weight normalization; G σs represents the spatial domain kernel, which measures the distance between p and q. The farther the distance, the lower the weight; G σr represents the pixel domain kernel, which measures the similarity between the pixel values of p and q. The more similar, the greater the weight; p and q represent two pixels in space, S represents the standard deviation, and I(p) and I(q) represent the pixel values of p and q respectively.

[0048] The calculation method of the above spatial domain kernel G σs is as follows:

[0049]

[0050] The calculation method of the above pixel domain kernel G σr is as follows:

[0051]

[0052] The calculation formula of the above W p is as follows:

[0053]

[0054] Where σs and σr are known, are the input pre-set values, and the other i, j, m, n are values determined in the traversal. Where (i, j) represents the window center value, (m, n) represents a value in the sliding window.

[0055] Further, the median filter can be used, and the median value is obtained by sorting the pixels in a certain neighborhood range of the current pixel point, so as to eliminate the noise. Because the image gray scale is gradually changed, the high-frequency noise can be eliminated by taking the median value. This method performs very well in removing spot noise and salt and pepper noise. At the same time, the weighted least square filter algorithm can be used, which can extract good detail information on multiple scales and prevent possible artifacts. The mathematical expression of the weighted least square filter algorithm is:

[0056]

[0057] Wherein, is a weight coefficient. Here, p represents the spatial position of the pixel point. The first term (u P -g P ) 2 represents that the input image u and the output image g are more similar, and the second term is a regularization term, which minimizes the partial derivative of u to make the output image g smoother.

[0058] S3: input the processed image into the neural network for power line image segmentation processing;

[0059] In view of the characteristics of the power line, such as long and thin shape, thin structure and weak texture, in the embodiment, the collected binocular images are segmented by using an encoding-decoding network. Specifically, the following steps are included: fixing the parameters of the encoder, only optimizing the parameters of the decoder (including: activation function, loss function, learning rate optimization algorithm), inputting the image set after filtering and denoising into the neural network for training, obtaining a segmentation model, segmenting the image, and obtaining the image of the segmented power line.

[0060] The present application uses VGG19 network as the encoder network, VGG19 contains 19 hidden layers (16 convolutional layers and 3 fully connected layers), and introduces a lightweight asymmetric decoder instead of the previous complex symmetric decoder network, which can be trained with limited data set, while significantly reducing the calculation amount and saving the training time. One layer of transposed convolution plus one de-pooling operation is used to gradually restore the image size and improve the training speed. The parameters of the encoder are fixed during the training process, and only the parameters of the decoder are optimized, including:

[0061] The activation function is a ReLU function, and its mathematical expression is as follows:

[0062] f(x)=max(0,x)

[0063] The loss function, used to evaluate the difference between the labeled transmission lines and the predicted transmission lines in the image, is chosen as the pixel-wise loss, which performs well in neural networks. The expression for the loss function is as follows:

[0064]

[0065] Where y is the binary value (label) of the corresponding pixel, and It is the predicted probability of that pixel.

[0066] The learning rate optimization algorithm is used to iteratively update the weight parameters using the Adam algorithm. Updating the weight parameters through sample training allows the obtained weight parameters to be closer to the ideal values, thus minimizing the loss function. The update process can be represented as follows:

[0067] m t+1 =αm t +(1-α)Δx t

[0068] v t+1 =βm t +(1-β)(Δx t ) 2

[0069]

[0070] Where α and β represent the default exponential decay rates of the first and second moment estimates of the Adam optimization algorithm, respectively. In this embodiment, α = 0.9 and β = 0.999, and ε is a parameter used to maintain numerical stability during the optimization process, with a default value of ε = 10. -8 lr represents the learning rate during training, m t m represents the exponential moving average of the gradient update by the optimization algorithm at time t. t+1 Let v represent the exponential moving average at time t+1. t Let v represent the average gradient at time t. t+1 Let Δx represent the average gradient at time t+1. t Let Δx represent the gradient at time step t. t+1 This represents the gradient at time step t+1.

[0071] Furthermore, depending on the different usage scenarios and computing power requirements, region-selection-based RCNN, fastRCNN, and faster RCNN networks can be used to segment the image of the target region of the power transmission line. At the same time, feature-encoding-based VGG, ResNet, and YOLO networks can also be used to segment the image of the target region of the power transmission line.

[0072] S4: superimpose the segmented image containing the power line position information and the original image into a neural network, perform binocular stereo matching processing, and output a depth map;

[0073] In view of the characteristics of the power line, such as long and thin shape, thin structure, and weak texture, in the embodiment, a twin residual neural network model is used as a trained neural network model. Global information of the superimposed image is used for stereo matching of the power line. Specifically, the following steps are included: superimposing the segmented image and the original image in the channel number, extracting features, constructing a 4D cost volume through disparity translation splicing, explaining image space information in multiple dimensions, unfolding the 4D cost volume through 3D convolution, regressing a disparity map through a soft argmin function, and optimizing the disparity map through a depth smoothing loss function.

[0074] In this step, the depth map prediction task is regarded as an output image task, which is used to process the images of the left and right original images and the segmented image stacked in the channel. First, 2D convolution operation is used to extract the feature spectrum of the left and right images. The feature spectrum is shifted and spliced to obtain a 4D cost aggregation body. 3D convolution is used for further feature extraction and transformation. 3D transpose convolution is used to unfold the feature tensor. Finally, a soft argmin function is used to regress a depth map, and a depth smoothing loss function is used to process the depth map. The expression of the soft argmin function is as follows:

[0075]

[0076] where d represents the disparity value, represents the gradient change, and σ(·) represents the softmax function. max represents the maximum value of the disparity value.

[0077] The specific expression of the depth smoothing loss function is as follows:

[0078]

[0079] where N represents the number of image pixels, I represents the pixel value of the input image, d represents the disparity value, and respectively represent the left and right input image data. represents the gradient change.

[0080] Further, different target detection algorithms can be used for different application scenarios, including: feature matching algorithm, region matching algorithm, dynamic programming matching algorithm, belief propagation matching algorithm, and simulated annealing matching algorithm.

[0081] S5: performing mathematical calculation on the depth map to obtain an estimated distance;

[0082] In this embodiment, the similar triangle method is used for depth map to distance information conversion.

[0083] In this step, the distance information is one of the key links, first need to determine the left and right optical center of binocular camera, focal length, baseline and other numerical value, and then through the construction of the optical center and the target, imaging plane and the similar triangle of the target, using the geometric knowledge to get the distance information.

[0084] Further, the distance can be estimated by using the angle value and the trigonometric function, and the different distance information corresponding to the different pixel gray scale in the depth map can also be used.

[0085] S6: compare the estimated distance with the standard safety distance, and trigger the alarm function if the distance is unsafe;

[0086] As shown in Figure 2 , in this embodiment, the threshold method is used to trigger the alarm function. The specific steps of the safety distance alarm include determining the safety distance of the corresponding grade transmission line, setting the safety distance under different voltage grades as the threshold, and then making the difference between the threshold and the estimated distance, that is, whether to alarm = threshold-estimated distance. The result is negative, which means the safety distance and no alarm is needed. The result is positive, which means the unsafe distance and the alarm is triggered.

[0087] S7: as shown in Figure 3 , the estimated distance and the safety distance are displayed on the liquid crystal screen for real-time monitoring. The image display is a visual display of the distance result after the binocular image interpretation, which can directly view the distance between the camera and the transmission line. A high-resolution liquid crystal display is selected, and the liquid crystal screen is driven through the VGA interface. The data signals RED, GREEN, and BLUE, and the control signals HSYNC and VSYNC are used for display control.

[0088] In this embodiment, a high-resolution liquid crystal display is used, which is connected to the FPGA through the VGA interface to achieve the purpose of distance measurement result display.

[0089] In this embodiment, considering that the original image directly collected by the binocular camera contains complex background and a large amount of irrelevant interference information, and the environmental factors also cause interference, a filtering algorithm is used to denoise the original image, reducing the problem of robustness decline caused by noise. Through image segmentation, the position of the transmission line in the whole image is identified and positioned, which is sent into the stereo matching network to solve the problem of low stereo precision caused by the shape of the transmission line. The twin residual network is used for the stereo matching task of the transmission line binocular image, which uses global information to construct a deep network, avoiding the low matching precision caused by the shape of the transmission line, and outputs a high-precision depth map. Figure 4The similar triangle geometry knowledge is shown, the depth map is converted into the distance map, and accurate distance information is obtained; the threshold method is used, the measured distance information is compared with the safety distance, and it is decided whether to trigger the alarm mechanism; the VGA interface is connected with the FPGA platform, the liquid crystal display screen is driven, and the data visualization purpose is achieved.

[0090] A kind of recognition system of safe distance between working machine and transmission line in construction operation based on binocular vision, such as Figure 5 As shown, the safe distance recognition device between working machine and transmission line is developed by FPGA programmable logic array platform, including hardware, software two parts, hardware part includes: binocular image acquisition module 1, image processing module 2, distance display module 3;Software part includes: target image segmentation module 4, binocular stereo matching module 5, distance measurement module 6, distance alarm module 7;

[0091] Binocular image acquisition module 1 is the input module of the whole system, and the input end of binocular image acquisition module 1 is connected with image processing module 2 through binocular camera image sensor to acquire transmission line image data;

[0092] Image processing module 2 carries out filtering denoising processing to image, for filtering denoising processing to target transmission line in image according to the image data input by binocular image acquisition module 1, the input end of image processing module 2 is connected with the output end of binocular image acquisition module 1, and the image generated by the output end of image processing module 2 is sent to target image segmentation module 5;

[0093] Distance display module 3 is the output module of the whole system, for visualizing the estimated distance between working machine and transmission line, and directly and quantitatively displaying distance. The input end of distance display module 3 is connected with the output end of distance measurement module 6 and distance alarm module 7;

[0094] Target image segmentation module 4 carries out segmentation processing to image, for identifying and positioning the position of transmission line in image according to the image after filtering denoising processing by image processing module 2. The input end of target image segmentation module 4 is connected with the output end of image processing module 2, and the output end of target image segmentation module 4 is connected with the input end of binocular stereo matching module 5;

[0095] Binocular stereo matching module 5 is used for binocular stereo matching to the image segmented by target image segmentation module 4, to obtain binocular image depth map, and the depth map is used for distance estimation. The position of transmission line in image is located by adding the channel number of original image and segmented image, to further improve the robustness of stereo matching. The input end of binocular stereo matching module 5 is connected with the output end of target image segmentation module 4, and the output end of binocular stereo matching module 5 is connected to the input end of distance measurement module 6.

[0096] The distance measurement module 6 is used for measuring the safe distance between the binocular camera embedded in the working machine and the power transmission line by using the different depths represented by the chroma of the pixels in the depth map obtained by the binocular stereo matching module 5 and the trigonometric function and the similar triangle principle; the input end of the distance measurement module 6 is connected with the output end of the binocular stereo matching module 5, and the output end of the distance measurement module 6 is used for being connected with the input end of the distance display module 3 and the input end of the distance alarm module 7;

[0097] The distance alarm module 7 is used for alarming the safe distance, so as to compare the distance obtained by the distance measurement module 6 with the standard safe distance by setting a threshold value, and alarm the distance range exceeding the safe distance. The input end of the distance alarm module 7 is connected with the output end of the distance measurement module 6, and the output end of the distance alarm module 7 is connected with the input end of the distance display module 3.

[0098] The binocular image acquisition module 1 adopts a binocular vision system to synchronously acquire binocular images containing the power transmission line. The image processing module 2 adopts a filtering algorithm, wherein the filtering algorithm includes a least square filtering algorithm, a bilateral filtering algorithm, a median filtering algorithm, a mean filtering algorithm and the like.

[0099] The distance display module 3 drives a display module through a VGA interface, wherein the display module includes an LED display module, an LCD display module and an OLED display module.

[0100] The target image segmentation module 4 adopts a neural network segmentation algorithm, wherein the neural network image segmentation algorithm includes a threshold-based image segmentation algorithm, an edge detection-based image segmentation algorithm, a genetic algorithm-based image segmentation algorithm, a feature coding-based image segmentation algorithm and a region selection-based image segmentation algorithm.

[0101] The binocular stereo matching module 5 adopts a neural network stereo matching algorithm, wherein the neural network stereo matching includes a feature matching algorithm, a region matching algorithm, a dynamic programming matching algorithm and an image segmentation matching algorithm.

[0102] The distance measurement module 6 interprets the image and the distance by a mathematical model to generate an accurate distance, including a trigonometric function method and a similar triangle method.

[0103] The distance alarm module 7 compares the actual measured distance with the standard safe distance as a threshold value, and identifies the distance greater than the threshold value as a safe distance and the distance less than the threshold value as an unsafe distance to trigger an alarm.

[0104] The present application has the advantages of saving labor cost, improving operation safety, reliability and recognition accuracy.

[0105] The above merely describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying the safe distance of machinery in near-electric work based on binocular vision, characterized in that: Includes the following steps: S1: Use a binocular camera to acquire raw images containing the power transmission lines; S2: Perform filtering and noise reduction processing on the original image acquired in step S1; S3: Input the image processed in step S2 into neural network model one for power transmission line image segmentation processing; S4: The segmented image containing the power line location information after segmentation in step S3 is superimposed on the original image, and then input into the second neural network model for stereo matching processing to output a depth map; specifically, the following steps are included: superimposing the segmented image with the original image by channel number, extracting features, constructing a 4D cost volume by disparity translation stitching, interpreting the spatial information of the image in multiple dimensions, unfolding the 4D cost volume using 3D convolution, regressing the disparity map by soft argmin, and optimizing the disparity map by depth smoothing loss function; S5: Based on the depth map, obtain the estimated distance; S6: Compare the estimated distance with the standard safe distance. If the estimated distance is within the safe distance, the distance between the working machinery and the power line is safe. Otherwise, the distance between the working machinery and the power line is unsafe and triggers the alarm function. S7: The estimated distance and safe distance are displayed on the LCD screen for real-time monitoring.

2. The method for identifying the safe distance of near-electric work machinery based on binocular vision as described in claim 1, characterized in that: In step S1, a calibrated binocular stereo camera is used to simultaneously capture images of the same target, obtaining left and right eye images from different angles, thus forming the original image.

3. The method for identifying the safe distance of near-electric work machinery based on binocular vision as described in claim 1, characterized in that: In step S2, a bilateral filtering algorithm is used to filter and denoise the spatial proximity and pixel similarity in the original image.

4. The method for identifying the safe distance of near-electric work machinery based on binocular vision as described in claim 1, characterized in that: The neural network model in step S3 is an encoder-decoder network.

5. The method for identifying the safe distance of near-electric work machinery based on binocular vision as described in claim 1, characterized in that: The second neural network model in step S4 is a twin residual network, which uses the global information of the superimposed image to perform stereo matching of the transmission line.

6. The method for identifying the safe distance of near-electric work machinery based on binocular vision as described in claim 1, characterized in that: In step S5, the specific steps to obtain distance information from the depth map include: determining the camera optical center, baseline, and focal length, and using the pixel triangle formed by the imaging plane and the object, and the optical center and the object, to obtain the distance information.

7. A binocular vision-based identification system for safe distance recognition of near-electric work machinery, characterized in that: include: The hardware and software components are as follows: the hardware component includes a binocular image acquisition module (1), an image processing module (2), and a distance display module (3); the software component includes a target image segmentation module (4), a binocular stereo matching module (5), a distance measurement module (6), and a distance alarm module (7). The binocular image acquisition module (1) is the input module of the whole system. The input end of the binocular image acquisition module (1) acquires the original image containing the transmission line through the binocular camera image sensor and obtains the image data. The output end of the binocular image acquisition module (1) is used to connect to the image processing module (2). The image processing module (2) is used to filter and denoise the target transmission line in the image based on the image data input by the binocular image acquisition module (1). The input end of the image processing module (2) is connected to the output end of the binocular image acquisition module (1), and the image generated by the output end of the image processing module (2) is sent to the target image segmentation module (4). The distance display module (3) is the output module of the whole system. It is used to visualize the estimated distance between the working machinery and the power transmission line and to display the distance in a clear and quantitative manner. The input end of the distance display module (3) is connected to the output end of the distance measurement module (6) and the distance alarm module (7). The target image segmentation module (4) is used to identify and locate the position of the transmission line in the image based on the image after the image processing module (2) has been filtered and denoised, and to perform segmentation processing; the input end of the target image segmentation module (4) is connected to the output end of the image processing module (2), and the output end of the target image segmentation module (4) is connected to the input end of the binocular stereo matching module (5); The binocular stereo matching module (5) is used to perform binocular stereo matching on the image segmented by the target image segmentation module (4) to obtain a binocular image depth map. The depth map is used for distance estimation. By superimposing the channel numbers of the original image and the segmented image, the position of the transmission line in the image is located, further improving the robustness of stereo matching. Specifically, it includes the following steps: superimposing the channel numbers of the segmented image and the original image, extracting features, constructing a 4D cost body by disparity translation stitching, interpreting the image spatial information in multiple dimensions, unfolding the 4D cost body using 3D convolution, regressing the disparity map by soft argmin, and optimizing the disparity map by the depth smoothing loss function. The input end of the binocular stereo matching module (5) is connected to the output end of the target image segmentation module (4), and the output end of the binocular stereo matching module (5) is connected to the input end of the distance measurement module (6). The distance measurement module (6) is used to measure the safe distance between the binocular camera embedded in the construction machinery and the power transmission line by using the depth map obtained by the binocular stereo matching module (5) and the different depths represented by the pixel chromaticity in the depth map, as well as the trigonometric function and similar triangle principle. The input end of the distance measurement module (6) is connected to the output end of the binocular stereo matching module (5), and the output end of the distance measurement module (6) is connected to the input end of the distance display module (3) and the distance alarm module (7). The distance alarm module (7) is used to alarm the safe distance. It is convenient to set a threshold to compare the estimated distance obtained by the distance measurement module (6) with the preset standard safe distance and to alarm the distance that exceeds the standard safe distance range. The input terminal of the distance alarm module (7) is connected to the output terminal of the distance measurement module (6), and the output terminal of the distance alarm module (7) is connected to the input terminal of the distance display module (3).

8. The identification system for safe distance of near-electric work machinery based on binocular vision as described in claim 7, characterized in that: The target image segmentation module (4) adopts a neural network segmentation algorithm.

9. The identification system for safe distance of near-electric work machinery based on binocular vision as described in claim 7, characterized in that: The binocular stereo matching module (5) uses a neural network stereo matching algorithm.

Images