Power operation safety risk identification method and system based on fine-grained detection

Abstract

The invention discloses a method and system for identifying safety risks of power operations based on fine-grained detection, which relates to the field of power operation safety management, solves the technical problems that it is difficult to obtain the proportion of material area on the surface of equipment through image analysis software, it is also difficult to obtain the disparity value of target pixel points through SAD algorithm, and there is a lack of a convolutional neural network as a basic framework to build a model for regional judgment and double early warning. The method and system include the following steps: obtaining the corrosion area ratio and the wear area ratio through image analysis software to obtain the comprehensive damage ratio of equipment; using a high-resolution binocular camera to collect video data of the power operation site; obtaining the disparity value and the corresponding disparity map of the target pixel point, and obtaining the depth value and three-dimensional coordinates of the corresponding point after obtaining the disparity value through the SAD algorithm; using the convolutional neural network as the basic framework to build a model to determine the vehicle type, and judging whether to trigger a warning signal according to the two judgment conditions.

Description

Technical Field

[0001] The present invention belongs to the field of power operation safety management, and specifically relates to a power operation safety risk identification method and system based on fine-grained detection. Background Art

[0002] With the rapid development of the power industry, the safety of power operations has received increasing attention. Therefore, a more scientific, efficient and accurate method and system for identifying safety risks in power operations is needed. The features of the pre-processed image are extracted by image analysis software, the area ratio of corrosive substances and hard particles on the surface of the equipment is identified, and the comprehensive damage ratio of the equipment is obtained. The SAD algorithm of the real-time video stream analysis of the near-power operation of the substation is used to obtain the disparity value of the target pixel, and the corresponding disparity image is obtained. The depth value of all points in the image is obtained according to the disparity result obtained by binocular stereo matching. Based on the resolution, sensor size and center point coordinates of the camera, the pixel points in the two-dimensional image are converted into coordinate points in three-dimensional space to generate point cloud data. The pre-trained convolutional neural network is selected as the basic feature extractor to build a model to determine the category of the vehicle, select the area of ​​interest, and judge whether the warning signal needs to be triggered according to the distance between the two feature points and the preset safety distance. Then, it is judged whether the first-level warning signal needs to be triggered according to the safety risk assessment index and the standard safety risk index.

[0003] Although the existing methods for identifying safety risks in power operations have achieved safety risk identification to a certain extent, they mainly rely on manual experience and simple rule judgments, and have problems of low efficiency and poor accuracy. It is difficult to obtain the proportion of material area on the surface of the equipment through image analysis software to obtain the comprehensive damage ratio of the equipment, and it is also difficult to obtain the disparity value of the target pixel point through the SAD algorithm to construct a disparity image, and combine the internal parameters of the high-resolution binocular camera to obtain the depth value of each pixel point, and then convert the pixel coordinates under the two-dimensional coordinates into three-dimensional coordinates. There is a lack of a model built with a convolutional neural network as the basic framework to determine the category of the vehicle, and to perform double warnings based on the two preset judgment conditions. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a method and system for identifying safety risks of power operations based on fine-grained detection, which are used to solve the following technical problems:

[0005] Although the existing methods for identifying safety risks in power operations have achieved safety risk identification to a certain extent, they mainly rely on manual experience and simple rule judgments, and have problems of low efficiency and poor accuracy. It is difficult to obtain the proportion of material area on the surface of the equipment through image analysis software to obtain the comprehensive damage ratio of the equipment, and it is also difficult to obtain the disparity value of the target pixel point through the SAD algorithm to construct a disparity image, and combine the internal parameters of the high-resolution binocular camera to obtain the depth value of each pixel point, and then convert the pixel coordinates under the two-dimensional coordinates into three-dimensional coordinates. There is a lack of a model built with a convolutional neural network as the basic framework to determine the category of the vehicle, and to perform double warnings based on the two preset judgment conditions.

[0006] To solve the above problems, a first aspect of the present invention provides a method for identifying safety risks of power operations based on fine-grained detection, comprising the following steps:

[0007] S1: Use a high-resolution camera to take images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment;

[0008] S2: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data;

[0009] S3: Obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data;

[0010] S4: A model is constructed using a convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether a first-level warning signal needs to be triggered;

[0011] The method of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in three-dimensional space according to the acquired data includes the following steps:

[0012] Determine the camera's resolution pixels, the sensor size is Millimeters, determine the coordinates of the center point of the image, for the pixel point (u, v), its position in the image is converted into the actual physical coordinates:

[0013] ; ; ;

[0014] in,( , ) is the two-dimensional coordinate of the pixel point, ( , , ) is the three-dimensional coordinate of the pixel point in three-dimensional space, ( , ) is the coordinate of the center point of the image, is the width of the sensor, is the height of the sensor, is the number of horizontal pixels of the image, is the number of vertical pixels of the image, is the pixel depth value;

[0015] The safety risk assessment index is obtained by the following formula:

[0016]

[0017] in, is the security risk assessment index value, The preset safety distance is is the distance between two feature points, is the comprehensive damage ratio of equipment, and are the weight coefficients corresponding to the difference and the comprehensive equipment damage ratio respectively.

[0018] As a further solution of the present invention: Step S1 comprises the following steps:

[0019] Use a high-resolution camera to capture images of equipment in the substation, pre-process the collected images, including size standardization, contrast enhancement, and noise removal, and use image analysis software to extract features from the pre-processed images, including identifying the location, shape, and size features of corrosive substances and hard particles in the images. Use image analysis software to calculate the areas occupied by corrosive substances and hard particles on the equipment surface, respectively, and compare them with the total area of ​​the equipment to obtain the corrosion area ratio and the wear area ratio. The corrosion area ratio and the wear area ratio are assigned corresponding weights and added to obtain the comprehensive damage ratio of the equipment;

[0020] Calculation formula for comprehensive equipment damage ratio:

[0021]

[0022] in, is the comprehensive damage ratio of equipment, is the corrosion area ratio, is the wear area ratio, and are the weight coefficients corresponding to the corrosion area ratio and the wear area ratio respectively.

[0023] As a further solution of the present invention: the step S2 comprises the following steps:

[0024] High-resolution binocular cameras are installed at the upper left and upper right corners of the detected object respectively, and connected to the FPGA hardware platform through the corresponding interface. The binocular cameras are started and used to collect video data of power operation sites in the construction area. The collected video data are processed in real time, including denoising and distortion correction preprocessing operations. The preprocessed video data are divided into independent image sequences, and grayscale, image smoothing and image enhancement operations are performed on the acquired image sequences.

[0025] As a further solution of the present invention: the obtaining of the disparity value of the target pixel point to obtain the corresponding disparity image includes the following steps:

[0026] Feature points are extracted from the left and right view images, and feature matching is performed using a matching algorithm. After determining the corresponding position of each feature point in the left and right view images, the matching points are screened. Based on the result of feature matching, the disparity value of each pixel is calculated using the SAD algorithm. The disparity values ​​of all pixels are integrated into a disparity image, and the disparity image is optimized using a stereo matching algorithm.

[0027] As a further solution of the present invention: the disparity value of each pixel is calculated by the SAD algorithm, and the disparity values ​​of all pixels are integrated into a disparity image, including the following steps:

[0028] S301: Set a The window size of pixels is used to create a blank disparity image with the same size as the left view image. For each pixel in the left view image, a Pixel window area;

[0029] S302: In the right-view image, determine a corresponding search area according to the current disparity value;

[0030] S303: traverse and calculate the SAD value under each disparity value within the given disparity range, and take the disparity value with the smallest SAD value as the best matching disparity value of the pixel point;

[0031] S304: Fill the best matching disparity value of each pixel point into the corresponding position in the disparity image, and post-process the generated disparity image to output a final disparity image.

[0032] As a further solution of the present invention: the obtaining of the internal parameters of the high-resolution binocular camera, reading the disparity value of each pixel in the disparity image generated by the SAD algorithm, and obtaining the depth values ​​of all pixels in the disparity image include the following steps:

[0033] Obtain the internal parameters of the high-resolution binocular camera, including focal length and baseline distance, read the disparity image generated by the SAD algorithm, and combine the disparity value of each pixel in the disparity image to obtain the corresponding depth value;

[0034] Pixel depth value calculation formula:

[0035]

[0036] in, is the pixel depth value, is the focal length of the camera, is the baseline distance, is the parallax value.

[0037] As a further solution of the present invention: the step of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in three-dimensional space according to the acquired data further includes:

[0038] After converting the two-dimensional coordinates of pixels in the two-dimensional image into coordinate points in three-dimensional space, point cloud data is generated.

[0039] As a further solution of the present invention: the step S4 comprises the following steps:

[0040] Use stereo cameras installed on drones and cameras to collect on-site three-dimensional data of power equipment and its surrounding environment, perform denoising and correction preprocessing operations on the collected data, select a pre-trained convolutional neural network as the basic feature extractor, the input layer is used to receive the pre-processed image data, the feature extraction layer is used to extract image features, including the color, outline and feature points of the vehicle, the fully connected layer is used to map the extracted features to the vehicle category space, and the output layer is used for the softmax function to output the probability distribution of various types of vehicles. Split the data set into several batches, determine the number of training sets in each batch, use the optimizer to update the model parameters, and use the model to output the vehicle category in the image;

[0041] Determine an area of ​​interest from the point cloud data, select the feature points of interest, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and display the calculated distance and relative position in a visual way; compare the relative position and distance results with the preset safety distance to determine whether it is necessary to trigger an early warning signal;

[0042] Euclidean distance formula:

[0043]

[0044] in, is the distance between two feature points, ( , , )and( , , ) are the coordinates of two specific feature points;

[0045] like ≥ the preset safety distance, the warning signal will not be triggered;

[0046] like < the preset safety distance, the second-level warning signal will be triggered;

[0047] exist Under the condition of the preset safety distance, the distance value between the two feature points, the preset safety distance value and the comprehensive damage ratio of the equipment are standardized, and the difference between the distance value of the two devices and the preset safety distance is combined with the comprehensive damage ratio of the equipment, and the weighted sum is performed according to their respective weights to obtain the final combined result, which is used as the safety risk assessment index. According to the obtained safety risk assessment index value, the early warning signal is triggered and determined;

[0048] The average damaged area ratio of damaged equipment in the construction area is obtained by counting the ratio of the comprehensive damaged area of ​​damaged equipment and dividing it by the number of damaged equipment. The standard safety risk index is obtained by weighted addition of the preset safety distance, where the comprehensive damaged area includes the corrosion area ratio and the wear area ratio.

[0049] Compare the safety risk assessment index with the standard safety risk index to determine whether a first-level warning signal needs to be triggered;

[0050] like ≥ standard safety risk index, a first-level warning signal is triggered;

[0051] like <standard safety risk index, the first-level warning signal will not be triggered.

[0052] The present invention also provides a power operation safety risk identification system based on fine-grained detection, comprising the following modules:

[0053] Equipment surface damage analysis and assessment module: Use a high-resolution camera to capture images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment;

[0054] Power operation site video processing and segmentation module: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data;

[0055] Disparity image depth calculation and three-dimensional coordinate conversion module: obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data;

[0056] Three-dimensional space safety risk assessment and early warning module based on convolutional neural network: The model is built with convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, and use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between the two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether it is necessary to trigger a first-level warning signal.

[0057] Beneficial effects of the present invention:

[0058] The present invention analyzes the image of the power action site through image analysis software, identifies the corrosive substances and hard particles on the surface of the equipment, and calculates the area ratios occupied respectively, combines them according to the corresponding weights, and obtains the comprehensive damage ratio of the equipment, thereby realizing the automatic inspection, quantification and risk assessment of the surface damage of the power equipment; obtains the disparity value of the target pixel point through the SAD algorithm, obtains the corresponding disparity image, and obtains the depth value of all points in the disparity image in combination with the internal parameters of the binocular camera, determines the resolution of the camera, the sensor size and the coordinates of the center point of the image, and converts the data coordinates in the two-dimensional coordinate system in the image into three-dimensional coordinates in the three-dimensional space coordinate system based on the above data, and generates point cloud data. The three-dimensional reconstruction of the work site is achieved; a pre-trained convolutional neural network is selected as the basic feature extractor to build the model. After the optimizer is used to update the model parameters, the model outputs the vehicle category in the image, and the Euclidean distance formula is used to obtain the distance between the two selected feature points. After comparing it with the preset safety distance, it is determined whether an alarm needs to be triggered. If the distance between the two feature points is less than the preset safety distance, the difference between the two is combined with the comprehensive damage ratio of the equipment according to the corresponding weights, and the obtained safety risk assessment index is compared with the standard safety risk index to determine whether a first-level warning signal needs to be triggered. This achieves real-time monitoring of the vehicle status and issues a double warning when necessary. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0061] Figure 2 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 As shown, the present invention is a method for identifying safety risks of power operations based on fine-grained detection, comprising the following steps:

[0064] S1: Use a high-resolution camera to take images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment;

[0065] S2: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data;

[0066] S3: Obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data;

[0067] S4: A model is constructed using a convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether a first-level warning signal needs to be triggered;

[0068] The method of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in three-dimensional space according to the acquired data includes the following steps:

[0069] Determine the camera's resolution pixels, the sensor size is Millimeters, determine the coordinates of the center point of the image, for the pixel point (u, v), its position in the image is converted into the actual physical coordinates:

[0070] ; ; ;

[0071] in,( , ) is the two-dimensional coordinate of the pixel point, ( , , ) is the three-dimensional coordinate of the pixel point in three-dimensional space, ( , ) is the coordinate of the center point of the image, is the width of the sensor, is the height of the sensor, is the number of horizontal pixels of the image, is the number of vertical pixels of the image, is the pixel depth value;

[0072] The safety risk assessment index is obtained by the following formula:

[0073]

[0074] in, is the security risk assessment index value, The preset safety distance is is the distance between two feature points, is the comprehensive damage ratio of equipment, and are the weight coefficients corresponding to the difference and the comprehensive equipment damage ratio respectively.

[0075] Specifically, use the tools in the image analysis software to calculate the area occupied by the corrosive material and hard particle areas respectively, multiply the number of pixels multiplied by the actual area represented by each pixel on the equipment surface, and compare them with the total area of ​​the equipment to obtain the corrosion area ratio and the wear area ratio, and obtain the comprehensive equipment damage ratio by weighted addition; start the binocular camera, start collecting video data of the power operation site in the construction area, perform real-time denoising and distortion correction on the collected data, and divide the continuous video data into independent image sequences; obtain the internal parameters of the high-resolution binocular camera, including focal length and baseline distance, traverse all pixel point data in the disparity image obtained by the SAD algorithm, and combine them with the internal parameter data to obtain the corresponding depth value, determine the camera resolution, sensor size and image center point coordinates, convert its position in the image into actual physical coordinates, and generate corresponding point cloud data; select the pre-trained convolutional neural network as the basic feature extractor to build a model to identify the vehicle category, and determine whether the corresponding warning signal needs to be triggered based on the two set judgment conditions.

[0076] In one embodiment of the present invention, the step S1 comprises the following steps:

[0077] Use a high-resolution camera to capture images of equipment in the substation, pre-process the collected images, including size standardization, contrast enhancement, and noise removal, and use image analysis software to extract features from the pre-processed images, including identifying the location, shape, and size features of corrosive substances and hard particles in the images. Use image analysis software to calculate the areas occupied by corrosive substances and hard particles on the equipment surface, respectively, and compare them with the total area of ​​the equipment to obtain the corrosion area ratio and the wear area ratio. The corrosion area ratio and the wear area ratio are assigned corresponding weights and added to obtain the comprehensive damage ratio of the equipment;

[0078] Calculation formula for comprehensive equipment damage ratio:

[0079]

[0080] in, is the comprehensive damage ratio of equipment, is the corrosion area ratio, is the wear area ratio, and are the weight coefficients corresponding to the corrosion area ratio and the wear area ratio respectively.

[0081] Specifically, key equipment in the substation is photographed, including large vehicles, medium-sized vehicles and small vehicles. After all images are adjusted to a uniform size and resolution, the image contrast is improved and image denoising technology is applied to reduce random noise in the image. Image analysis software is used to extract features from the pre-processed images, including identifying the location, shape and size characteristics of corrosive substances and hard particles in the image. The tools in the image analysis software are used to calculate the number of pixels in the corrosive substance and hard particle areas multiplied by the actual area represented by each pixel to represent the area occupied on the equipment surface. The areas are compared with the total area of ​​the equipment to obtain the corrosion area ratio and the wear area ratio, and the comprehensive equipment damage ratio is obtained by weighted addition.

[0082] in, and Both are 0.5.

[0083] In one embodiment of the present invention, the step S2 comprises the following steps:

[0084] High-resolution binocular cameras are installed at the upper left and upper right corners of the detected object respectively, and connected to the FPGA hardware platform through the corresponding interface. The binocular cameras are started and used to collect video data of power operation sites in the construction area. The collected video data are processed in real time, including denoising and distortion correction preprocessing operations. The preprocessed video data are divided into independent image sequences, and grayscale, image smoothing and image enhancement operations are performed on the acquired image sequences.

[0085] Specifically, high-resolution binocular cameras are installed at the upper left and upper right corners of the object to be inspected, respectively, to ensure that the two cameras start working at the same time, and time synchronization control is achieved through the synchronization mechanism on the FPGA platform. The binocular cameras are started to collect video data of the power operation site in the construction area, and the collected data are subjected to real-time denoising and distortion correction. The continuous video data is divided into independent image sequences, the acquired image data is grayed, and image smoothing technology is used to reduce image noise and enhancement algorithms are used to enhance the image.

[0086] In one embodiment of the present invention, the step of obtaining the disparity value of the target pixel point to obtain the corresponding disparity image includes the following steps:

[0087] Feature points are extracted from the left and right view images, and feature matching is performed using a matching algorithm. After determining the corresponding position of each feature point in the left and right view images, the matching points are screened. Based on the result of feature matching, the disparity value of each pixel is calculated using the SAD algorithm. The disparity values ​​of all pixels are integrated into a disparity image, and the disparity image is optimized using a stereo matching algorithm.

[0088] Specifically, a feature extraction algorithm is used to extract feature points, including key points and descriptors, from the left and right view images. A matching algorithm is used to match the feature points in the left and right view images. By comparing the descriptors of the feature points, the corresponding feature points in the left and right view images are found, and the matching results are screened to remove incorrect matches or low-quality matches. For each pair of matched feature points, a SAD algorithm is used to calculate the disparity value between them, and the disparity value is assigned to the corresponding pixel point to obtain a preliminary disparity image. The preliminary disparity image is optimized by exchanging the positions of the left and right view images, repeating stereo matching, and comparing the consistency of the two matching results. If the disparity values ​​are consistent within a certain threshold, the disparity point is retained; otherwise, the disparity point is discarded.

[0089] In one embodiment of the present invention, the method of calculating the disparity value of each pixel point by using the SAD algorithm and integrating the disparity values ​​of all the pixels into a disparity image includes the following steps:

[0090] S301: Set a The window size of pixels is used to create a blank disparity image with the same size as the left view image. For each pixel in the left view image, a Pixel window area;

[0091] S302: In the right-view image, determine a corresponding search area according to the current disparity value;

[0092] S303: traverse and calculate the SAD value under each disparity value within the given disparity range, and take the disparity value with the smallest SAD value as the best matching disparity value of the pixel point;

[0093] S304: Fill the best matching disparity value of each pixel point into the corresponding position in the disparity image, and post-process the generated disparity image to output a final disparity image.

[0094] Specifically, set a The window size of pixels is used to cover the pixels and their neighborhoods in the image. A blank disparity image with the same size as the left view image is created to store the best matching disparity of each pixel. For each pixel in the left view image, a Pixel window area, in the right perspective image, according to the initial disparity value of 0, determine the corresponding search area, in the search area of ​​the right perspective image, select an area of ​​the same size as the left perspective image window, calculate the absolute value and difference between the corresponding pixels of the left and right windows, within the given disparity range, and traverse all disparity values, calculate the SAD value under each disparity value, find the disparity value that minimizes the SAD value, as the best matching disparity of the pixel point, fill it into the corresponding position of the disparity image, and post-process the generated disparity image, including left and right consistency check, elimination of small connected domains and uniqueness detection, and output the final disparity image.

[0095] In one embodiment of the present invention, the process of obtaining the internal parameters of the high-resolution binocular camera, reading the disparity value of each pixel in the disparity image generated by the SAD algorithm, and obtaining the depth values ​​of all pixels in the disparity image includes the following steps:

[0096] Obtain the internal parameters of the high-resolution binocular camera, including focal length and baseline distance, read the disparity image generated by the SAD algorithm, and combine the disparity value of each pixel in the disparity image to obtain the corresponding depth value;

[0097] Pixel depth value calculation formula:

[0098]

[0099] in, is the pixel depth value, is the focal length of the camera, is the baseline distance, is the parallax value.

[0100] Specifically, the internal parameters of the high-resolution binocular camera are obtained, including the focal length and baseline distance. All pixel data in the disparity image obtained by the SAD algorithm are traversed and combined with the internal parameter data to obtain the corresponding depth value. All calculated depth values ​​are integrated into a depth map to reflect the depth information of each pixel.

[0101] In one embodiment of the present invention, the step of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in a three-dimensional space according to the acquired data further includes:

[0102] After converting the two-dimensional coordinates of pixels in the two-dimensional image into coordinate points in three-dimensional space, point cloud data is generated.

[0103] Specifically, determine the camera resolution as pixels, the sensor size is mm, the image center point is usually located at the center of the image, and the coordinates are ( , ),in, is equal to one-half of W, Equal to half of H, assuming that the coordinates of the pixel point in the image are ( , ), first convert the pixel coordinates into coordinates in a coordinate system with the center of the image as the origin , and then according to the camera resolution and sensor size, the depth information is combined Value, convert the coordinates into actual physical coordinates ( , , ).

[0104] In one embodiment of the present invention, the step S4 comprises the following steps:

[0105] Use stereo cameras installed on drones and cameras to collect on-site three-dimensional data of power equipment and its surrounding environment, perform denoising and correction preprocessing operations on the collected data, select a pre-trained convolutional neural network as the basic feature extractor, the input layer is used to receive the pre-processed image data, the feature extraction layer is used to extract image features, including the color, outline and feature points of the vehicle, the fully connected layer is used to map the extracted features to the vehicle category space, and the output layer is used for the softmax function to output the probability distribution of various types of vehicles. Split the data set into several batches, determine the number of training sets in each batch, use the optimizer to update the model parameters, and use the model to output the vehicle category in the image;

[0106] Determine an area of ​​interest from the point cloud data, select the feature points of interest, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and display the calculated distance and relative position in a visual way; compare the relative position and distance results with the preset safety distance to determine whether it is necessary to trigger an early warning signal;

[0107] Euclidean distance formula:

[0108]

[0109] in, is the distance between two feature points, ( , , )and( , , ) are the coordinates of two specific feature points;

[0110] like ≥ the preset safety distance, the warning signal will not be triggered;

[0111] like < the preset safety distance, the second-level warning signal will be triggered;

[0112] exist Under the condition of the preset safety distance, the distance value between the two feature points, the preset safety distance value and the comprehensive damage ratio of the equipment are standardized, and the difference between the distance value of the two devices and the preset safety distance is combined with the comprehensive damage ratio of the equipment, and the weighted sum is performed according to their respective weights to obtain the final combined result, which is used as the safety risk assessment index. According to the obtained safety risk assessment index value, the early warning signal is triggered and determined;

[0113] The average damaged area ratio of damaged equipment in the construction area is obtained by counting the ratio of the comprehensive damaged area of ​​damaged equipment and dividing it by the number of damaged equipment. The standard safety risk index is obtained by weighted addition of the preset safety distance, where the comprehensive damaged area includes the corrosion area ratio and the wear area ratio.

[0114] Compare the safety risk assessment index with the standard safety risk index to determine whether a first-level warning signal needs to be triggered;

[0115] like ≥ standard safety risk index, a first-level warning signal is triggered;

[0116] like <standard safety risk index, the first-level warning signal will not be triggered.

[0117] Specifically, a pre-trained convolutional neural network is selected as the basic feature extractor, the input layer is used to receive the pre-processed image data, the feature extraction layer is used for the pre-trained convolutional neural network to extract image features, including the color, outline and feature points of the vehicle, where the feature points include the front, rear and door of the vehicle, the fully connected layer is used to map the extracted features to the vehicle category space, and the output layer is used for the softmax function to output the probability distribution of each type of vehicle. The data set is split into several batches, the number of training sets in each batch is determined, and the model parameters are updated using the optimizer until the model can accurately output the vehicle category in the image; in the point cloud data, an area of ​​interest is determined, such as the area around the vehicle in the construction area, which is achieved by setting spatial filtering conditions, selecting feature points of interest, including the color, outline and feature points of the vehicle, and using the Euclidean distance formula in three-dimensional space to calculate specific features The relative position and distance between the points are visualized. The calculated relative position and distance are represented by color coding by drawing lines between the feature points to represent different distance ranges. The relative position and distance results are compared with the preset safety distance to determine whether to trigger the second-level warning. When the distance between the feature points is less than the preset safety distance, the safety risk assessment index is obtained by assigning corresponding weights to the difference between the two and the comprehensive damage ratio of the equipment. The index is compared with the standard safety risk index to determine whether the first-level warning needs to be triggered. The comprehensive damaged area ratio of the damaged equipment in the construction area is calculated by counting the comprehensive damaged area ratio. The comprehensive damaged area is obtained by image analysis software. The comprehensive damaged area includes the corrosion area ratio and the wear area ratio. The average equipment damaged area ratio is obtained by dividing it by the number of damaged equipment. The standard safety risk index is obtained by combining the preset safety distance.

[0118] in, is 3, is 1.5.

[0119] See also Figure 2 As shown, the present invention is a power operation safety risk identification system based on fine-grained detection, including the following modules:

[0120] Equipment surface damage analysis and assessment module: Use a high-resolution camera to capture images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment;

[0121] Power operation site video processing and segmentation module: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data;

[0122] Disparity image depth calculation and three-dimensional coordinate conversion module: obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data;

[0123] Three-dimensional space safety risk assessment and early warning module based on convolutional neural network: The model is built with convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, and use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between the two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether it is necessary to trigger a first-level warning signal.

[0124] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for identifying safety risks of power operations based on fine-grained detection, characterized in that: The following steps are involved: S1: Use a high-resolution camera to take images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment; S2: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data; S3: Obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data; S4: A model is constructed using a convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether a first-level warning signal needs to be triggered; The method of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in three-dimensional space according to the acquired data includes the following steps: Determine the camera's resolution pixels, the sensor size is Millimeters, determine the coordinates of the center point of the image, for the pixel point (u, v), its position in the image is converted into the actual physical coordinates: ； ； ； in, is the two-dimensional coordinate of the pixel point, is the three-dimensional coordinate of the pixel point in three-dimensional space, is the coordinate of the center point of the image, is the width of the sensor, is the height of the sensor, is the number of horizontal pixels of the image, is the number of vertical pixels of the image, is the pixel depth value; The safety risk assessment index is obtained by the following formula: ； in, is the security risk assessment index value, The preset safety distance is is the distance between two feature points, is the comprehensive damage ratio of equipment, and are the weight coefficients corresponding to the difference and the comprehensive equipment damage ratio respectively.

2. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The step S1 comprises the following steps: Use a high-resolution camera to capture images of equipment in the substation, pre-process the collected images, including size standardization, contrast enhancement, and noise removal, and use image analysis software to extract features from the pre-processed images, including identifying the location, shape, and size features of corrosive substances and hard particles in the images. Use image analysis software to calculate the areas occupied by corrosive substances and hard particles on the equipment surface, respectively, and compare them with the total area of ​​the equipment to obtain the corrosion area ratio and the wear area ratio. The corrosion area ratio and the wear area ratio are assigned corresponding weights and added to obtain the comprehensive damage ratio of the equipment; Calculation formula for comprehensive equipment damage ratio: ； in, is the comprehensive damage ratio of equipment, is the corrosion area ratio, is the wear area ratio, and are the weight coefficients corresponding to the corrosion area ratio and the wear area ratio respectively.

3. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The step S2 comprises the following steps: High-resolution binocular cameras are installed at the upper left and upper right corners of the detected object respectively, and connected to the FPGA hardware platform through the corresponding interface. The binocular cameras are started and used to collect video data of power operation sites in the construction area. The collected video data are processed in real time, including denoising and distortion correction preprocessing operations. The preprocessed video data are divided into independent image sequences, and grayscale, image smoothing and image enhancement operations are performed on the acquired image sequences.

4. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The method of obtaining the disparity value of the target pixel point and obtaining the corresponding disparity image includes the following steps: Feature points are extracted from the left and right view images, and feature matching is performed using a matching algorithm. After determining the corresponding position of each feature point in the left and right view images, the matching points are screened. Based on the result of feature matching, the disparity value of each pixel is calculated using the SAD algorithm. The disparity values ​​of all pixels are integrated into a disparity image, and the disparity image is optimized using a stereo matching algorithm.

5. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 4 is characterized in that: The method of calculating the disparity value of each pixel point by using the SAD algorithm and integrating the disparity values ​​of all the pixels into a disparity image includes the following steps: S301: Set a The window size of pixels is used to create a blank disparity image with the same size as the left view image. For each pixel in the left view image, a Pixel window area; S302: In the right-view image, determine a corresponding search area according to the current disparity value; S303: traverse and calculate the SAD value under each disparity value within the given disparity range, and take the disparity value with the smallest SAD value as the best matching disparity value of the pixel point; S304: Fill the best matching disparity value of each pixel point into the corresponding position in the disparity image, and post-process the generated disparity image to output a final disparity image.

6. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The method of obtaining the internal parameters of the high-resolution binocular camera, reading the disparity value of each pixel in the disparity image generated by the SAD algorithm, and obtaining the depth values ​​of all pixels in the disparity image includes the following steps: Obtain the internal parameters of the high-resolution binocular camera, including focal length and baseline distance, read the disparity image generated by the SAD algorithm, and combine the disparity value of each pixel in the disparity image to obtain the corresponding depth value; Pixel depth value calculation formula: ； in, is the pixel depth value, is the focal length of the camera, is the baseline distance, is the parallax value.

7. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The step of converting the two-dimensional coordinates of the pixel points into three-dimensional coordinates in three-dimensional space according to the acquired data further includes: After converting the two-dimensional coordinates of pixels in the two-dimensional image into coordinate points in three-dimensional space, point cloud data is generated.

8. The method for identifying safety risks of electric power operations based on fine-grained detection according to claim 1 is characterized in that: The step S4 comprises the following steps: Use stereo cameras installed on drones and cameras to collect on-site three-dimensional data of power equipment and its surrounding environment, perform denoising and correction preprocessing operations on the collected data, select a pre-trained convolutional neural network as the basic feature extractor, the input layer is used to receive the pre-processed image data, the feature extraction layer is used to extract image features, including the color, outline and feature points of the vehicle, the fully connected layer is used to map the extracted features to the vehicle category space, and the output layer is used for the softmax function to output the probability distribution of various types of vehicles. Split the data set into several batches, determine the number of training sets in each batch, use the optimizer to update the model parameters, and use the model to output the vehicle category in the image; Determine an area of ​​interest from the point cloud data, select the feature points of interest, use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and display the calculated distance and relative position in a visual way; compare the relative position and distance results with the preset safety distance to determine whether it is necessary to trigger an early warning signal; Euclidean distance formula: ； in, is the distance between two feature points, and are the coordinates of two specific feature points; like ≥ the preset safety distance, the warning signal will not be triggered; like < the preset safety distance, the second-level warning signal will be triggered; exist Under the condition of the preset safety distance, the distance value between the two feature points, the preset safety distance value and the comprehensive damage ratio of the equipment are standardized, and the difference between the distance value of the two devices and the preset safety distance is combined with the comprehensive damage ratio of the equipment, and the weighted sum is performed according to their respective weights to obtain the final combined result, which is used as the safety risk assessment index. According to the obtained safety risk assessment index value, the early warning signal is triggered and determined; The average damaged area ratio of damaged equipment in the construction area is obtained by counting the ratio of the comprehensive damaged area of ​​damaged equipment and dividing it by the number of damaged equipment. The standard safety risk index is obtained by weighted addition of the preset safety distance, where the comprehensive damaged area includes the corrosion area ratio and the wear area ratio. Compare the safety risk assessment index with the standard safety risk index to determine whether a first-level warning signal needs to be triggered; If R ≥ standard safety risk index, a first-level warning signal is triggered; If R is less than the standard safety risk index, the first-level warning signal will not be triggered.

9. A power operation safety risk identification system based on fine-grained detection, using a power operation safety risk identification method based on fine-grained detection as claimed in any one of claims 1 to 8, characterized in that: Includes the following modules: Equipment surface damage analysis and assessment module: Use a high-resolution camera to capture images of equipment in the substation, and use image analysis software to calculate the area occupied by corrosive substances and hard particles on the equipment surface, obtain the corrosion area ratio and wear area ratio, assign corresponding weights to each, and add them up to obtain the comprehensive damage ratio of the equipment; Power operation site video processing and segmentation module: Use a high-resolution binocular camera to collect video data of power operation sites in the construction area, perform video segmentation to obtain image data, and pre-process the obtained image data; Disparity image depth calculation and three-dimensional coordinate conversion module: obtain the disparity value of the target pixel point, obtain the corresponding disparity image, obtain the internal parameters of the high-resolution binocular camera, read the disparity value of each pixel point in the disparity image generated by the SAD algorithm, obtain the depth value of all pixels in the disparity image, and convert the two-dimensional coordinates of the pixel point into three-dimensional coordinates in three-dimensional space according to the obtained data; Three-dimensional space safety risk assessment and early warning module based on convolutional neural network: The model is built with convolutional neural network as the basic framework to determine the vehicle category, select feature points of interest from the point cloud data, and use the Euclidean distance formula in three-dimensional space to calculate the relative position and distance between specific feature points, and compare them with the preset safety distance. When the distance between the two points is less than the preset safety distance, the safety risk assessment index obtained by combining the distance value with the comprehensive damage ratio of the equipment is compared with the standard safety risk index to determine whether it is necessary to trigger a first-level warning signal.

Images