Power distribution cabinet safety monitoring method and system based on image recognition

Abstract

The application relates to the technical field of electrical equipment state monitoring, and discloses a power distribution cabinet safety monitoring method and system based on image recognition. The method comprises the following steps: deploying a double-waveband image acquisition device in a key area of a power distribution cabinet and first acquiring an infrared thermal image; pre-processing the image and performing temperature analysis, and triggering an alarm when an abnormal area is found; adjusting the device posture to synchronously acquire high-definition visible light and infrared images of the abnormal area; registering the two images and extracting apparent and temperature distribution feature vectors, and then fusing the feature vectors; finally, inputting the fused feature vectors into a diagnosis model for fault identification, comparing high-confidence results with benchmark images, and generating a diagnosis conclusion after analysis. The application can improve the reliability and efficiency of power distribution cabinet monitoring.

Description

Technical Field

[0001] This application relates to the field of electrical equipment condition monitoring technology, and in particular to a method and system for safety monitoring of distribution cabinets based on image recognition. Background Technology

[0002] As a key component of the power system, the operating status of the distribution cabinet directly affects the reliability and security of power supply. Traditional inspection methods mainly rely on manual periodic inspections or monitoring by a single sensor (such as a temperature sensor), which has problems such as low efficiency, limited coverage, and inability to intuitively obtain information on the correlation between the equipment's apparent condition and its internal thermal state.

[0003] In recent years, infrared thermal imaging technology has been used to detect potential overheating hazards in electrical equipment. However, thermal images alone are insufficient to distinguish between surface defects such as dirt, corrosion, and mechanical deformation, and overheating-related faults such as overload and poor contact, easily leading to misdiagnosis or missed diagnosis. Therefore, there is an urgent need for an intelligent monitoring method that can comprehensively utilize visible light and infrared image information to achieve accurate diagnosis and early warning of the status of power distribution cabinets.

[0004] Therefore, the present invention provides a method and system for safety monitoring of power distribution cabinets based on image recognition. Summary of the Invention

[0005] This application provides a method and system for safety monitoring of power distribution cabinets based on image recognition, which can be used to achieve accurate identification, location and diagnosis of power distribution cabinet faults.

[0006] In a first aspect, this application provides a method for safety monitoring of power distribution cabinets based on image recognition, the method comprising:

[0007] Step S1: Deploy image acquisition equipment in the key area of ​​the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, where the first wavelength is the visible light band and the second wavelength is the infrared imaging band. First, acquire the infrared thermal image of the power distribution cabinet.

[0008] Step S2: Preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, proceed to step S3. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered simultaneously.

[0009] Step S3: Obtain the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device according to the location information to place the anomaly area in the center of the image capture area, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area.

[0010] Step S4: Register the acquired visible light image and infrared thermal image, extract features from the registered visible light image to obtain the apparent feature vector, extract features from the registered infrared thermal image to obtain the temperature distribution feature vector, and fuse the apparent feature vector and the temperature distribution feature vector to obtain the fused feature vector.

[0011] Step S5: Input the fused feature vector into the pre-trained diagnostic model to identify the fault, obtain the fault type and the corresponding fault probability, compare and analyze the visible light image with the corresponding pre-stored benchmark image with the fault probability greater than the preset confidence threshold, and take corresponding measures based on the analysis results.

[0012] Secondly, this application provides a power distribution cabinet safety monitoring system based on image recognition, the system comprising:

[0013] The acquisition module is used to deploy image acquisition equipment in key areas of the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, the first of which is the visible light band and the second of which is the infrared imaging band. First, the infrared thermal image of the power distribution cabinet is acquired.

[0014] The analysis module is used to preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, the adjustment module is executed. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered at the same time.

[0015] The adjustment module is used to acquire the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device to place the anomaly area in the center of the image based on the location information, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area.

[0016] The processing module is used to register the acquired visible light image and infrared thermal image, extract features from the registered visible light image to obtain the apparent feature vector, extract features from the registered infrared thermal image to obtain the temperature distribution feature vector, and fuse the apparent feature vector and the temperature distribution feature vector to obtain the fused feature vector.

[0017] The identification module is used to input the fused feature vector into the pre-trained diagnostic model for fault identification, obtain the fault type and the corresponding fault probability, compare and analyze the visible light image with the corresponding pre-stored benchmark image with the fault probability greater than the preset confidence threshold, and take corresponding measures based on the analysis results.

[0018] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0019] The technical solution provided in this application employs a process that triggers high-definition acquisition and registration fusion of visible light images through preliminary screening of infrared images. This ensures both real-time performance and the accuracy of subsequent analysis data. Image processing steps such as adaptive image enhancement based on prior knowledge, multi-scale feature extraction, and fusion feature vector construction effectively improve feature quality, providing better input for subsequent fault identification and thus enhancing the accuracy of fault classification. By comparing the model's diagnostic results with a benchmark image library, refined analysis of fault diagnosis is achieved, generating diagnostic conclusions that include the severity of the fault, helping staff to better respond to challenges. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an embodiment of the image recognition-based power distribution cabinet safety monitoring method in this application.

[0022] Figure 2 These are three adjacent difference images in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of one embodiment of the image recognition-based power distribution cabinet safety monitoring system in this application. Detailed Implementation

[0024] This application provides a method and system for safety monitoring of power distribution cabinets based on image recognition. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1One embodiment of the image recognition-based power distribution cabinet safety monitoring method in this application includes:

[0026] Step S1: Deploy image acquisition equipment in the key area of ​​the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, where the first wavelength is the visible light band and the second wavelength is the infrared imaging band. First, acquire the infrared thermal image of the power distribution cabinet.

[0027] Specifically, in order to conduct preliminary temperature screening, image acquisition equipment is deployed in key areas of the distribution cabinet to be monitored, such as busbar connections, circuit breaker terminals, and switch contacts. This image acquisition equipment integrates two image sensors of different wavelengths: one is a visible light wavelength sensor, and the other is an infrared imaging wavelength sensor. At the start of monitoring, infrared thermal images of the distribution cabinet are acquired first for preliminary temperature screening.

[0028] Step S2: Preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, proceed to step S3. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered simultaneously.

[0029] Specifically, in order to enhance the edge and detail information of the temperature distribution, a detail enhancement algorithm based on guided filtering is used to preprocess the acquired raw infrared thermal image. Temperature analysis is then performed on the preprocessed infrared thermal image. The specific temperature analysis method will be explained in detail later. The temperature analysis determines whether there are abnormal temperature areas. If so, step S3 is executed. If the temperature value of the abnormal temperature area is greater than the alarm threshold, it means that the temperature is too high and relevant measures need to be taken in time. Therefore, an alarm signal is triggered at the same time as step S3 to facilitate timely response by staff.

[0030] If there is no abnormal temperature area, there is no need to perform subsequent steps. Just perform steps S1 and S2 to continuously collect infrared thermal images of the distribution cabinet and perform safety monitoring of the distribution cabinet based on the infrared thermal images.

[0031] Step S3: Obtain the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device according to the location information to place the anomaly area in the center of the image capture area, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area.

[0032] Specifically, if an abnormal temperature area exists, it indicates a potential hazard in the distribution cabinet. To more accurately determine if a fault exists, it is essential to first ensure that a clear image is captured. To ensure that the target area is clearly imaged in the visible light image, the location information of the abnormal temperature area in the infrared thermal image is first obtained, such as the center point of the outer rectangle of the abnormal temperature area. Based on this location information, the image acquisition device is rotated and tilted to center the abnormal area in the captured image. Simultaneously, based on the area and shape of the abnormal temperature area, such as calculating the aspect ratio of the outer rectangle and the proportion of its area to the entire image, the optical zoom of the visible light sensor is adaptively adjusted, and the image acquisition frequency is increased, such as from once per minute to once per second. The automatic focusing and supplementary lighting functions of the visible light sensor are also activated simultaneously to overcome the problem of insufficient light inside the distribution cabinet. Finally, high-definition visible light and infrared thermal images of the abnormal area are acquired simultaneously to facilitate anomaly identification based on the newly acquired images.

[0033] Step S4: Register the acquired visible light image and infrared thermal image, extract features from the registered visible light image to obtain the apparent feature vector, extract features from the registered infrared thermal image to obtain the temperature distribution feature vector, and fuse the apparent feature vector and the temperature distribution feature vector to obtain the fused feature vector.

[0034] Specifically, to address the issue of the same physical target appearing in different positions in two images due to parallax and lens differences, the acquired visible light image and infrared thermal image are registered. The registered visible light image is then used to extract a visual feature vector, which describes the texture, shape, color, and other appearance information of the visible light image. The registered infrared thermal image is then used to extract a temperature distribution feature vector, which includes multi-dimensional temperature statistical features such as mean temperature, standard deviation, skewness, and kurtosis. Finally, the visual feature vector and the temperature distribution feature vector are fused to obtain a fused feature vector. This method spatially aligns image information from different sensors, extracts the most essential feature information from each, and finally fuses them into a fused feature vector that comprehensively and complementaryly describes the state of the same target, providing high-quality, information-rich input data for subsequent fault diagnosis.

[0035] Step S5: Input the fused feature vector into the pre-trained diagnostic model to identify the fault, obtain the fault type and the corresponding fault probability, compare and analyze the visible light image with the corresponding pre-stored benchmark image with the fault probability greater than the preset confidence threshold, and take corresponding measures based on the analysis results.

[0036] Specifically, the fused feature vector is input into a pre-trained diagnostic model for fault identification. The diagnostic model is a multimodal fusion deep learning network, which includes parallel convolutional neural network branches that process two types of features respectively, and performs feature fusion and fault classification through fully connected layers. Based on the diagnostic model, the fault type and fault probability are obtained. For fault probabilities greater than a preset confidence threshold, such as 0.8, the visible light image corresponding to the fused feature vector is compared and analyzed with images in a pre-established benchmark image library to obtain the analysis results. The benchmark image library includes standard case images of the same distribution cabinet under different typical faults. The comparative analysis method will be explained in detail later. Based on the analysis results, staff can take corresponding countermeasures, such as immediately repairing serious abnormalities to prevent them from further aggravating the impact on the continuous operation of the distribution cabinet, and paying close attention to moderate abnormalities.

[0037] In one specific embodiment, the infrared thermal image is preprocessed, which specifically includes the following steps:

[0038] Using a pre-established template of key components of the power distribution cabinet, the key areas are initially located on the original infrared thermal image through image registration. The key areas refer to the rough areas of the key components. At the same time, the background area is defined to generate a binary mask image.

[0039] Guided filtering is performed in the critical region using a first filtering radius and a first regularization parameter, and in the background region using a second filtering radius and a second regularization parameter. After performing guided filtering, the base layer image is obtained, and the base layer image is further processed to obtain the enhanced image.

[0040] Specifically, in order to concentrate subsequent computing resources on the real critical areas and avoid ineffective processing or false enhancement of the background areas, the rough areas of these critical components (such as circuit breakers, terminals, cable joints, etc.) are initially located on the original infrared thermal image using image registration methods by using pre-established templates of key components of the distribution cabinet. At the same time, the background area is defined and a binary mask image is generated. In the binary mask image, the white area represents the critical area and the black area represents the background area.

[0041] To achieve regionalized, edge-preserving noise filtering, different guided filtering parameters are set for key and background regions based on the binary mask image. In the key region, a smaller first filtering radius and a smaller second regularization parameter are used for guided filtering, smoothing out minute noise while preserving the clear edges and fine textures of the components to the maximum extent. In the background region, a larger second filtering radius and a larger second regularization parameter are used for guided filtering, strongly smoothing the background region and suppressing random noise and irrelevant details as much as possible. After guided filtering the entire original infrared thermal image using these adaptive parameters, a base layer image is obtained. This base layer image achieves superior detail preservation in the key region compared to traditional global filtering, while also achieving better noise suppression in the background region. Further processing of the base layer image yields an enhanced image.

[0042] In one specific embodiment, the base layer image is further processed to obtain an enhanced image, specifically including the following steps:

[0043] The initial detail layer is obtained by subtracting the base layer from the original infrared thermal image. The initial detail layer is then filtered by a linear filter with a preset radius to obtain a large-scale detail layer. The large-scale detail layer is then subtracted from the initial detail layer to obtain a small-scale detail layer. In key regions, the large-scale detail layer is enhanced while the small-scale detail layer is weakened. All detail layers are set to zero in the background region. The processed layers are then added together to obtain the enhanced infrared image.

[0044] Specifically, to separate detailed information by scale for better differential processing, a preliminary detail layer is obtained by subtracting the base layer image from the original infrared thermal image. This preliminary detail layer contains all high-frequency information, from minute noise to true edges. Then, a simple linear filter, such as mean filtering, is used based on a preset large radius to filter the preliminary detail layer, resulting in a large-scale detail layer. Mean filtering is a simple linear low-pass filter that smooths out subtle noise and texture while preserving slowly changing major edges and contours over a large area. Mean filtering yields a large-scale detail layer that represents significant fault characteristics. To achieve effective separation of detailed information, from the preliminary... Subtracting the large detail layer representing the main features from the total detail layer leaves noise representing noise and subtle textures. In other words, subtracting the large-scale detail layer from the initial detail layer yields a small-scale detail layer. Then, selective detail enhancement is performed under constraints in key regions. Specifically, within key regions, large-scale detail layers are enhanced, while small-scale detail layers are weakened. In background regions, all detail layers are set to zero. Finally, the processed layers are summed to obtain the final enhanced infrared thermal image. The formula is: Enhanced Image = Base Detail Layer Image + Binary Mask Image ⊙(Processed Large-Scale Detail Layer + Processed Small-Scale Detail Layer), where ⊙ represents pointwise multiplication. The enhanced infrared thermal image is an optimized image with a clean background, clear outlines of key components, and prominently displayed fault hotspots. This provides a reliable and clear data source for subsequent temperature analysis and fault diagnosis, thereby improving the accuracy and reliability of the monitoring system.

[0045] The above method differs from the traditional method of uniformly processing the entire image. It introduces component region segmentation and multi-scale processing based on the power distribution cabinet, and achieves more targeted noise suppression and detail enhancement.

[0046] In one specific embodiment, temperature analysis of the preprocessed infrared thermal image includes the following steps:

[0047] Historical infrared thermal images of the power distribution cabinet under normal conditions on similar dates, during similar time periods, and under similar ambient temperatures are obtained. The average temperature and standard deviation of the corresponding key areas in the historical infrared thermal images are calculated. The average temperature plus twice the standard deviation is taken as the upper limit of the temperature that the corresponding key areas may reach under normal conditions. The temperature margin is set based on electrical safety standards and component characteristics. The upper limit of temperature plus the temperature margin is taken as the abnormal threshold.

[0048] Traverse each pixel in the infrared thermal image, compare the pixel's temperature value with an anomaly threshold. If the temperature value is greater than or equal to the anomaly threshold, mark the corresponding pixel as an anomaly and set the pixel value at the corresponding position in the result mask image to 1. Otherwise, mark it as a normal point and set the pixel value at the corresponding position in the result mask image to 0. Obtain the result mask image, perform an opening operation on the result mask image to smooth the boundaries of the anomaly region, find all connected regions in the result mask image, and ignore connected regions with an area less than or equal to a preset threshold.

[0049] Specifically, in order to determine what constitutes a normal temperature level under current conditions and avoid false alarms in high-temperature environments or missed alarms in low-temperature environments, historical infrared thermal images of the distribution cabinet under normal conditions at similar dates, time periods, and ambient temperatures are acquired. The average temperature and standard deviation of the corresponding key areas in the historical infrared thermal images are calculated. The result obtained by adding twice the standard deviation to the average temperature is taken as the upper limit of the temperature that the corresponding key area may reach under normal conditions. This upper limit of temperature serves as a benchmark reference value. To improve the accuracy and reliability of the alarm, a temperature margin is set above the benchmark reference value based on electrical safety standards and component characteristics (such as the allowable temperature rise of metal conductors). The temperature margin is a safety buffer, such as 5°C or 3°C. The result obtained by adding the temperature margin to the benchmark reference value is taken as the abnormal threshold, which serves as a clear abnormal boundary.

[0050] To convert the infrared thermal image into a result mask image to identify suspicious anomalies, each pixel in the infrared thermal image is traversed, and the temperature value of each pixel is compared with an anomaly threshold. If the temperature value is greater than or equal to the anomaly threshold, it indicates that the temperature of this pixel is abnormal, so the corresponding pixel is marked as an anomaly and the pixel value at the corresponding position in the result mask image is set to 1. Otherwise, it is marked as a normal point and the pixel value at the corresponding position in the result mask image is set to 0. The result mask image is a binary image, and the white area represents the identified abnormal high temperature area.

[0051] To ensure more accurate and reliable location of the abnormal region in the final output, avoid false triggering due to image noise, and guarantee the accuracy of the alarm, an opening operation is performed on the resulting mask image to eliminate isolated white pixels (noise) caused by noise and smooth the boundaries of the abnormal region. Then, all connected regions in the resulting mask image are identified, and connected regions containing only a few (e.g., 9) pixels are ignored, as these may be noise. Only regions with an area greater than a preset threshold are retained as true abnormal regions. Through the above method, more accurate abnormal region mask, location, and area information can be obtained.

[0052] Opening is an operation that first involves erosion and then dilation. First, a structuring element is set, typically a small 3×3 matrix. Erosion is then performed: the structuring element is slid across the resulting mask image. For each pixel at the center of the structuring element, the central pixel is retained as white only if all pixels covered by the structuring element are white; otherwise, it is set to black. Erosion eliminates tiny isolated points, shrinks and smooths boundaries. Then, dilation is performed: using the same structuring element, it is slid across the eroded mask image. For each white pixel at the center of the structuring element, all pixels covered by the structuring element are set to white. Dilation expands the area and fills in concave corners. Opening can remove false anomalies caused by noise while preserving as many genuine abnormal heating areas as possible.

[0053] In one specific embodiment, the registration of the acquired visible light image and infrared thermal image includes the following steps:

[0054] Visible light images and infrared thermal images are convolved with multiple Gaussian kernels of varying standard deviations from small to large to generate a multi-scale convolved image sequence consisting of images with different degrees of blur. The convolved images between adjacent scales are then differentially processed to obtain multiple differential image sequences. Local extrema are identified based on the differential image sequences, and these local extrema are used as candidate feature points. The candidate feature points are then filtered to obtain key feature points.

[0055] For each key feature point, a corresponding descriptor vector is generated. For each descriptor vector in the infrared thermal image, the first distance and the second distance corresponding to the descriptor vector with the closest Euclidean distance and the second closest descriptor vector with the visible light image are obtained. The ratio of the first distance and the second distance is calculated, and all matching point pairs with a ratio less than a preset ratio threshold are obtained.

[0056] Based on all matching point pairs, the RNASC algorithm is used to estimate the matrix parameters to obtain the optimal affine transformation matrix. The affine transformation matrix is ​​then used to perform spatial transformation on the infrared thermal image to achieve accurate registration with the visible light image.

[0057] Specifically, to achieve accurate registration between visible light images and infrared thermal images, key feature points are first extracted. To detect feature points that are stable at different scales, Gaussian kernels with different standard deviations (such as σ, kσ, k) are applied to both the visible light images and the infrared thermal images. 2Convolution is performed on images of different scales (k, ..., k), where k is a positive integer greater than 1 and σ is a preset standard deviation. This generates a multi-scale convolutional image sequence consisting of images with different degrees of blur. Difference is performed between adjacent scales of the multi-scale convolutional image sequence to obtain multiple difference image sequences. For example, subtracting a convolutional image of scale kσ from a convolutional image of scale kσ yields a difference image. All difference images constitute a difference image sequence. Local extrema are found based on the difference image sequence. These local extrema are used as potential candidate feature points. Candidate feature points may contain a large number of useless or duplicate feature points, resulting in computational redundancy and high matching noise. To ensure the high quality and high matching reliability of the final feature points, the candidate feature points are filtered to obtain key feature points.

[0058] For each key feature point, a descriptor vector is generated. Specifically, SIFT or SURF can be used to generate the descriptor vector. The descriptor vector is a fixed-length array, and each value in the array represents the intensity of a certain visual feature in the neighborhood of the key feature point. The similarity between two descriptor vectors can be calculated to determine whether the key feature points correspond to the same physical point, which is the basis of image registration.

[0059] To efficiently select the most likely correct matching point pairs, for each descriptor vector in the infrared thermal image, the descriptor vector with the closest Euclidean distance and the second closest distance are found in the feature descriptor set of the entire visible light image. The ratio of the first distance (closest distance) to the second distance (second closest distance) is calculated. If the ratio is small, it means that the best matching point is much better than the second best point, and this match is likely to be correct. If the ratio is large, it means that the similarity between the best and second best matching points is about the same, indicating that this match is very ambiguous (possibly in a region of repeated texture, such as the heat dissipation mesh of a power distribution cabinet), and it is impossible to determine which is correct. Therefore, these matching point pairs are discarded, and only matching point pairs with a ratio less than a preset threshold are obtained. The above method can effectively filter out a large number of erroneous matches caused by image noise, repeated textures, modal differences, etc., providing reliable matching point pairs for subsequent steps.

[0060] Then, based on all matching point pairs, the RNASC algorithm is used to estimate the matrix parameters to obtain the optimal affine transformation matrix. The RNASC algorithm does not rely on all matching point pairs being correct, but rather uses random sampling to estimate the correct mathematical model from a large number of matching point pairs, thus obtaining the optimal affine transformation matrix. The obtained optimal affine transformation matrix is ​​then used to perform spatial transformation on the infrared thermal image to achieve accurate registration with the visible light image.

[0061] In one specific embodiment, finding local extrema based on a difference image sequence includes the following steps:

[0062] Select a pixel from the difference image sequence, compare the gray value of this pixel with its eight neighboring points in the same difference image, compare it with its nine corresponding neighboring points in the previous adjacent difference image, and compare it with its nine corresponding neighboring points in the next adjacent difference image. When the gray value of this pixel is the maximum value among these twenty-seven points, this pixel is taken as a local extremum.

[0063] Specifically, such as Figure 2 As shown, these are three adjacent difference images. Assuming the selected pixel is point O in the second image, the grayscale values ​​of point O are compared with those of its eight neighboring points O1-O8. Furthermore, the grayscale values ​​of point O are compared with... Figure 2 The grayscale values ​​of O11-O19 in the first image are compared, and then it is also compared with... Figure 2 The gray values ​​of O21 and O29 in the third image are compared. Only when the pixel is the maximum value among these 27 points is the pixel considered a local extremum. The above method ensures that the found point is not only prominent on the two-dimensional image plane, but also stable in multiple scale dimensions by searching for extrema in a multi-dimensional scale space. This ensures that the found point is a good feature point in both clearer and blurrier images.

[0064] In one specific embodiment, filtering candidate feature points to obtain key feature points specifically includes the following steps:

[0065] Obtain the gray values ​​of candidate feature points in the difference image, and remove candidate feature points whose gray values ​​are less than the empirical threshold;

[0066] For the remaining candidate feature points, obtain the points corresponding to the candidate feature points on the larger convolutional image, obtain a preset neighborhood window with this point as the center point, calculate the gradients of all pixels in the x and y directions within this neighborhood window, calculate the gradient covariance matrix of this window, calculate the trace and determinant of the gradient covariance matrix, and remove candidate feature points whose result of dividing the square of the trace of the matrix by the determinant is greater than the preset first threshold.

[0067] For the remaining candidate feature points, determine whether there are other candidate feature points in the neighboring regions of the candidate feature point in the difference image. If so, retain the candidate feature point with the largest gray value in the neighboring region, remove all other candidate feature points in the neighboring region, and take all remaining candidate feature points as key feature points.

[0068] Specifically, in order to quickly screen out key feature points, firstly, if the gray value of a candidate feature point is less than an empirical threshold, it means that the difference between the candidate feature point and its surrounding area is very small, and it is likely not a stable feature point, but rather generated by noise. Therefore, by removing these candidate feature points, unstable points that are sensitive to noise can be eliminated. The empirical threshold is determined by visual analysis and quantitative evaluation based on the specific image dataset in the power distribution cabinet detection scenario.

[0069] For the remaining candidate feature points, obtain the corresponding points on the larger convolutional image. Candidate feature points are determined based on difference images; each candidate feature point corresponds to a difference image, which is obtained by differencing convolutional images of adjacent scales. Therefore, the difference image corresponds to two convolutional images of different scales. Here, the convolutional image corresponding to the larger convolutional kernel is obtained. A preset neighborhood window is obtained with this point as the center, for example, a 3×3 or 5×5 neighborhood window. The gradients of all pixels within this neighborhood window in the x and y directions are calculated, for example, using simple operators such as Sobel. Calculate the gradient covariance matrix of the window, then calculate the trace and determinant of the gradient covariance matrix. Divide the square of the trace by the determinant to obtain a result value. The magnitude of this result value reflects the distribution of the gradient in the neighborhood of the center point. If the result value is less than or equal to the preset first threshold, it means that the result value is relatively small, indicating that this is a corner region with gradient changes in all directions, and it is a stable feature point. If the result value is greater than or equal to the first threshold, it means that the result value is relatively large, indicating that this is an edge region with gradient dominance in only one direction, and it is an unstable feature point. Points located on the edge are removed through this step.

[0070] In a very small area, to avoid retaining multiple duplicate points pointing to the same feature and to make the key feature points more evenly distributed, for the remaining candidate feature points, it is determined whether there are other candidate feature points in the neighboring regions of the candidate feature point in the difference image. If so, the candidate feature point with the largest gray value in the neighboring region is retained, and all other candidate feature points in the neighboring region are removed. This step ensures that each obvious feature is represented by only one most prominent key feature point, reducing redundancy and improving candidate matching efficiency.

[0071] In one specific embodiment, a comparative analysis is performed, which includes the following steps:

[0072] Calculate the global similarity index between a visible light image and its corresponding reference image;

[0073] The system also performs difference calculations on the visible light image and the corresponding reference image to obtain a difference image, generates a binarized image based on the difference image, performs morphological operations on the binarized image, and obtains the changed regions based on the processed binarized image.

[0074] The analysis results are generated based on a comprehensive decision-making process using a global similarity index and the region of change.

[0075] Specifically, in order to refine the correction and verification of faults, the global similarity index (SSIM) between the visible light image and the corresponding reference image is first calculated. The calculation method of the global similarity index is existing technology and will not be explained in detail here. The global similarity index is used to evaluate the macroscopic similarity between the visible light image and the reference image in terms of overall structure, brightness and contrast. The closer the global similarity index is to 1, the more similar the two images are. A value much less than 1 indicates that the two images are very different.

[0076] To accurately pinpoint the location, shape, and size of pixel-level differences between two images, a difference image is obtained by performing difference calculations on the visible light image and the reference image. This difference image is then thresholded to generate a binarized image. For example, if the grayscale value of the difference image is greater than a preset threshold, the corresponding pixel value is set to 1 (white); otherwise, it is set to 0 (black). Morphological operations, such as closing and dilation followed by erosion, are then performed on the binarized image. Opening is a process of erosion followed by dilation, while closing is a process of dilation followed by erosion. Closing removes isolated white pixels caused by image noise, creating continuous and complete connected regions from the actual changes, thus smoothing the boundaries. All white connected regions in the processed binarized image are considered as the changes. These changes allow for the calculation of their areas, the acquisition of corresponding outer rectangles and center coordinates, facilitating subsequent fault location.

[0077] The method for generating comprehensive decision-making analysis results based on global similarity index and change region will be explained in detail later.

[0078] In one specific embodiment, a comparative analysis is performed, which includes the following steps:

[0079] If the global similarity index is greater than the preset similarity threshold, it is confirmed that there is a typical fault of the type, and the fault severity is set to normal. If the global similarity index is less than or equal to the similarity threshold, the area of ​​the largest change region is obtained. If the area is greater than the preset area threshold, the corresponding fault severity is set to severe; otherwise, the corresponding fault severity is set to moderate.

[0080] Specifically, if the global similarity index is greater than the preset similarity threshold, it means that the current visible light image is very similar to the typical fault case image, so there is a corresponding fault. The severity of the fault is similar to that of the typical fault, which is based on historical processing experience. Therefore, the fault severity is set to normal, and corresponding processing can be carried out based on historical experience. If the global similarity index is less than or equal to the similarity threshold, it means that the current visible light image is not similar to the typical fault case image, but the fault model determines that there is a corresponding fault. At this time, the difference image can be used to determine whether the fault has worsened. If it has worsened, the change area may be very obvious and large. Therefore, the change area is obtained. There may be multiple change areas. The area of ​​the largest change area is obtained. If the area is greater than the preset area threshold, it means that the appearance fault has worsened on the basis of the previous fault. At this time, it needs to be focused on. Therefore, the corresponding fault severity is set to severe to remind relevant personnel to pay close attention. If the area is less than or equal to the preset area threshold, it means that the appearance fault has worsened on the basis of the previous fault, but since the area of ​​the change area is not that large, it means that the fault has not worsened much. Therefore, the fault severity is set to moderate.

[0081] The above describes the image recognition-based power distribution cabinet safety monitoring method in the embodiments of this application. The following describes the image recognition-based power distribution cabinet safety monitoring system in the embodiments of this application. Please refer to [link / reference]. Figure 3 One embodiment of the image recognition-based power distribution cabinet safety monitoring system in this application includes:

[0082] The acquisition module is used to deploy image acquisition equipment in key areas of the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, the first of which is the visible light band and the second of which is the infrared imaging band. First, the infrared thermal image of the power distribution cabinet is acquired.

[0083] The analysis module is used to preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, the adjustment module is executed. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered at the same time.

[0084] The adjustment module is used to acquire the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device to place the anomaly area in the center of the image based on the location information, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area.

[0085] The processing module is used to register the acquired visible light image and infrared thermal image, extract features from the registered visible light image to obtain the apparent feature vector, extract features from the registered infrared thermal image to obtain the temperature distribution feature vector, and fuse the apparent feature vector and the temperature distribution feature vector to obtain the fused feature vector.

[0086] The identification module inputs the fused feature vector into the pre-trained diagnostic model to identify faults, obtains the fault type and the corresponding fault probability, compares and analyzes visible light images with fault probabilities greater than a preset confidence threshold with corresponding pre-stored benchmark images, and takes corresponding measures based on the analysis results.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for safety monitoring of power distribution cabinets based on image recognition, characterized in that, The method includes: Step S1: Deploy image acquisition equipment in the key area of ​​the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, where the first wavelength is the visible light band and the second wavelength is the infrared imaging band. First, acquire the infrared thermal image of the power distribution cabinet. Step S2: Preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, proceed to step S3. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered simultaneously. Step S3: Obtain the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device according to the location information to place the anomaly area in the center of the image capture area, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area. Step S4: Register the acquired visible light image and infrared thermal image, including: convolving the visible light image and infrared thermal image with multiple Gaussian kernels of varying standard deviations from small to large, generating a multi-scale convolutional image sequence composed of images with different degrees of blur, and performing a difference operation on the convolutional images between adjacent scales to obtain multiple difference image sequences. Based on the difference image sequences, find local extrema points, use the local extrema points as candidate feature points, and filter the candidate feature points to obtain key feature points, including: obtaining the gray values ​​of the candidate feature points in the difference images, and filtering the gray values ​​less than an empirical value. Candidate feature points are eliminated based on a threshold. For the remaining candidate feature points, each candidate feature point corresponds to a difference image. These difference images are obtained by subtracting convolutional images at adjacent scales; therefore, each difference image corresponds to a convolutional image at two scales. The point corresponding to the candidate feature point on the larger convolutional image is obtained. A preset neighborhood window is obtained with this point as the center. The gradients of all pixels within this neighborhood window in the x and y directions are calculated, and the gradient covariance matrix of this window is calculated. The trace and determinant of the gradient covariance matrix are calculated. The result obtained by dividing the square of the trace by the determinant is greater than a preset value. Candidate feature points are eliminated based on a first threshold. For the remaining candidate feature points, it is determined whether other candidate feature points exist in the neighboring regions of the candidate feature point in the difference image. If so, the candidate feature point with the largest gray value in the neighboring region is retained, and all other candidate feature points in the neighboring region are eliminated. All remaining candidate feature points are used as key feature points. A corresponding descriptor vector is generated for each key feature point. For each descriptor vector in the infrared thermal image, the first distance and the second distance corresponding to the descriptor vector closest to this descriptor vector and the second closest descriptor vector in the visible light image are obtained. The ratio of the first distance and the second distance is calculated, and all matching point pairs with a ratio less than a preset ratio threshold are obtained. Based on all matching point pairs, the matrix parameters are estimated using the RANSAC algorithm to obtain the optimal affine transformation matrix. The affine transformation matrix is ​​used to perform spatial transformation on the infrared thermal image to achieve accurate registration with the visible light image. Feature extraction is performed on the registered visible light image to obtain the apparent feature vector. Feature extraction is performed on the registered infrared thermal image to obtain the temperature distribution feature vector. The apparent feature vector and the temperature distribution feature vector are fused to obtain the fused feature vector. Step S5: Input the fused feature vector into the pre-trained diagnostic model for fault identification to obtain the fault type and corresponding fault probability. Compare and analyze the visible light image with a fault probability greater than a preset confidence threshold with the corresponding pre-stored benchmark image, including: calculating the global similarity index between the visible light image and the corresponding benchmark image; performing difference calculation on the visible light image and the corresponding benchmark image to obtain a difference image; generating a binarized image based on the difference image; performing morphological operations on the binarized image; obtaining the changed regions based on the processed binarized image; and generating a comprehensive decision based on the global similarity index and the changed regions to generate analysis results, including: if the global similarity index is greater than a preset similarity threshold, confirming the existence of a typical fault of the type, and setting the fault severity to normal; if the global similarity index is less than or equal to the similarity threshold, obtaining the area of ​​the largest changed region; if the area is greater than a preset area threshold, setting the corresponding fault severity to severe; otherwise, setting the corresponding fault severity to moderate; and taking corresponding measures based on the analysis results.

2. The method according to claim 1, characterized in that, Preprocessing of infrared thermal images includes: Using a pre-established template of key components of the power distribution cabinet, the key areas are initially located on the original infrared thermal image through image registration. The key areas refer to the rough areas of the key components. At the same time, the background area is defined to generate a binary mask image. Guided filtering is performed in the critical region using a first filtering radius and a first regularization parameter, and in the background region using a second filtering radius and a second regularization parameter. After performing guided filtering, the base layer image is obtained, and the base layer image is further processed to obtain the enhanced image.

3. The method according to claim 2, characterized in that, Further processing of the base layer image yields the enhanced image, including: The initial detail layer is obtained by subtracting the base layer from the original infrared thermal image. The initial detail layer is then filtered by a linear filter with a preset radius to obtain a large-scale detail layer. The large-scale detail layer is then subtracted from the initial detail layer to obtain a small-scale detail layer. In key regions, the large-scale detail layer is enhanced while the small-scale detail layer is weakened. All detail layers are set to zero in the background region. The processed layers are then added together to obtain the enhanced infrared image.

4. The method according to claim 1, characterized in that, Temperature analysis is performed on the preprocessed infrared thermal image, including: Historical infrared thermal images of the power distribution cabinet under normal conditions on similar dates, during similar time periods, and under similar ambient temperatures are obtained. The average temperature and standard deviation of the corresponding key areas in the historical infrared thermal images are calculated. The average temperature plus twice the standard deviation is taken as the upper limit of the temperature that the corresponding key areas may reach under normal conditions. The temperature margin is set based on electrical safety standards and component characteristics. The upper limit of temperature plus the temperature margin is taken as the abnormal threshold. Traverse each pixel in the infrared thermal image, compare the pixel's temperature value with an anomaly threshold. If the temperature value is greater than or equal to the anomaly threshold, mark the corresponding pixel as an anomaly and set the pixel value at the corresponding position in the result mask image to 1. Otherwise, mark it as a normal point and set the pixel value at the corresponding position in the result mask image to 0. Obtain the result mask image, perform an opening operation on the result mask image to smooth the boundaries of the anomaly region, find all connected regions in the result mask image, and ignore connected regions with an area less than or equal to a preset threshold.

5. The method according to claim 1, characterized in that, Finding local extrema based on difference image sequences includes: Select a pixel from the difference image sequence, compare the gray value of this pixel with its eight neighboring points in the same difference image, compare it with its nine corresponding neighboring points in the previous adjacent difference image, and compare it with its nine corresponding neighboring points in the next adjacent difference image. When the gray value of this pixel is the maximum value among these twenty-seven points, this pixel is taken as a local extremum.

6. A power distribution cabinet safety monitoring system based on image recognition, used to implement the power distribution cabinet safety monitoring method based on image recognition as described in any one of claims 1-5, characterized in that, The system includes: The acquisition module is used to deploy image acquisition equipment in key areas of the power distribution cabinet to be monitored. The image acquisition equipment includes two image sensors with different wavelengths, the first of which is the visible light band and the second of which is the infrared imaging band. First, the infrared thermal image of the power distribution cabinet is acquired. The analysis module is used to preprocess the infrared thermal image and perform temperature analysis on the preprocessed infrared thermal image. If an abnormal temperature area is found, the second acquisition module is executed. If the temperature value of the abnormal area is greater than or equal to the alarm temperature, an alarm signal is triggered simultaneously. The adjustment module is used to acquire the location information of the temperature anomaly area in the infrared thermal image, control the image acquisition device to place the anomaly area in the center of the image based on the location information, increase the image acquisition frequency, and simultaneously activate the focusing and supplementary lighting functions of the visible light sensor to simultaneously acquire high-definition visible light images and infrared thermal images of the anomaly area. The processing module is used to register the acquired visible light images and infrared thermal images. This includes: convolving the visible light images and infrared thermal images with multiple Gaussian kernels of varying standard deviations (from smallest to largest) to generate a multi-scale convolutional image sequence consisting of images with different degrees of blur. Then, it performs a difference operation on the convolutional images between adjacent scales to obtain multiple difference image sequences. Based on the difference image sequences, it identifies local extrema, uses these local extrema as candidate feature points, and filters these candidate feature points to obtain key feature points. This includes: acquiring the grayscale values ​​of the candidate feature points in the difference images, and filtering out grayscale values ​​smaller than the standard deviation. Candidate feature points are eliminated based on a threshold. For the remaining candidate feature points, each candidate feature point corresponds to a difference image, which is obtained by differencing convolutional images at adjacent scales. Therefore, the difference image corresponds to two convolutional images at different scales. The point corresponding to the candidate feature point on the larger convolutional image is obtained. A preset neighborhood window is obtained with this point as the center point. The gradients of all pixels within this neighborhood window in the x and y directions are calculated, and the gradient covariance matrix of this window is calculated. The trace and determinant of the gradient covariance matrix are calculated. The result obtained by dividing the square of the trace by the determinant is greater than the threshold value. Candidate feature points are eliminated based on a first threshold. For the remaining candidate feature points, it is determined whether other candidate feature points exist in the neighboring regions of the candidate feature point in the difference image. If so, the candidate feature point with the largest gray value in the neighboring region is retained, and all other candidate feature points in the neighboring region are eliminated. All remaining candidate feature points are used as key feature points. A corresponding descriptor vector is generated for each key feature point. For each descriptor vector in the infrared thermal image, the first distance and the second distance corresponding to the descriptor vector closest to this descriptor vector and the second closest descriptor vector in the visible light image are obtained. The ratio of the first distance and the second distance is calculated, and all matching point pairs with a ratio less than a preset ratio threshold are obtained. Based on all matching point pairs, the matrix parameters are estimated using the RANSAC algorithm to obtain the optimal affine transformation matrix. The affine transformation matrix is ​​used to perform spatial transformation on the infrared thermal image to achieve accurate registration with the visible light image. Feature extraction is performed on the registered visible light image to obtain the apparent feature vector. Feature extraction is performed on the registered infrared thermal image to obtain the temperature distribution feature vector. The apparent feature vector and the temperature distribution feature vector are fused to obtain the fused feature vector. The identification module is used to input fused feature vectors into a pre-trained diagnostic model for fault identification, obtain fault types and corresponding fault probabilities, and compare visible light images with fault probabilities greater than a preset confidence threshold with corresponding pre-stored benchmark images. This includes: calculating the global similarity index between the visible light image and the corresponding benchmark image; performing difference calculations on the visible light image and the corresponding benchmark image to obtain a difference image; generating a binarized image based on the difference image; performing morphological operations on the binarized image; and obtaining the changed regions based on the processed binarized image. Based on the global similarity index and the changed regions, a comprehensive decision is made to generate the analysis results, including: if the global similarity index is greater than a preset similarity threshold, a typical fault of the corresponding type is confirmed, and the fault severity is set to "normal"; if the global similarity index is less than or equal to the similarity threshold, the area of ​​the largest changed region is obtained; if the area is greater than a preset area threshold, the corresponding fault severity is set to "severe"; otherwise, the corresponding fault severity is set to "moderate". Corresponding measures are then taken based on the analysis results.

Images