Fault detection method and device based on high-temperature-resistant camera

Abstract

The invention relates to the technical field of fault detection, and discloses a fault detection method and device based on a high-temperature-resistant camera, and the method comprises the steps: obtaining an infrared image and a visible light image of the surface of high-temperature equipment, and carrying out the registration processing, and obtaining fusion image data; performing temperature field gradient analysis according to the fused image data to obtain temperature gradient distribution data; according to the temperature gradient distribution data, identifying and positioning a temperature abnormal area to obtain temperature abnormal area data; obtaining texture features of the visible light image, and performing similarity analysis to obtain equipment surface deformation area data; and calculating the correlation between the temperature anomaly region data and the equipment surface deformation region data according to the temperature anomaly region data and the equipment surface deformation region data, judging whether a causal relationship exists or not, and if the causal relationship exists and the equipment surface deformation region data is greater than a preset area threshold, determining that the equipment has a fault risk, and outputting a fault detection result. According to the method, accurate fault early warning and space positioning of the high-temperature equipment can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fault detection, and in particular to a fault detection method and device based on a high-temperature-resistant camera. BACKGROUND

[0002] In modern industrial production, real-time state monitoring of key equipment operating in extreme environments such as high temperature and high pressure is crucial, as it is directly related to production safety and efficiency. Temperature abnormalities or physical deformations on the surface of equipment are often early signs of potential faults, so real-time and accurate online monitoring of these equipment is a key link to ensure production safety, extend equipment life and improve economic efficiency. At present, visual detection technology, as a non-contact online monitoring method, has been widely used in equipment fault warning field due to its efficiency and intuitiveness.

[0003] In one prior art, a single detection method is usually used to monitor high-temperature equipment, for example, only an infrared thermal imager is used for temperature measurement to determine whether there is an abnormality by capturing high-temperature points on the surface. However, such a method has significant limitations when dealing with complex industrial site environments. Due to the lack of simultaneous observation of the physical form of the equipment surface, when a temperature anomaly is detected, the system has difficulty in determining whether the anomaly is caused by a real equipment crack, material thinning, or other physical deformation, or is caused by false positives due to complex textures, stains or background thermal radiation interference on the equipment surface. The lack of multi-source information fusion capability makes the detection system unable to accurately establish the causal relationship between temperature changes and physical deformations, thereby reducing the accuracy and reliability of fault diagnosis.

[0004] The prior art has the problem of insufficient recognition accuracy of complex fault characteristics. SUMMARY

[0005] The present application provides a fault detection method and device based on a high-temperature-resistant camera to solve the problem of insufficient recognition accuracy of complex fault characteristics.

[0006] In a first aspect, to solve the above technical problems, the present application provides a fault detection method based on a high-temperature-resistant camera, comprising: obtaining infrared images and visible light images of the surface of a high-temperature equipment and performing registration processing to obtain fusion image data; performing temperature field gradient analysis according to the fusion image data to obtain temperature gradient distribution data; performing spatial continuity compensation and noise suppression on the temperature gradient distribution data, and if the gradient amplitude exceeds a preset noise threshold, judging it as an effective temperature jump signal to obtain a first temperature jump signal; Regionally scanning the first temperature jump signal, calculating a segmentation threshold and identifying a spatial continuous feature to obtain a first candidate abnormal region; According to the first candidate abnormal region, calculating a spatial aggregation degree of the abnormal region and performing an expansion operation to obtain a second candidate abnormal temperature region; From the second candidate abnormal temperature region, obtaining corresponding texture features, calculating the similarity of the texture features and the preset normal device surface texture, and if the similarity is less than a preset similarity threshold, determining it as a deformation region to obtain a device surface deformation region; Calculating the correlation coefficient of the device surface deformation region and the second candidate abnormal temperature region, if the correlation coefficient is greater than a preset correlation threshold, and the abnormal temperature region is greater than a preset area lower limit value, determining that the device has a failure risk, and outputting a failure detection result.

[0007] Preferably, infrared images and visible light images of the high-temperature device surface are obtained and registration processing is performed to obtain fused image data, including: Obtaining infrared radiation information of the high-temperature device surface; According to the infrared radiation information, image acquisition is performed to obtain infrared images and visible light images; According to the infrared images and visible light images, correction and registration are performed to obtain fused image data.

[0008] Preferably, according to the fused image data, temperature field gradient analysis is performed to obtain temperature gradient distribution data, including: From the fused image data, temperature information is extracted and convolution operation is performed to obtain first and second order gradient distribution of the temperature field; According to the first and second order gradient distribution, normalization processing and convolution operation are performed to obtain multi-level temperature gradient representation; According to the multi-level temperature gradient, weight factors are calculated and distributed, and weighted fusion is performed to obtain temperature gradient distribution data.

[0009] Preferably, the temperature gradient distribution data is subjected to spatial continuity compensation and noise suppression, and if the gradient amplitude exceeds a preset noise threshold, it is judged as an effective temperature jump signal to obtain a first temperature jump signal, including: According to the temperature gradient distribution data, spatial continuity compensation is performed to obtain a gradient vector field distribution map; According to the gradient vector field distribution map, noise suppression processing is performed to obtain a set of denoised gradient data; According to the set of denoised gradient data, threshold detection and clustering processing are performed to obtain the first temperature jump signal.

[0010] Preferably, the region scanning on the first temperature jump signal, calculating a segmentation threshold and identifying a spatially continuous feature to obtain a first candidate abnormal region, comprises: performing window sliding scanning on the first temperature jump signal and calculating a segmentation threshold to obtain a segmentation threshold parameter; According to the segmentation threshold parameter, the spatially continuous feature of the temperature gradient is identified to obtain a spatially continuous feature label map; According to the spatially continuous feature label map, region growing is performed to obtain the first candidate abnormal region.

[0011] Preferably, the calculating the spatial aggregation degree of the abnormal region and performing an expansion operation according to the first candidate abnormal region to obtain a second candidate abnormal temperature region, comprises: extracting temperature abnormal points from the first candidate abnormal region, and performing spatial position analysis on the temperature abnormal points to obtain a region aggregation density distribution map; According to the region aggregation density distribution map, find the density peak position and mark it as a seed candidate point to obtain a seed candidate point coordinate set; According to the seed candidate point coordinate set, an expansion operation is performed to obtain an abnormal region growth result; According to the abnormal region growth result, region tracking is performed to obtain the second candidate abnormal temperature region.

[0012] Preferably, the corresponding texture feature is obtained from the second candidate abnormal temperature region, the similarity between the texture feature and the preset normal device surface texture is calculated, and if the similarity is less than a preset similarity threshold, it is determined as a deformation region to obtain a device surface deformation region, comprising: obtaining a visible light image region corresponding to the second candidate abnormal temperature region from the second candidate abnormal temperature region; extracting texture features from the visible light image region to obtain region texture features; Calculate the similarity between the region texture features and the preset normal device surface texture, and if the similarity is less than a preset similarity threshold, it is determined as a deformation region to obtain a device surface deformation region.

[0013] Preferably, the correlation coefficient between the device surface deformation region and the second candidate abnormal temperature region is calculated, if the correlation coefficient is greater than a preset correlation threshold, and the abnormal temperature region is greater than a preset area lower limit value, it is determined that the device has a fault risk, and the fault detection result is output, comprising: Calculate the spatial overlap degree of the device surface deformation region and the second candidate abnormal temperature region to obtain a correlation coefficient; The correlation coefficient is compared with a preset correlation threshold value, if the correlation coefficient is greater than the preset correlation threshold value, it is judged that the temperature anomaly and the deformation exist a causal relationship, and abnormal region boundary data is obtained; A total pixel area is extracted from the abnormal region boundary data, if the total pixel area is greater than a preset area lower limit value, it is determined that the equipment exists a failure risk, and a failure detection result is output.

[0014] In a second aspect, the present application provides a failure detection device based on a high-temperature-resistant camera, comprising: A data acquisition and registration module is configured to acquire infrared images and visible light images of a high-temperature equipment surface, and perform registration processing to obtain fused image data; A gradient analysis module is configured to perform temperature field gradient analysis according to the fused image data to obtain temperature gradient distribution data; A signal identification module is configured to perform spatial continuity compensation and noise suppression on the temperature gradient distribution data, if the gradient amplitude exceeds a preset noise threshold value, it is judged that it is an effective temperature jump signal, and a first temperature jump signal is obtained; An abnormal region identification module is configured to perform region scanning on the first temperature jump signal, calculate a segmentation threshold value, and identify spatial continuity features to obtain a first candidate abnormal region; An abnormal region positioning module is configured to calculate the spatial aggregation degree of the abnormal region according to the first candidate abnormal region and perform an expansion operation to obtain a second candidate abnormal temperature region; A deformation region identification module is configured to acquire corresponding texture features from the second candidate abnormal temperature region, calculate the similarity of the texture features and a preset normal equipment surface texture, if the similarity is less than a preset similarity threshold value, it is determined as a deformation region, and an equipment surface deformation region is obtained; A failure risk judgment module is configured to calculate the correlation coefficient of the equipment surface deformation region and the second candidate abnormal temperature region, if the correlation coefficient is greater than a preset correlation threshold value, and the abnormal temperature region is greater than a preset area lower limit value, it is determined that the equipment exists a failure risk, and a failure detection result is output.

[0015] In a third aspect, the present application further provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, when the processor executes the computer program, the above-mentioned failure detection method based on a high-temperature-resistant camera is realized.

[0016] In a fourth aspect, the present application also provides a computer readable storage medium comprising a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the fault detection method based on the high-temperature-resistant camera according to any one of the above aspects when the computer program is running.

[0017] Compared with the prior art, the present application has the following beneficial effects: (1) The present application synchronously acquires infrared thermal images and visible light images of high-temperature equipment, and realizes pixel-level spatial alignment by using image registration and distortion correction technology, thereby constructing a dual-modal data basis that fuses temperature and texture information; further, the present application realizes accurate capture of weak temperature changes under different scales by using multi-scale gradient operators and pyramid structures for fine analysis of temperature fields, thereby solving the problem of low signal-to-noise ratio and being easily disturbed by complex surface textures and background thermal radiation of the equipment caused by the prior art relying on a single data source, and providing comprehensive and reliable data support for subsequent fault identification.

[0018] (2) The present application establishes a coarse-to-fine abnormal region positioning system by using adaptive threshold segmentation, spatial continuity judgment, and a secondary region growing algorithm based on cluster density statistics, which can accurately separate and lock real and continuous temperature abnormal regions from complex background noise, thereby solving the problem of inaccurate fault positioning or missed detection caused by the prior art being difficult to effectively distinguish real abnormalities from random noise due to the use of a fixed threshold, and significantly improving the accuracy and reliability of abnormal region positioning.

[0019] (3) The present application extracts visible light image texture features corresponding to the positioned temperature abnormal regions to identify physical deformation, and further calculates the spatial overlap and correlation coefficient of the temperature abnormal region and the deformation region, thereby establishing a quantitative judgment mechanism of the causal relationship between temperature abnormalities and physical deformation. This cross-validation of multi-modal information solves the core pain point that the prior art cannot confirm whether the temperature abnormality is caused by real physical damage, realizes the leap from simple "phenomenon alarm" to high-confidence "fault confirmation", and greatly reduces the fault false alarm rate. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of a fault detection method based on a high-temperature-resistant camera provided by the first embodiment of the present application; Figure 2 is a structural diagram of a fault detection device based on a high-temperature-resistant camera provided by the second embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of the present application.

[0022] With reference to Figure 1 The first embodiment of the present application provides a fault detection method based on a high-temperature-resistant camera, comprising the following steps: S11, acquiring infrared images and visible light images of a high-temperature equipment surface and performing registration processing to obtain fusion image data; S12, performing temperature field gradient analysis according to the fusion image data to obtain temperature gradient distribution data; S13, performing spatial continuity compensation and noise suppression on the temperature gradient distribution data, and if the gradient amplitude exceeds a preset noise threshold, determining that it is an effective temperature jump signal to obtain a first temperature jump signal; S14, performing regional scanning on the first temperature jump signal, calculating a segmentation threshold and identifying spatial continuity features to obtain a first candidate abnormal region; S15, calculating the spatial aggregation degree of the abnormal region according to the first candidate abnormal region and performing an expansion operation to obtain a second candidate abnormal temperature region; S16, acquiring corresponding texture features from the second candidate abnormal temperature region, calculating the similarity of the texture features and a preset normal equipment surface texture, and if the similarity is less than a preset similarity threshold, determining that it is a deformation region to obtain an equipment surface deformation region; S17, calculating the correlation coefficient of the equipment surface deformation region and the second candidate abnormal temperature region, and if the correlation coefficient is greater than a preset correlation threshold and the abnormal temperature region is greater than a preset area lower limit value, determining that the equipment has a fault risk and outputting a fault detection result.

[0023] In step S11, the infrared images and visible light images of the high-temperature equipment surface are acquired, and registration processing is performed to obtain fusion image data, comprising: acquiring infrared radiation information of the high-temperature equipment surface; performing image acquisition according to the infrared radiation information to obtain infrared images and visible light images; performing correction and registration according to the infrared images and visible light images to obtain fusion image data.

[0024] In an implementation, the step is realized by a dual-light camera system consisting of a high-resolution long-wave infrared thermal imager and a visible light industrial camera. First, the infrared thermal imager collects the raw digital signal of the infrared radiation of the equipment in real time, and converts it into actual temperature values through the built-in temperature calibration algorithm. When the temperature of a continuous NxN pixel area on the surface of the equipment is detected to exceed the preset temperature trigger threshold (which is set according to the equipment safety operation specification, such as 80 degrees Celsius), the system will immediately send a synchronous acquisition instruction to trigger the visible light industrial camera to take a picture at the same millisecond level time.

[0025] After obtaining the time-synchronized dual-mode image information, spatial alignment is needed. This process is realized through image registration technology, and a feature-based matching method can be used. For example, the Harris corner point detection algorithm extracts the corner features of the hot spot area in the infrared image. The algorithm calculates the gray level change after moving the pixel points in each direction, and the response function is where is the 2x2 gradient covariance matrix, and the sensitivity factor is usually set to between 0.04 and 0.06. At the same time, the Canny edge detection algorithm is used to extract the edge features of the equipment outline in the visible light image. By calculating the descriptors (such as SIFT or SURF) of these feature points and matching them, a series of matching point pairs can be obtained. If the number of matching point pairs is less than the preset minimum feature matching threshold (such as 50), it indicates that there are not enough feature points. According to these accurate correspondence relationships, the affine transformation matrix H required to map the infrared image to the visible light image coordinate system can be calculated.

[0026] Before registration, geometric distortion correction is also needed to eliminate the image distortion caused by the lens itself. This process can use the intrinsic parameters obtained during camera calibration, and the radial distortion correction formula is where is the distorted coordinate, is the distance from the point to the center of the image, , , is the radial distortion coefficient. In a specific setting, if the degree of distortion in the edge area of the image is detected to be greater than the maximum distortion threshold (such as 2 pixels), a tangential distortion correction term will be added for compensation.

[0027] Finally, the distortion-corrected infrared image is mapped to the coordinate space of the visible light image based on the calculated affine transformation matrix H, and a pixel-level correspondence is established between the two images. The pixel-level correspondence here means that for any pixel coordinate in the visible light image, the corresponding pixel coordinate in the infrared image observing the same physical point of the device can be found through the transformation matrix H. By performing a weighted fusion process on the infrared radiation intensity information and the visible light brightness information, the first fused image data containing accurate temperature information and rich texture details is ultimately generated. During the weighted fusion processing stage, the system dynamically adjusts the fusion weights based on the characteristics of the image content: for areas with higher temperatures, the system assigns a higher weight to the infrared information to highlight temperature anomalies; for areas with rich texture details, the system increases the contribution of the visible light information to preserve structural details.

[0028] It's important to note that this dual-modal image acquisition and fusion method provides a unified, information-rich perspective for subsequent fault analysis. Time-synchronized acquisition ensures data immediacy, while precise geometric correction and alignment ensure spatial accuracy. Registration accuracy reaches sub-pixel levels, laying the foundation for precise correlation between temperature anomalies and physical deformation.

[0029] In step S12, temperature field gradient analysis is performed based on the fused image data to obtain temperature gradient distribution data, including: Extracting temperature information from the fused image data, performing a convolution operation, and obtaining first-order and second-order gradient distributions of the temperature field; According to the first-order and second-order gradient distributions, normalization processing and convolution operations are performed to obtain a multi-level temperature gradient representation; According to the multi-level temperature gradient, weight factors are calculated and assigned, and weighted fusion is performed to obtain temperature gradient distribution data.

[0030] In one implementation, this step first extracts temperature information from the fused image data to form a two-dimensional temperature matrix. Specifically, since the fused image data contains the temperature information collected by the infrared thermal imager, the system can directly parse the temperature value corresponding to each pixel from the fused data, thereby obtaining a two-dimensional temperature matrix. Each element in the matrix represents the temperature value of a specific physical location on the surface of the device, in degrees Celsius. Then, the Sobel operator is used to convolve the temperature field to obtain the first-order partial derivative matrix of the temperature field in the horizontal and vertical directions. and The gradient amplitude is given by the formula At the same time, the Laplace operator is used to perform a quadratic differential operation on the temperature field to obtain a second-order gradient distribution, which is particularly sensitive to the inflection points and extreme points of temperature changes.

[0031] After obtaining the original gradient distribution, in order to analyze on different scales, a multi-level temperature gradient representation is created by constructing a Gaussian pyramid. The pyramid construction here is an iterative, dimension-reducing process from high resolution to low resolution, which is implemented as follows: First, the gradient amplitude data after maximum-minimum value normalization is taken as the 0th layer of the pyramid, denoted as Second, the first layer of the pyramid (L1) is constructed. The L0 image is subjected to Gaussian smoothing, i.e., a Gaussian filter with a specific Gaussian kernel standard deviation (e.g., = 0.8) is convolved with to obtain a blurred image. Then, the blurred image is down-sampled, specifically, all even rows and even columns of the image are removed, making its width and height both reduced to one half of the original size. This blurred and down-sampled image with a size of one quarter of the original size is the first layer of the pyramid Then, the second layer of the pyramid (L2) is constructed. The first layer image is taken as input, and the above process is repeated. That is, a Gaussian filter with a Gaussian kernel standard deviation (e.g., = 1.6) is convolved with , and then the result is down-sampled (even rows and even columns are removed). The resulting smaller-sized image is the second layer of the pyramid Finally, the third layer of the pyramid (L3) is constructed. The second layer image is taken as input, and the process is repeated again, using a Gaussian filter with a Gaussian kernel standard deviation (e.g., = 3.2) for convolution smoothing, and down-sampling, to finally obtain the third layer L3 of the pyramid.

[0032] Finally, the three layers of temperature gradient representation are fused by weighting. The allocation of weight factors is based on the change amplitude of each layer of gradient representation: if the change rate of a layer of gradient exceeds a preset gradient intensity threshold (which is usually set through statistical analysis of historical data of a large number of normally operating equipment, e.g., 0.6), it is considered that the features of this layer are more prominent, and a higher weight factor is assigned to it (e.g., = 0.5); otherwise, a lower weight is assigned (e.g., = 0.3 or = 0.2). The weighted sum formula is Finally, a second gradient intensity matrix, which integrates multi-scale information and highlights the key temperature change region, i.e. temperature gradient distribution data, is obtained.

[0033] In step S13, the temperature gradient distribution data is subjected to spatial continuity compensation and noise suppression. If the gradient amplitude exceeds a preset noise threshold, it is determined as an effective temperature jump signal, and a first temperature jump signal is obtained, including: According to the temperature gradient distribution data, spatial continuity compensation is performed to obtain a gradient vector field distribution map; According to the gradient vector field distribution map, noise suppression processing is performed to obtain a denoised gradient data set; According to the denoised gradient data set, threshold detection and clustering processing are performed to obtain the first temperature jump signal.

[0034] In an implementation manner, the second gradient intensity matrix is first subjected to spatial continuity compensation. A bilinear interpolation algorithm is used to fill the blank areas between the discrete data points in the matrix with numerical values. In a specific implementation, the algorithm is a process of estimating the value of a target point through the values of the four nearest neighbor known data points around the target point. The core idea is to first perform two independent linear interpolations in the first direction (e.g. horizontal direction) to obtain the estimated values of two intermediate points; and then perform a linear interpolation in the second direction (e.g. vertical direction) according to the values of the two intermediate points to calculate the final value of the target point.

[0035] Next, in order to suppress random noise, noise detection is performed on the gradient vector field distribution map. This process is based on an adaptive Wiener filter, which dynamically adjusts the filter parameters by analyzing the local gradient change rate. For example, when the gradient fluctuation frequency of a certain region is higher than the background noise frequency, the system will automatically reduce the filtering strength of this region. The filter estimates the true signal by minimizing the mean square error of the signal and noise, and finally obtains a denoised gradient data set.

[0036] Finally, threshold detection and clustering processing are performed on the gradient data set after noise reduction to obtain the first temperature jump signal. This process adopts double determination criteria. First, when the gradient amplitude of a certain data point is greater than the preset noise threshold (for example, 5.2, which is obtained by statistically analyzing the gradient data of the normally operating equipment in the fault-free state and taking the mean value plus three times the standard deviation), the system further checks the duration space of the signal. If the number of consecutive pixel points exceeds the minimum signal width requirement (for example, 6 pixel units), the data point is marked as a candidate temperature jump position. Then, a connected domain labeling algorithm is used to regionally cluster adjacent candidate jump positions, and if the distance between adjacent candidate positions is less than the preset aggregation radius (for example, 10 pixel units), they are merged into the same temperature jump event. The accurate position coordinates of each jump event are determined by the centroid calculation method to obtain the first temperature jump signal.

[0037] In step S14, texture features of the visible light image are obtained from the temperature anomaly region, and similarity analysis is performed to obtain the device surface deformation region data, including: The first temperature jump signal is regionally scanned, the segmentation threshold is calculated, and the spatial continuous feature is identified to obtain the first candidate abnormal region, including: The first temperature jump signal is windowed and scanned, and the segmentation threshold is calculated to obtain the segmentation threshold parameter. According to the segmentation threshold parameter, the spatial continuous feature of the temperature gradient is identified to obtain the spatial continuous feature label map. According to the spatial continuous feature label map, region growing is performed to obtain the first candidate abnormal region.

[0038] In an implementation manner, first, the spatial coordinate information of the first temperature jump signal is obtained, and a fixed-size rectangular window is used to slide scan the data region row by row and column by column. If the current window position exceeds the data boundary range (for example, the data boundary is 512*384 pixels), the system automatically adjusts the window size to adapt to the boundary constraint to obtain a complete region scanning path sequence. According to the numerical distribution of the temperature data in each window, the fluctuation amplitude of the temperature data in the window is analyzed by a statistical variance calculator. If the fluctuation amplitude is higher than the preset reference value (for example, 6.0), the threshold adjustment factor value is increased by a preset linear adjustment formula, for example, new threshold adjustment factor = original threshold adjustment factor * (fluctuation amplitude / preset reference value), to obtain the dynamically adjusted segmentation threshold parameter.

[0039] Next, according to the dynamically adjusted segmentation threshold parameter, a pixel adjacency detector is used to verify the continuity of the temperature data points in each scanning window. Specifically, if the temperature difference of three adjacent pixels is less than the segmentation threshold and the spatial distance is within a preset neighborhood radius (e.g., 2 pixel units), the pixel combination is marked as a continuous temperature gradient feature, and a spatial continuity feature label map is obtained.

[0040] Finally, according to the label distribution pattern in the spatial continuity feature label map, a region growing algorithm is used to aggregate the pixel points with the same label attribute. The seed point selection rule of this algorithm is to randomly select an unaggregated pixel point from the label map as the initial seed point. Then, the algorithm searches for pixel points with the same label attribute in the eight adjacent directions of the seed point. If the number of aggregated pixel points exceeds a preset minimum region area threshold (e.g., 300 pixels), the aggregated region is identified as a candidate abnormal temperature region, and a first candidate abnormal region set is obtained.

[0041] In step S15, according to the first candidate abnormal region, the spatial aggregation degree of the abnormal region is calculated and an expansion operation is performed to obtain a second candidate abnormal temperature region, including: Extracting temperature abnormal points from the first candidate abnormal region, performing spatial position analysis on the temperature abnormal points, and obtaining a region aggregation density distribution map; According to the region aggregation density distribution map, finding the density peak position and marking it as a seed candidate point to obtain a seed candidate point coordinate set; According to the seed candidate point coordinate set, performing an expansion operation to obtain an abnormal region growth result; According to the abnormal region growth result, performing region tracking to obtain the second candidate abnormal temperature region.

[0042] In an implementation, pixel coordinate data of each region in the first set of candidate abnormal regions is first acquired. Then, spatial position analysis is performed on the temperature abnormal points in the region, which is achieved by the following steps: if the Euclidean distance between the abnormal points is less than a preset aggregation radius (e.g., 5.0 pixels) and the number of abnormal points exceeds a minimum aggregation number threshold (e.g., 12), a kernel density estimation method is used to calculate the aggregation density parameter value of the region to obtain a region aggregation density distribution map. Specifically, kernel density estimation is a non-parametric estimation method for estimating the probability density function of a random variable. The calculation process is to calculate the density contribution value of each pixel point in the region according to the number and distance of the temperature abnormal points around the pixel point, and finally generate a region aggregation density distribution map reflecting the spatial aggregation degree of the abnormal points. Then, according to the density value distribution in the region aggregation density distribution map, the density peak position is found in each candidate region, and if the density value of the current pixel point is greater than the density values of all pixel points in its eight-neighborhood range, the pixel point is marked as a seed candidate point to obtain a set of seed candidate point coordinates.

[0043] The extension operation is achieved by the following steps: the seed candidate point is taken as a starting position to perform an extension operation on the surrounding pixel points, and if the temperature difference of adjacent pixel points in the 8-connected neighborhood is less than a growth threshold (e.g., 2.5°C), the adjacent pixel points are attributed to the growth region of the current seed point. It should be noted that the growth region is a dynamically aggregated pixel set, which includes the starting seed point and all adjacent pixel points meeting the growth condition. The process continues until there is no new pixel point meeting the growth condition, and finally the abnormal region growth result is obtained. Finally, the region tracking is achieved by performing single-pixel-precision contour tracking on the region edges in the abnormal region growth result, so as to obtain the second candidate abnormal temperature region data. Specifically, the process first identifies the starting point of the growth region edge, and then traverses and records all pixel points on the region boundary in a preset direction (e.g., clockwise) pixel by pixel to obtain a sequence containing all boundary pixel coordinates, thereby achieving single-pixel-precision contour extraction.

[0044] In step S16, the corresponding texture features are acquired from the second candidate abnormal temperature region, the similarity between the texture features and the preset normal device surface texture is calculated, and if the similarity is less than a preset similarity threshold, the region is determined as a deformation region, and a device surface deformation region is obtained, including: acquiring a visible light image region from the second candidate abnormal temperature region; extracting texture features from the visible light image region to obtain region texture features; Calculate the similarity between the region texture feature and the preset normal device surface texture, and if the similarity is less than the preset similarity threshold, determine that it is a deformation region, and obtain the device surface deformation region.

[0045] First, according to the image registration mapping relationship established in step S11, the system maps the pixel coordinate range of the second candidate abnormal temperature region to the corresponding visible light image, thereby accurately obtaining the visible light image region to be analyzed.

[0046] The texture feature refers to a set of texture statistical feature parameters calculated by a local binary pattern (LBP) descriptor and a gray level co-occurrence matrix (GLCM). These features are used to quantify the local microscopic patterns and global statistical properties of the device surface texture. In specific implementation, first, the local binary pattern (LBP) coding is performed on the region to be analyzed of the visible light image. This process compares the gray value of each pixel point with the gray values of its eight neighborhood pixel points. If the neighborhood pixel gray value is greater than or equal to the center pixel, it is marked as 1, otherwise it is marked as 0. Finally, a binary coding sequence is generated as the local texture descriptor of the pixel point. Then, the gray level co-occurrence matrix (GLCM) is constructed. This matrix describes the global characteristics of the texture by counting the frequency of pixel pairs with a specific spatial relationship (e.g. 1 pixel unit distance, 0 degree direction) in the image. That is, the value of each element (i, j) in the matrix represents the number of times that the pixel pairs with gray levels i and j appear with a specific spatial relationship in the image. Based on this matrix, multiple texture statistical parameters such as texture contrast, entropy, angular second moment and correlation are calculated, which together form a set of region texture statistical feature parameters.

[0047] In order to determine whether the region is a deformation region, the system pre-establishes a normal device surface texture feature library. This library is a set of standard texture feature vectors extracted and stored by the same method when the device is in a known normal state. Then, through the similarity measure, the texture statistical feature parameter set of the current region is compared with all the standard feature vectors in the feature library, and the Pearson correlation coefficient between them is calculated as a quantitative index to measure the similarity. If all the calculated correlation coefficients are less than the preset similarity threshold (e.g. 0.65), it indicates that the texture feature of the current region deviates significantly from the normal state, and the system determines that the region is a surface deformation anomaly, and outputs the device surface deformation region position data.

[0048] In step S17, the correlation coefficient between the device surface deformation region and the second candidate abnormal temperature region is calculated, and if the correlation coefficient is greater than the preset correlation threshold and the abnormal temperature region is greater than the preset area lower limit value, it is determined that the device has a risk of failure, and the failure detection result is output, including: calculate a spatial overlap degree between the device surface deformation region and the second candidate abnormal temperature region to obtain a correlation coefficient; compare the correlation coefficient with a preset correlation threshold value, and if the correlation coefficient is greater than the preset correlation threshold value, determine that the temperature abnormality and the deformation have a causal relationship, and obtain abnormal region boundary data; extract a total pixel area from the abnormal region boundary data, and if the total pixel area is greater than a preset area lower limit value, determine that the device has a failure risk, and output a failure detection result.

[0049] First, the system obtains device surface deformation region position data and second candidate abnormal temperature region data, calculates the spatial overlap degree of the two, and in specific implementation, this process is based on geometric analysis of region boundary coordinates, and the spatial correlation of the two regions is quantified by calculating the ratio of the intersection and union areas of the two regions.

[0050] The correlation coefficient is calculated by the Pearson correlation algorithm, and the core idea is to use a statistical linear correlation analysis method. In specific implementation, the system extracts the spatial distribution feature vector of the deformation region and the temperature abnormal region, including region center coordinates, length-width ratio, boundary regularity, etc. Then, a quantitative correlation coefficient is calculated by the Pearson correlation formula.

[0051] Next, the correlation coefficient is compared with a preset correlation threshold value (for example, the threshold value is 0.75), and if the correlation coefficient is greater than the threshold value, the system determines that the temperature abnormality and the surface deformation have a strong causal relationship, confirms that the two belong to a composite abnormal phenomenon caused by the same fault source, and merges the boundaries of the two causally related abnormal regions to obtain the final abnormal region boundary data.

[0052] Finally, the system counts the total pixel area of the abnormal region by a pixel counting function. When the system processes an abnormal region with a boundary range of [195, 145] to [270, 205], the counter will scan all the pixel points in this rectangular range row by row, and count the number of valid pixels belonging to the abnormal region. After complete traversal, if the total pixel area (for example, 4125 pixels) obtained by counting is greater than a preset area lower limit value (for example, 3000 pixels), it is determined that the device has a failure risk, and the failure detection result and spatial position coordinate calibration information are output.

[0053] It should be noted that the causal relationship is defined as a multi-stage judgment rule. In the first stage, the spatial overlap degree of the temperature abnormal region A and the deformation region B is calculated, and the Jaccard coefficient can be used, that is, If If the spatial overlap is greater than a preset spatial overlap threshold T_overlap (for example, T_overlap = 0.5), the second stage is entered. In the second stage, the spatial distribution feature vectors (for example, vectors containing the coordinates of the center of mass, the aspect ratio, and the boundary regularity) of the two regions are extracted respectively, and the Pearson correlation coefficient r of the two vectors is calculated. It should be noted that the boundary regularity refers to the smoothness or regularity of the region boundary, and the calculation method is the ratio of the perimeter of the region to the perimeter of a circle with the same area. The greater the ratio, the more irregular the boundary. If r is greater than a preset feature correlation threshold T_corr (for example, T_corr = 0.75), it is determined that there is a strong causal relationship between the temperature anomaly and the surface deformation.

[0054] In summary, the present application fuses the temperature jump of the infrared image and the surface deformation of the visible light image, and establishes a causal relationship judgment mechanism therebetween, thereby realizing precise fault early warning and spatial positioning of high-temperature equipment.

[0055] Reference Figure 2 The second embodiment of the present application provides a fault detection device based on a high-temperature-resistant camera, comprising: A data acquisition and registration module is configured to acquire infrared images and visible light images of the surface of the high-temperature equipment, and perform registration processing to obtain fused image data; A gradient analysis module is configured to perform temperature field gradient analysis based on the fused image data to obtain temperature gradient distribution data; A signal recognition module is configured to perform spatial continuity compensation and noise suppression on the temperature gradient distribution data. If the gradient amplitude exceeds a preset noise threshold, it is determined to be an effective temperature jump signal, and a first temperature jump signal is obtained. An abnormal region recognition module is configured to perform region scanning on the first temperature jump signal, calculate a segmentation threshold, and recognize spatial continuity features to obtain a first candidate abnormal region. An abnormal region positioning module is configured to calculate the spatial aggregation degree of the abnormal region based on the first candidate abnormal region and perform an expansion operation to obtain a second candidate abnormal temperature region. A deformation region recognition module is configured to obtain corresponding texture features from the second candidate abnormal temperature region, calculate the similarity of the texture features and a preset normal equipment surface texture, and determine a deformation region if the similarity is less than a preset similarity threshold, thereby obtaining a device surface deformation region. A fault risk judgment module is configured to calculate the correlation coefficient of the device surface deformation region and the second candidate abnormal temperature region. If the correlation coefficient is greater than a preset correlation threshold, and the abnormal temperature region is greater than a preset area lower limit value, it is determined that the device has a fault risk, and a fault detection result is output.

[0056] It should be noted that the high-temperature-resistant camera-based fault detection device provided in the embodiments of the present application is used to execute all process steps of the high-temperature-resistant camera-based fault detection method provided in the above embodiments, and the working principles and beneficial effects of the two are one-to-one correspondence, thus not being described in detail.

[0057] The embodiments of the present application also provide an electronic device. The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, for example, a high-temperature-resistant camera fault detection program. The processor implements the steps in the above various high-temperature-resistant camera-based fault detection method embodiments when executing the computer program, for example Figure 1 The processor implements the functions of the modules / units in the above various device embodiments when executing the computer program, for example, a signal identification module.

[0058] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0059] The electronic device can be a desktop computer, a notebook computer, a palm computer, a smart tablet, and the like. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the above, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like.

[0060] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like, and the processor is the control center of the electronic device, which connects all parts of the electronic device through various interfaces and lines.

[0061] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0062] The modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0063] It should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separate, and the units displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0064] The above specific embodiments further illustrate the purpose, technical scheme and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A fault detection method based on a high temperature resistant camera, characterized in that: include: Acquire infrared images and visible light images of the surface of high-temperature equipment, perform registration processing, and obtain fused image data; Performing temperature field gradient analysis based on the fused image data to obtain temperature gradient distribution data; Performing spatial continuity compensation and noise suppression on the temperature gradient distribution data, and determining that it is a valid temperature jump signal if the gradient amplitude exceeds a preset noise threshold, thereby obtaining a first temperature jump signal; Performing a regional scan on the first temperature jump signal, calculating a segmentation threshold and identifying spatially continuous features to obtain a first candidate abnormal region; Based on the first candidate abnormal area, calculating the spatial aggregation degree of the abnormal area and performing an expansion operation to obtain a second candidate abnormal temperature area; Acquire corresponding texture features from the second candidate abnormal temperature region, calculate the similarity between the texture features and a preset normal device surface texture, and if the similarity is less than a preset similarity threshold, determine it as a deformation region, thereby obtaining the device surface deformation region; Calculating a correlation coefficient between the deformation area on the device surface and the second candidate abnormal temperature area; if the correlation coefficient is greater than a preset correlation threshold, and the abnormal temperature area is greater than a preset lower area limit, determining that the device has a fault risk and outputting a fault detection result; The step of calculating the spatial aggregation degree of the abnormal region and performing an expansion operation based on the first candidate abnormal region to obtain the second candidate abnormal temperature region includes: Extracting temperature anomaly points from the first candidate anomaly region, performing spatial position analysis on the temperature anomaly points, and obtaining a regional cluster density distribution map; According to the regional cluster density distribution map, the density peak position is found and marked as a seed candidate point, and a set of seed candidate point coordinates is obtained; Performing an expansion operation based on the seed candidate point coordinate set to obtain an abnormal region growth result; Based on the abnormal region growth result, region tracking is performed to obtain the second candidate abnormal temperature region.

2. A fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The infrared image and visible light image of the surface of the high-temperature equipment are acquired and registered to obtain fused image data, including: Obtain infrared radiation information from the surface of high-temperature equipment; Perform image acquisition based on the infrared radiation information to obtain an infrared image and a visible light image; Correction and registration are performed based on the infrared image and the visible light image to obtain fused image data.

3. The fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The step of performing temperature field gradient analysis based on the fused image data to obtain temperature gradient distribution data includes: Extracting temperature information from the fused image data, performing a convolution operation, and obtaining first-order and second-order gradient distributions of the temperature field; According to the first-order and second-order gradient distributions, normalization processing and convolution operations are performed to obtain a multi-level temperature gradient representation; According to the multi-level temperature gradient, weighted fusion is performed to obtain temperature gradient distribution data.

4. The fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The performing of spatial continuity compensation and noise suppression on the temperature gradient distribution data, and determining that it is a valid temperature jump signal if the gradient amplitude exceeds a preset noise threshold, to obtain a first temperature jump signal, includes: Performing spatial continuity compensation according to the temperature gradient distribution data to obtain a gradient vector field distribution map; performing noise suppression processing according to the gradient vector field distribution map to obtain a noise-reduced gradient data set; Threshold detection and clustering processing are performed on the de-noised gradient data set to obtain a first temperature jump signal.

5. The fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The performing of regional scanning on the first temperature jump signal, calculating a segmentation threshold, and identifying spatially continuous features to obtain a first candidate abnormal region includes: Performing a window sliding scan on the first temperature jump signal, and calculating a segmentation threshold to obtain a segmentation threshold parameter; According to the segmentation threshold parameter, identifying the spatial continuous features of the temperature gradient, and obtaining a spatial continuous feature labeling map; Region growing is performed based on the spatial continuous feature labeling map to obtain a first candidate abnormal region.

6. The fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The step of obtaining a corresponding texture feature from the second candidate abnormal temperature region, calculating a similarity between the texture feature and a preset normal device surface texture, and determining the region as a deformed region if the similarity is less than a preset similarity threshold, and obtaining the device surface deformed region, includes: acquiring a corresponding visible light image region from the second candidate abnormal temperature region; Extracting texture features from the visible light image region to obtain regional texture features; The similarity between the texture feature of the region and a preset normal device surface texture is calculated. If the similarity is less than a preset similarity threshold, the region is determined to be a deformed region, thereby obtaining a device surface deformed region.

7. The fault detection method based on a high temperature resistant camera according to claim 1, characterized in that: The calculating of the correlation coefficient between the deformation area on the device surface and the second candidate abnormal temperature area, and if the correlation coefficient is greater than a preset correlation threshold and the abnormal temperature area is greater than a preset area lower limit, determining that the device has a failure risk, and outputting a failure detection result, includes: Calculating the spatial overlap between the deformation area on the device surface and the second candidate abnormal temperature area to obtain a correlation coefficient; Comparing the correlation coefficient with a preset correlation threshold, if the correlation coefficient is greater than the preset correlation threshold, it is determined that there is a causal relationship between the temperature anomaly and the deformation, and abnormal area boundary data is obtained; The total pixel area is extracted from the abnormal region boundary data. If the total pixel area is greater than a preset area lower limit, it is determined that there is a risk of equipment failure, and a fault detection result is output.

8. A fault detection device based on a high temperature resistant camera, characterized in that: The method for implementing the fault detection method based on a high-temperature resistant camera according to any one of claims 1 to 7 comprises: The data acquisition and registration module is used to acquire infrared images and visible light images of the surface of high-temperature equipment, and perform registration processing to obtain fused image data; A gradient analysis module, configured to perform temperature field gradient analysis based on the fused image data to obtain temperature gradient distribution data; a signal recognition module, configured to perform spatial continuity compensation and noise suppression on the temperature gradient distribution data, and determine that the data is a valid temperature jump signal if the gradient amplitude exceeds a preset noise threshold, thereby obtaining a first temperature jump signal; an abnormal region identification module, configured to perform regional scanning on the first temperature jump signal, calculate a segmentation threshold, and identify spatially continuous features to obtain a first candidate abnormal region; an abnormal region positioning module, configured to calculate the spatial aggregation degree of the abnormal region and perform an expansion operation based on the first candidate abnormal region to obtain a second candidate abnormal temperature region; a deformation region identification module, configured to obtain corresponding texture features from the second candidate abnormal temperature region, calculate the similarity between the texture features and a preset normal device surface texture, and determine the region as a deformation region if the similarity is less than a preset similarity threshold, thereby obtaining the device surface deformation region; A fault risk judgment module is used to calculate the correlation coefficient between the surface deformation area of ​​the equipment and the second candidate abnormal temperature area. If the correlation coefficient is greater than a preset association threshold and the abnormal temperature area is greater than a preset area lower limit, it is determined that the equipment has a fault risk and the fault detection result is output.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the fault detection method based on a high-temperature resistant camera according to any one of claims 1 to 7.

Images