Single-point anomaly detection method, system and readable medium based on infrared image of air blower

Abstract

The application relates to a kind of based on blower infrared image single-point anomaly detection method, system and readable medium, wherein method is by collecting the infrared video data of detected blower, from infrared video data, filter the frame image of preset quantity as image sequence, can obtain more comprehensive temperature information, more suitable for single-point anomaly detection;Using improved unsupervised anomaly detection network analyzes image sequence to obtain anomaly distribution map, anomaly distribution map can contain global temperature information, can effectively detect the case of abnormal high temperature and abnormal low temperature.Through the component region divided based on semantic segmentation network, the pixel of the anomaly distribution map is processed, and according to a plurality of single-point anomaly characteristics, the suspected point is finely detected, a plurality of different types of single-point anomaly characteristics and threshold segmentation are used, more information about image and target is captured, not just the difference in brightness or color, thereby improving the detection performance of complex scenes and objects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of anomaly detection, in particular to a single-point anomaly detection method based on an infrared image of a blower, a system and a readable medium. BACKGROUND

[0002] A high-speed direct-drive blower is a high-efficiency, energy-saving and environmentally friendly blower, which is widely used in pharmaceutical, power, chemical, waste treatment and other industries. The most prominent feature of single-point anomaly is that it produces obvious temperature anomaly points on certain areas of the infrared image. These anomaly points are usually small in area and do not cause significant changes to the overall temperature, but may have cases of broken volute, broken cooling pipe or foreign matter blockage, which can also cause serious consequences if not discovered in time.

[0003] In industry, thermocouples are often used to directly measure the surface temperature of a machine. This method can obtain accurate temperature values, but the thermocouple device is expensive and can only obtain the temperature of a certain point, which is not suitable for detecting single-point anomalies that may occur anywhere. The infrared image obtained by the infrared imager can capture the complete temperature information of the object surface, which is the best choice of data for detecting single-point anomalies.

[0004] Common methods of using infrared images for anomaly detection usually combine deep learning and image processing techniques. However, most of them detect single images by detecting abnormal high-temperature areas, which cannot detect abnormal low-temperature areas.

[0005] In the prior art, there is also a method of using SimpleBlobDetector to detect the anomaly points of the blower. SimpleBlobDetector detects the bright or colored spots on the blower that have a significant difference from the surrounding area. Once a spot is detected, the position, size, brightness and other features of the spot can be extracted. These features can be used to locate related components or defects on the blower. However, the performance of SimpleBlobDetector is usually limited to cases where the brightness difference is large and the spots are obvious. It can only detect simple spots, and when the detection task involves more complex targets, such as objects with different textures, shapes or structures, it cannot handle them. SUMMARY

[0006] Therefore, it is necessary to provide a single-point anomaly detection method based on an infrared image of a blower, a system and a readable medium to solve the problem that the existing SimpleBlobDetector method cannot detect complex targets.

[0007] The present application provides a single-point anomaly detection method based on an infrared image of a blower, which comprises:

[0008] acquire infrared video data of the detected blower, and screen a preset number of frame images from the infrared video data as an image sequence;

[0009] input the image sequence into an improved unsupervised anomaly detection network, run the improved unsupervised anomaly detection network, and obtain an anomaly distribution map corresponding to each frame image in the image sequence output by the improved unsupervised anomaly detection network;

[0010] perform pixel processing on the anomaly distribution map based on the component region divided by the semantic segmentation network, and obtain a coarse detection result containing suspected points;

[0011] perform fine detection of single-point anomaly on the suspected points according to a plurality of single-point anomaly features, and obtain a fine detection result.

[0012] The application also provides a blower infrared image-based single-point anomaly detection system, comprising:

[0013] an infrared acquisition device configured to acquire infrared video data of the detected blower;

[0014] a data processing device configured to execute the blower infrared image-based single-point anomaly detection method as described above.

[0015] The application also provides a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the blower infrared image-based single-point anomaly detection method as described above.

[0016] The application relates to a blower infrared image-based single-point anomaly detection method, system and readable medium, wherein the method can acquire more comprehensive temperature information and is more suitable for single-point anomaly detection by acquiring infrared video data of the detected blower, screening a preset number of frame images from the infrared video data as an image sequence, using an improved unsupervised anomaly detection network to analyze the image sequence to obtain an anomaly distribution map, and the anomaly distribution map can contain global temperature information and can effectively detect abnormal high temperature and abnormal low temperature. In addition, by performing pixel processing on the anomaly distribution map based on the component region divided by the semantic segmentation network and performing fine detection of single-point anomaly on the suspected points according to a plurality of single-point anomaly features, a plurality of different types of single-point anomaly features and threshold segmentation are used, more information about the image and the target can be captured, not just the difference in brightness or color, thereby improving the detection performance of complex scenes and objects. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of a blower infrared image-based single-point anomaly detection method provided by an embodiment of the application is shown.

[0018] Figure 2 A live photo when acquiring infrared video data of the detected blower.

[0019] Figure 3 An infrared original image of a frame image in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0020] Figure 4 An abnormality distribution diagram corresponding to a frame image in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0021] Figure 5 A coarse detection result diagram corresponding to an infrared original image of a frame image in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0022] Figure 6 A variance change diagram of an abnormality block and a non-abnormality block in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0023] Figure 7 A distance diagram of different block center pixels and four direction edges in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0024] Figure 8 A gradient distribution diagram of different blocks in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0025] Figure 9 A diagram of each quadrant and main direction in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application.

[0026] Figure 10 A comparison diagram of SimpleBlobDetector, a coarse detection result and a fine detection result in a single-point abnormality detection method based on infrared images of a blower provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0028] The present application provides a single-point abnormality detection method based on infrared images of a blower.

[0029] As shown in the drawings, Figure 1 in an embodiment of the present application, the single-point abnormality detection method based on infrared images of a blower includes S100 to S400 as follows.

[0030] S100, acquire infrared video data of the detected blower, and screen a preset number of frame images from the infrared video data as an image sequence.

[0031] S200, input the image sequence into an improved unsupervised anomaly detection network, run the improved unsupervised anomaly detection network, and obtain an abnormal distribution map corresponding to each frame image in the image sequence output by the improved unsupervised anomaly detection network.

[0032] Specifically, using the unsupervised anomaly detection network can solve the problems of lack of abnormal data and no label, and can learn the semantic information in the image space and between sequences to remove the interference caused by normal dark spots, bright spots and slight changes in viewing angle. However, in the original unsupervised anomaly detection network Meta Prototype Network (MPN) proposed by Lv et al., the output is a single abnormal value.

[0033] In order to obtain the abnormal situation of each pixel and the position information of the abnormality, first, the output of the original unsupervised anomaly detection network is improved, so that the output result changes from the original single abnormal score to a two-dimensional abnormal distribution map. The abnormal feature map obtained for each image is directly normalized to solve the problem of variable numerical range. Then, the pixel value is mapped to the range of 0-255, and the result is output and stored in the form of a gray image as an abnormal distribution map, as shown in Figure 4 Fig. 1 shows the infrared original image corresponding to the abnormal distribution map. Figure 3 Fig. 1 shows the infrared original image corresponding to the abnormal distribution map.

[0034] S300, based on the component region divided by the semantic segmentation network, performing pixel processing on the abnormal distribution map to obtain a coarse detection result containing suspected points.

[0035] When using the unsupervised anomaly detection network for detection, each input frame image will obtain a corresponding abnormal distribution map. In the abnormal distribution map, the larger the pixel value, the greater the possibility of abnormal points at that pixel position. However, in the abnormal distribution map obtained by the unsupervised anomaly detection network, it is difficult to obtain a highly generalized pattern due to the non-fixed background, resulting in that the abnormal score of the background part is usually large. In order to solve this problem, it is necessary to combine the semantic segmentation network for subsequent processing according to the component region to remove interference. The segmentation network can be any semantic segmentation network, which can be used after being trained using the blower data.

[0036] To remove the occasional cases in single frame image, the application verifies the abnormal results of continuous multiple images. If the abnormal value at a certain position is large and repeatedly appears, that is, the mean value of the pixels of N images at this position is greater than a certain threshold, it is determined that there is a high possibility of abnormality at this position, the position is recorded and the pixel value at this position is set to 1, otherwise it is set to 0, and the result is as follows Figure 5 The same processing is performed on all image sequences, and a binary coarse detection result image can be obtained for each image. The calculation process of a certain position is as follows:

[0037]

[0038] In formula a, represents the pixel value of a certain position in the abnormal distribution map, and φ dPrel represents a threshold value, and N represents the number of frames verified continuously.

[0039] S400, according to a plurality of single-point abnormality features, fine detection of single-point abnormality of the suspected point is performed to obtain a fine detection result.

[0040] Specifically, the single-point abnormality features include one or more of local variance change trend features, local edge four-direction distance features, and local gradient field features. The suspected points verified by one single-point abnormality feature or simultaneously verified by multiple single-point abnormality features are regarded as abnormal points.

[0041] In this embodiment, by collecting infrared video data of the detected blower, a preset number of frame images are selected from the infrared video data as an image sequence, more comprehensive temperature information can be obtained, which is more suitable for single-point abnormality detection; the improved unsupervised anomaly detection network is used to analyze the image sequence to obtain an abnormal distribution map. The abnormal distribution map can contain global temperature information, and can effectively detect abnormal high temperature and abnormal low temperature conditions. In addition, by processing the pixels of the abnormal distribution map based on the component regions divided by the semantic segmentation network, and by fine detection of the suspected points according to a plurality of single-point abnormality features, a plurality of different types of single-point abnormality features and threshold segmentation are used, which can capture more information about the image and the target, not just the difference in brightness or color, thereby improving the detection performance of complex scenes and objects.

[0042] In an embodiment of the present application, the training method of the improved unsupervised anomaly detection network comprises:

[0043] 1a) According to the preset angle, distance and position, normal blower infrared video data and simulated single-point abnormal blower infrared video data are respectively shot.

[0044] Specifically, the infrared video data of the blower fan is captured by a DM10 infrared thermal imager from Dali Science and Technology. The infrared temperature measurement range is -20°C to +150°C, the temperature measurement accuracy is ±2°C, the infrared resolution of a single frame is 1024x768 pixels, the video frame rate is 20 frames per second, the video code rate is 8192 kb / s, the encoding format is.H264, the video content is a grayscale infrared image without pseudo-color processing, and the final storage format is.Mp4.

[0045] During measurement, the height of the infrared thermal imager is flush with the center of the fan, the horizontal distance is 0.6 meters or 1 meter, and the measurement is performed at an angle of 45° to 135° in the direction of the fan (the direction of the blower fan inlet) and an angle of 60° to 135° in the opposite direction (the direction of the blower fan motor). The rotation change is 15° for each measurement, and the video length is 10 to 40 minutes. The initial rotation speed of the fan is 6000 r / min, and the rotation speed of the fan is adjusted to simulate temperature changes. The shooting site is shown in Figure 2

[0046] 2a) Select appropriate partial frame images from normal blower fan infrared video data to make normal image sequences as the training set of the improved unsupervised anomaly detection network.

[0047] Specifically, appropriate partial frame images are selected from the video data and arranged in order to obtain image sequences. In this algorithm, the next image is predicted based on the first five images, so each image sequence contains at least five frame images. In addition, the improved unsupervised anomaly detection network can use a separate prototype to represent similar data, but when the angle or position difference is too large, the prediction error will also increase.

[0048] To improve accuracy, the training data needs to be as diverse as possible, including different angles, distances, and positions. Therefore, for data on the same side of the blower fan, at least three different angles and two different distances of normal data at different angles are selected to enable the improved unsupervised anomaly detection network to extract more comprehensive feature representations from the data. After careful screening, the training set used to train the improved unsupervised anomaly detection network finally contains 4931 images.

[0049] 3a) Similarly, the blower fan infrared video data simulating single-point anomalies is made into abnormal image sequences as the test set of the improved unsupervised anomaly detection network.

[0050] Specifically, the same operations are performed on the blower fan infrared video data simulating single-point anomalies as described above, and the test set contains a total of 1658 images. Each image input into the network has a size of 512x384 pixels. The abnormal points are manually segmented and binary labeled as ground truth (true data).

[0051] ​4a) training and testing the improved unsupervised anomaly detection network using the training set and the validation set.

[0052] In this embodiment, first, the infrared video data of the blower containing normal and simulated single-point anomalies is shot using an infrared thermal imager, then suitable parts are selected from the normal data to make image sequences as the training set of the improved unsupervised anomaly detection network, and similarly, the simulated anomaly data is made into image sequences as the test set, realizing the construction of the single-point anomaly detection data set.

[0053] The three single-point anomaly features of the present application are designed according to the pixel features of the single-point anomaly neighborhood, combined with the idea of small target detection, to further screen the coarse detection results, obtain fine detection results and segment the anomaly region.

[0054] Specifically, since the volume of the blower is usually large, the local anomaly point occupies a very small proportion of pixels on the complete infrared image captured by the infrared camera. In the large target detection data set COCO, objects with an area less than 1024 pixels, i.e. 32x32 pixels, are usually called small targets. The data used in the present application for anomaly point detection is 512x384 pixels, wherein the area of the single-point anomaly region is about 100-700 pixels, accounting for 0.05%-0.36% of the whole image, which can be considered as a small target problem. Therefore, three single-point anomaly features are designed in combination with small target detection to further analyze the pixels in the neighborhood of these suspected points.

[0055] The idea of each single-point anomaly feature is specifically introduced below.

[0056] 1b) Local variance trend

[0057] The variance in the image can be used to measure the degree of change of the pixel value, and therefore can be used to describe the texture information or contrast of the image. The greater the variance, the greater the degree of change of the pixel value, and the more details and texture information the image contains. Conversely, the smaller the variance, the smaller the degree of change of the pixel value, and the smoother the image.

[0058] In the infrared image of the blower, the single-point anomaly usually appears as a dark or bright spot, and the pixel value on the anomaly point has a relatively obvious difference with the pixel value of the surrounding area, so the variance of the nearby area is large.

[0059] The center pixels of the anomaly and non-anomaly regions in the infrared image of the blower are taken, a neighborhood patch with a margin of r is taken with the center pixel as the center, and the pixel variance is calculated. The value of r is increased until a set maximum value is reached. A series of variance values are obtained during the increase of the margin. When the average size of the variances is similar, the change degree near the anomaly point x is more violent, and in the tail, it presents a gradually stable trend. The change near the non-anomaly point is more gentle.

[0060] Neighborhood graph and corresponding variance values ​​are as follows Figure 6 As shown, the corresponding order of the neighborhood graph is from right to left in the first row and from left to right in the second row, and the corresponding order of the variance values ​​is from left to right.

[0061] 2b) Local edge four-way distance

[0062] Edge detection is a commonly used image feature extraction method. Due to the significant difference in pixel values ​​between a single outlier region and its surrounding area, a basically closed edge can be obtained around it. By analyzing the distances from the center pixel to the edges in four directions, it can be determined whether the center pixel is inside the outlier.

[0063] Furthermore, the distance obtained can be used to estimate the size of outliers, guiding the size of subsequent patches and yielding more refined results during threshold segmentation.

[0064] To better adapt to data in industrial scenarios, this application uses the Canny operator to calculate edge information within the tile, and then studies the distances of the center pixel to the nearest boundary in the 0°, 90°, 180°, and 270° directions. It can be observed that when the center pixel is inside an anomaly, due to the presence of clear edge lines around it, the distances from the center pixel to the four directions are small and relatively close. However, when the center pixel is not inside an anomaly, edge information is absent in more than one direction.

[0065] like Figure 7 As shown, patches a and b are patches without single-point anomalies, while patch c is a patch with single-point anomalies. Therefore, the four-way distance of the local edge is used as one of the features for judgment.

[0066] 3b) Local gradient field characteristics

[0067] Gradient field features are one of the commonly used features in small object detection, and different selection formulas can be designed according to requirements. The gradient field of an image presents the gradient vector of each pixel, reflecting the direction and degree of the fastest change in pixel value, and can effectively describe different texture and shape features in the image.

[0068] Near a single anomaly, the pixel value change rate is highest at the edge, the gradient magnitude is largest and the direction is perpendicular to the edge. As the distance from the edge increases, the gradient gradually decreases.

[0069] Calculate the gradient field using patches with and without outliers. For example... Figure 8As shown, for the abnormal tile a, the gradient amplitude is large, and the direction is roughly directed to the target center area, showing a radial distribution trend. For the background tile b without abnormality and relatively flat, the gradient amplitude is small, and the direction has no obvious rule. In the tile c without abnormality but containing sharp edges, the gradient amplitude is also large, but the direction is relatively uniform.

[0070] By analyzing the gradient field form of the tile, the correct abnormal point can be further screened out, and therefore the gradient field form is adopted as one of the characteristics for judging the abnormal point.

[0071] In an embodiment of the present application, the S400 comprises the following S410 to S440.

[0072] S410, screening the suspected points in the coarse detection result by using the local variance change trend feature.

[0073] S420, for the suspected points screened by the local variance change trend feature, further screening is performed by using the local edge four-direction distance feature.

[0074] S430, for the suspected points screened by the local edge four-direction distance feature, further screening is performed by using the local gradient field feature to obtain a screening result.

[0075] S440, performing threshold segmentation on all screening results to obtain a fine detection result.

[0076] In the embodiment, the suspected points in the coarse detection result are screened by using the local variance change trend feature, the local edge four-direction distance feature and the local gradient field feature in sequence, and the pixel points screened by the three abnormal features are considered as abnormal points to obtain the final fine detection result.

[0077] In an embodiment of the present application, the S410 comprises the following S411 to S412.

[0078] S411, judging whether the suspected points in the coarse detection result satisfy the conditions defined by the following formulas 2 to 4 at the same time:

[0079]

[0080]

[0081]

[0082] Wherein, N represents a neighborhood tile with the suspected point as the center and the edge distance r, and in the process of continuously increasing the value of r until reaching the set maximum value, the total number of a series of variance values obtained.v idenotes the i-th variance value, V denotes a variance sequence composed of all variance values, max(V) denotes the maximum value in the variance sequence, min(V) denotes the minimum value in the variance sequence, φ var denotes the increment threshold, max(V a:b ) denotes the maximum value of all variance values between the a-th number and the b-th number in the variance sequence, min(V a:b ) denotes the minimum value of all variance values between the a-th number and the b-th number in the variance sequence.

[0083] Specifically, formula 2 indicates that the variance as a whole tends to increase in the process of increasing the neighborhood, formula 3 indicates that the increment value of the variance needs to reach a certain degree, and formula 4 indicates that in the process of variance change, the increment rate of the front part is higher than that of the rear part, that is, the rear part of the data tends to be stable compared with the beginning stage.

[0084] S412, if the suspected point in the coarse detection result meets the conditions defined in formulas 2 to 4 at the same time, it is considered to pass the screening of the local variance change trend feature.

[0085] In an embodiment of the present application, the S420 includes the following S421 to S422.

[0086] S421, it is judged whether the suspected point passing the screening of the local variance change trend feature meets the conditions defined in the following formulas 5 to 7 at the same time:

[0087] D = Dis(E) = Dis(Canny(I ROI )) Formula 5

[0088] num(d i > 0) ≥ 3, i = 1, 2, 3, 4 Formula 6

[0089]

[0090] Wherein, I ROI denotes the obtained tile containing the suspected point, E denotes a binary edge map calculated using a Canny operator, Dis denotes the calculation of four-direction distance, D is the set of four-direction distances obtained, num(·) denotes the number of data satisfying the condition ·, d i represents the distance from a certain direction to the nearest boundary, r apx is the estimated radius of the suspected point, μ is a compensation value for fine-tuning the radius, and μ is generally taken as 1 to 3.

[0091] Specifically, formula 5 is used to calculate the Canny edge and the four-direction distance, and in the present application, if there is no edge in a certain direction or the center is located on the edge, both are assigned a value of 0. Formula 6 is a judgment condition, and formula 7 is used to estimate the radius of the abnormal point.

[0092] S422, if the suspected points screened by the local variance change trend feature simultaneously meet the conditions defined by formulas 5 to 7, they are considered to be screened by the local edge four-way distance feature.

[0093] In one embodiment of this application, S430 includes the following S431 to S434.

[0094] S431, perform reverse processing on the block to be judged according to the following formula 8:

[0095]

[0096] Among them, I ROI This represents the original image patch before reverse processing. GP represents the maximum grayscale level, which is typically set to 2. bit -1 represents the bit depth of each pixel. This represents the tile after reverse processing.

[0097] Since the gradient directions are different in bright spots and dark spots, the gradient directions are unified through the reverse processing described above.

[0098] S432, For the original tile I before reverse processing ROI And the corresponding reverse-processed tiles Simultaneously calculate the magnitude and direction of the gradient:

[0099]

[0100]

[0101] Among them, g x G represents the gradient in the horizontal direction. y θ represents the gradient in the vertical direction, g represents the magnitude of the gradient, and θ represents the direction angle of the gradient.

[0102] like Figure 9 As shown, the gradient direction angle gradually increases from 0 to 360° in the order of first quadrant Q1, second quadrant Q2, third quadrant Q3 to fourth quadrant Q4.

[0103] S433, determine whether the magnitude and direction of the gradient are uniformly distributed in several preset directions.

[0104] Specifically, this application determines whether the gradient field distribution meets the conditions by setting eight main directions and then calculating whether the distribution of gradient magnitude and direction is uniform.

[0105] S434, if the magnitude and direction of the gradient are uniformly distributed in the preset direction, it is considered to have passed the screening of local gradient field features, and the suspected points in the map are regarded as abnormal points.

[0106] In an embodiment of the present application, the S433 comprises the following S433a to S433e.

[0107] S433a, counting the number of gradient vectors and the average amplitude within the estimated radius r of the suspected point apx Resizing the tile.

[0108] S433b, counting the number of gradient vectors and the average amplitude within the effective angle of each quadrant, setting a main direction range and two secondary direction ranges in each quadrant, and determining the main direction and secondary direction according to the set weight.

[0109] Specifically, the weight in the main direction is set to 0.5, and the weight in the secondary direction is set to 0.25.

[0110] S433c, counting the gradient direction score according to formula 11, and counting the amplitude score according to formula 12:

[0111]

[0112]

[0113] wherein, represents the gradient direction score in the kth quadrant, represents the gradient amplitude score in the kth quadrant, represents the number of gradient directions contained in the specified direction range of the kth quadrant, represents the average gradient amplitude contained in the specified direction range of the kth quadrant.

[0114] For example, as shown in Figure 7 , taking the first quadrant Q1 as an example, the total direction range is 0 to 90°, the main direction range is 20° to 70° (i.e. the yellow area), the left secondary direction range is 0 to 20°, and the right secondary direction range is 70° to 90°.

[0115] S433d, determining whether the gradient direction score and the gradient amplitude score satisfy the conditions defined by the following formula 13 to formula 16 at the same time:

[0116] min(S dir )>0 formula 13

[0117]

[0118]

[0119]

[0120] wherein, φ SD , φ dir , φval respectively, are adjustable thresholds.

[0121] Specifically, formula 13 and formula 14 ensure that the gradient direction is uniformly distributed in the four quadrants, and formula 15 and formula 16 are used to distinguish the background.

[0122] S433e, if the gradient direction score and the gradient magnitude score simultaneously satisfy the conditions defined by formula 13 to formula 16, the magnitude and direction of the gradient are uniformly distributed.

[0123] The following tests are performed on all test data, and the practical indicators are the average intersection over union MIoU, the center point distance DotD, the recognition rate RR, and the false alarm rate FA.

[0124] The average intersection over union MIoU is calculated according to formula 17:

[0125]

[0126] where M is the number of classes, TP i is the number of true positives of the i-th class, FN i is the number of false negatives of the i-th class, FP i is the number of false positives of the i-th class.

[0127] The center point distance DotD is calculated according to formula 18:

[0128]

[0129] In formula 18, ED is calculated according to formula 19, is calculated according to formula 20:

[0130]

[0131]

[0132] where (x A ,y B ) and (x B ,y B ) represent the coordinates of the center points of two adjacent frames of images in the image sequence, M represents the number of images, N i represents the number of labeled bounding boxes in the i-th image, w ij represents the width of the j-th bounding box in the i-th image, h ij represents the height of the j-th bounding box in the i-th image.

[0133] The recognition rate RR is calculated according to formula 21:

[0134]

[0135] where S T represents the number of correctly detected images, S ALL represents the number of all frame images in the image sequence. The false alarm rate FA is calculated according to formula 22:

[0136]

[0137] where FP represents the number of false positive samples, and TN represents the number of true negative samples.

[0138] The detection results are compared with the SimpleBlobDetector method in the python toolkit, the coarse detection results and the fine detection results Figure 10 The specific average intersection over union MIoU, center point distance DotD, recognition rate RR and false alarm rate FA calculation results are shown in Table 1.

[0139] Table 1-Comparison of SimpleBlobDetector, coarse detection results and fine detection results

[0140] Detection method MIoU↑ DotD↑ RR↑ FA↓ SimpleBlobDetector 0.2550 0.2325 0.5212 0.6576 Coarse detection 0.3697 0.5697 0.6780 0.7636 Fine detection 0.6417 0.9187 0.9403 0.0154

[0141] Since SimpleBlobDetector only considers morphological information, and the coarse detection results only consider semantic information, the detection accuracy is low, and the false alarm rate is high, more than 0.65, and the real anomaly cannot be well detected.

[0142] The fine detection results consider both semantic information and morphological information, and the MIoU, DotD and RR reach 0.6417, 0.9187 and 0.9403, which are improved by 0.4883 and 0.2854 compared with the results of SimpleBlobDetector and coarse detection, and the false alarm rate is less than 0.02.

[0143] Experiments prove that the fine detection can accurately detect single-point anomalies contained in the image, remove more false detection results, and the accuracy of positioning and area estimation is also high. And the average running time of the algorithm for detecting a single image is 0.4040 seconds, which has certain real-time performance and certain practicality in industrial application scenarios.

[0144] The application also discloses a single-point anomaly detection system based on an air blower infrared image.

[0145] In an embodiment of the application, the single-point anomaly detection system based on the air blower infrared image comprises an infrared acquisition device and a data processing device.

[0146] Specifically, the infrared acquisition device is configured to acquire infrared video data of the detected blower. Optionally, the infrared acquisition device is an infrared thermal imager. The processing device is configured to perform the single-point abnormality detection method based on infrared images of a blower as described above.

[0147] The application also discloses a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the single-point abnormality detection method based on infrared images of a blower as described above.

[0148] The technical features of the above embodiments can be combined in any manner, and the execution sequence of the method steps is not limited. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the description.

[0149] The above embodiments only express several implementation manners of the application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which all belong to the protection scope of the application. Therefore, the protection scope of the application should be subject to the appended claims.

Claims

1. A method for single-point anomaly detection based on blower infrared image, characterized in that, The single-point anomaly detection method based on the blower infrared image comprises the following steps: Obtaining infrared video data of a blower to be detected, and screening a preset number of frame images from the infrared video data as an image sequence; Inputting the image sequence into an improved unsupervised anomaly detection network, running the improved unsupervised anomaly detection network, and obtaining an abnormal distribution map corresponding to each frame image in the image sequence output by the improved unsupervised anomaly detection network; Performing pixel processing on the abnormal distribution map based on a component region divided by a semantic segmentation network to obtain a coarse detection result containing suspected points; Verifying abnormal results of a plurality of continuous images, if an abnormal value at a certain position is large and repeatedly appears, that is, a pixel mean value of N images at the position is greater than a certain threshold, it is determined that there is a high possibility of abnormality at the position, the position is recorded, and the pixel value at the position is set to 1, otherwise, it is set to 0, and the same processing is performed on all image sequences, and a binary coarse detection result image can be obtained for each image; Performing fine detection of single-point anomaly on the suspected points according to a plurality of single-point anomaly features to obtain a fine detection result.

2. The single-point anomaly detection method based on the infrared image of the air blower according to claim 1, characterized in that, The training method of the improved unsupervised anomaly detection network comprises the following steps: According to a preset angle, distance and position, a normal blower infrared video data and a blower infrared video data simulating single-point anomaly are respectively shot; Selecting appropriate frame images from the normal blower infrared video data to make a normal image sequence as a training set of the improved unsupervised anomaly detection network; Similarly, the blower infrared video data simulating single-point anomaly is made into an abnormal image sequence as a test set of the improved unsupervised anomaly detection network; Training and testing the improved unsupervised anomaly detection network by using the training set and the test set.

3. The single-point anomaly detection method based on the infrared image of the air blower according to claim 1, characterized in that, The single-point anomaly features include one or more of a local variance change trend feature, a local edge four-direction distance feature and a local gradient field feature.

4. The method of claim 3, wherein the method is based on a single point anomaly detection of an infrared image of a blast furnace. The fine detection of single-point anomaly on the suspected points according to a plurality of single-point anomaly features to obtain a fine detection result comprises the following steps: Screening the suspected points in the coarse detection result by using the local variance change trend feature; Further screening the suspected points screened by the local variance change trend feature by using the local edge four-direction distance feature; Further screening the suspected points screened by the local edge four-direction distance feature by using the local gradient field feature to obtain a screening result; Threshold segmentation is performed on all screening results to obtain a fine detection result.

5. The method of claim 4, wherein the method is based on a single point anomaly detection of an infrared image of a blast furnace. The screening of the suspected points in the coarse detection result by using the local variance change trend feature comprises the following steps: Judging whether the suspected points in the coarse detection result satisfy the conditions defined in the following formulas 2 to 4 at the same time: Formula 2; Formula 3; Formula 4; N represents a neighborhood block centered at the suspect point with a margin of r, and a total number of a series of variance values is obtained in a process of continuously increasing the value of r until a set maximum value is reached; represents the i-th variance value, represents a variance sequence composed of all variance values, represents a maximum value in the variance sequence, represents a minimum value in the variance sequence, represents an increment threshold value, represents a maximum value of all variance values between the a-th number and the b-th number in the variance sequence, represents a minimum value of all variance values between the a-th number and the b-th number in the variance sequence. If the suspected points in the coarse detection result satisfy the conditions defined in the formulas 2 to 4 at the same time, it is considered that the screening by the local variance change trend feature is passed.

6. The single-point anomaly detection method based on the infrared image of the air blower according to claim 5, characterized in that, The further screening of the suspected points screened by the local variance change trend feature by using the local edge four-direction distance feature comprises the following steps: Judging whether the suspected points screened by the local variance change trend feature satisfy the conditions defined in the following formulas 5 to 7 at the same time: Formula 5: Formula 6; Formula 7; wherein, represents the obtained patch containing suspected points, represents the binary edge map calculated using Canny operator, represents the calculation of four-directional distance, D is the obtained set of four-directional distances, represents the number of data satisfying the condition , represents the distance from the nearest boundary in a certain direction, is the estimated radius of suspected points, is the compensation value, used to fine-tune the radius; If the suspected point screened by the local variance change trend feature satisfies the conditions defined by the formulas 5 to 7, it is considered to pass the screening of the local edge four-direction distance feature.

7. The single-point anomaly detection method based on the infrared image of the air blower according to claim 6, characterized in that, The suspected point screened by the local edge four-direction distance feature is further screened using the local gradient field feature to obtain a screening result including: The block containing the suspected point is reversely processed according to the following formula 8: Formula 8; wherein, represents the original patch before the inverse processing, represents the maximum value of the number of gray levels, represents the patch after the inverse processing; original patch before inverse processing and the patch after corresponding inverse processing simultaneously compute the magnitude of the gradient and the direction of the gradient: Formula 9; Formula 10; wherein, denotes the gradient in the horizontal direction, denotes the gradient in the vertical direction, denotes the magnitude of the gradient, denotes the direction angle of the gradient; Determine whether the amplitude and direction of the gradient are uniformly distributed in a plurality of preset directions; If the amplitude and direction of the gradient are uniformly distributed in the preset directions, it is considered to pass the screening of the local gradient field feature, and the suspected point in the block is considered to be an abnormal point.

8. The single-point anomaly detection method based on the infrared image of the air blower according to claim 7, characterized in that, The determination of whether the amplitude and direction of the gradient are uniformly distributed in a plurality of preset directions includes: According to the estimated radius of the suspect point Re-cropping the tile; Count the number and average amplitude of the gradient vectors in each quadrant effective angle, set a main direction range and two secondary direction ranges in each quadrant, and determine the main direction and secondary direction according to the set weights; According to the formula 11, count the gradient direction score, and according to the formula 12, count the amplitude score: Formula 11; Formula 12; in, Indicates the first Gradient direction fractions in each quadrant Indicates the first Gradient magnitude fractions in each quadrant Indicates included in the Each quadrant specifies the directional range The number of gradient directions within the space, Indicates included in the Each quadrant specifies the directional range The average gradient magnitude within the range; Determine whether the gradient direction score and the gradient amplitude score satisfy the conditions defined by the following formulas 13 to 16: Formula 13; Formula 14; Formula 15; Formula 16; wherein are adjustable thresholds, respectively. If the gradient direction score and the gradient amplitude score satisfy the conditions defined by the formulas 13 to 16, the amplitude and direction of the gradient are uniformly distributed.

9. A single-point anomaly detection system based on blower infrared images, characterized by, Including: An infrared acquisition device for acquiring infrared video data of the detected blower; A data processing device for executing the single-point anomaly detection method based on the infrared image of the blower according to any one of claims 1 to 8.

10. A computer readable medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the single-point anomaly detection method based on the infrared image of the blower according to any one of claims 1 to 8.

Images