A Method and System for Monitoring Heat Sources of Blowers Based on Dual-Photon Fusion and Segmentation

Abstract

This application relates to a method for monitoring the heat source of a blower based on dual-light fusion and segmentation. Infrared and visible light images are acquired using a multimodal sensor. A processor preprocesses the infrared and visible light images respectively. The preprocessed infrared and visible light images are then fused and segmented to obtain fused and segmented images. The mean and variance of the infrared image region corresponding to each labeled region in the segmented image are calculated. Thresholds for the mean and variance are designed, and the number of labels is optimized based on these thresholds. This method pre-plans the number of labels to be used, solving the problem that the label requirements for the segmented blower section always fluctuate in real time and are difficult to determine.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for monitoring the heat source of a blower based on dual-light fusion and segmentation. Background Technology

[0002] Dual-light image fusion refers to an image processing technique that extracts information from images of the same target acquired through visible light and infrared channels using certain rules or methods, resulting in enhanced image quality. Based on multimodal sensors, both infrared and visible light images of the same target object can be obtained. Visible light images, with their high spatial resolution, provide rich information about the object's appearance and detailed textures. However, they cannot accurately reflect the target object in environments with insufficient lighting or fog. In contrast, infrared images, which image the thermal radiation emitted by the object, provide temperature information while being less affected by changes in lighting or artifacts, overcoming the obstacles of nighttime target detection. However, the spatial resolution of infrared images is generally lower than that of visible light images. Therefore, the complementarity of infrared and visible light images allows them to present almost all the inherent characteristics of an object, meaning that the fusion of infrared and visible light images provides a richer and more complete amount of information compared to a single-modal image.

[0003] Regarding the fusion of infrared and visible light images, some traditional fusion algorithms include multi-scale transformation-based methods and subspace-based methods. Multi-scale transformation-based methods decompose the infrared and visible light images into a series of multi-scale representations, then fuse each pair of representations according to a given fusion rule. Finally, a corresponding multi-scale inverse transformation is performed on the fused image to obtain the final fused image. Subspace-based methods aim to project high-dimensional images onto low-dimensional spaces or subspaces using principal component analysis and independent component analysis. However, the fusion rules of these methods are often too simple, resulting in limited fusion performance and the presence of artifacts in the results.

[0004] On the other hand, due to the advantages of neural networks in image feature extraction, many deep learning fusion algorithms have been proposed, but they usually assume that the source image is noise-free, which is obviously unrealistic in actual imaging.

[0005] On the other hand, commonly used algorithms that specify the number of image segmentation labels cannot adaptively detect heat source temperature labels that match the temperature differences of the blower's heat source. They often over-divide the image into multiple heat source temperature labels or divide it into too few labels, increasing the false negative rate. Therefore, traditional blower monitoring methods struggle to adaptively adjust labels during segmentation, making it difficult to determine the appropriate number of layered labels for abnormal blower heat source warnings. Summary of the Invention

[0006] Therefore, it is necessary to provide a blower heat source monitoring method and system based on dual-light fusion and segmentation to address the problems of traditional blower monitoring methods, such as the difficulty in adaptively adjusting labels in the segmented part and the difficulty in determining the number of labels in the image layer for early warning of blower heat source anomalies.

[0007] This application provides a method for monitoring the heat source of a blower based on dual-light fusion and segmentation, including:

[0008] Obtain infrared and visible light images of the target object in the blower, and perform preprocessing;

[0009] Set the initial value for the total number of tags K;

[0010] Based on K labels, the preprocessed infrared image and the preprocessed visible light image are fused and segmented to obtain a segmented image;

[0011] Calculate the mean and variance of the infrared image region corresponding to each labeled region in the segmented image;

[0012] Set thresholds for the mean value and variance of infrared image tags;

[0013] Based on the infrared image label mean threshold and the infrared image label variance threshold, the total number of labels K is continuously adjusted so that the mean of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, or the variance of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0014] This application relates to a method for monitoring the heat source of a blower based on dual-light fusion and segmentation. It acquires infrared and visible light images using a multimodal sensor, preprocesses both images separately, and then fuses and segments them to obtain fused and segmented images. The mean and variance of the infrared image region corresponding to each labeled region in the segmented image are calculated. Thresholds for the mean and variance are designed, and the number of labels is optimized based on these thresholds. This method pre-plans the number of labels to be used, solving the problem that the label requirements for the segmented blower section always fluctuate in real time and are difficult to determine. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a method for monitoring the heat source of a blower based on dual-light fusion and segmentation, provided in an embodiment of this application.

[0016] Figure 2 Visible image in a blower heat source monitoring method based on dual-light fusion and segmentation provided in an embodiment of this application.

[0017] Figure 3 Infrared image in a blower heat source monitoring method based on dual-light fusion and segmentation provided in an embodiment of this application.

[0018] Figure 4 In a blower heat source monitoring method based on dual-light fusion and segmentation provided in one embodiment of this application, when K=8, the following will be used: Figure 2 and Figure 3 The segmented image obtained after fusion.

[0019] Figure 5 In a blower heat source monitoring method based on dual-light fusion and segmentation provided in one embodiment of this application, when K=4, the following will be used: Figure 2 and Figure 3 The segmented image obtained after fusion.

[0020] Figure 6 This is a schematic diagram showing the connection between a multimodal sensor and a processor in a blower heat source monitoring method based on dual-light fusion and segmentation, provided in an embodiment of this application.

[0021] Figure label:

[0022] 100 - Multimode sensor; 110 - Infrared camera; 120 - Visible light camera; 200 - Processor. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] like Figure 1 As shown, this application provides a method for monitoring the heat source of a blower based on dual-light fusion and segmentation, including the following steps S100 to S600:

[0025] S100: Obtain infrared and visible light images of the target object in the blower and perform preprocessing.

[0026] S200 sets the initial value of the total number of tags K.

[0027] S300 performs image fusion and segmentation on the preprocessed infrared image and the preprocessed visible light image based on K tags to obtain a segmented image.

[0028] S400 calculates the mean and variance of the infrared image region corresponding to each labeled region in the segmented image.

[0029] S500 sets the infrared image tag mean threshold and infrared image tag variance threshold.

[0030] S600 continuously adjusts the total number of labels K based on the infrared image label mean threshold and the infrared image label variance threshold, so that the mean of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, or the variance of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0031] Specifically, the initial value of K is determined based on the image to be segmented, and the initial value of K is generally 8-10. S600 can be executed using prior models and hyperparameter estimation methods within a Bayesian framework.

[0032] This application relates to a method for monitoring the heat source of a blower based on dual-light fusion and segmentation. Infrared and visible light images are acquired using a multimodal sensor. A processor preprocesses the infrared and visible light images respectively. The preprocessed infrared and visible light images are then fused and segmented to obtain fused and segmented images. The mean and variance of the infrared image region corresponding to each labeled region in the segmented image are calculated. Thresholds for the mean and variance are designed, and the number of labels is optimized based on these thresholds. This method pre-plans the number of labels to be used, solving the problem that the label requirements for the segmented blower section always fluctuate in real time and are difficult to determine.

[0033] In one embodiment of this application, S100 includes the following S110 and S120:

[0034] S110 acquires infrared and visible light images of the target object.

[0035] S120 performs pre-registration and normalization of infrared and visible light images.

[0036] Specifically, infrared images can be acquired using an infrared camera. Visible light images can be acquired using a visible light camera under normal daytime lighting conditions.

[0037] In this embodiment, the preprocessing first determines the visible light image as the reference image, registers the infrared image to the visible light image, then uses the cpselect function in MATLAB to determine three pairs of registration points. Next, it calculates the scaling ratio, translation scale, and rotation angle for the selected registration points. Finally, it scales, rotates, and translates the infrared image accordingly to obtain the registered image. Generally, 3-5 registration points are determined when using the cpselect function.

[0038] Normalization specifically includes:

[0039] Using formulas for infrared image pixels and visible light image pixels

[0040]

[0041] Calculations are performed to ensure that the pixel value range is [0, 1].

[0042] In one embodiment of this application, S300 includes the following S310 to S360:

[0043] S310, Create an infrared image pixel set g1, and add each pixel from the preprocessed infrared image to the infrared image pixel set g1, g1 = {g 1j Let g1 = {j = 1, ..., M}, where M is the total number of pixels in the preprocessed infrared image and j is the pixel number of each pixel in the pixel set g1 of the infrared image.

[0044] S320, Create a visible light image pixel set g2, and add each pixel from the preprocessed visible light image to the visible light image pixel set g2, g2 = {g 2j ,j=1,...,N}, where N is the total number of pixels in the preprocessed visible light image, and 2j is the pixel number of each pixel in the visible light image pixel set g2.

[0045] S330, Given an initial segmented image z1 with K labels.

[0046] S340, extract the pixels belonging to each tag from the infrared image pixel set g1, denoted as infrared tag pixel g. 1k k is the label number, k = 1, ... K.

[0047] S350, extract the visible light pixels belonging to each label from the visible light image pixel set g2, and denote them as visible light label pixels g. 2k k is the label number, k = 1, ... K.

[0048] S360, defines the fusion rules under the Bayesian framework, and assigns the corresponding infrared tag pixels g 1k and visible light tag pixels g 2k By performing fusion, k = 1, 2, ..., K, a complete fused image z is obtained. k ={z ik {k = 1, ..., K}, z = {z} ik Let i be the pixel index, k be the label index, K be the total number of labels, and z be the number of pixels. k To segment the label region with label k in image z.

[0049] Specifically, g 1k and g 2kThese represent pixel regions belonging to category k in infrared and visible light images, respectively. All pixels with the same label share some attributes, such as average gray level, average variance, and the same correlation structure.

[0050] In this embodiment, taking i=1 and K=8 as an example,

[0051] Infrared image g1 contains pixels g 11 g 12 g 13 g 14 g 15 g 16 g 17 and g 18 .

[0052] The visible light image g2 contains pixels g 21 g 22 g 23 g 24 g 25 g 26 g 27 and g 28 .

[0053] The fused image z obtained by fusing the infrared image g1 and the visible light image g2 contains:

[0054] z 11 ,z 12 ,z 13 ,z 14 ,z 15 ,z 16 ,z 17 and z 18 .

[0055] Visible light images such as Figure 2 As shown, the infrared image is as follows Figure 3 As shown, when K=8, the fused segmented image is as follows: Figure 4 As shown, the left vertical coordinate and the bottom horizontal coordinate both represent the position of each pixel in the image. For example, a(i,j), where a represents an image, a(i,j) is a pixel in the i-th row (left vertical coordinate) and j-th column (horizontal coordinate).

[0056] The right-hand vertical axis is a color bar that displays color levels. The coordinate range is set to [0.2, 0.8], which can map colors to the pixel values ​​they represent. For example, when the pixel value is 0.2, it will display as dark blue in the image.

[0057] In one embodiment of this application, S400 includes the following S410 to S420:

[0058] S410, Calculate the mean value of the infrared image pixels corresponding to each label region in the segmented image z according to Formula 1, create a mean set m, and include all means in the mean set m, m = {m1, m2, ..., m k}, m1<m2,...,<m k .

[0059]

[0060] Where, m k Let g be the mean value of the infrared image pixels of the infrared image region corresponding to the label region with label k in the segmented image z. 1ki n is the pixel value of the pixel belonging to label k in the infrared image. k 1ki represents the total number of pixels in the infrared image that belong to label k, and 1ki is the index of the pixel in the infrared image that belongs to label k.

[0061] S420, Calculate the variance of the infrared image pixel values ​​corresponding to each label region in the segmented image z according to Formula 2, create a variance set v, and include all means in the variance set v, v = {v1, v2, ..., v...} k}, v1<v2,...,<v k .

[0062]

[0063] Among them, v k Let g be the variance of the pixel values ​​of the infrared image region corresponding to the label region with label k in segmented image z. 1ki n is the pixel value of the pixel belonging to label k in the infrared image. k Let 1ki be the total number of pixels labeled k in the segmented image z, where 1ki is the index of the pixel in the infrared image that belongs to label k, and m is the total number of pixels in the segmented image z. k It represents the average value of the infrared image pixels corresponding to the label region with label k in the segmented image z.

[0064] Specifically, the pixel values ​​of the infrared image reflect the temperature of the labeled area. Temperature information can reflect the operating status of the blower equipment, so the operating status of the blower equipment can be determined based on the pixel values.

[0065] In this embodiment, taking K=8 as an example, we can obtain Table 1:

[0066] Table 1 - Statistical table of mean and variance of infrared image pixel values ​​when K is 8.

[0067] mean m variance v 0.2433 7.3e-04 0.2745 3.2e-03 0.3749 8.9e-03 0.3827 1.1e-03 0.5564 5.9e-03 0.6158 1.6e-03 0.7466 1.1e-03 0.7624 1.0e-03

[0068] In one embodiment of this application, S500 includes the following S510 to S530:

[0069] S510, obtain the maximum value m of all means in the mean set m. max and minimum value m min .

[0070] S520, obtain the maximum value v of all variances in the variance set v. max and minimum value v min .

[0071] S530, calculate the threshold values ​​for infrared image tag parameters and visible light image tag parameters according to Formula 3.

[0072]

[0073] Where, m max ,m min v represents the maximum and minimum values ​​of all means in the mean set m, respectively. max ,v min d1 and d2 are the maximum and minimum values ​​of all variances in the variance set v, respectively. d1 is the first preset adjustment factor and d2 is the second preset adjustment factor. d1, d2 ∈ (0, 1).

[0074] Specifically, d1 and d2 can be adjusted as needed. The larger d1 and d2 are, the higher the threshold and the more obvious the differences between different label regions in the segmented image.

[0075] In this embodiment, thresholds a and b are designed to ensure that different label areas can provide differentiated temperature information to meet the requirements of actual industrial applications.

[0076] In one embodiment of this application, S600 includes the following S610 to S650:

[0077] S610, determine whether the mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, and whether the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0078] S620: If there is no infrared image region with a mean difference less than the infrared image label mean threshold and no infrared image region with a variance difference less than the infrared image label variance threshold in the segmented image corresponding to K, then output K and the segmented image corresponding to K.

[0079] S630, if the mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label mean threshold, and at the same time the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label variance threshold, then the value of K is reduced by 1.

[0080] S640, two label regions whose mean difference between merged infrared image regions is greater than the infrared image label parameter threshold and / or two label regions whose mean difference between merged visible light image regions is greater than the visible light image label parameter threshold.

[0081] S650, return to S300.

[0082] Specifically, there are four different possible outcomes for the judgment in S610:

[0083] (1) The mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, and the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0084] (2) The mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, and the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label variance threshold.

[0085] (3) The mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label mean threshold, and the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0086] (4) The difference in mean values ​​of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label mean threshold, and the difference in variance values ​​of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label variance threshold.

[0087] When the judgment result is (1), (2), (3), it proves that each label category in the segmented image obtained by using the prior model and hyperparameter estimation method under the Bayesian framework is meaningful. This means that each label region obtained by using the prior model and hyperparameter estimation method under the Bayesian framework has a significant temperature difference. Different labels represent different temperature distributions, which can help detect whether the wind turbine is running at an abnormally high temperature. At this time, K does not need to be iterated anymore, and K and the segmented image corresponding to K can be output, that is, S620 is executed.

[0088] When the judgment result is (4), it means that K is not suitable and it needs to continue iterating. K is reduced by 1, that is, S630 is executed.

[0089] In this embodiment, the data of the final K segmented image obtained after dynamic clustering criterion transformation is shown in Table 2:

[0090] Table 2 shows the values ​​of the mean and variance of the infrared image regions corresponding to different label regions in the segmented image corresponding to K when the mean of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold and / or the mean of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold.

[0091] mean m variance v 0.2474 9.6e-04 0.4235 6.1e-03 0.5515 1.0e-02 0.7496 1.6e-03

[0092] For example, after executing S610 to S650, the final iteration result is K = 4, then Table 2 corresponds to the data when K = 4. At this point, when K = 4, [the data will be...]. Figure 2 and Figure 3 The segmented image obtained after fusion is as follows Figure 5 As shown.

[0093] In one embodiment of this application, S610 includes:

[0094] Define p = 2, ..., K.

[0095] Substitute different values ​​of p into |m p -m p-1 |and|v p -v p-1 In |m, determine if |m exists. p -m p-1 |<a and|v p -v p-1 |<b.

[0096] In this embodiment, taking K=4 as an example,

[0097] Calculate |m4-m3|=f1, |m3-m2|=f2, |m2-m1|=f3.

[0098] Calculate |v4-v3|=h1, |v3-v2|=h2, |v2-v1|=h3.

[0099] Determine whether f1 < a, f2 < a, f3 < a and h1 < b, h2 < b, h3 < b are simultaneously satisfied.

[0100] In one embodiment of this application, S620 includes:

[0101] If |m does not exist i -m i-1 |<a and|v i -v i-1 If | < b, then output K and the segmented image corresponding to K.

[0102] In this embodiment, when f1 < a, f2 < a, f3 < a and h1 < b, h2 < b, h3 < b are not satisfied, K and the segmented image corresponding to K are output.

[0103] In one embodiment of this application, S630 includes:

[0104] If |m exists i -m i-1 |<a and|v i -v i-1 If | < b, then the value of K will be reduced by 1.

[0105] In this embodiment, when f1 < a, f2 < a, f3 < a and h1 < b, h2 < b, h3 < b are satisfied, the value of K is reduced by 1.

[0106] This application also claims protection for a system for fusing and segmenting infrared and visible light images of a blower.

[0107] In this embodiment, the system includes a multimodal sensor 100 and a processor 200.

[0108] A multimodal sensor 100 is provided, which is equipped with an infrared camera 110 and a visible light camera 120 for acquiring infrared images and visible light images.

[0109] The processor 200, connected to the multimodal sensor 100, is used to perform the fusion and segmentation method of the blower infrared image and visible light image as mentioned in any of the foregoing embodiments.

[0110] It should be noted that, for the sake of brevity, the reference numerals for the multimodal sensor 100, processor 200, infrared camera 110 and visible light camera 120 mentioned in this application are only used in this embodiment, that is, only in the embodiment of the fusion and segmentation system of the blower infrared image and visible light image, and not in the embodiment of the aforementioned fusion and segmentation method of the blower infrared image and visible light image.

[0111] In this embodiment, infrared and visible light images of the target object are acquired by a multimodal sensor and transmitted to the processor. The processor then processes the infrared and visible light images after receiving them.

[0112] The technical features of the above embodiments can be combined arbitrarily, and the execution order of the method steps is not restricted. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for monitoring the heat source of a blower based on dual-light fusion and segmentation, characterized in that: Obtain infrared and visible light images of the target object in the blower, and perform preprocessing; Set the initial value for the total number of tags K; Based on K labels, the preprocessed infrared image and the preprocessed visible light image are fused and segmented to obtain a segmented image; Calculate the mean and variance of the infrared image region corresponding to each labeled region in the segmented image; Set thresholds for the mean value and variance of infrared image tags; Based on the infrared image label mean threshold and the infrared image label variance threshold, the total number of labels K is continuously adjusted so that the mean of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, or the variance of the infrared image region corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold. The process of fusing and segmenting the preprocessed infrared image and the preprocessed visible light image based on K tags to obtain the segmented image includes: Create an infrared image pixel set g1, and include each pixel from the preprocessed infrared image into the infrared image pixel set g1. M is the total number of pixels in the preprocessed infrared image, and 1j is the pixel number of each pixel in the infrared image pixel set g1. Create a visible light image pixel set g2, and include each pixel from the preprocessed visible light image into the visible light image pixel set g2. N is the total number of pixels in the preprocessed visible light image, and 2j is the pixel number of each pixel in the visible light image pixel set g2; Given an initial segmentation image with K labels ; Extract the pixels belonging to each tag from the infrared image pixel set g1, and denote them as infrared tag pixels. k is the tag number. ; Extract the visible light pixels belonging to each label from the visible light image pixel set g2, and denote them as visible light label pixels. k is the tag number. ; Create a blank image; Define fusion rules under the Bayesian framework and combine the corresponding infrared tag pixels. and visible light tag pixels By performing fusion, k=1, 2, ..., K, a complete fused image is obtained. ; Where i is the pixel index, k is the label index, and K is the total number of labels. To segment the image The label region with the label 'k' in the middle; Calculate image segmentation Create a mean set by taking the mean pixel values ​​of the infrared image region corresponding to each label region. ; Calculate image segmentation Create a variance set by taking the variance of the infrared image pixel values ​​of the infrared image region corresponding to each label region. ; The setting of the infrared image tag mean threshold and the infrared image tag variance threshold includes: Get the maximum value of all means in the set of means m. and minimum value ; Obtain the variance set The maximum value of all variances and minimum value ; Calculate the infrared image tag mean threshold and infrared image tag variance threshold according to Formula 3; Formula 3 in Let be the maximum and minimum values ​​of all means in the mean set m, respectively. They are the variance sets The maximum and minimum values ​​of all variances in the range. As the first preset adjustment factor, This is the second preset adjustment factor. ; definition ; Substitute different values ​​of p in sequence and In, and determine whether it exists. and ; If it does not exist and Then the output is K and the segmented image corresponding to K.

2. The method for monitoring the heat source of a blower based on dual-light fusion and segmentation according to claim 1, characterized in that, The process of obtaining infrared and visible light images of the target object in the blower and performing preprocessing includes: Acquire infrared and visible light images of the target object; Pre-registration and normalization of infrared and visible light images.

3. The method for monitoring the heat source of a blower based on dual-light fusion and segmentation according to claim 1, characterized in that, The calculation of the mean and variance of the infrared image region corresponding to each labeled region in the segmented image includes: According to Formula 1, all means are included in the mean set. , , ; Formula 1: in, To segment the image The mean value of the infrared image pixels in the infrared image region corresponding to the label region with label k. For infrared images belonging to tags The pixel value of the pixel. For infrared images belonging to tags The total number of pixels, 1ki is the number of pixels belonging to the tag in the infrared image. The pixel number; According to Formula 2, all variances are included in the variance set. , , ; Formula 2: in, To segment the image The variance of pixel values ​​in the infrared image of the region corresponding to the label region with label k. For infrared images belonging to tags The pixel value of the pixel. To segment the image Chinese label The total number of pixels, 1ki is the number of pixels belonging to the tag in the infrared image. The pixel number, To segment the image The average value of the infrared image pixels in the infrared image region corresponding to the label region with label k.

4. The method for monitoring the heat source of a blower based on dual-light fusion and segmentation according to claim 1, characterized in that, The step of continuously adjusting the total number of labels K based on the infrared image label mean threshold and the infrared image label variance threshold, such that the mean of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, or the variance of the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold, includes: Determine whether the mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label mean threshold, and whether the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is greater than or equal to the infrared image label variance threshold. If there is no infrared image region corresponding to different label regions in the segmented image corresponding to K whose mean difference is less than the infrared image label mean threshold and whose variance difference is less than the infrared image label variance threshold, then output K and the segmented image corresponding to K. If the mean difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label mean threshold, and at the same time the variance difference between the infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label variance threshold, then the value of K is reduced by 1. Two label regions whose mean difference between merged infrared image regions is less than the infrared image label mean threshold and / or two label regions whose variance difference between merged infrared image regions is less than the infrared image label variance threshold; The image is then fused and segmented based on K labels to obtain a segmented image.

5. The method for monitoring the heat source of a blower based on dual-light fusion and segmentation according to claim 4, characterized in that, If the mean difference between infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label mean threshold, and simultaneously the variance difference between infrared image regions corresponding to different label regions in the segmented image corresponding to K is less than the infrared image label variance threshold, then the value of K is reduced by 1, including: If it exists and If so, then the value of K will be reduced by 1.

6. A blower heat source monitoring system based on dual-light fusion and segmentation, characterized in that, include: A multimodal sensor, which is equipped with an infrared camera and a visible light camera for acquiring infrared and visible light images; The processor, connected to the multimodal sensor, is used to perform the blower heat source monitoring method based on dual-light fusion and segmentation as described in any one of claims 1-5.

Images