Temperature monitoring method and system based on dual-light fusion image

Abstract

The invention provides a temperature monitoring method based on a dual-light fusion image. The method comprises the following steps: S11, synchronously acquiring a visible light image and an infrared thermal image inside and outside a switch cabinet in an intelligent station building; s12, performing local adaptive convolution on the visible light image and the infrared thermal image, extracting local shallow layer features, and performing heterogeneous association to obtain heterogeneous association degree; s13, based on an anisotropic diffusion mechanism, decomposing the infrared thermal image interfered by sunlight in the switch cabinet to obtain a detail layer image and a base layer image, and performing enhanced fusion on texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous association degree of textures in the visible light image to obtain an enhanced infrared thermal image; and S14, inputting the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtaining a temperature abnormal region, and carrying out early warning on a temperature characteristic value. According to the invention, the heterogeneous correlation enhancement of the dual-light fusion image is realized, so that the reliability of temperature monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a temperature monitoring method based on dual-light fusion images and a system thereof. Background Art

[0002] In the power system, the switch cabinet of the smart station is the core equipment, responsible for the distribution, control and protection of electric energy. The switch cabinet contains several important components such as the cable room, circuit breaker room, busbar compartment, etc. These components will generate heat due to the flow of current during operation. If the temperature is too high and not detected in time, it may cause equipment failure or even cause serious accidents such as fire. Therefore, real-time monitoring of temperature changes inside and outside the switch cabinet is a key measure to ensure the safe, stable and reliable operation of high-voltage room equipment.

[0003] In the prior art, traditional temperature monitoring methods usually rely on a single infrared temperature measurement technology, which can infer the temperature of an object by measuring the thermal radiation on the surface of the object. However, infrared temperature measurement technology has certain limitations in some complex environments, especially under strong light interference, the accuracy of the measurement results is significantly affected. Specifically, infrared thermal imaging instruments detect temperature by receiving infrared radiation emitted by the surface of the object, and the intensity of infrared radiation is directly related to the temperature of the object. In an environment with good lighting conditions (such as direct sunlight), the infrared thermal imager may simultaneously receive radiation signals from other objects in the environment (such as sunlight, surrounding reflected light, etc.). The strong sunlight radiation and the infrared radiation of the target object are superimposed on each other, resulting in the thermal imager being unable to accurately distinguish the heat emitted by the target object from the radiation of the external environment. In this case, the measured temperature value may be higher than the actual value, or even a false alarm of abnormal temperature may occur. By fusing visible light and infrared image information, the ability to resist ambient light interference can be effectively improved. Therefore, how to achieve heterogeneous correlation enhancement of dual-light fusion images to improve the reliability of temperature monitoring has become a difficult problem faced by the industry. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a temperature monitoring method based on dual-light fusion images, which improves the reliability of temperature monitoring by realizing heterogeneous correlation enhancement of dual-light fusion images.

[0005] To achieve the above-mentioned object of the invention, the first aspect of the present invention provides a temperature monitoring method based on dual-light fusion images, the method comprising:

[0006] S11. Synchronously collect visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room;

[0007] S12, performing local adaptive convolution on the visible light image and the infrared thermal image respectively, extracting local shallow features of the visible light image and the infrared thermal image and performing heterogeneous association, so as to obtain a heterogeneous association degree between the visible light image and the infrared thermal image;

[0008] S13, based on the anisotropic diffusion mechanism, decomposing the infrared thermal image disturbed by sunlight in the switch cabinet to obtain a detail layer image and a base layer image, and then enhancing and merging the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image;

[0009] S14, inputting the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtaining an abnormal temperature area, and issuing an early warning according to the temperature characteristic value in the abnormal temperature area.

[0010] Furthermore, in step S12, local shallow features of the visible light image and the infrared thermal image are extracted, specifically including:

[0011] S21, dividing the visible light image and the infrared thermal image into local areas by sliding a window according to a preset window size;

[0012] S22, calculating the texture variance and average value of all local areas in the visible light image and the infrared thermal image in each window in step S21 by variance;

[0013] S23, sorting the average values ​​of texture variances, selecting the average value of the minimum texture variance, and using it as the convolution kernel size of the feature recognition model;

[0014] S24. Perform convolution operations on the visible light image and the infrared thermal image respectively through the feature recognition model to extract local shallow features of the visible light image and the infrared thermal image.

[0015] Furthermore, in step S12, heterogeneous association is performed on the local shallow features of the visible light image and the infrared thermal image to obtain the heterogeneous association degree between the visible light image and the infrared thermal image, which specifically includes:

[0016] S31, performing feature alignment of all local shallow features in the same coordinate system to obtain shallow feature label pairs of the same local area in the visible light image and the infrared thermal image;

[0017] S32, calculating the Euclidean distance between the pixel values ​​in the shallow feature label pair, and taking the reciprocal of the Euclidean distance as the feature similarity of the shallow feature label pair;

[0018] S33, taking the ratio of the feature similarity of the shallow feature label pair to the total feature similarity as the weight of the shallow feature label pair;

[0019] S34, performing weighted averaging on the feature similarities of all shallow feature label pairs to obtain the heterogeneous correlation between the visible light image and the infrared thermal image.

[0020] Furthermore, in step S13, based on the anisotropic diffusion mechanism, the infrared thermal image interfered by sunlight in the switch cabinet is decomposed to obtain a detail layer image and a base layer image, which specifically includes:

[0021] S41, performing non-uniform smoothing processing on the infrared thermal image using an anisotropic diffusion algorithm to obtain an infrared thermal smoothed image;

[0022] S42, using the infrared thermal smoothed image as the input of the filter, using the low-frequency information output by the filter as the base layer image, and using the high-frequency information output by the filter as the detail layer image.

[0023] Furthermore, in step S13, the texture features of the detail layer image and the base layer image are enhanced and fused by using the distribution gradient of the texture in the visible light image and the heterogeneous correlation degree to obtain an enhanced infrared thermal image, which specifically includes:

[0024] S51, using a gradient operator to calculate the rate of change of each pixel in the visible light image in the horizontal direction and the vertical direction;

[0025] S52, the change rate in step S51 forms a gradient image according to the position information of the corresponding pixels as the distribution gradient of the texture in the visible light image;

[0026] S53, based on the image enhancement model obtained by image fusion network training, the distribution gradient and heterogeneous correlation degree are used as constraint parameters of the image enhancement model, and the texture features of the detail layer image and the base layer image are enhanced by the image enhancement model to obtain an enhanced detail layer image and an enhanced base layer image;

[0027] S54. Fusing the enhanced detail layer image and the enhanced base layer image based on an image fusion network to obtain an enhanced infrared thermal image.

[0028] Furthermore, in step S14, the enhanced infrared thermal image is input into a pre-trained temperature detection model to identify and obtain the temperature abnormality area, which specifically includes:

[0029] A pre-trained temperature detection model is constructed based on a convolutional neural network, abnormal regions are detected on the enhanced infrared thermal image using the temperature detection model, and the detected abnormal regions are used as temperature abnormal regions.

[0030] Further, in step S14, an early warning is issued according to the temperature characteristic value in the temperature abnormality area, specifically including:

[0031] The temperature characteristic value is judged by the threshold. When the temperature characteristic value exceeds the set threshold, the system automatically triggers an early warning.

[0032] Furthermore, a CMOS image sensor is used to capture visible light images.

[0033] Furthermore, an infrared thermal imaging sensor is used to collect infrared thermal images.

[0034] A second aspect of the present invention provides a temperature monitoring system based on dual-light fusion images, the system comprising:

[0035] Image acquisition module, used to synchronously acquire visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room;

[0036] An image association module is used to perform local adaptive convolution on the visible light image and the infrared thermal image, respectively, extract local shallow features of the visible light image and the infrared thermal image, and perform heterogeneous association to obtain a heterogeneous association degree between the visible light image and the infrared thermal image;

[0037] An image enhancement module is used to decompose the infrared thermal image interfered by sunlight in the switch cabinet based on an anisotropic diffusion mechanism to obtain a detail layer image and a base layer image, and then enhance and fuse the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image;

[0038] The temperature warning module is used to input the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtain the temperature abnormality area, and issue a warning according to the temperature characteristic value in the temperature abnormality area.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a temperature monitoring method and system based on dual-light fusion images, which enhances and fuses the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image, and further identifies the temperature abnormality area in the enhanced infrared thermal image based on the temperature detection model. Among them, firstly, local adaptive convolution is used to extract the local shallow features of the image, which can effectively mine the detail texture information in the visible light image and the infrared thermal image, and through the calculation of heterogeneous correlation, the relevant detail texture information in the visible light image and the infrared thermal image is compared and fused, which can reduce the error caused by the single modality image.

[0041] Then, when disturbed by sunlight, the infrared thermal image is decomposed based on the anisotropic diffusion mechanism, and the infrared image is divided into a detail layer image and a base layer image, which can effectively reduce the influence of ambient light and retain important temperature information. The detail layer and the base layer images are further enhanced and fused in combination with the texture distribution gradient and heterogeneous correlation in the visible light image, which can highlight the details of temperature changes and make the enhanced infrared thermal image more recognizable and accurate, thereby improving the accuracy of temperature anomaly detection. Finally, the enhanced infrared thermal image is analyzed for anomalies using a pre-trained temperature detection model, which can accurately identify temperature anomaly areas and issue warnings based on the temperature characteristic values ​​of these areas. In summary, the present invention can achieve heterogeneous correlation enhancement of dual-light fusion images, thereby improving the reliability of temperature monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 A schematic flow chart of a temperature monitoring method based on dual-light fusion images provided by an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the structure of a temperature monitoring system based on dual-light fusion images provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0046] Reference Figure 1 In a first aspect, this embodiment provides a temperature monitoring method based on dual-light fusion images, the method comprising:

[0047] S11. Synchronously collect visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room. Use CMOS image sensors to collect visible light images; use infrared thermal imaging sensors to collect infrared thermal images.

[0048] It should be noted that in the power system, the switch cabinet, as a key equipment in the intelligent station house, undertakes the important responsibility of distributing, controlling and protecting electric energy. Its interior includes multiple key components such as the cable room, circuit breaker room, busbar compartment, etc. These components will generate heat due to the passage of current during operation. If the temperature is too high and is not discovered in time, it may cause equipment failure or even cause serious consequences such as fire. Therefore, in this embodiment, the temperature inside and outside the switch cabinet is monitored based on the dual-light fusion image to ensure the safe operation of the power system.

[0049] In specific implementation, a dual-camera system that supports both visible light and infrared thermal imaging can be used to take photos and monitor the temperature environment inside and outside the switch cabinet in the intelligent station at a preset frequency to obtain visible light images and infrared thermal images. Among them, a CMOS (CMOS, Complementary Metal Oxide Semiconductor) image sensor can be used to collect visible light images, and an infrared thermal imaging sensor can be used to collect infrared thermal images. It should be noted that in this embodiment, a hardware synchronization interface (such as a trigger signal) can be used to ensure that the acquisition time of the visible light image and the infrared thermal image is consistent. In other embodiments, a software synchronization method can also be used to accurately synchronize the acquisition time of the visible light image and the infrared thermal image through software, and use timestamps for post-processing alignment, which is not limited here.

[0050] S12, performing local adaptive convolution on the visible light image and the infrared thermal image respectively, extracting local shallow features of the visible light image and the infrared thermal image and performing heterogeneous association to obtain a heterogeneous association degree between the visible light image and the infrared thermal image.

[0051] In step S12, local shallow features of the visible light image and the infrared thermal image are extracted, specifically including:

[0052] S21. Divide the visible light image and the infrared thermal image into local areas by sliding a window according to a preset window size.

[0053] S22, respectively calculating the texture variance and average value of all local areas in the visible light image and the infrared thermal image under each window in step S21 through variance.

[0054] S23. Sort the average values ​​of texture variances, and select the average value of the minimum texture variance as the convolution kernel size of the feature recognition model.

[0055] S24. Perform convolution operations on the visible light image and the infrared thermal image respectively through the feature recognition model to extract local shallow features of the visible light image and the infrared thermal image.

[0056] In the specific implementation, first, the visible light image and the infrared thermal image can be divided into local areas according to windows of different sizes (such as 3×3, 5×5, 7×7, etc.) by means of a sliding window, and then the texture variance of all local areas in the visible light image and the infrared thermal image under each window is calculated by using the variance calculation formula, and the average value of all texture variances under each window is further calculated, and the average value of the texture variance under each window is sorted, and the average value of the minimum texture variance is screened out, and the window size corresponding to the average value of the texture variance is used as the convolution kernel size of the feature recognition model; then, the convolution kernel of the feature recognition model is used to perform convolution operations on the visible light image and the infrared thermal image respectively. During the convolution process, each local area of ​​the image will receive different convolution processing according to its texture characteristics, and the local shallow features in the image (such as edges, textures, color changes, etc.) are extracted, and then the feature map obtained by the convolution operation is pooled to reduce the dimension of the feature map and retain important feature information, so as to obtain the local shallow features of the visible light image and the infrared thermal image.

[0057] It should be noted that the local shallow features in this application refer to fine-grained features extracted from local areas of an image. Such features usually reflect the basic structural information of the image, such as edges, textures, shapes, color changes, etc. Compared with deep features (such as high-level semantic information of objects or scenes), local shallow features focus on capturing small local changes in the image, and they are usually more sensitive to local details of the image; in addition, the texture variance in this application can reflect the degree of change of pixel values ​​in the image area, that is, the detailed information in the area.

[0058] In step S12, heterogeneous association is performed on the local shallow features of the visible light image and the infrared thermal image to obtain the heterogeneous association degree between the visible light image and the infrared thermal image, which specifically includes:

[0059] S31, aligning all local shallow features in the same coordinate system to obtain shallow feature label pairs of the same local area in the visible light image and the infrared thermal image.

[0060] S32, calculating the Euclidean distance between the pixel values ​​in the shallow feature label pair, and taking the reciprocal of the Euclidean distance as the feature similarity of the shallow feature label pair.

[0061] S33. The ratio of the feature similarity of the shallow feature label pair to all feature similarities is used as the weight of the shallow feature label pair.

[0062] S34, performing weighted averaging on the feature similarities of all shallow feature label pairs to obtain the heterogeneous correlation between the visible light image and the infrared thermal image.

[0063] In specific implementation, first, the image registration technology (such as feature point-based registration) can be used to align the visible light image and the infrared thermal image into the same coordinate system. In the aligned image, local areas of the same size are delineated. The shallow features corresponding to these local areas will have the same coordinates in the visible light image and the infrared thermal image. The local shallow features in the aligned visible light image and the infrared thermal image are extracted, and a shallow feature label is assigned to each local area. The shallow feature label represents the specific features of the local area, such as texture, color or edge intensity. Then, the shallow feature labels corresponding to the same local areas in the visible light image and the infrared thermal image are combined into a set as the shallow feature label pairs of the corresponding local areas, so that the shallow feature label pairs of all the same local areas in the visible light image and the infrared thermal image can be obtained; then, for each shallow feature label pair, the Euclidean distance between the pixel values ​​in the shallow feature label pair is calculated, and the inverse of the Euclidean distance is used as the feature similarity of the shallow feature label pair.

[0064] Finally, the weight of the shallow feature label pair is assigned based on the strength of the feature similarity itself, that is, the ratio of the feature similarity of the shallow feature label pair to all feature similarities is used as the weight of the shallow feature label pair, and then the feature similarities of all shallow feature label pairs are weighted averaged to obtain the heterogeneous correlation between the visible light image and the infrared thermal image. It should be noted that the heterogeneous correlation in this application is an indicator to measure the similarity of corresponding regions in images of different modalities. The higher the heterogeneous correlation value, the more similar the features of the visible light image and the infrared thermal image in the corresponding region, and vice versa.

[0065] S13. Based on the anisotropic diffusion mechanism, the infrared thermal image disturbed by sunlight in the switch cabinet is decomposed to obtain a detail layer image and a base layer image, and then the texture features of the detail layer image and the base layer image are enhanced and fused through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image.

[0066] It should be noted that when the switch cabinet is performing temperature monitoring, sunlight interference will cause inaccurate temperature information in the infrared thermal image. The radiation of sunlight and the temperature distribution of the switch cabinet itself may overlap or introduce additional thermal noise, making the temperature data in the infrared thermal image untrue, thereby affecting the accuracy and reliability of temperature monitoring. Therefore, when the switch cabinet is subjected to sunlight interference during temperature monitoring, in order to ensure that the temperature monitoring system can accurately reflect the actual temperature status of the switch cabinet and avoid false alarms or missed alarms due to interference, in this embodiment, the infrared thermal image is enhanced based on the visible light image to reduce the interference of sunlight on temperature monitoring.

[0067] In step S13, based on the anisotropic diffusion mechanism, the infrared thermal image interfered by sunlight in the switch cabinet is decomposed to obtain a detail layer image and a base layer image, which specifically includes:

[0068] S41, performing non-uniform smoothing processing on the infrared thermal image using an anisotropic diffusion algorithm to obtain an infrared thermal smoothed image.

[0069] S42, using the infrared thermal smoothed image as the input of the filter, using the low-frequency information output by the filter as the base layer image, and using the high-frequency information output by the filter as the detail layer image.

[0070] It should be noted that the base layer image in this application contains the low-frequency information of the infrared thermal smooth image, which reflects the overall texture distribution information of the image; the detail layer image contains the high-frequency information of the infrared thermal smooth image, which reflects the detail texture distribution information of the image. In specific implementation, first, the Perona-Malik model can be used to perform non-uniform smoothing according to the local structural information (such as edges and textures) of the infrared thermal image, and the image obtained by the smoothing process is used as the infrared thermal smooth image; then, the infrared thermal smooth image can be used as the input of the filter, the low-frequency information output by the filter is used as the base layer image, and the high-frequency information output by the filter is used as the detail layer image.

[0071] In step S13, the texture features of the detail layer image and the base layer image are enhanced and fused by using the distribution gradient of the texture in the visible light image and the heterogeneous correlation degree to obtain an enhanced infrared thermal image, which specifically includes:

[0072] S51. Use a gradient operator to calculate the rate of change of each pixel in the visible light image in the horizontal and vertical directions.

[0073] S52 . The change rate in step S51 forms a gradient image according to the position information of the corresponding pixels as the distribution gradient of the texture in the visible light image.

[0074] S53. Based on the image enhancement model obtained by image fusion network training, the distribution gradient and heterogeneous correlation degree are used as constraint parameters of the image enhancement model, and the texture features of the detail layer image and the base layer image are enhanced through the image enhancement model to obtain an enhanced detail layer image and an enhanced base layer image.

[0075] S54. Fusing the enhanced detail layer image and the enhanced base layer image based on an image fusion network to obtain an enhanced infrared thermal image.

[0076] In the specific implementation, first, the Sobel operator can be used to calculate the change rate of each pixel in the visible light image in the horizontal and vertical directions, and all the change rates are used to form a gradient image according to the position information of the corresponding pixels, and then the gradient image is used as the distribution gradient of the texture in the visible light image; then, an image enhancement model is initialized, and the image enhancement model is obtained based on image fusion network training, and the distribution gradient and the heterogeneous correlation degree are used as constraint parameters of the image enhancement model, and the detail layer image and the base layer image are used as initialization parameters. The texture features of the detail layer image and the base layer image are enhanced through the image enhancement model, and the enhanced detail layer image is used as an enhanced detail layer image, and the enhanced base layer image is used as an enhanced base layer image.

[0077] It should be noted that in this application, the distribution gradient and heterogeneous correlation are used as constraint parameters of the image enhancement model, aiming to optimize the texture features of the image by guiding the image enhancement model, so that when enhancing the detail layer and base layer images, the enhanced image can better retain the key texture information and structural features. The distribution gradient provides local texture information in the visible light image, which can help the image enhancement model identify and highlight important changes in the detail layer image and the base layer image, while the heterogeneous correlation describes the correlation between the visible light image and the infrared thermal image, ensuring that during the enhancement process, the images of the two modes can maintain consistency and complement each other. Through the constraints of the distribution gradient and heterogeneous correlation, the image enhancement model can more accurately adjust the local features of the image, so that the final enhanced infrared thermal image is clearer and more realistic in both details and temperature features.

[0078] S14, inputting the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtaining an abnormal temperature area, and issuing an early warning according to the temperature characteristic value in the abnormal temperature area.

[0079] In step S14, the enhanced infrared thermal image is input into a pre-trained temperature detection model to identify and obtain the temperature abnormality area, which specifically includes:

[0080] A pre-trained temperature detection model is constructed based on a convolutional neural network, and the enhanced infrared thermal image is subjected to abnormal area detection through the temperature detection model, and the detected abnormal area is used as the temperature abnormal area. In specific implementation, first, a temperature detection model can be trained through a large number of infrared thermal images, and the temperature detection model can be constructed based on a convolutional neural network. In other embodiments, other algorithm structures can also be used for construction, which is not limited here; then, the enhanced infrared thermal image can be used as an input parameter of the pre-trained temperature detection model, and the detected abnormal area is output through the temperature detection model, and finally the detected abnormal area is used as the temperature abnormal area.

[0081] In step S14, an early warning is issued according to the temperature characteristic value in the temperature abnormality area, specifically including:

[0082] The temperature characteristic value is judged by a threshold value, and when the temperature characteristic value exceeds the set threshold value, the system automatically triggers an early warning. It should be noted that the early warning based on the temperature characteristic value in the temperature abnormal area refers to the threshold value judgment of the temperature characteristic value. When the temperature characteristic value exceeds the set threshold value, the system automatically triggers an early warning and reminds the staff to take further measures, such as checking the equipment or repairing it, to avoid possible safety hazards or equipment damage.

[0083] Reference Figure 2 The second aspect of the present invention provides a temperature monitoring system based on dual-light fusion images, the system comprising:

[0084] The image acquisition module is used to synchronously acquire visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room.

[0085] The image association module is used to perform local adaptive convolution on the visible light image and the infrared thermal image respectively, extract local shallow features of the visible light image and the infrared thermal image, and perform heterogeneous association to obtain the heterogeneous association degree between the visible light image and the infrared thermal image.

[0086] The image enhancement module is used to decompose the infrared thermal image interfered by sunlight in the switch cabinet based on the anisotropic diffusion mechanism to obtain a detail layer image and a base layer image, and then enhance and fuse the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image.

[0087] The temperature warning module is used to input the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtain the temperature abnormality area, and issue a warning according to the temperature characteristic value in the temperature abnormality area.

[0088] Specifically, the image acquisition module of this embodiment can use a dual camera system that supports both visible light and infrared thermal imaging to take photos and monitor the temperature environment inside and outside the switch cabinet in the intelligent station at a preset frequency to obtain visible light images and infrared thermal images. The image association module uses local adaptive convolution to extract local shallow features of the image, which can effectively mine the detail texture information in the visible light image and the infrared thermal image, and compare and fuse the relevant detail texture information in the visible light image and the infrared thermal image through the calculation of heterogeneous correlation, which can reduce the error caused by the single modal image. The image enhancement module decomposes the infrared thermal image based on the anisotropic diffusion mechanism, and divides the infrared image into a detail layer image and a base layer image, which can effectively reduce the influence of ambient light and retain important temperature information. The texture distribution gradient and heterogeneous correlation in the visible light image are further combined to enhance and fuse the detail layer and the base layer images, which can highlight the details of the temperature change, making the enhanced infrared thermal image more recognizable and accurate, thereby improving the accuracy of temperature anomaly detection. The temperature warning module uses a pre-trained temperature detection model to perform anomaly analysis on the enhanced infrared thermal image. It can accurately identify areas with abnormal temperature and issue warnings based on the temperature characteristic values ​​of these areas.

[0089] It can be seen that the present application enhances and fuses the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image, so as to obtain an enhanced infrared thermal image, and further identifies the temperature abnormality area in the enhanced infrared thermal image based on the temperature detection model; wherein, firstly, local adaptive convolution is used to extract the local shallow features of the image, which can effectively mine the detail texture information in the visible light image and the infrared thermal image, and through the calculation of heterogeneous correlation, the relevant detail texture information in the visible light image and the infrared thermal image is compared and fused, which can reduce the error caused by the single modality image.

[0090] Then, when affected by strong sunlight, the infrared thermal image is decomposed based on the anisotropic diffusion mechanism, and the infrared image is divided into a detail layer image and a base layer image, which can effectively reduce the influence of ambient light and retain important temperature information. The detail layer and the base layer images are further enhanced and fused in combination with the texture distribution gradient and heterogeneous correlation in the visible light image, which can highlight the details of the temperature change, making the enhanced infrared thermal image more recognizable and accurate, thereby improving the accuracy of temperature anomaly detection; finally, the enhanced infrared thermal image is analyzed for anomalies using a pre-trained temperature detection model, which can accurately identify temperature anomaly areas and issue early warnings based on the temperature characteristic values ​​of these areas; in summary, the present application scheme can achieve heterogeneous correlation enhancement of dual-light fusion images, thereby improving the reliability of temperature monitoring.

[0091] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0093] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A temperature monitoring method based on dual-light fusion images, characterized in that: The method comprises: S11. Synchronously collect visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room; S12, performing local adaptive convolution on the visible light image and the infrared thermal image respectively, extracting local shallow features of the visible light image and the infrared thermal image and performing heterogeneous association, so as to obtain a heterogeneous association degree between the visible light image and the infrared thermal image; S13, based on the anisotropic diffusion mechanism, decomposing the infrared thermal image disturbed by sunlight in the switch cabinet to obtain a detail layer image and a base layer image, and then enhancing and merging the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image; S14, inputting the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtaining an abnormal temperature area, and issuing an early warning according to the temperature characteristic value in the abnormal temperature area.

2. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: In step S12, local shallow features of the visible light image and the infrared thermal image are extracted, specifically including: S21, dividing the visible light image and the infrared thermal image into local areas by sliding a window according to a preset window size; S22, calculating the texture variance and average value of all local areas in the visible light image and the infrared thermal image in each window in step S21 by variance; S23, sorting the average values ​​of texture variances, selecting the average value of the minimum texture variance, and using it as the convolution kernel size of the feature recognition model; S24. Perform convolution operations on the visible light image and the infrared thermal image respectively through the feature recognition model to extract local shallow features of the visible light image and the infrared thermal image.

3. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: In step S12, heterogeneous association is performed on the local shallow features of the visible light image and the infrared thermal image to obtain the heterogeneous association degree between the visible light image and the infrared thermal image, which specifically includes: S31, performing feature alignment of all local shallow features in the same coordinate system to obtain shallow feature label pairs of the same local area in the visible light image and the infrared thermal image; S32, calculating the Euclidean distance between the pixel values ​​in the shallow feature label pair, and taking the reciprocal of the Euclidean distance as the feature similarity of the shallow feature label pair; S33, taking the ratio of the feature similarity of the shallow feature label pair to the total feature similarity as the weight of the shallow feature label pair; S34, performing weighted averaging on the feature similarities of all shallow feature label pairs to obtain the heterogeneous correlation between the visible light image and the infrared thermal image.

4. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: In step S13, based on the anisotropic diffusion mechanism, the infrared thermal image interfered by sunlight in the switch cabinet is decomposed to obtain a detail layer image and a base layer image, which specifically includes: S41, performing non-uniform smoothing processing on the infrared thermal image using an anisotropic diffusion algorithm to obtain an infrared thermal smoothed image; S42, using the infrared thermal smoothed image as the input of the filter, using the low-frequency information output by the filter as the base layer image, and using the high-frequency information output by the filter as the detail layer image.

5. The temperature monitoring method based on dual-light fusion images according to claim 1, characterized in that: In step S13, the texture features of the detail layer image and the base layer image are enhanced and fused by using the distribution gradient of the texture in the visible light image and the heterogeneous correlation degree to obtain an enhanced infrared thermal image, which specifically includes: S51, using a gradient operator to calculate the rate of change of each pixel in the visible light image in the horizontal direction and the vertical direction; S52, the change rate in step S51 forms a gradient image according to the position information of the corresponding pixels as the distribution gradient of the texture in the visible light image; S53, based on the image enhancement model obtained by image fusion network training, the distribution gradient and heterogeneous correlation degree are used as constraint parameters of the image enhancement model, and the texture features of the detail layer image and the base layer image are enhanced by the image enhancement model to obtain an enhanced detail layer image and an enhanced base layer image; S54. Fusing the enhanced detail layer image and the enhanced base layer image based on an image fusion network to obtain an enhanced infrared thermal image.

6. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: In step S14, the enhanced infrared thermal image is input into a pre-trained temperature detection model to identify and obtain the temperature abnormality area, which specifically includes: A pre-trained temperature detection model is constructed based on a convolutional neural network, abnormal regions are detected on the enhanced infrared thermal image using the temperature detection model, and the detected abnormal regions are used as temperature abnormal regions.

7. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: In step S14, an early warning is issued according to the temperature characteristic value in the temperature abnormality area, specifically including: The temperature characteristic value is judged by the threshold. When the temperature characteristic value exceeds the set threshold, the system automatically triggers an early warning.

8. The temperature monitoring method based on dual-light fusion images according to claim 1 is characterized in that: A CMOS image sensor is used to capture visible light images.

9. The temperature monitoring method based on dual-light fusion images according to claim 1, characterized in that: Use infrared thermal imaging sensors to collect infrared thermal images.

10. A temperature monitoring system based on dual-light fusion images, characterized in that: The system comprises: Image acquisition module, used to synchronously acquire visible light images and infrared thermal images inside and outside the switch cabinet in the intelligent station room; An image association module is used to perform local adaptive convolution on the visible light image and the infrared thermal image, respectively, extract local shallow features of the visible light image and the infrared thermal image, and perform heterogeneous association to obtain a heterogeneous association degree between the visible light image and the infrared thermal image; An image enhancement module is used to decompose the infrared thermal image interfered by sunlight in the switch cabinet based on an anisotropic diffusion mechanism to obtain a detail layer image and a base layer image, and then enhance and fuse the texture features of the detail layer image and the base layer image through the distribution gradient and heterogeneous correlation of the texture in the visible light image to obtain an enhanced infrared thermal image; The temperature warning module is used to input the enhanced infrared thermal image into a pre-trained temperature detection model for identification, obtain the temperature abnormality area, and issue a warning according to the temperature characteristic value in the temperature abnormality area.

Images