Infrared thermal imaging and acoustic imaging linked electric power communication machine room equipment fault monitoring and three-dimensional positioning method and system

Abstract

The invention discloses an infrared thermal imaging and acoustic imaging linked electric power communication machine room equipment fault monitoring and three-dimensional positioning method and system. The method comprises the following steps: performing preprocessing, off-line calibration and registration fusion on a thermal image and a sound field cloud picture to obtain a fused image; establishing a power communication machine room equipment fault monitoring and fault area positioning model; accurate three-dimensional positioning of a fault point is realized; through innovative linkage of infrared thermal imaging and acoustic imaging, the three-dimensional fault point is intelligently monitored and positioned in real time in combination with an unsupervised learning algorithm, the fault monitoring efficiency and the fault positioning accuracy are improved, and the system has the advanced capability of monitoring unknown faults. The fault monitoring accuracy and efficiency of the power communication machine room equipment are remarkably improved, early fault diagnosis can be achieved, potential system faults are avoided, the real-time monitoring capability is achieved, and continuous and stable operation of a power system is ensured; the identification of the fault point is more accurate, the maintenance response time is greatly shortened, and the maintenance cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring and positioning of electric power communication room equipment, and in particular to a method for monitoring and three-dimensional positioning of electric power communication room equipment faults by linking infrared thermal imaging with acoustic imaging. Background Art

[0002] The normal operation of equipment in power communication rooms is the key to ensuring data security and system reliability, but existing fault monitoring technologies mostly rely on limited electrical parameter detection, making it difficult to fully diagnose potential internal faults of equipment and usually lack real-time monitoring capabilities.

[0003] In recent years, non-contact detection technology has gained attention. Infrared thermal imaging can detect the surface temperature distribution of equipment for thermal fault diagnosis, and acoustic imaging can capture sound waves to analyze internal anomalies. The two are often used independently and lack linkage, which limits the comprehensiveness and accuracy of monitoring. Unsupervised learning technology can automatically identify and extract patterns in data, and is very suitable for processing large-scale infrared thermal imaging and acoustic imaging data. This method can reveal abnormal patterns in equipment operation and realize the diagnosis of unknown faults.

[0004] Although patent application number 202310307598.8 proposes a cable fault location method and system using a microphone and infrared thermal imaging for joint positioning, it has made progress in two-dimensional positioning and only uses the fusion image of the infrared thermal image and the sound field cloud map as the basis for fault analysis. It does not use intelligent image recognition technology to identify the fault, does not perform three-dimensional positioning of the fault point, and also ignores the fault information of the single data of the infrared thermal image or the sound field cloud map. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method for fault monitoring and three-dimensional positioning of power communication room equipment by linking infrared thermal imaging with acoustic imaging. The method strengthens the fusion analysis of infrared thermal imaging and acoustic imaging data by introducing unsupervised learning technology, realizes more accurate fault feature recognition and three-dimensional positioning, and provides more reliable technical support for the stable operation of the power system.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: The method for monitoring and three-dimensionally locating equipment faults in a power communication room by combining infrared thermal imaging with acoustic imaging provided by the present invention comprises the following steps: S1, collecting and preprocessing the image pairs consisting of thermal images and acoustic field cloud images of the normal operating status of the equipment in the power communication room; S2, offline calibration of infrared thermal imager and acoustic camera; S3, registering and fusing the preprocessed image pairs to obtain a fused image; S4, build a model for equipment fault monitoring and fault area location in the power communication room; S5, accurately locate the fault point in three dimensions according to the thermal image fault area, the acoustic field cloud image fault area, and the fusion image fault area.

[0007] Further, the preprocessing includes the following steps: The image pairs are converted into single-channel grayscale image pairs by using weighted averaging method; The grayscale range of the single-channel grayscale image pair is stretched using a piecewise linear grayscale transformation method to obtain a grayscale image pair with prominent contour features; The grayscale image pairs with prominent contour features are denoised using Gaussian filters; Furthermore, the offline calibration includes the following steps: Make a rectangular wooden calibration board with an array of equally spaced buzzers; An infrared thermal imager and an acoustic camera are used to collect image pairs consisting of thermal images and acoustic field cloud images of the calibration plate at different positions when the buzzer array is working; The preprocessing step is performed on the image pair consisting of the thermal image and the acoustic field cloud image of the calibration plate in different positions; Based on the threshold, the grayscale image pair with prominent contour features after denoising is processed into a binary image to obtain an image pair with a temperature area and a sound field area segmented; According to the gray values ​​of the temperature area and the sound field area obtained by segmentation, the calibration points are detected and located using the image moment; The calibration points of multiple frames of images at different angles are sorted using the calibration point co-linearity detection algorithm; Calculate the intrinsic parameter matrix of the infrared thermal imager based on the coordinates of the located and sorted calibration points and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix .

[0008] Further, the registration comprises the following steps: Determine the viewing angles of thermal imaging cameras and acoustic cameras; According to the internal parameter matrix of the infrared thermal imager and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix Calculate the homography matrix that transforms the image with a smaller viewing angle into the coordinate system of another image; Traverse all pixels of the image with a smaller viewing angle range, use the homography matrix to transform them to the viewing angle of the image with a larger viewing angle range, and use the interpolation method to fill in the blank pixels when some pixels have no corresponding pixels.

[0009] Furthermore, the fusion adopts a pixel weighted average fusion method to perform linear weighted average on the grayscale values ​​of the pixels of the thermal image and the acoustic field cloud image obtained by the alignment.

[0010] Furthermore, constructing an unsupervised learning fault monitoring and fault area location model includes the following steps: The pre-trained feature extractor is used to extract the low-level visual features of the registered thermal image, sound field cloud image and fusion image of the normal operation status of the power communication room equipment and perform L2 norm normalization. Using non-parametric clustering algorithm, the mean vector of each cluster is taken as the normal prototype to obtain the normal thermal image feature prototype, the normal sound field cloud image feature prototype and the normal fusion feature prototype; In the monitoring stage, the thermal image and the sound field cloud map are collected in real time, and the preprocessing step and the registration step are adopted to obtain the low-level visual features of the real-time thermal image, the real-time sound field cloud map and the real-time fusion image and perform L2 norm normalization; The normal thermal image feature prototype, the normal sound field cloud image feature prototype and the normal fusion feature prototype normalized by L2 norm are respectively used as the kernel parameters of 1x1 convolution and the inner product of the corresponding low-level visual features of the real-time thermal image, the low-level visual features of the real-time sound field cloud image and the low-level visual features of the real-time fusion image; The inner product result is applied to the channel maximum pooling to obtain the similarity between the pixels at each position in the normal thermal image feature prototype and the real-time thermal image, the normal sound field cloud image feature prototype and the real-time sound field cloud image, and the normal fusion feature prototype and the real-time fusion image; A fault score threshold is set. When the similarity is lower than the threshold, the pixel position is determined as a fault point. If a fault point exists, it is determined that there is a fault area in the real-time image.

[0011] Furthermore, accurate three-dimensional positioning of the fault point includes the following steps: Determine an accurate two-dimensional fault point criterion, and calculate the accurate two-dimensional fault point using the image moment; Based on the result of the offline calibration, the precise two-dimensional fault point is mapped to the three-dimensional space.

[0012] The invention provides an electric power communication room equipment fault monitoring and three-dimensional positioning system that combines infrared thermal imaging with acoustic imaging, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. The method is implemented when the processor executes the program.

[0013] The beneficial effects of the present invention are: The present invention provides a method for monitoring and three-dimensionally locating equipment faults in an electric power communication room by linking infrared thermal imaging with acoustic imaging. The method uses an infrared thermal imager and an acoustic camera to collect electric infrared thermal images and acoustic field cloud images to obtain temperature and sound information of the equipment in the electric power communication room, and locates the fault point by linking thermal images, acoustic field cloud images and fused images. The fused image can locate the fault point more accurately when both the temperature and sound of the equipment are abnormal, and link single fault information at the same time to avoid the situation in which, when only the temperature or only the sound is abnormal, the threshold is not reached in the fault area determination process of the fused image and it is determined to be a normal point, thereby further improving the accuracy of fault location.

[0014] This method realizes image registration based on the parameters obtained by offline calibration of the camera during the process of converting the thermal image and the acoustic field cloud map into a unified coordinate system for fusion. The traditional camera uses a checkerboard calibration plate that cannot obtain acoustic and temperature characteristics and cannot calibrate the thermal imager and the acoustic camera. This method produces a special rectangular wooden calibration plate with an equidistantly arranged buzzer array during the offline calibration process to complete the calibration of the thermal imager and the acoustic camera. The calibrated parameters can be used to convert the two-dimensionally located fault point into a three-dimensional world coordinate system.

[0015] This method adopts an unsupervised learning algorithm in the process of fault judgment, and performs similarity analysis on the monitored real-time thermal image features, sound field cloud map features and fusion image features with the normal thermal image feature prototype, normal sound field cloud map feature prototype and normal fusion feature prototype respectively, so as to further judge the fault, realize intelligent real-time monitoring and judgment of the fault, improve the efficiency of fault monitoring of equipment in the power communication room, and help monitor unknown faults.

[0016] This method includes collecting and preprocessing image pairs consisting of thermal images and acoustic field cloud images of the normal operating status of the power communication room equipment; offline calibration of the infrared thermal imager and the acoustic camera; fusing the preprocessed image pairs after registration to obtain a fused image; building a fault monitoring and fault area positioning model for the power communication room equipment; and accurately locating the fault point in three dimensions according to the thermal image fault area, the acoustic field cloud image fault area, and the fused image fault area. The present invention uses the innovative linkage of infrared thermal imaging and acoustic imaging, combined with an unsupervised learning algorithm, to intelligently monitor and locate three-dimensional fault points in real time, thereby improving the efficiency of fault monitoring and the accuracy of fault location, while also having the advanced ability to monitor unknown faults.

[0017] The more accurate positioning of the present invention includes two levels. The first level: for the fused image, due to the combination of the information of the thermal image and the sound field cloud map, the fault point is located more accurately from two dimensions; the second level: the information of a single thermal image and the sound field cloud map is linked to avoid the situation in which the fused image fault area is judged as a normal point because the threshold is not reached during the judgment process of the fault area of ​​the fused image when only the temperature or only the sound is abnormal, thereby further improving the accuracy.

[0018] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration: Figure 1 The present invention is a simplified schematic diagram of the process.

[0020] Figure 2 It is a framework diagram of a specific implementation method.

[0021] Figure 3 This is a comparison chart of ROC curves. DETAILED DESCRIPTION

[0022] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0023] refer to Figure 1 and Figure 2 , Figure 1 It is a simplified schematic diagram of the process of the present invention, Figure 2 This is a specific implementation framework diagram. The embodiment of the present invention discloses a method for monitoring and three-dimensional positioning of equipment faults in a power communication room by combining infrared thermal imaging with acoustic imaging, including the following steps: S1, collecting and preprocessing the image pairs consisting of thermal images and acoustic field cloud images of the normal operating status of the equipment in the power communication room; It should be noted that the thermal image and sound field cloud map of the normal operating status of the power communication room equipment described in S1 are collected by deploying a fixed-position infrared thermal imager and an acoustic camera in the room respectively, and the viewing angles of the deployed infrared thermal imager and acoustic camera must include at least one of the same power equipment. In this embodiment, 1031 samples of normal operating status are collected for learning the normal prototype, each of which contains a sound field cloud map and a thermal image, and 306 abnormal samples are collected, of which 113 samples have both abnormal thermal images and sound field cloud maps, 101 samples have only abnormal thermal images, and 102 samples have only abnormal sound field cloud maps; It should be noted that the preprocessing in S1 includes the following sub-steps: S11, converting the three-channel RGB image pair consisting of the collected thermal image and the acoustic field cloud image of the normal operation status of the power communication room equipment into a single-channel grayscale image pair by using a weighted average method, wherein the calculation formula of the weighted average is as follows: in Represents the pixel positions respectively. The gray value, red channel value, green channel value, and blue channel value after weighted average at Respectively represent the weighting coefficients of the three color channels. In this embodiment Take 0.3, Take 0.59, Take 0.11; S12, stretching the grayscale range of the single-channel grayscale image pair using a piecewise linear grayscale transformation method to obtain a grayscale image pair with prominent contour features, wherein the calculation formula of the piecewise linear grayscale transformation is as follows: in, and The pixel positions are The transformed grayscale value and the original grayscale value, ( )and( ) are two nodes used to divide the grayscale value into three segments; S13, using a Gaussian filter to denoise the grayscale image pair with prominent contour features.

[0024] S2, offline calibration of infrared thermal imager and acoustic camera; It should be noted that the offline calibration in S2 includes the following sub-steps: S21, making a rectangular wooden calibration board with an array of buzzers arranged equidistantly. In this embodiment, the specification of the wooden calibration board is 40 cm 40cm, place a buzzer that generates heat when working every 10cm, a total of 16 buzzers; S22, using an infrared thermal imager and an acoustic camera to collect image pairs consisting of thermal images and acoustic field cloud images of the calibration plate at different positions when the buzzer array is working. In this embodiment, the positions of the infrared thermal imager and the acoustic camera remain unchanged. By changing the placement position of the calibration plate, image pairs consisting of thermal images and acoustic field cloud images of the calibration plate at different positions are obtained, and at least three image pairs with different positions are obtained; S23, performing the preprocessing step on the image pair consisting of the thermal image and the acoustic field cloud image of the calibration plate in different positions; S24, processing the grayscale image pair of the prominent contour features after denoising into a binary image based on a threshold value to obtain an image pair segmenting a temperature region and a sound field region; S25, detecting and locating the calibration points using image moments according to the grayscale values ​​of the temperature region and the sound field region obtained by segmentation, wherein the calculation formula of the image moment is as follows: in, is the x-coordinate of the image moment of the segmented area, is the y coordinate of the image moment of the segmented area, that is, the pixel coordinate of the image moment in the area, is the gray value of the current point; S26, using the calibration point collinearity detection algorithm to sort the calibration points of the multi-frame images at different angles. The specific process of the calibration point collinearity detection algorithm used in this embodiment is as follows: first, select any point P1 among all the calibration points, iteratively calculate the point P2 closest to P1, the distance between P1 and P2 is d, draw a circular area with a radius of 1.5d, and bring all the points in the circular area into the following formula one by one: Will The point is determined as the point that is collinear with P1 and P2, all the calibration points are traversed to obtain the position relationship, and all the calibration points are sorted according to the position relationship; S27, calculate the internal parameter matrix of the infrared thermal imager based on the coordinates of the calibration points after positioning and sorting and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix In this embodiment, the distortion coefficient of the camera is not considered. The intrinsic parameter matrix and the extrinsic parameter matrix are solved according to the mapping relationship between the world coordinate system and the pixel coordinate system of the calibration plate. The conversion formula between the pixel coordinate system and the world coordinate system is as follows: Among them, (u, v) represents the coordinates in the pixel coordinate system, s represents the scale factor, represents the camera intrinsic parameter matrix, represents the camera external parameters, ( ) represents the world coordinate system, and the calibration plate is located in the world coordinate system plane.

[0025] S3, registering and fusing the preprocessed image pairs to obtain a fused image; It should be noted that the registration in S3 includes the following sub-steps: S31, determine the viewing angle range of the infrared thermal imager and the acoustic camera; S32, according to the internal parameter matrix of the infrared thermal imager and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix Calculate the homography matrix of an image with a preset viewing angle range (smaller viewing angle range) transformed into the coordinate system of another image. The conversion process is as follows: in, is the pixel coordinate after conversion, is the coordinate of the pixel before transformation, is the scale factor; In this embodiment, the viewing angle range of the infrared thermal imager is ( , ), the visual range of the acoustic camera is ( , ), so in this embodiment, the image with a smaller viewing angle range is an infrared thermal image, and the image with a larger viewing angle range is an acoustic field cloud image. The image with a preset viewing angle range in this embodiment refers to an image with a smaller viewing angle range.

[0026] S33, traverse all pixels of the image in a preset viewing angle range (smaller viewing angle range), transform them to the viewing angle of an image in another preset viewing angle range (larger viewing angle range) using a homography matrix, and fill in blank pixels using an interpolation method when some pixels have no corresponding pixels.

[0027] The viewing angle of an image in another preset viewing angle range here refers to the viewing angle of an image with a larger viewing angle range; It should be noted that the fusion in S3 uses a pixel weighted average fusion method to linearly weighted average the grayscale values ​​of the pixels of the thermal image and the acoustic field cloud image obtained by the alignment, wherein the pixel weighted average fusion formula is as follows: in, The pixel position is The pixel value after fusion is and The pixel positions of thermal image and sound field cloud map are The pixel value at and is the weighting coefficient. In this embodiment Take 0.5, Take 0.5.

[0028] S4, build a model for equipment fault monitoring and fault area location in the power communication room.

[0029] It should be noted that building an unsupervised learning fault monitoring and fault area location model in S4 includes the following sub-steps: S41, using a pre-trained feature extractor to extract the low-level visual features of the normal operating status of the registered power communication room equipment, the thermal image, the sound field cloud image and the fused image, and perform L2 norm normalization. The pre-trained feature extractor used in this embodiment is a Resnet network, and the dimension of the first layer output of the Resnet network is The feature map of is used as the low-level visual feature, where is a number, C is the dimension; S42, using a non-parametric clustering algorithm to take the mean vector of each cluster as a normal prototype to obtain a normal thermal image feature prototype, a normal sound field cloud image feature prototype and a normal fusion feature prototype. The non-parametric clustering algorithm used in this embodiment is the FINCH clustering algorithm, and the dimension can be obtained. C prototype, where K is the number of clusters and C is the dimension; S43, in the monitoring stage, real-time acquisition of thermal images and sound field cloud images is performed, and the preprocessing step and the registration step are performed to obtain low-level visual features of real-time thermal images, real-time sound field cloud images, and real-time fusion images, and L2 norm normalization is performed; S44, respectively taking the normal thermal image feature prototype, the normal sound field cloud image feature prototype and the normal fusion feature prototype normalized by L2 norm as the kernel parameters of 1x1 convolution and inner products with the corresponding low-level visual features of the real-time thermal image, the low-level visual features of the real-time sound field cloud image and the low-level visual features of the real-time fusion image; S45, applying the inner product result to the channel maximum pooling to obtain the similarity between the normal thermal image feature prototype and the real-time thermal image, the normal sound field cloud image feature prototype and the real-time sound field cloud image, and the normal fusion feature prototype and the real-time fusion image at each position pixel, wherein the similarity calculation formula is as follows: in, and Represent the similarity and normalized features at the pixel position (i, j), is the kth prototype vector, and cos is the cosine similarity.

[0030] S46, setting a fault score threshold, when the similarity is lower than the fault score threshold, the pixel position is determined as a fault point, and if there is a fault point, it is determined that there is a fault area in the real-time image; S5, accurately locate the fault point in three dimensions according to the fault area of ​​thermal image, fault area of ​​acoustic field cloud image and fault area of ​​fusion image; It should be noted that the accurate three-dimensional positioning of the fault point in S5 includes the following sub-steps: S51, compare the average similarity of the fault area, determine the accurate two-dimensional fault point criterion, and calculate the accurate two-dimensional fault point using the image moment, wherein the calculation formula of the image moment is as follows: in, is the x-coordinate of the image moment of the segmented area, is the y coordinate of the image moment of the segmented area, that is, the pixel coordinate of the image moment in the area, is the gray value of the current point; In this embodiment, when the power communication room equipment fails, there are three situations: only the temperature is abnormal, only the operating sound is abnormal, or both the temperature and the operating sound are abnormal. Therefore, the fault area includes only the abnormal , only exists , only exists , coexist and , coexist and , coexist , and There are six situations in total, including , and They represent the real-time thermal image fault area, the real-time sound field cloud image fault area and the real-time fusion image fault area respectively. For the above six situations, the infrared thermal image, sound field cloud image and fusion image linkage control are realized to determine the judgment criteria of the two-dimensional fault point: If there is only a single type of fault area, that is, only , only exists and only exist In the third case, the fault area is directly used as the criterion for the two-dimensional fault point; If there are two or more types of fault areas, that is, there are and , coexist and , coexist , and In the three cases, the average similarity of different types of fault areas is calculated, and the fault area with the smaller average similarity is used as the accurate two-dimensional fault point judgment criterion. The calculation of the average similarity of the fault area is as follows: Among them, S is the average similarity of the fault area Z, Represents the similarity at the pixel position (i, j), It represents the sum of similarities of all fault points in the fault area Z, and N represents the number of all fault points in the fault area.

[0031] The judgment criteria of two-dimensional fault points are determined by linking thermal images, sound field cloud maps and fused images. The fused image can accurately locate the fault point by combining the temperature information and sound information of the equipment fault, and link single fault information at the same time to avoid the situation where the fused image fault area judgment process fails to reach the threshold and is judged as a normal point when only the temperature or only the sound is abnormal. Therefore, it is generally not possible to use the fused image as the basis for judging the fault area.

[0032] S52, based on the offline calibration result, the precise two-dimensional fault point is mapped to the three-dimensional space. In this embodiment, the mapping process is as follows: Among them, (u,v) is the determined accurate two-dimensional fault point, and are scale factor, internal parameter matrix and external parameter matrix respectively. In this embodiment, and The specific value of depends on the criterion of the precise two-dimensional fault point. is the coordinate of the fault point in the world coordinate system. In this embodiment, the power communication room equipment is placed on the same plane, and the placement height of the power equipment is not involved, so the world coordinate system is not considered. The value of .

[0033] refer to Figure 3 , Figure 3 It is a comparison chart of ROC curves. The chart compares the ROC curves of monitoring using only thermal images, monitoring using only acoustic field cloud images, monitoring using fusion images, and monitoring using the three in combination. Among them, the ROC curve area of ​​the three-way linkage monitoring is the largest, indicating that the abnormal monitoring effect of the linkage monitoring is the best, which improves the accuracy of fault location.

[0034] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for monitoring and 3D positioning equipment faults in power communication rooms using infrared thermal imaging and acoustic imaging, characterized in that: The following steps are involved: S1, collecting and preprocessing the image pairs consisting of thermal images and acoustic field cloud images of the normal operating status of the equipment in the power communication room; S2, offline calibration of infrared thermal imager and acoustic camera; S3, registering and fusing the preprocessed image pairs to obtain a fused image; S4, build a model for equipment fault monitoring and fault area location in the power communication room; S5, three-dimensionally locate the fault point according to the thermal image fault area, the acoustic field cloud image fault area, and the fusion image fault area.

2. The method for monitoring and three-dimensional positioning of equipment faults in power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The pre-processing in step S1 comprises the following steps: The image pairs are converted into single-channel grayscale image pairs by using weighted averaging method; The grayscale range of the single-channel grayscale image pair is stretched using a piecewise linear grayscale transformation method to obtain a grayscale image pair with prominent contour features; Gaussian filter is used to denoise the grayscale image pairs that highlight contour features.

3. The method for monitoring and three-dimensional positioning of equipment faults in a power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The offline calibration in S2 includes the following steps: Make a rectangular wooden calibration board with an array of equally spaced buzzers; An infrared thermal imager and an acoustic camera are used to collect image pairs consisting of thermal images and acoustic field cloud images of the calibration plate at different positions when the buzzer array is working; The preprocessing step is performed on the image pairs consisting of the thermal image and the acoustic field cloud image of the calibration plate in different positions; Based on the threshold, the grayscale image pair with prominent contour features after denoising is processed into a binary image to obtain an image pair with a temperature area and a sound field area segmented; According to the grayscale values ​​of the temperature area and the sound field area obtained by segmentation, the calibration points are detected and located using the image moment; The calibration points of multiple frames of images at different angles are sorted using the calibration point co-linearity detection algorithm; Calculate the intrinsic parameter matrix of the infrared thermal imager based on the coordinates of the located and sorted calibration points and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix .

4. The method for monitoring and three-dimensional positioning of equipment faults in power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The registration in S3 includes the following steps: Determine the viewing angles of thermal imaging cameras and acoustic cameras; According to the internal parameter matrix of the infrared thermal imager and the external parameter matrix And the intrinsic parameter matrix of the acoustic camera and the external parameter matrix Calculate the homography matrix for transforming an image within a preset viewing angle range into a coordinate system of another image; All pixels of an image within a preset viewing angle range are traversed, and the pixels are transformed to the viewing angle of an image within another preset viewing angle range using a homography matrix.

5. The method for monitoring and three-dimensional positioning of equipment faults in a power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The fusion in S3 uses a pixel weighted average fusion method to perform linear weighted average on the grayscale values ​​of the pixels of the thermal image and the acoustic field cloud image obtained by the alignment.

6. The method for monitoring and three-dimensional positioning of equipment faults in a power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The construction of the unsupervised learning fault monitoring and fault area location model in S4 specifically includes the following steps: The pre-trained feature extractor is used to extract the low-level visual features of the registered thermal image, sound field cloud image and fusion image of the normal operation status of the power communication room equipment and perform L2 norm normalization. Using non-parametric clustering algorithm, the mean vector of each cluster is used as the normal prototype to obtain the normal thermal image feature prototype, the normal sound field cloud image feature prototype and the normal fusion feature prototype; In the monitoring stage, thermal images and sound field cloud images are collected in real time, and the preprocessing step and the registration step are adopted to obtain low-level visual features of the real-time thermal image, the real-time sound field cloud image and the real-time fusion image and perform L2 norm normalization; The normal thermal image feature prototype, the normal sound field cloud image feature prototype and the normal fusion feature prototype normalized by L2 norm are respectively used as the kernel parameters of 1x1 convolution and the inner product of the corresponding low-level visual features of the real-time thermal image, the low-level visual features of the real-time sound field cloud image and the low-level visual features of the real-time fusion image; The inner product result is applied to the channel maximum pooling to obtain the similarity between the pixels at each position in the normal thermal image feature prototype and the real-time thermal image, the normal sound field cloud image feature prototype and the real-time sound field cloud image, and the normal fusion feature prototype and the real-time fusion image; A fault score threshold is set. When the similarity is lower than the threshold, the pixel position is determined as a fault point. If a fault point exists, it is determined that there is a fault area in the real-time image.

7. The method for monitoring and three-dimensional positioning of equipment faults in power communication room by combining infrared thermal imaging with acoustic imaging according to claim 1 is characterized in that: The three-dimensional positioning of the fault point in S5 includes the following steps: Determine a two-dimensional fault point criterion, and calculate the two-dimensional fault point using the image moment; Based on the result of the offline calibration, the two-dimensional fault point is mapped to the three-dimensional space.

8. A power communication room equipment fault monitoring and three-dimensional positioning system that combines infrared thermal imaging with acoustic imaging, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images