A power equipment image processing method based on fiber bundle imaging, a storage medium and a system

Abstract

A kind of power equipment image processing method based on optical fiber bundle imaging, storage medium and system, method is: signal output by infrared fiber bundle is collected by infrared camera pixel matrix, unit in matrix is divided into effective unit and invalid unit, and generate unit division binary matrix data file;Again processing obtains temperature correction matrix data file, temperature segmented point corresponding response value matrix data file, and establish the correction value of response value matrix under different radiation scene;Through numerical fitting and interpolation compensation method, the video stream of optical fiber bundle transmission image is carried out real-time restoration and optimization, completes the inversion from thermal map to temperature field distribution;The present application also includes the storage medium and system of storing the power equipment image processing method based on optical fiber bundle imaging of running this, the present application carries out real-time restoration and optimization to the video stream of optical fiber bundle transmission image by numerical fitting and interpolation compensation method, completes the inversion from thermal map to temperature field distribution.

Description

Technical Field

[0001] This invention belongs to the field of online monitoring technology for power equipment, and specifically relates to a power equipment image processing method, storage medium, and system based on fiber optic bundle imaging. Background Technology

[0002] During the operation of power equipment, thermal signals constantly change. Monitoring equipment based on infrared flexible fiber optic bundles can comprehensively and in real-time acquire the dynamic characteristics of these thermal signals, enabling accurate assessment of the power equipment's operating status. The fiber optic bundle structure is used because it separates photoelectric signals, keeping the core of the detection equipment that performs photoelectric signal conversion away from the power equipment and maintaining a safe, insulated distance. A fiber optic bundle consists of a bundle of optical fibers. Due to manufacturing limitations, these fiber cores have varying sizes and shapes; the cores respond slightly differently to the same signal; and each fiber has a cladding that prevents signal transmission. Overall, gaps exist between the fibers, and the arrangement of all fibers is not entirely regular or orderly. These inherent characteristics of the fiber optic bundle result in a honeycomb-like grid in the final image, affecting the observation effect. Therefore, the grid should be suppressed or eliminated.

[0003] Current methods for processing honeycomb-like meshes include Fourier bandpass filtering, interpolation, inter-frame combination compensation, and intelligent algorithm correction such as neural networks. However, unlike fiber optic bundles used in other fields, the fiber optic bundles used in power equipment monitoring observe objects through a fixed window, and the relative position of the object and the window does not change frequently. Therefore, inter-frame combination compensation, which uses object movement to fill gaps, cannot be used. Moreover, the infrared energy radiated by the equipment needs to be mapped into temperature information, so the pixel response values ​​of the fiber core must be accurate and reliable. However, filtering methods would cause changes in this data, making them unsuitable. In summary, interpolation is the most reasonable and accurate method for image correction. The patent application CN117710250 A, entitled "A Method for Eliminating Honeycomb Structures in Fiber Optic Imaging," uses interpolation to filter images. This method is designed for endoscope systems and does not form fixed reference nodes and node weights at the interpolation points. It cannot meet the needs of high-speed acquisition and analysis of video frames during power equipment faults, nor does it match the application scenario of power equipment temperature field inversion. The paper "Research on Notch Filtering Elimination Method for Fiber Optic Bundle Endoscopic Imaging Grids" proposed a method to eliminate honeycomb-like grids using notch filters. However, while filtering removes noise, it also disturbs non-noise signals. Although the filtered image has good visual effects, it cannot meet the high requirements for accuracy and realism of key signals when retrieving temperature from image signals, thus leading to a decrease in the accuracy of temperature calculation. Therefore, this paper proposes an image processing method, storage medium, and system for power equipment based on fiber optic bundle imaging. This has significant guiding significance for the application of fiber optic bundles in the field of power equipment temperature measurement and the exploration of new operating characteristics of power equipment. Summary of the Invention

[0004] In order to overcome the problems existing in the prior art, the present invention aims to provide an image processing method, storage medium and system for power equipment based on fiber optic bundle imaging. Through numerical fitting and interpolation compensation methods, the video stream of the fiber optic bundle transmitted image is restored and optimized in real time, and the inversion from thermal image to temperature field distribution is completed, realizing a breakthrough in the fusion of fiber optic temperature measurement and fiber optic bundle imaging.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An image processing method for power equipment based on fiber optic bundle imaging includes the following steps:

[0007] (1) The signal output by the infrared fiber bundle is acquired by the pixel matrix of the focal plane of the infrared camera. Each pixel corresponds to a response value. Therefore, the pixel matrix is ​​mapped to the response value matrix. The units in the matrix are divided into valid units and invalid units, and a binary matrix data file of unit division is generated.

[0008] (2) Establish the mapping relationship between effective unit values ​​and heat source temperature, obtain temperature correction matrix data file and temperature segment point corresponding response value matrix data file, and establish the correction value of response value matrix under different radiation scenarios;

[0009] (3) The effective unit is the node to be referenced, and the invalid unit is the interpolation point. The invalid unit value is interpolated and compensated based on the effective unit value to obtain a cell array data file that stores the reference nodes and weights of each interpolation point.

[0010] (4) The loading unit divides the binary matrix data file into a binary matrix, the temperature correction matrix data file into a temperature correction matrix, the temperature segment point corresponding response value matrix data file into a segment point matrix, and the cell array data file into a cell array.

[0011] (5) Read the camera video stream. The response value matrix of each frame of the video stream is used as an acquisition matrix. Call the correction value, binary matrix, temperature correction matrix, segment point matrix and cell array under different radiation scenarios, and perform matrix operations with the acquisition matrix to obtain the temperature matrix that reflects the temperature field distribution of the camera observation scene.

[0012] (6) Assign the temperature matrix to the image matrix of the same spatial size. The image is displayed as a grayscale image. By stretching the image pixel values ​​and optimizing the image quality to meet the visual needs of the human eye, the video display is completed.

[0013] Step (1) specifically includes:

[0014] (1.1) Using a blackbody as the standard heat source, the high and low temperature planes of the blackbody are photographed by a camera to obtain the response value matrices at high and low temperatures. The difference matrix is ​​obtained by subtracting the corresponding unit values ​​of the two matrices.

[0015] (1.2) Threshold segmentation is performed on the difference matrix to generate a binary matrix, where the cell with a value of 1 is the valid cell corresponding to the fiber core, and the cell with a value of 0 is the invalid cell corresponding to the cladding and the gap.

[0016] Step (2) establishes the mapping relationship between effective unit values ​​and heat source temperatures, including:

[0017] (2.1) At the same temperature interval, the temperature of the surface source blackbody is increased from low temperature to high temperature, and the response value matrix at each temperature point is collected;

[0018] (2.2) Calculate the mean value of the effective unit values ​​in each response value matrix, and use it as the reference value for heat source temperature mapping;

[0019] (2.3) If the reference value increases linearly with the temperature of the heat source, the two-point correction method is used to establish the mapping function between the temperature and the reference value. If it increases nonlinearly, the temperature is piecewise adaptively fitted and the response matrix values ​​of the piecewise points are recorded.

[0020] (2.4) To ensure that all effective elements have the same value at the same heat source temperature, the effective element values ​​of the response value matrix are non-uniformly corrected in each temperature segment to obtain a correction matrix with the same spatial size as the response value matrix.

[0021] (2.5) Merge the correction matrix and the mapping function to establish a temperature correction matrix with the same spatial size as the response value matrix, where the cells with the same coordinates as the invalid cells of the response value matrix are assigned a value of 0.

[0022] Step (2) establishes the correction values ​​of the response value matrix under different radiation scenarios, including:

[0023] (2.6) Collect response value matrices for different radiating surfaces and calculate surface emissivity;

[0024] (2.7) Collect response value matrices at different background temperatures and calculate the background radiation influence value;

[0025] (2.8) Collect the response value matrix for different radiation propagation media and calculate the media attenuation rate.

[0026] Step (3) involves interpolating and compensating invalid cell values ​​based on valid cell values, including:

[0027] (3.1) Window the region near the interpolation point of the response value matrix;

[0028] (3.2) When the fiber core corresponds to an effective unit, extract the maximum value of each effective unit region in the response value matrix as the node to be referenced, and draw the first Thiessen polygon map based on the node to be referenced in the window and the second Thiessen polygon map after the interpolation points are added.

[0029] (3.3) Calculate the correlation ratio between the polygon where the interpolation point is located in the second image and the polygons it covers in the first image. Therefore, the referenced nodes in the covered polygons are the referenced nodes of the interpolation point, and the correlation ratio is the weight of the referenced nodes.

[0030] (3.4) When the fiber core corresponds to multiple valid cells, the interpolation point is interpolated using inverse distance weighting based on the reference node at the edge of the invalid cell region within the window;

[0031] (3.5) Define a 0 matrix with the same size as the window, assign the weights of the nodes used for the interpolation points to the corresponding cells of the 0 matrix, and establish a weight matrix for each interpolation point;

[0032] (3.6) Establish a cell array with the same spatial size as the response value matrix, and store the weight matrix of each interpolation point into a cell with the same coordinates as the interpolation point.

[0033] Step (5) involves performing matrix operations with the acquisition matrix, including:

[0034] (5.1) Correct the values ​​of each element in the acquisition matrix based on the radiation scene correction value;

[0035] (5.2) Perform the Hadamard product operation on the binary matrix and the acquisition matrix to obtain the new acquisition matrix after filtering;

[0036] (5.3) Traverse the valid cell coordinates of the new acquisition matrix where the value is not 0. Based on the cell value under the corresponding coordinate of the segmented matrix, determine the temperature segment where the cell value is located. Extract the corresponding coordinate cell value from the temperature correction matrix under the corresponding temperature segment and cover the cell value under the corresponding coordinate in the 0 matrix to obtain the new temperature correction matrix.

[0037] (5.4) Using matrix calculation, the valid cell values ​​in the new acquisition matrix are corrected to accurate temperature values ​​based on the new temperature correction matrix, and the invalid cell values ​​are 0;

[0038] (5.5) At each invalid cell in the new acquisition matrix, open a window in the new acquisition matrix. The size of the window is the same as the matrix space in each cell of the cell array to obtain the local new acquisition matrix.

[0039] (5.6) Allocate a GPU thread for each invalid cell, read in the cell array the weight matrix and the local new acquisition matrix corresponding to the invalid cell, obtain the matrix after the Hadamard product operation, and sum all the cell values ​​to get the invalid cell temperature value.

[0040] (5.7) Make the temperature values ​​of each invalid cell cover the corresponding cell values ​​in the newly acquired matrix to obtain the temperature matrix.

[0041] A computer-readable storage medium for power equipment image processing based on fiber optic bundle imaging stores a computer program. When the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the aforementioned power equipment image processing method based on fiber optic bundle imaging.

[0042] A power equipment image processing system based on fiber optic bundle imaging includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the power equipment image processing method based on fiber optic bundle imaging.

[0043] The beneficial effects of this invention are:

[0044] 1. Step (1) of this invention performs threshold segmentation based on the difference matrix, which reduces the bias caused by various complex factors in scene radiation compared to directly distinguishing valid and invalid units from the response value matrix. Threshold segmentation is based on grayscale histogram, which is more objective and accurate than manually selecting the grayscale value segmentation interface.

[0045] 2. Steps (2.2) to (2.4) of this invention calculate the average value and perform non-uniformity correction based on the extracted effective cell values ​​in the response value matrix. Previous infrared thermal imagers did not have fiber optic bundle structures, therefore all pixel responses were effective values. This step additionally considers the influence of the honeycomb grid caused by the fiber optic bundle, making the calculation process more consistent with the special optical path structure of the monitoring equipment, thus helping to improve the accuracy of the reference value.

[0046] 3. Steps (2.6) to (2.8) of the present invention divide the radiation effects in the scene into three categories and establish correction values ​​for each category; therefore, the application of correction values ​​is more flexible in the face of scene changes, effectively improving the calculation accuracy of heat source temperature values ​​inverted from response values.

[0047] 4. Step (3) of the present invention takes into account the coupling relationship between different fiber bundles and the focal plane pixels of the core, and selects two methods based on Thiessen polygons and inverse distance interpolation, making the image processing method more universal and more practical.

[0048] 5. Step (3.2) of the present invention adopts an interpolation calculation method based on Thiessen polygons, which is different from commonly used filtering, neural network and other honeycomb grid processing methods. The interpolation method can ensure that the effective unit value is not changed and maximize the preservation of the authenticity of the key values ​​of the scene radiation signal.

[0049] 6. In step (5.6) of this invention, a GPU thread is allocated for the interpolation calculation of each invalid unit. Previous video interpolation research has mostly focused on inserting a frame between two frames to make the video more coherent, while the optimization of each frame image has mostly adopted filtering or neural network methods. However, while filtering out noise signals, this method also blurs the key information of the effective signal. Therefore, for the special needs of image processing of power equipment based on fiber bundle imaging, this step flexibly applies GPU acceleration to the field of video image optimization, providing new possibilities for improving the speed of video image processing by interpolation.

[0050] In summary, this invention uses numerical fitting and interpolation compensation methods to restore and optimize the video stream of the fiber optic bundle transmission image in real time, completing the inversion from the thermal map to the temperature field distribution. It achieves a breakthrough in the fusion of fiber optic temperature measurement and fiber optic bundle imaging, filling the gap in fiber optic bundle image-based temperature measurement but lacking data processing methods, and providing a brand-new solution for physical field measurement and operational status analysis of power equipment. Attached Figure Description

[0051] Figure 1 A flowchart of an image processing method for power equipment based on fiber optic bundle imaging.

[0052] Figure 2 This is a schematic diagram of the non-uniformity correction technique for the response value matrix.

[0053] Figure 3 A schematic diagram illustrating the process of grayscale stretching and image quality optimization for an image matrix. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention provides, for example Figure 1 The illustrated method for image processing of power equipment based on fiber optic bundle imaging includes the following steps:

[0056] (1) The signal output by the infrared fiber bundle is acquired by the pixel matrix of the focal plane of the infrared camera. Each pixel corresponds to a response value. Therefore, the pixel matrix can be mapped to the response value matrix. The units in the matrix are divided into valid units and invalid units, and a binary matrix data file of unit division is generated.

[0057] Specifically, due to the gaps between the fiber core, cladding, and fibers within the fiber bundle, these gaps cannot effectively transmit optical signals. Therefore, when the signal is coupled to the focal plane of the camera core via the fiber output, it appears as a honeycomb grid. The response values ​​of the corresponding pixels cannot be accurately quantified. Thus, it is necessary to identify the effective cells corresponding to each fiber core and the ineffective cells corresponding to the cladding and gaps in the response value matrix mapped by the camera pixel matrix, and to perform subsequent data processing based on the values ​​of the effective cells.

[0058] (1.1) Using a blackbody as the standard heat source, the high and low temperature planes of the blackbody are photographed by a camera to obtain the response value matrices at high and low temperatures. The difference matrix is ​​obtained by subtracting the corresponding unit values ​​of the two matrices.

[0059] Specifically, using a surface-source blackbody as a standard uniform surface heat source, high-temperature T samples were collected respectively. H Low temperature T L The camera response matrix S in the plane H and S L And perform matrix subtraction ΔS = S H -S L .

[0060] (1.2) Threshold segmentation is performed on the difference matrix to generate a binary matrix, where the cell with a value of 1 is the effective cell corresponding to the fiber core, and the cell with a value of 0 is the invalid cell corresponding to the cladding and the gap.

[0061] Specifically, if the fiber bundle is in an ideal state, the optical signals output from the slots and cladding should not change with the temperature of the heat source. Therefore, compared to the effective cell values ​​corresponding to the fiber core, the changes in the ineffective cell values ​​corresponding to the slots and cladding are very small. Thresholding the cell values ​​in the difference matrix allows for direct cell classification and the generation of a binary matrix B, where effective cell values ​​are 1 and ineffective cell values ​​are 0.

[0062] (2) Establish the mapping relationship between effective unit values ​​and heat source temperature, obtain temperature correction matrix data file and temperature segment point corresponding response value matrix data file, and establish the correction value of response value matrix under different radiation scenarios;

[0063] Specifically, the purpose of collecting infrared signals is to invert the temperature field inside the power equipment in order to assess the equipment's operating status in a timely manner. Therefore, it is necessary to establish a mapping relationship between effective unit values ​​and heat source temperatures. The process for establishing this relationship is as follows: Figure 2 As shown

[0064] (2.1) At the same temperature interval, the temperature of the surface source blackbody is increased from low temperature to high temperature, and the response value matrix at each temperature point is collected;

[0065] Specifically, for example, the surface source blackbody temperature can be increased sequentially from low to high in 5°C intervals within the observed target temperature range, and then the response value matrix at each temperature point can be collected for subsequent data analysis.

[0066] (2.2) Calculate the mean value of the effective unit values ​​in each response value matrix, and use it as the reference value for heat source temperature mapping;

[0067] Specifically, since the reference value for heat source temperature mapping varies with the optical system and the observation scene, there is no unified standard. Therefore, we must first ignore the influence of complex factors and calculate the mean value of the effective element value of the response value matrix generated by the surface source blackbody as the reference value for heat source temperature mapping. for:

[0068]

[0069] In the formula, s ij The values ​​of each element in the response value matrix are given, and the matrix space is m rows and n columns.

[0070] (2.3) If the reference value increases linearly with the temperature of the heat source, the two-point correction method is used to establish the mapping function between the temperature and the reference value. If it increases nonlinearly, the temperature is piecewise adaptively fitted and the response matrix values ​​of the piecewise points are recorded.

[0071] Specifically, since the unit values ​​of the response value matrix mapped after the optical signal is transmitted through the optical system and converted into photoelectric signals by the mechanism do not strictly increase linearly with temperature, and using complex fitting would reduce the processing speed of the video stream, piecewise linear fitting can be used to ensure both calculation accuracy and speed. Taking the two-point correction method to establish the mapping function between temperature and the reference response value as an example, the known heat source temperature T corresponding to the reference value is:

[0072]

[0073] In the formula, a is the mapping slope, and b is the mapping intercept. Therefore:

[0074]

[0075]

[0076] In the formula, T H T represents the temperature at which the heat source reaches a high temperature. L This refers to the temperature at which the heat source is at a low temperature.

[0077] (2.4) To ensure that all effective elements have the same value at the same heat source temperature, the effective element values ​​of the response value matrix are non-uniformly corrected in each temperature segment to obtain a correction matrix with the same spatial size as the response value matrix.

[0078] The non-uniformity of the response value matrix of the infrared focal plane pixel matrix mapping of the camera mechanism manifests as both additive noise imbalance and multiplicative noise response non-uniformity. Even when all pixels are input with the same signal and the same voltage, the values ​​of each element in the response matrix are not completely consistent, resulting in an image with non-uniform grayscale values. Therefore, non-uniformity correction is necessary. Within a certain range, the noise and element values ​​can be considered to have a linear relationship; therefore, two-point correction is taken as an example. The correction function for each element is:

[0079]

[0080] In the formula, a i ′ j ′ represents the correction slope, b i ′ j ′ represents the correction slope. Therefore:

[0081]

[0082]

[0083] In the formula, is These are the reference values ​​for high and low temperatures of the heat source, s Hij s Lij These represent the values ​​of each element in the response matrix at high and low temperatures of the heat source.

[0084] (2.5) Merge the correction matrix and the mapping function to establish a temperature correction matrix with the same spatial size as the response value matrix, where the cells with the same coordinates as the invalid cells of the response value matrix are assigned a value of 0;

[0085] Specifically, the temperature correction matrix ST is:

[0086] ST = A⊙S + B

[0087] In the formula, A is the slope matrix and B is the intercept matrix.

[0088] (2.6) Collect response value matrices for different radiating surfaces and calculate surface emissivity;

[0089] (2.7) Collect response value matrices at different background temperatures and calculate the background radiation influence value;

[0090] (2.8) Collect response value matrices for different radiation propagation media and calculate the media attenuation rate;

[0091] (3) The effective unit is the node to be referenced, and the invalid unit is the interpolation point. The invalid unit value is interpolated and compensated based on the effective unit value to obtain a cell array data file that stores the reference nodes and weights of each interpolation point.

[0092] (3.1) Window the region near the interpolation point of the response value matrix;

[0093] Specifically, by opening a window in the vicinity of the interpolation point of the response value matrix and interpolating within a small neighboring region, the amount of interpolation calculation can be effectively reduced.

[0094] (3.2) When the fiber core corresponds to an effective unit, extract the maximum value of each effective unit region in the response value matrix as the node to be referenced, and draw the first Thiessen polygon map based on the node to be referenced in the window and the second Thiessen polygon map after the interpolation points are added.

[0095] (3.3) Calculate the correlation ratio between the polygon where the interpolation point is located in the second image and the polygons it covers in the first image. Therefore, the referenced nodes in the covered polygons are the referenced nodes of the interpolation point, and the correlation ratio is the weight of the referenced nodes.

[0096] Specifically, when the fiber core corresponds to one pixel, and the effective cell regions are uniformly distributed, the cell with the highest value in the region corresponds to the fiber core. The maximum value in all effective cell regions of the response value matrix is ​​extracted as the node to be referenced during interpolation. The interpolation point calculation methods can be divided into two types: interpolation by establishing a fitting function and interpolation by assigning weights to neighboring nodes. Since the temperature field distribution of the observed target is dynamically changing, a fixed fitting function is not applicable; therefore, the interpolation method by assigning weights to neighboring nodes is adopted.

[0097] (3.4) When the fiber core corresponds to multiple valid cells, the interpolation point is interpolated using inverse distance weighting based on the reference node at the edge of the invalid cell region within the window;

[0098] (3.5) Define a 0 matrix with the same size as the window, assign the weights of the nodes used for the interpolation points to the corresponding cells of the 0 matrix, and establish a weight matrix for each interpolation point;

[0099] Specifically, the weight matrix is ​​W ij .

[0100] (3.6) Establish a cell array with the same spatial size as the response value matrix, and store the weight matrix of each interpolation point into a cell with the same coordinates as the interpolation point;

[0101] Specifically, the cell array W m×n for:

[0102]

[0103] (4) Initialize the application program, load the unit partition binary matrix data file as a binary matrix, the temperature correction matrix data file as a temperature correction matrix, the temperature segment point corresponding response value matrix data file as a segment point matrix, and the cell array data file as a cell array;

[0104] Specifically, the temperature correction matrix is ​​ST, and the cell array is W.

[0105] (5) Read the camera video stream in the application. The response value matrix of each frame of the video stream is used as a collection matrix. Call the correction value, binary matrix, temperature correction matrix, segment point matrix and cell array under different radiation scenarios, and perform matrix operations with the collection matrix to obtain the temperature matrix that reflects the temperature field distribution of the camera observation scene.

[0106] (5.1) Correct the values ​​of each element in the acquisition matrix based on the radiation scene correction value;

[0107] (5.2) Perform the Hadamard product operation on the binary matrix and the acquisition matrix to obtain the new acquisition matrix after filtering;

[0108] (5.3) Traverse the valid cell coordinates of the new acquisition matrix where the value is not 0. Based on the cell value under the corresponding coordinate of the segmented matrix, determine the temperature segment where the cell value is located. Extract the corresponding coordinate cell value from the temperature correction matrix under the corresponding temperature segment and cover the cell value under the corresponding coordinate in the 0 matrix to obtain the new temperature correction matrix.

[0109] (5.4) Using matrix calculation, the valid cell values ​​in the new acquisition matrix are corrected to accurate temperature values ​​based on the new temperature correction matrix, and the invalid cell values ​​are 0;

[0110] Specifically, the effective unit temperature value is t ij .

[0111] (5.5) At each invalid cell in the new acquisition matrix, open a window in the new acquisition matrix. The size of the window is the same as the matrix space in each cell of the cell array to obtain the local new acquisition matrix.

[0112] Specifically, the local new acquisition matrix at each invalid cell is S. Pij , where i and j are the coordinates of the invalid unit in the new acquisition matrix.

[0113] (5.6) Allocate a GPU thread for each invalid cell, read in the cell array the weight matrix and the local new acquisition matrix corresponding to the invalid cell, obtain the matrix after the Hadamard product operation, and sum all the cell values ​​to get the invalid cell temperature value.

[0114] Specifically, to ensure continuous video display, at least 30 frames per second are typically required for real-time display. Therefore, video data processing speed needs to be carefully considered, and acceleration methods should be adopted to achieve real-time interpolation of the video stream.

[0115] The interpolation calculations for all invalid cell values ​​can be performed in parallel, so a GPU thread is allocated for each invalid cell. The matrix T after the Hadamard product of the weight matrix and the locally acquired matrix is... ij 'for:

[0116] T ij ′=W ij ⊙S Pij

[0117] After that, for T ij Summing all element values ​​yields the temperature value of the invalid element, i.e.:

[0118]

[0119] (5.7) Make the temperature values ​​of each invalid cell cover the corresponding cell values ​​in the newly acquired matrix to obtain the temperature matrix.

[0120] (6) Assign the temperature matrix to the image matrix of the same spatial size. The image is displayed as a grayscale image. By stretching the image pixel values ​​and optimizing the image quality to meet the visual needs of the human eye, the video display is completed.

[0121] Specifically, the pixel value stretching and image quality optimization process is as follows: Figure 3 As shown. First, the values ​​p of each unit in the image matrix are... ij After normalizing the stretch, we have:

[0122]

[0123] In the formula, p max p min These are the maximum and minimum unit values ​​in the current image matrix, respectively.

[0124] Because the human eye is more sensitive to low light, gamma correction is still used for LDR. ij By stretching the image to conform to human visual perception, we obtain:

[0125]

[0126] Finally, noise reduction and detail enhancement are performed on the stretched matrix to achieve image quality optimization.

[0127] The present invention also includes a computer-readable storage medium for power equipment image processing based on fiber optic bundle imaging, which stores a computer program. When the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the aforementioned power equipment image processing method based on fiber optic bundle imaging.

[0128] The present invention also includes a power equipment image processing system based on fiber optic bundle imaging, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the power equipment image processing method based on fiber optic bundle imaging.

[0129] In summary, this invention uses numerical fitting and interpolation compensation methods to restore and optimize the video stream of the fiber optic bundle transmission image in real time, completing the inversion from the thermal map to the temperature field distribution. It achieves a breakthrough in the fusion of fiber optic temperature measurement and fiber optic bundle imaging, filling the gap in fiber optic bundle image-based temperature measurement but lacking data processing methods, and providing a brand-new solution for physical field measurement and operational status analysis of power equipment.

Claims

1. A method for image processing of power equipment based on fiber optic bundle imaging, characterized in that, Includes the following steps: (1) The signal output by the infrared fiber bundle is acquired by the pixel matrix of the focal plane of the infrared camera. Each pixel corresponds to a response value. Therefore, the pixel matrix is ​​mapped to the response value matrix. The units in the matrix are divided into valid units and invalid units, and a binary matrix data file of unit division is generated. (2) Establish the mapping relationship between effective unit values ​​and heat source temperature, obtain temperature correction matrix data file and temperature segment point corresponding response value matrix data file, and establish the correction value of response value matrix under different radiation scenarios; (3) The valid cell is the node to be referenced, and the invalid cell is the interpolation point. The invalid cell value is interpolated and compensated based on the valid cell value to obtain a cell array data file that stores the referenced nodes and weights of each interpolation point; (4) The loading unit divides the binary matrix data file into a binary matrix, the temperature correction matrix data file into a temperature correction matrix, the temperature segment point corresponding response value matrix data file into a segment point matrix, and the cell array data file into a cell array; (5) Read the camera video stream. The response value matrix of each frame of the video stream is used as an acquisition matrix. Call the correction value, binary matrix, temperature correction matrix, segment point matrix and cell array under different radiation scenarios, and perform matrix operation with the acquisition matrix to obtain the temperature matrix that reflects the temperature field distribution of the camera observation scene. (6) Assign the temperature matrix to the image matrix of the same spatial size. The image is displayed as a grayscale image. By stretching the image pixel values ​​and optimizing the image quality to meet the visual needs of the human eye, the video display is completed. Step (5) involves matrix operations with the acquisition matrix, including: (5.1) Correct the values ​​of each element in the acquisition matrix based on the radiation scene correction value; (5.2) Perform the Hadamard product operation on the binary matrix and the acquisition matrix to obtain the new acquisition matrix after filtering; (5.3) Traverse the valid cell coordinates of the new acquisition matrix where the value is not 0. Based on the cell value under the corresponding coordinate of the segmented matrix, determine the temperature segment where the cell value is located. Extract the corresponding coordinate cell value from the temperature correction matrix under the corresponding temperature segment and cover the cell value under the corresponding coordinate in the 0 matrix to obtain the new temperature correction matrix. (5.4) Using matrix calculation, the valid cell values ​​in the new acquisition matrix are corrected to accurate temperature values ​​based on the new temperature correction matrix, and the invalid cell values ​​are 0; (5.5) At each invalid cell in the new acquisition matrix, open a window in the new acquisition matrix. The size of the window is the same as the matrix space in each cell of the cell array to obtain the local new acquisition matrix. (5.6) Allocate a GPU thread for each invalid cell, read in the cell array the weight matrix and the local new acquisition matrix corresponding to the invalid cell, obtain the matrix after the Hadamard product operation, and sum all the cell values ​​to get the invalid cell temperature value; (5.7) Make the temperature values ​​of each invalid cell cover the corresponding cell values ​​in the newly acquired matrix to obtain the temperature matrix.

2. The image processing method for power equipment based on fiber optic bundle imaging according to claim 1, characterized in that, Step (1) specifically includes: (1.1) Using a blackbody as the standard heat source, the high and low temperature planes of the blackbody are photographed by a camera to obtain the response value matrices at high and low temperatures. The difference matrix is ​​obtained by subtracting the corresponding unit values ​​of the two matrices. (1.2) Threshold segmentation is performed on the difference matrix to generate a binary matrix, where the cell with a value of 1 is the effective cell corresponding to the fiber core, and the cell with a value of 0 is the invalid cell corresponding to the cladding and the gap.

3. The image processing method for power equipment based on fiber optic bundle imaging according to claim 1, characterized in that, Step (2) establishes the mapping relationship between effective unit values ​​and heat source temperatures, including: (2.1) At the same temperature interval, the temperature of the surface source blackbody is increased from low temperature to high temperature, and the response value matrix at each temperature point is collected; (2.2) Calculate the mean value of the effective unit values ​​in each response value matrix, and use it as the reference value for heat source temperature mapping; (2.3) If the reference value increases linearly with the temperature of the heat source, the two-point correction method is used to establish the mapping function between the temperature and the reference value. If it increases nonlinearly, the temperature is piecewise adaptively fitted and the response matrix values ​​of the piecewise points are recorded. (2.4) In order to ensure that all effective elements have the same value at the same heat source temperature, the non-uniformity correction of the effective element values ​​of the response value matrix is ​​performed in each temperature segment to obtain a correction matrix with the same spatial size as the response value matrix. (2.5) Merge the correction matrix and the mapping function to establish a temperature correction matrix with the same spatial size as the response value matrix, where the cells with the same coordinates as the invalid cells of the response value matrix are assigned a value of 0.

4. The image processing method for power equipment based on fiber optic bundle imaging according to claim 1, characterized in that, Step (2) establishes the correction values ​​of the response value matrix under different radiation scenarios, including: (2.6) Collect response value matrices for different radiating surfaces and calculate the surface emissivity; (2.7) Collect response value matrices at different background temperatures and calculate the background radiation influence value; (2.8) Collect the response value matrix for different radiation propagation media and calculate the media attenuation rate.

5. The image processing method for power equipment based on fiber optic bundle imaging according to claim 1, characterized in that, Step (3) involves interpolating and compensating invalid cell values ​​based on valid cell values, including: (3.1) Window the region near the interpolation point of the response value matrix; (3.2) When the fiber core corresponds to an effective unit, extract the maximum value of each effective unit region in the response value matrix as the node to be referenced, and draw the first Thiessen polygon map based on the node to be referenced in the window and the second Thiessen polygon map after adding the interpolation points. (3.3) Calculate the correlation ratio between the polygon where the interpolation point is located in the second image and the polygons it covers in the first image. Therefore, the reference nodes in the covered polygons are the reference nodes of the interpolation point, and the correlation ratio is the weight of the reference nodes. (3.4) When the fiber core corresponds to multiple valid cells, the interpolation points are interpolated using inverse distance weighting based on the reference nodes at the edge of the invalid cell region within the window; (3.5) Define a 0 matrix with the same size as the window, assign the weights of the nodes used for the interpolation points to the corresponding cells of the 0 matrix, and establish a weight matrix for each interpolation point; (3.6) Establish a cell array with the same spatial size as the response value matrix, and store the weight matrix of each interpolation point into a cell with the same coordinates as the interpolation point.

6. A computer-readable storage medium for storing a computer program, characterized in that, When the computer program runs, it controls the device containing the computer-readable storage medium to execute the power equipment image processing method based on fiber optic bundle imaging as described in any one of claims 1-5.

7. A system, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the image processing method for power equipment based on fiber bundle imaging as described in any one of claims 1-5.

Images