Intelligent AR glasses infrared thermal image information processing method based on image processing technology

Abstract

The invention belongs to the technical field of infrared thermal image information processing, and particularly discloses an intelligent AR glasses infrared thermal image information processing method based on an image processing technology, and the method comprises the steps: collecting an infrared light image and a visible light image of target equipment, and then extracting parameters, angle correction information and pixel information of a fused image of the target equipment are obtained by analyzing parameters, so that the fused image is obtained, temperature early warning analysis of the target equipment is performed according to the temperature of each part of the target equipment in the fused image and the monitored external temperature, temperature early warning information of the target equipment is obtained, and corresponding feedback is performed at the same time; according to the method, the visible light image and the infrared light image of the target equipment are acquired through the AR glasses and are fused to obtain the fused image of the target equipment, and the image of the target equipment is comprehensively acquired in multiple dimensions, so that the accuracy of fused image acquisition of the target equipment is guaranteed, and the integrity of fused image acquisition of the target equipment is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of infrared thermal imaging information processing, and relates to an infrared thermal imaging information processing method for intelligent AR glasses based on image processing technology. Background Art

[0002] AR glasses infrared thermal imaging refers to the infrared thermal imaging technology integrated in augmented reality (AR) glasses. It can detect infrared radiation on the surface of the target object, convert it into a visual thermal image and present it on the display screen of the glasses. It can penetrate environmental interference and clearly display the temperature distribution of the target device or scene. In terms of troubleshooting of industrial equipment and inspection of power lines, it provides users with intuitive and critical temperature information with its non-contact temperature measurement and real-time imaging characteristics, helping users to quickly and accurately detect anomalies, assess conditions and make decisions.

[0003] For example, the Chinese invention patent with publication number CN112634346A discloses a method and system for obtaining the size of real objects based on AR glasses, including: an original real image acquisition step: the AR glasses obtain the real-time image of the real environment in front of the AR glasses through a camera to obtain the original real image; a thermal feature distribution real image acquisition step: the AR glasses obtain the real-time image of the real environment in front of the AR glasses through an infrared imaging device to obtain a thermal feature distribution real image; a real object size acquisition step: the outline of the real object is obtained according to the thermal feature distribution, and the size of the real object is obtained according to the outline of the real object. The present invention provides a new method for obtaining real objects that is different from traditional image recognition.

[0004] The above existing technologies have the following deficiencies: 1. Currently, the thermal feature distribution real image is obtained only based on the real-time image of the real environment in front of the AR glasses, without considering the infrared image and visible light image collected by the AR glasses and fusing them to obtain a comprehensive real image. The image acquisition dimension is single, and thus the accuracy of the real image acquisition cannot be guaranteed, and thus the integrity of the real image acquisition cannot be guaranteed, which reduces the comprehensiveness and effectiveness of the subsequent real image analysis.

[0005] 2. Currently, the outline of real objects is obtained only based on the distribution of thermal features, without considering the limitations of obtaining image edges based on thermal features. The accuracy of target device information acquisition cannot be guaranteed, which increases the risk of misjudgment of target device information and causes the loss of detail points of target device information, making it impossible to accurately reflect the authenticity of the target device information. Summary of the invention

[0006] In view of this, in order to solve the problems raised in the above background technology, a method for processing infrared thermal imaging information of smart AR glasses based on image processing technology is proposed.

[0007] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides an intelligent AR glasses infrared thermal imaging information processing method based on image processing technology, including: step one, device image acquisition: acquiring infrared light images and visible light images of the target device.

[0008] Step 2: Device image extraction: Based on the infrared image and visible light image of the target device, extract the position coordinates of each reference point of the target device in the infrared image and visible light image as well as the image contrast, image resolution and image pixel value.

[0009] Step 3: Image angle confirmation: According to the position coordinates of each reference point of the target device in the infrared image and the visible light image, confirm the angle correction information of the target device fusion image.

[0010] Step 4: Image pixel confirmation: According to the image contrast, image resolution and image pixel value of the target device in the infrared light image and the visible light image, the image pixel information of the target device fusion image is confirmed.

[0011] Step 5: Confirmation of fused image: Based on the angle correction information and image pixel information of the fused image of the target device, the infrared light image and the visible light image of the target device are fused to obtain the fused image of the target device.

[0012] Step 6: External temperature monitoring: Monitor the external temperature of the target device.

[0013] Step 7: External temperature analysis: Analyze the external temperature influencing factors of the target device based on the external temperature of the target device and the fused image.

[0014] Step 8: Temperature warning analysis: Based on the fused image of the target device and the external temperature influencing factors, a temperature warning analysis of the target device is performed to obtain the temperature warning information of the target device.

[0015] Step 9: Temperature warning feedback: Provide corresponding feedback based on the temperature warning information of the target device.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention obtains the infrared light image and the visible light image of the target device through AR glasses, and fuses them to obtain the fused image of the target device, and comprehensively collects the image of the target device in multiple dimensions, thereby ensuring the accuracy of the fused image acquisition of the target device, thereby ensuring the integrity of the fused image acquisition of the target device, and at the same time improving the comprehensiveness and effectiveness of the subsequent analysis of the fused image of the target device.

[0017] (2) The present invention avoids the limitation of the current image edge blur defect that does not consider thermal features by fusing the infrared light image and the visible light image of the target device, thereby ensuring the accuracy of the acquisition of the target device image information, thereby reducing the risk of misjudgment of the target device image information, and improving the reliability of the acquisition of the target device image information.

[0018] (3) When fusing the infrared light image and the visible light image of the target device, the present invention improves the consistency of the target device image fusion by considering the angle correction information and image pixel information of the target device fused image, thereby improving the accuracy of the subsequent temperature anomaly analysis, thereby ensuring the reliability of the target device temperature warning.

[0019] (4) The present invention analyzes the external temperature influencing factors of the target device and performs temperature warning analysis on the target device, thereby improving the effectiveness of temperature anomaly analysis of target device components, thereby improving the timeliness of temperature anomaly warning, and thus improving the rationality of subsequent target device maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0021] Figure 1 It is a schematic diagram of the connection of each step of the method of the present invention.

[0022] Figure 2 It is a connection diagram of the angle correction information confirmation step of the present invention.

[0023] Figure 3 It is a connection schematic diagram of the statistical step of abnormal temperature component ratio in the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1 As shown, the present invention provides an intelligent AR glasses infrared thermal imaging information processing method based on image processing technology, the method comprising: step one, device image acquisition: acquiring infrared light images and visible light images of the target device.

[0026] It should be added that the infrared image and visible light image acquisition process of AR glasses is as follows: AR glasses are equipped with auxiliary alignment functions, such as displaying crosshairs or other aiming marks on the display screen, and using these marks to accurately align the target device to ensure that the target device is within the shooting range of the infrared thermal imager and in a suitable position. When the target device is aligned, the infrared thermal imager is triggered to acquire images. The infrared thermal imager will detect the infrared radiation on the surface of the target device according to its working principle, and convert it into a corresponding temperature distribution image, namely an infrared light image. For example, when acquiring an infrared light image of a running electronic device, the infrared thermal imager will display the temperature distribution of each component inside the device, with high-temperature areas displayed in brighter colors and low-temperature areas displayed in darker colors. At the same time, the appearance, color, texture and other information of the target device are captured through a visible light camera to generate ordinary color or grayscale images, and the acquired infrared light images and visible light images are stored in the storage device of the AR glasses or transmitted to an external device connected to it (such as a mobile phone or computer).

[0027] Step 2: Device image extraction: Based on the infrared image and visible light image of the target device, extract the position coordinates of each reference point of the target device in the infrared image and visible light image as well as the image contrast, image resolution and image pixel value.

[0028] Step 3: Image angle confirmation: According to the position coordinates of each reference point of the target device in the infrared image and the visible light image, confirm the angle correction information of the target device fusion image.

[0029] See also Figure 2 As shown, exemplarily, the confirmation of the angle correction information of the target device fusion image includes: G1, analyzing the deviation angle θ′ of the target device in the infrared light image and the visible light image according to the position coordinates of each reference point of the target device in the infrared light image and the visible light image.

[0030] Further, the analysis of the deviation angle of the target device in the infrared light image and the visible light image includes: G1-1, according to the position coordinates of each reference point of the target device in the infrared light image, counting the normalized position vector of each reference point of the target device in the infrared light image i is the reference point number, i=1,2,......,n.

[0031] It should be added that the reason for counting the normalized position vector of each reference point is: in an image, the position vectors of different reference points may have different scales, which depends on factors such as the size and resolution of the image and the distribution range of the objects represented by the reference points in the image. For example, in a large image with high resolution, the distance between the reference points may be large and the length of the position vector is also long, while in a small image with low resolution, the length of the reference point position vector corresponding to the same object will be shorter. By obtaining the normalized position vector, these position vectors can be unified within a standard scale range, so that the reference point position information of subsequent infrared images and visible light images can be compared and analyzed at the same scale.

[0032] Furthermore, the statistical normalized position vector of each reference point of the target device in the infrared light image includes: G1-1-1, marking the position coordinate of each reference point of the target device in the infrared light image as η i (x i ,y i ).

[0033] G1-1-2. Count the position vectors of each reference point of the target device in the infrared image

[0034] G1-1-3. Count the normalized position vectors of each reference point of the target device in the infrared image

[0035] G1-2. According to the statistical method of the normalized position vector of each reference point of the target device in the infrared light image, the normalized position vector of each reference point of the target device in the visible light image is obtained by statistics and recorded as

[0036] G1-3. Count the deviation angle θ between the infrared image and the visible light image at each reference point i ,

[0037] G1-4. Count the deviation angle θ′ of the target device in the infrared light image and the visible light image. n is the number of reference points.

[0038] In a specific embodiment, in the infrared light image, one reference point is η1(100,100), and another corresponding reference point is η2(150,100), and in the visible light image, the corresponding reference points are η1′(200,200) and η2′(300,200).

[0039] In infrared images, the vector is Then the normalized position vector is In a visible light image, the vector is Then the normalized position vector is

[0040]

[0041] According to the dot product formula Then, the deviation angle between the infrared light image and the visible light image at the reference point is θ1=arccos(1)=0°.

[0042] Assuming that the calculated deviation angles of the infrared image and the visible light image at each reference point are 0°, 2°, 3°, 0°, 1°, 4°, 2°, and 3° respectively, the deviation angle of the target device between the infrared image and the visible light image is

[0043] G2. Compare the deviation angle of the target device in the infrared light image and the visible light image with the set allowable deviation angle.

[0044] G3. If the deviation angle between the infrared light image and the visible light image of the target device is less than or equal to the set allowable deviation angle, no angle correction will be performed as the angle correction information of the target device fusion image.

[0045] G4. If the deviation angle between the infrared light image and the visible light image of the target device is greater than the set allowable deviation angle, the visible light image is rotated in the opposite direction of the deviation angle by the deviation angle as the angle correction information of the target device fusion image.

[0046] Step 4: Image pixel confirmation: According to the image contrast, image resolution and image pixel value of the target device in the infrared light image and the visible light image, the image pixel information of the target device fusion image is confirmed.

[0047] Exemplarily, the step of confirming the image pixel information of the target device fused image includes: H1, recording the image contrast of the target device in the infrared light image and the visible light image as C IR and C VIS .

[0048] H2, the contrast ratio β of the target device fusion image is counted,

[0049] H3. Upsample the image pixel values ​​of the target device in the infrared image and the visible light image by bilinear interpolation to obtain the upsampled image pixel values ​​of the target device in the infrared image and the visible light image, and record them as I and IR (x,y) and I VIS (x,y).

[0050] It should be added that the definition of upsampled image pixel value is: upsampled image pixel value is a concept involved in image processing operations. In the image field, upsampling refers to the process of converting a low-resolution image into a high-resolution image. The image pixel value of the new image pixel generated in this process is the upsampled image pixel value. It is based on a certain algorithm to add image pixels on the basis of the original image pixels and assign them corresponding values, so that the image size becomes larger and the details are richer.

[0051] It should be added that the specific process of obtaining the upsampled image pixel values ​​of the target device in the infrared image and the visible light image through bilinear interpolation upsampling is as follows: first, the upsampling multiple is determined, that is, the resolution of the infrared image is increased to the same magnification as the visible light image. Assume that the resolution of the infrared image is M IR *N IR , the resolution of the visible light image is M VIS *N VIS , then the upsampling factor in the horizontal direction is Vertical upsampling factor

[0052] The image pixel in the infrared image is I I ' R (x′, y′)(x′ represents the column coordinate, y′ represents the row coordinate, and the coordinate ranges are 0 to N IR -1 and 0 to M IR -1).

[0053] Based on the upsampling multiples in the horizontal and vertical directions, determine the neighboring image pixels corresponding to the original image pixels in the high-resolution space. Suppose the image pixel coordinates in the high-resolution space after upsampling are (x′, y′), then the corresponding coordinates in the original infrared image are ( is rounded down),

[0054] The coordinates (x, y) in the high-resolution space after upsampling are calculated as follows: x = x'*k x , y=y′*k y .

[0055] Bilinear interpolation is used to calculate the upsampled image pixel values. Assume that the coordinates of the 2*2 neighborhood image pixels of the upsampled image pixels in the original infrared image are (ξ,ψ), (ξ+1,ψ), (ξ,ψ+1), and (ξ+1,ψ+1). The corresponding image pixel values ​​are I IR (ξ,ψ),I IR (ξ+1,ψ),I IR (ξ,ψ+1),I IR(ξ+1,ψ+1), which are respectively denoted as and

[0056] Upsample image pixel value I IR The calculation formula for (x,y) is: in And follow I IR The statistical method of (x, y) is similar to the statistical method to obtain the upsampled image pixel value of the visible light image, which is recorded as I VIS (x,y).

[0057] In a specific embodiment, if it is assumed that the resolution of the infrared image is 100*100 and the resolution of the visible light image is 200*200, the upsampling factor in the horizontal direction is The upsampling factor in the vertical direction is

[0058] If the image pixel in the infrared image is I I ' R (3,2), then the corresponding coordinates in the original infrared image are

[0059] The upsampled image pixel value (x, y) of the infrared image is calculated as follows: x=3*2=6, y=2*2=4, and the upsampled coordinates are (6, 4).

[0060] Bilinear interpolation is used to calculate the upsampled image pixel values. Assuming that the coordinates of the 2*2 neighborhood image pixels of the upsampled image pixels in the original infrared image are (1,1), (2,1), (1,2), (2,2), the corresponding image pixel values ​​are I IR (1,1),I IR (2,1),I IR (1,2),I IR (2,2), then

[0061] The upsampled image pixel value I of the infrared image IR The calculation formula for (x,y) is: IR (6,4)=(1-2)*(1-1)*10+2*(1-1)*12+(1-2)*1*8+2*1*9=10.

[0062] H4. Count the contrast adjustment image pixel values ​​of the target device fusion image I F (x,y),I F (x,y)=β*I IR (x,y)+(1-β)*I VIS (x,y).

[0063] H5. According to the resolution of the target device in the infrared image and the visible light image and the upsampled image pixel value, the resolution of the target device fusion image is counted and the image pixel value I′ is adjusted F (x,y).

[0064] Furthermore, the resolution adjustment of the image pixel value of the statistical target device fusion image includes: H5-1, extracting the number of rows and columns of the infrared light image from the infrared light image, and extracting the number of rows and columns of the visible light image from the visible light image.

[0065] H5-2. The ratio of the number of visible light image columns to the number of infrared light image columns is used as the resolution scaling factor of the image in the horizontal direction, denoted as k. x .

[0066] H5-3. The ratio of the number of visible light image lines to the number of infrared light image lines is used as the resolution scaling factor in the vertical direction of the image, denoted as k. y .

[0067] H5-4. Count the resolution of the fused image of the target device and adjust the image pixel value I′ F (x,y),I′ F (x,y)=k x *I IR (x,y)+k y *I VIS (x,y).

[0068] H6. Count the image pixel values ​​I′ of the fused image of the target device F ′(x,y),I′ F ′(x,y)=I F (x,y)*r1+I′ F (x, y)*r2, r1 and r2 are the weights of the contrast adjustment image pixel value and the resolution adjustment image pixel value, r1+r2=1, r1>r2, and I′ F ′(x,y) is the pixel information of the target device fused image.

[0069] The embodiment of the present invention avoids the limitation of the current image edge blur defect that does not consider the thermal feature to obtain the target device image information by fusing the infrared light image and the visible light image of the target device, thereby ensuring the accuracy of the target device image information acquisition, thereby reducing the risk of misjudgment of the target device image information, and improving the reliability of the target device image information acquisition.

[0070] It should be added that in equipment detection scenarios, such as fault detection and status assessment, image pixel values ​​adjusted based on contrast are crucial for detecting abnormal conditions. Many equipment faults are manifested as local temperature changes (reflected in infrared images) or appearance changes (reflected in visible light images). By adjusting the contrast, these changes can be amplified, making fault features easier to detect. For example, when detecting whether there is an overheating fault in power equipment, a high-contrast fused image can clearly show areas with abnormally high temperatures, which may be key clues to the fault. Resolution adjustment is mainly to increase the detail information of the image. However, the effective presentation of these details often depends on contrast. If there is not enough contrast, even if the resolution is high, the details may be submerged in similar image pixel values. For example, in a high-resolution target device fused image, if the contrast between image pixel values ​​is low, then the tiny circuit elements inside the device (in the device fused image after infrared-visible light fusion) may not be clearly observed because their image pixel values ​​are too close to the surrounding areas. Therefore, r1>r2 is set. For ease of analysis, r1 can be specifically taken as 0.6 and r2 can be specifically taken as 0.4.

[0071] The embodiment of the present invention acquires the infrared light image and the visible light image of the target device through AR glasses, and fuses them to obtain the fused image of the target device, and comprehensively acquires the image of the target device in multiple dimensions, thereby ensuring the accuracy of the acquisition of the fused image of the target device, thereby ensuring the integrity of the acquisition of the fused image of the target device, and at the same time improving the comprehensiveness and effectiveness of the subsequent analysis of the fused image of the target device.

[0072] Step 5: Confirmation of fused image: Based on the angle correction information and image pixel information of the fused image of the target device, the infrared light image and the visible light image of the target device are fused to obtain the fused image of the target device.

[0073] When fusing the infrared light image and the visible light image of the target device, the embodiment of the present invention improves the consistency of the target device image fusion by considering the angle correction information and image pixel information of the target device fused image, thereby improving the analysis accuracy of the subsequent temperature anomaly analysis, thereby ensuring the reliability of the target device temperature warning.

[0074] Step 6: External temperature monitoring: Monitor the external temperature of the target device.

[0075] It should be added that the external temperature of the target device is monitored by a temperature sensor installed near the target device.

[0076] Step 7: External temperature analysis: Analyze the external temperature influencing factors of the target device based on the external temperature of the target device and the fused image.

[0077] Exemplarily, the analyzing the external temperature influencing factor of the target device includes: extracting the temperature of each device component in the target device from the fused image of the target device, denoted as T j , j is the equipment part number, j=1,2,......,m.

[0078] Let the external temperature of the target device be T'.

[0079] Subtract the temperature of each device component in the target device from the external temperature to obtain the temperature difference between each device component in the target device and the external temperature, recorded as ΔT j .

[0080] ΔT j Perform mean calculation to obtain the average temperature difference between the target device and the external temperature, recorded as

[0081] Statistical temperature standard deviation σ between the target device and the external temperature, m is the number of equipment components.

[0082] Statistical analysis of external temperature influencing factors of target devices

[0083] The embodiment of the present invention analyzes the external temperature influencing factors of the target device and performs temperature warning analysis on the target device, thereby improving the effectiveness of temperature anomaly analysis of target device components, thereby improving the timeliness of temperature anomaly warning, and thus improving the rationality of subsequent target device maintenance.

[0084] Step 8: Temperature warning analysis: Based on the fused image of the target device and the external temperature influencing factors, a temperature warning analysis of the target device is performed to obtain the temperature warning information of the target device.

[0085] Exemplarily, the temperature warning analysis of the target device includes: A1, extracting the temperature value of each device component in the target device from the fused image of the target device, and then calculating the ratio K1 of the target device's temperature abnormal components.

[0086] See also Figure 3 As shown, further, the counting of the abnormal temperature component ratio of the target device includes: A1-1, comparing the temperature value of each device component in the target device with its set temperature threshold.

[0087] A1-2. If the temperature value of a device component in the target device is greater than the set temperature threshold, the device component of the target device is recorded as a device component with abnormal temperature.

[0088] A1-3. Count the number of equipment components with abnormal temperatures and the number of equipment components, and then use the ratio of the number of equipment components with abnormal temperatures to the number of equipment components as the abnormal temperature component ratio of the target equipment, denoted as K1.

[0089] A2. Extract the position coordinates of each device component in the target device from the fused image of the target device, and then calculate the temperature anomaly gradient ratio K2 of the target device.

[0090] Furthermore, the method of counting the temperature anomaly gradient ratio of the target device includes: A2-1, combining each device component in the target device with its remaining device components in pairs to obtain each device component group in the target device.

[0091] A2-2. Subtract the temperature of the target device in each device component group to obtain the temperature difference of each device component group in the target device, recorded as ΔT p , p is the equipment component group number, p = 1, 2, ... q.

[0092] A2-3. The position coordinates of the target device in each device component group are calculated by the Euclidean distance formula to obtain the distance between each device component group in the target device, which is recorded as d p .

[0093] A2-4. Count the temperature gradient values ​​of each equipment component group in the target equipment

[0094] A2-5. Compare the temperature gradient value of each equipment component group with the set temperature gradient threshold.

[0095] A2-6. If the temperature gradient value of a certain equipment component group is greater than the set temperature gradient threshold, the equipment component group is recorded as a temperature gradient deviation equipment component group.

[0096] A2-7. Count the number of temperature gradient deviation equipment component groups and the number of equipment component groups, and then use the ratio of the number of temperature gradient deviation equipment component groups to the number of equipment component groups as the temperature abnormality gradient ratio of the target equipment, and record it as K2.

[0097] In a specific embodiment, assume that there is a small air-conditioning outdoor unit as the target device, which has main components such as a compressor, a condenser, and a fan motor. If the coordinate coordinates of the compressor are (0,0), the temperature T1 = 80°C, the position coordinates of the condenser are (20,10), the temperature T2 = 60°C, the position coordinates of the fan motor are (-10,15), and the temperature T3 = 50°C.

[0098] The temperature difference between the compressor and condenser equipment components in the target equipment is ΔT1=T1-T2=80-60=20°C, and the distance between the compressor and condenser equipment components in the target equipment is Then the temperature gradient value of the compressor and condenser equipment component group in the target equipment is

[0099] The temperature difference of the compressor and fan motor equipment component group in the target device is ΔT2 = T1-T3 = 80-50 = 30°C, and the distance between the compressor and fan motor equipment component group in the target device is The temperature gradient value of the condenser and fan motor equipment component group in the target equipment is

[0100] The temperature difference between the condenser and the fan motor equipment component group in the target device is ΔT3 = T2-T3 = 60-50 = 10°C, and the distance between the condenser and the fan motor equipment component group in the target device is The temperature gradient value of the condenser and fan motor equipment component group in the target equipment is

[0101] If the temperature gradient threshold set for the air conditioner outdoor unit is 1.3°C, the temperature abnormality gradient ratio of the target device is

[0102] A3. Statistical temperature warning coefficient δ of target equipment. z1 and z2 are the weights of the set temperature anomaly component ratio and temperature anomaly gradient ratio, respectively, z1+z2=1, z1>z2.

[0103] It should be added that the temperature abnormal component ratio refers to the ratio of the number of temperature abnormal equipment components to the number of equipment components. This ratio can intuitively reflect the temperature abnormality of the entire equipment. If this ratio is high, it means that a large proportion of equipment components have temperature abnormalities, which may mean that there is a problem with the overall cooling system of the equipment, or that the equipment is under abnormal workload, etc. For example, in a server cabinet, there are 50 server nodes. If the temperature abnormal equipment component ratio reaches 40% (that is, 20 nodes have abnormal temperatures), this indicates that the air conditioning and refrigeration system of the cabinet may fail or the ambient temperature of the computer room may be too high, which may cause the temperature of most server nodes to rise. This overall abnormality may have a serious impact on the overall operation of the equipment. The temperature abnormal gradient ratio mainly reflects the unreasonable degree of temperature changes between components. Compared with the temperature abnormal component ratio, it is more sensitive to local heat conduction failures or local performance degradation. For example, on a circuit board, even if there are only a few adjacent electronic components There is a large temperature gradient anomaly between the components, which may be because the solder joints between the components are loose or the local heat dissipation channels are blocked. Although this may be small in proportion to the overall components, it can accurately indicate the local fault point. If this local fault is not handled in time, it may gradually spread and eventually lead to more serious problems, such as component damage or even circuit short circuit. In the actual equipment temperature warning and fault diagnosis process, we usually focus on the overall operating status of the equipment first. This is because the overall abnormality often has a more serious impact on the equipment, such as equipment performance degradation, shutdown, etc. Therefore, the weight of the temperature abnormal component ratio should be higher, which allows us to quickly determine whether the equipment is in a relatively serious overall abnormal state. Once it is determined that there is a problem with the overall state, the temperature abnormal gradient ratio is used to further locate the specific fault point or abnormal area for more detailed investigation and repair. Therefore, z1>z2 is set. For the convenience of analysis, z1 can be specifically taken as 0.6 and z2 can be specifically taken as 0.4.

[0104] A4. Match and compare the temperature warning coefficient of the target device with the temperature warning coefficient corresponding to the warning level of each device to obtain the device warning level of the target device, and use the warning level of the target device as the temperature warning information of the target device.

[0105] Step 9: Temperature warning feedback: Provide corresponding feedback based on the temperature warning information of the target device.

[0106] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for processing infrared thermal image information of smart AR glasses based on image processing technology, characterized in that: The method includes: Step 1: Device image acquisition: Acquire infrared light images and visible light images of the target device; Step 2: Device image extraction: Based on the infrared image and visible light image of the target device, the position coordinates of each reference point of the target device in the infrared image and the visible light image as well as the image contrast, image resolution and image pixel value are extracted; Step 3: Image angle confirmation: According to the position coordinates of each reference point of the target device in the infrared image and the visible light image, confirm the angle correction information of the target device fusion image; Step 4: Image pixel confirmation: according to the image contrast, image resolution and image pixel value of the target device in the infrared image and the visible light image, confirm the pixel information of the target device fused image; Step 5: Confirmation of fused image: According to the angle correction information and pixel information of the fused image of the target device, the infrared image and the visible light image of the target device are fused to obtain the fused image of the target device; Step 6: External temperature monitoring: monitor the external temperature of the target device; Step 7: External temperature analysis: Analyze the external temperature influencing factors of the target device according to the external temperature of the target device and the fused image; Step 8: Temperature warning analysis: Based on the fusion image of the target device and the external temperature influencing factors, the temperature warning analysis of the target device is performed to obtain the temperature warning information of the target device; Step 9: Temperature warning feedback: Provide corresponding feedback based on the temperature warning information of the target device.

2. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 1 is characterized in that: The step of confirming the angle correction information of the fused image of the target device includes: G1. Analyze the deviation angle θ′ of the target device in the infrared light image and the visible light image according to the position coordinates of each reference point of the target device in the infrared light image and the visible light image; G2. Compare the deviation angle of the target device in the infrared light image and the visible light image with the set allowable deviation angle; G3. If the deviation angle between the infrared image and the visible light image of the target device is less than or equal to the set permissible deviation angle, no angle correction will be performed as the angle correction information of the target device fusion image; G4. If the deviation angle between the infrared light image and the visible light image of the target device is greater than the set allowable deviation angle, the visible light image is rotated in the opposite direction of the deviation angle by the deviation angle as the angle correction information of the target device fusion image.

3. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 2 is characterized in that: The analyzing the deviation angle of the target device in the infrared light image and the visible light image includes: According to the position coordinates of each reference point of the target device in the infrared image, the normalized position vector of each reference point of the target device in the infrared image is counted i is the reference point number, i=1,2,......,n; According to the statistical method of the normalized position vector of each reference point of the target device in the infrared light image, the normalized position vector of each reference point of the target device in the visible light image is obtained by statistics and recorded as Statistical deviation angle θ between infrared image and visible light image at each reference point i , Statistically calculate the deviation angle θ′ of the target device in the infrared light image and the visible light image, n is the number of reference points.

4. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 3 is characterized in that: The method of counting the normalized position vectors of the reference points of the target device in the infrared light image includes: The position coordinates of each reference point of the target device in the infrared light image are marked as η i (x i ,y i ); Count the position vectors of each reference point of the target device in the infrared image Count the normalized position vectors of each reference point of the target device in the infrared image 5. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 1, characterized in that: The step of confirming pixel information of the fused image of the target device includes: H1. The image contrast of the target device in the infrared image and the visible light image is recorded as C IR and C VIS ; H2, the contrast ratio β of the target device fusion image is counted, H3. Upsample the image pixel values ​​of the target device in the infrared image and the visible light image by bilinear interpolation to obtain the upsampled image pixel values ​​of the target device in the infrared image and the visible light image, and record them as I and IR (x,y) and I VIS (x,y); H4. Count the contrast adjustment image pixel values ​​of the target device fusion image I F (x,y),I F (x,y)=β*I IR (x,y)+(1-β)*I VIS (x,y); H5. According to the resolution of the target device in the infrared image and the visible light image and the upsampled image pixel value, the resolution of the target device fusion image is counted and the image pixel value I′ is adjusted F (x,y); H6. Count the image pixel values ​​I′ of the fused image of the target device F ′(x,y),I′ F ′(x,y)=I F (x,y)*r1+I′ F (x, y)*r2, r1 and r2 are the weights of the contrast adjustment image pixel value and the resolution adjustment image pixel value, r1+r2=1, r1>r2, and I′ F ′(x,y) is the pixel information of the target device fused image.

6. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 5 is characterized in that: The counting of the resolution adjustment image pixel values ​​of the target device fusion image includes: Extracting the number of rows and columns of the infrared image from the infrared image, and extracting the number of rows and columns of the visible light image from the visible light image; The ratio of the number of visible light image columns to the number of infrared light image columns is taken as the resolution scaling factor of the image in the horizontal direction, denoted as k x ; The ratio of the number of visible light image lines to the number of infrared light image lines is taken as the resolution scaling factor in the vertical direction of the image, denoted as k y ; Count the resolution of the fused image of the target device and adjust the image pixel value I′ F (x,y),I′ F (x,y)=k x *I IR (x,y)+k y *I VIS (x,y).

7. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 1, characterized in that: The analyzing the external temperature influencing factors of the target device includes: The temperature of each device component in the target device is extracted from the fused image of the target device, denoted as T j , j is the equipment part number, j = 1, 2, ..., m; Let the external temperature of the target device be T′; Subtract the temperature of each device component in the target device from the external temperature to obtain the temperature difference between each device component in the target device and the external temperature, recorded as ΔT j ; ΔT j Perform mean calculation to obtain the average temperature difference between the target device and the external temperature, recorded as Statistical temperature standard deviation σ between the target device and the external temperature, m is the number of equipment components; Statistical analysis of external temperature influencing factors of target devices 8. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 1, characterized in that: The temperature warning analysis of the target device includes: A1. Extract the temperature value of each device component in the target device from the fusion image of the target device, and then calculate the ratio K1 of the abnormal temperature components of the target device; A2, extracting the position coordinates of each device component in the target device from the fusion image of the target device, and then calculating the temperature abnormality gradient ratio K2 of the target device; A3. Statistical temperature warning coefficient δ of target equipment. z1 and z2 are the weights of the set temperature anomaly component ratio and temperature anomaly gradient ratio, respectively, z1+z2=1, z1>z2; A4. Match and compare the temperature warning coefficient of the target device with the temperature warning coefficient corresponding to the warning level of each device to obtain the device warning level of the target device, and use the warning level of the target device as the temperature warning information of the target device.

9. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 8 is characterized in that: The counting of the abnormal temperature component ratio of the target device includes: Compare the temperature value of each device component in the target device with the set temperature threshold; If the temperature value of a device component in the target device is greater than the set temperature threshold, the device component of the target device is recorded as a device component with abnormal temperature; The number of equipment components with abnormal temperatures and the number of equipment components are counted, and then the ratio of the number of equipment components with abnormal temperatures to the number of equipment components is used as the temperature abnormal component ratio of the target equipment, which is recorded as K1.

10. The method for processing infrared thermal image information of smart AR glasses based on image processing technology according to claim 8, characterized in that: The statistical abnormal temperature gradient ratio of the target device includes: Combining each device component in the target device with its remaining device components in pairs to obtain each device component group of the target device; The temperature difference of each device component group in the target device is obtained by subtracting the temperature of each device component group in the target device, which is recorded as ΔT p , p is the equipment component group number, p = 1, 2, ... q; The position coordinates of the target device in each device component group are calculated using the Euclidean distance formula to obtain the distance between each device component group in the target device, which is recorded as d p ; Statistical temperature gradient value θ of each equipment component group in the target equipment p , Compare the temperature gradient value of each equipment component group with the set temperature gradient threshold; If the temperature gradient value of a certain equipment component group is greater than the set temperature gradient threshold, the equipment component group is recorded as a temperature gradient deviation equipment component group; The number of temperature gradient deviation equipment component groups and the number of equipment component groups are counted, and then the ratio of the number of temperature gradient deviation equipment component groups to the number of equipment component groups is used as the temperature abnormality gradient ratio of the target equipment, and is recorded as K2.

Images