Computer vision image processing method based on artificial intelligence

Abstract

The present invention discloses a computer vision image processing method based on artificial intelligence, which relates to the technical field of computer image processing. The method synchronously collects visible light, infrared thermal images, vibration spectrum and reference temperature data to ensure temporal and spatial consistency. The method extracts the main frequency and direction angle based on the vibration spectrum, directly drives the deconvolution and deblurring of the visible light image, and restores edge details. The edge gradient map of the deblurred image and the normalized temperature difference map are dynamically weighted and fused to generate a dual-channel high signal-to-noise ratio feature map. The defect candidate area is cross-validated by fractal dimension and heat conduction law to generate a high-confidence mask. The method adjusts the direction angle based on dynamic feedback of the mask edge fracture ratio, and iteratively optimizes until the defect edge continuity meets the standard to adapt to complex vibration working conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a computer vision image processing method based on artificial intelligence. Background Art

[0002] With the continuous advancement of industrial automation and intelligence, computer vision-based equipment condition monitoring and defect detection technologies are playing an increasingly important role in key sectors such as electricity, energy, rail transportation, and aerospace. In particular, surface crack detection using visible light images and overheating defect identification using infrared thermal images have become important tools for predictive maintenance.

[0003] However, actual industrial application scenarios, especially large rotating machinery, high-voltage electrical equipment, or equipment under complex working conditions, generally have significant vibration problems caused by the equipment itself or the environment: equipment vibration causes relative movement between the imaging device and the photographed target, causing motion blur in the captured visible light image, seriously interfering with the clarity of image details and reducing the accuracy of traditional edge detection algorithms.

[0004] Existing solutions often use vibration sensor data to compensate for image blur caused by vibration. However, when the device vibrates, it also generates heat. The heat generated by vibration not only causes pixel displacement and blur in the thermal image obtained by the infrared thermal imager, but also generates non-target heat sources such as frictional heat on the device surface, making it difficult to accurately distinguish between real defects and environmental noise. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a computer vision image processing method based on artificial intelligence.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention discloses a computer vision image processing method based on artificial intelligence, comprising the following steps:

[0008] Obtain visible light images, infrared thermal images, vibration spectrum data, and reference temperature data of the target device surface at the same timestamp;

[0009] Extracting the main vibration frequency and direction angle according to the vibration spectrum data and performing deconvolution processing on the visible light image to obtain a deblurred image;

[0010] Obtaining a normalized temperature difference map based on the reference temperature data and the deblurred image;

[0011] Extracting an edge gradient map from the deblurred image, and performing weighted fusion of the edge gradient map and the corrected normalized temperature difference map to generate a dual-channel fusion feature map;

[0012] Performing local signal-to-noise ratio analysis on the dual-channel fusion feature map, and screening areas with a signal-to-noise ratio greater than a preset defect threshold as defect candidate areas;

[0013] Perform fractal dimension verification and heat conduction law verification on each defect candidate area to generate a defect confidence mask;

[0014] Extracting the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determining whether the fracture length ratio exceeds a preset length threshold;

[0015] If the result of the determination is yes, the direction angle is adjusted according to the fracture length and the total edge length until the fracture length ratio in the defect confidence mask generated according to the adjusted direction angle is lower than the preset length threshold;

[0016] If the judgment result is no, the defect confidence mask is fused with the dual-channel fusion feature map to generate a three-channel result map including the normalized temperature difference map, the deblurred edge map, and the defect confidence mask.

[0017] In a second aspect, the present invention discloses an artificial intelligence-based computer vision image processing system, which is applied with the above-mentioned artificial intelligence-based computer vision image processing method, including:

[0018] A data processing module is used to obtain visible light images, infrared thermal images, vibration spectrum data and reference temperature data of the target device surface at the same time stamp;

[0019] a visible light image processing module, configured to extract the main vibration frequency and direction angle according to the vibration spectrum data and perform deconvolution processing on the visible light image to obtain a deblurred image;

[0020] A temperature difference map acquisition module, configured to obtain a normalized temperature difference map based on the reference temperature data and the deblurred image;

[0021] A dual-channel fusion module is used to extract an edge gradient map from the deblurred image, and generate a dual-channel fusion feature map after weighted fusion of the edge gradient map and the corrected normalized temperature difference map;

[0022] A defect recognition module is used to perform local signal-to-noise ratio analysis on the dual-channel fusion feature map and select areas with a signal-to-noise ratio greater than a preset defect threshold as defect candidate areas;

[0023] The mask generation module is used to perform fractal dimension verification and heat conduction law verification on each defect candidate area to generate a defect confidence mask;

[0024] Verification feedback module, which is used to perform the following steps:

[0025] Extracting the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determining whether the fracture length ratio exceeds a preset length threshold;

[0026] If the result of the determination is yes, the direction angle is adjusted according to the fracture length and the total edge length until the fracture length ratio in the defect confidence mask generated according to the adjusted direction angle is lower than the preset length threshold;

[0027] If the judgment result is no, the defect confidence mask is fused with the dual-channel fusion feature map to generate a three-channel result map including the normalized temperature difference map, the deblurred edge map, and the defect confidence mask.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. Through vibration spectrum deconvolution deblurring, optical flow compensation for thermal image offset, and adaptive directional angle optimization, the core problem of image quality degradation in strong vibration environments is systematically solved;

[0030] 2. The integration of clear geometric edges, enhanced thermal features, and dual verification based on physical laws significantly improves defect detection accuracy while significantly reducing false detection rates.

[0031] 3. The closed-loop feedback optimization mechanism enables the solution to automatically adapt to changes in equipment vibration patterns, ensuring long-term stable and reliable operation in complex and ever-changing industrial sites;

[0032] 4. Visible light, infrared thermal imaging, vibration spectrum, and temperature data interact deeply and enhance each other in multiple links, achieving a mutually enhanced synergistic effect and surpassing the performance limits of a single modality or simple fusion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0034] Figure 1 is a flow chart of the method of the present invention;

[0035] Figure 2 It is a data flow diagram of the present invention;

[0036] Figure 3 It is a system module diagram of the present invention. DETAILED DESCRIPTION

[0037] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention and should not be regarded as the entire invention or as a limitation or restriction of the technical solution of the present invention.

[0038] Application Overview:

[0039] In traditional equipment condition monitoring systems, visible light images and infrared thermal images often experience coupling interference due to equipment vibration. Equipment surface vibration causes relative motion blur between the visible light imaging device and the object being imaged. Frictional heat generated by vibration also creates non-target heat source noise in the infrared thermal image. Insufficient decoupling of vibration spectrum data from the heat conduction model makes it difficult to simultaneously optimize image deblurring and temperature field correction. This reduces the spatial alignment between edge gradient features and thermal anomaly areas, leading to inaccurate signal-to-noise ratio threshold settings when screening defect candidate areas.

[0040] For example, in the scenario of detecting cracks on the surface of high-voltage circuit breaker contacts, the high-frequency mechanical vibrations caused by the circuit breaker's opening and closing operations cause directional blurring in the visible light image, and the main frequency component of the vibration exhibits non-steady-state changes over time. The Joule heat generated by the infrared thermal imager due to changes in contact resistance is superimposed on the vibration friction heat, resulting in pixel-level displacement errors in the thermal image. There is a spatial misalignment between the gradient features extracted by the edge detection algorithm in the blurred image and the normalized temperature difference map after displacement compensation. The local signal-to-noise ratio calculation of the dual-channel fusion feature map is biased, and the misjudgment rate of the fracture length ratio in the crack area increases.

[0041] If these issues are not addressed, edge fractures will be miscompensated during defect confidence mask generation, leading to inaccurate crack growth trend predictions. Pseudo-defect regions caused by non-target heat sources cannot be effectively filtered out, and the synergistic effect of fractal dimension verification and heat conduction law verification will be lost. This ultimately affects the real-time and reliability of equipment health assessments and increases the risk of unplanned downtime for critical equipment.

[0042] When faced with the above problems, the present application first considers how to achieve the coordinated optimization of vibration blur correction and heat conduction compensation. Traditional methods deal with visible light image blur or infrared thermal image displacement errors separately, without considering the coupling relationship between the vibration spectrum and the temperature field. The present application found that the main frequency component of the device vibration directly affects the directional characteristics of the point spread function, and the reference temperature data and the optical flow displacement vector can jointly correct the pixel-level error of the thermal image. To this end, the present application attempts to invert the vibration spectrum data into a point spread function matrix, restore the details of the visible light image through deconvolution, and use the theoretical temperature field model and optical flow displacement compensation to eliminate the spatial dislocation of the thermal image. Furthermore, the present application realizes that the dual-channel feature fusion of edge gradient and temperature anomaly requires dynamic adjustment of weights, and eliminates pseudo-defect interference through dual verification of fractal dimension and heat conduction law. Finally, it is chosen to use the directional angle iterative adjustment mechanism to optimize the vibration parameters in real time during the defect confidence mask generation stage to ensure the consistency of edge continuity and heat conduction direction.

[0043] like Figure 1 As shown, this application proposes a computer vision image processing method based on artificial intelligence, comprising the following steps:

[0044] Obtaining visible light images, infrared thermal images, vibration spectrum data, and reference temperature data of the target device surface at the same timestamp; obtaining visible light images, infrared thermal images, vibration spectrum data, and reference temperature data of the target device surface at the same timestamp means obtaining multimodal data at the same time point through synchronized acquisition devices. This can be achieved specifically by using a timestamp synchronization module to ensure time alignment of data from different sensors and eliminate errors caused by time differences.

[0045] The main vibration frequency and direction angle are extracted according to the vibration spectrum data, and the visible light image is subjected to deconvolution processing to obtain a deblurred image; the main vibration frequency and direction angle refer to identifying the frequency component with the largest energy and its vibration direction from the vibration spectrum, which can be specifically implemented by using a fast Fourier transform combined with a peak detection algorithm, and is used to quantify the degree of influence of vibration on image blur. The point spread function matrix is ​​calculated based on the main vibration frequency and direction angle, and the visible light image is subjected to deconvolution processing using the point spread function matrix to obtain a deblurred image. The point spread function matrix refers to a mathematical model that describes the image blur caused by vibration, which can be specifically generated by calculating the motion trajectory caused by vibration through the main vibration frequency and direction angle, and is used to restore a clear image in the deconvolution algorithm.

[0046] A normalized temperature difference map is obtained based on the reference temperature data and the deblurred image. The normalized temperature difference map refers to a standardized temperature difference distribution map after comparing the infrared thermal image with the theoretical temperature field model. Specifically, it can be achieved by calculating the theoretical temperature field through the heat conduction equation and performing pixel-level difference operations to eliminate environmental thermal noise interference.

[0047] An edge gradient map is extracted from the deblurred image, and the edge gradient map is weightedly fused with the corrected normalized temperature difference map to generate a dual-channel fusion feature map; the weighted fusion of the edge gradient map and the corrected normalized temperature difference map refers to superimposing the edge information of the visible light image and the temperature difference information according to the signal-to-noise ratio. Specifically, it can be implemented using a linear weighted algorithm to enhance the feature significance of the defect area.

[0048] A local signal-to-noise ratio analysis is performed on the dual-channel fusion feature map, and areas with a signal-to-noise ratio greater than a preset defect threshold are screened as defect candidate areas. Local signal-to-noise ratio analysis refers to dividing a detection window on the fusion feature map and calculating the energy ratio of the signal to noise within the window. Specifically, it can be implemented by using a sliding window combined with a signal-to-noise ratio calculation formula to screen out high-probability defect areas.

[0049] Fractal dimension verification and heat conduction law verification are performed on each defect candidate area to generate a defect confidence mask. Fractal dimension verification involves calculating the fractal dimension of the binary image of the defect candidate area to determine whether it conforms to the crack morphology characteristics. This can be achieved using box counting or differential box dimension methods to distinguish real defects from noise artifacts. Heat conduction law verification involves analyzing whether the temperature gradient direction outside the defect area conforms to the direction predicted by the heat conduction equation. This can be achieved using gradient direction statistics and angle calculation to eliminate temperature differences caused by non-heat conduction anomalies.

[0050] Extracting the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determining whether the fracture length ratio exceeds a preset length threshold;

[0051] If the result of the determination is yes, the direction angle is adjusted according to the fracture length and the total edge length until the fracture length ratio in the defect confidence mask generated according to the adjusted direction angle is lower than the preset length threshold;

[0052] If the result is negative, the defect confidence mask is fused with the two-channel fusion feature map to generate a three-channel result image consisting of a normalized temperature difference map, a deblurred edge map, and a defect confidence mask. Dynamic adjustment of the azimuth angle refers to the reverse optimization of the vibration azimuth angle parameters based on the edge breakage ratio. This is achieved by iteratively updating the azimuth angle and reconstructing the point spread function. This is used to eliminate edge breakage artifacts in the deconvolution image.

[0053] The core innovation of this application lies in: improving image clarity through the simultaneous acquisition and fusion processing of multimodal data, combining vibration fuzzy modeling and deconvolution algorithm, and using the dual verification mechanism of fractal dimension and heat conduction law to dynamically optimize the directional angle parameters, thereby achieving high-precision detection and anti-interference verification of equipment surface defects in complex vibration environments.

[0054] like Figure 2As shown, it is a data flow diagram of the present application; as a preferred embodiment, the solution of the present application is specifically implemented as follows:

[0055] Obtain visible light images, infrared thermal images, vibration spectrum data, and baseline temperature data of the target device surface at the same timestamp. The visible light images are collected by a high-speed camera, the infrared thermal images are collected by an infrared thermal imager, the vibration spectrum data is collected by an acceleration sensor, and the baseline temperature data is collected by a temperature sensor.

[0056] The primary vibration frequency and azimuth angle are extracted from the vibration spectrum data. The vibration spectrum data is analyzed using a fast Fourier transform to identify the frequency component with the highest energy as the primary vibration frequency and calculate its corresponding azimuth angle. The point spread function matrix is ​​calculated based on the primary vibration frequency and azimuth angle. A two-dimensional Gaussian function model is constructed using the primary vibration frequency and azimuth angle to generate the point spread function matrix. The point spread function matrix is ​​used to deconvolve the visible light image to obtain a deblurred image. Deconvolution is performed using the Wiener filter algorithm to restore image details.

[0057] A normalized temperature difference map is generated based on the baseline temperature data and the deblurred image. A heat conduction equation model is established using the baseline temperature data to calculate the theoretical temperature field. The infrared thermal image and the theoretical temperature field are normalized to generate a temperature difference map. Optical flow displacement vectors are calculated based on successive frames of the deblurred image, and pixel displacement compensation is performed on the temperature difference map.

[0058] Extract the edge gradient map from the deblurred image. Use the Sobel operator to calculate the image gradient and obtain the edge gradient map.

[0059] The edge gradient map and the corrected normalized temperature difference map are weightedly fused to generate a dual-channel fusion feature map. The signal-to-noise ratio of the edge gradient map and the temperature difference map is calculated, and the fusion weight is dynamically adjusted based on the signal-to-noise ratio.

[0060] Perform local signal-to-noise ratio analysis on the dual-channel fusion feature map and select areas with a signal-to-noise ratio greater than the preset defect threshold as defect candidate areas. Use the sliding window method to calculate the local signal-to-noise ratio and mark areas above the threshold as candidate areas.

[0061] Perform fractal dimension and heat conduction law verification on each defect candidate area to generate a defect confidence mask. Calculate the fractal dimension of the candidate area to determine whether it meets the crack characteristics. Analyze the temperature gradient vector around the candidate area to verify whether it meets the heat conduction law.

[0062] Extract the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determine whether the fracture length ratio exceeds the preset length threshold.

[0063] If the result is positive, the orientation angle is adjusted based on the fracture length and total edge length until the fracture length ratio in the defect confidence mask generated based on the adjusted orientation angle falls below a preset length threshold. Edge continuity is improved by iteratively optimizing the orientation angle.

[0064] If the judgment result is no, the defect confidence mask is fused with the dual-channel fusion feature map to generate a three-channel result map including the normalized temperature difference map, the deblurred edge map, and the defect confidence mask.

[0065] Through the above scheme, the present application realizes the coordinated optimization of vibration blur correction and heat conduction compensation. By inverting the vibration spectrum data into a point spread function matrix, the accurate deblurring of the visible light image is achieved. The spatial dislocation of the thermal image is eliminated by using the theoretical temperature field model and optical flow displacement compensation. The dual-channel feature fusion of edge gradient and temperature anomaly adopts dynamic weight adjustment to improve the accuracy of feature fusion. The dual verification of fractal dimension and heat conduction law effectively eliminates the interference of pseudo-defects. The iterative adjustment mechanism of the direction angle ensures the consistency of edge continuity and heat conduction direction. The comprehensive application of these technical means significantly improves the accuracy and reliability of equipment surface defect detection, and provides more accurate data support for predictive maintenance of industrial equipment.

[0066] This application further proposes to calculate the edge continuity scores of all connected domains in the defect confidence mask, perform fast Fourier transform on the vibration spectrum data to extract the primary vibration frequency and secondary vibration frequency and calculate the primary-secondary frequency energy ratio, generate an adjustment amount based on the edge continuity score and the primary-secondary frequency energy ratio, and update the direction angle.

[0067] Adjusting the direction angle based on the fracture length and the total edge length includes:

[0068] Calculate the edge continuity score of all connected regions in the defect confidence mask :

[0069] , where is the breaking length, is the total edge length;

[0070] Perform fast Fourier transform on the vibration spectrum data to extract the main vibration frequency and secondary vibration frequency , and calculate the energy ratio of the primary and secondary frequencies ;

[0071] According to edge continuity score Energy ratio of primary and secondary frequencies Generate adjustment amount :

[0072] ,in, 、 All are prefabricated weight coefficients;

[0073] Update direction angle :

[0074] , is the direction angle before updating, is the updated direction angle.

[0075] Extracting the main vibration frequency of the vibration spectrum by fast Fourier transform and sub-vibration frequencies , calculate the energy ratio of the primary and secondary frequencies , combined with the edge continuity score of the defect confidence mask Generate adjustment amount .

[0076] For example, when the sub-vibration frequency When the energy ratio increases, the energy ratio of the primary and secondary frequencies Reduce, adjust middle As the proportion increases, the adjustment range of the direction angle changes accordingly.

[0077] By pre-set weight coefficient 、 Score edge continuity Energy ratio of primary and secondary frequencies This fusion allows the azimuth angle update process to simultaneously address both edge fracture repair requirements and the dynamic characteristics of the vibration spectrum. The updated azimuth angle is then fed into the point spread function matrix reconstruction process to optimize the deconvolution process, thereby enhancing the edge continuity of subsequent defect confidence masks.

[0078] For example, in one embodiment, assuming that the total edge length of a connected region in the defect confidence mask is 100 pixels, break length is 20 pixels, the edge continuity score After the vibration spectrum data is transformed by fast Fourier transform, the energy of the main vibration frequency is 100, the energy of the secondary vibration frequency is 30, and the energy ratio of the main and secondary frequencies is is 0.3. Set to 0.6, Set to 0.4. The adjustment amount can be calculated from this for:

[0079] 0.6*(1-0.8)+0.4*(30 / 100)=0.24.

[0080] Assume the direction angle before update is 45°, then the updated direction angle for:

[0081] 45+0.24=45.24°.

[0082] Through the above technical solution, the present application can dynamically adjust the azimuth angle based on the edge continuity of the defect confidence mask and the characteristics of the vibration spectrum data. This adjustment mechanism takes into account the integrity of the defect edge and the complexity of the equipment vibration, making the azimuth angle update more accurate. This improves the accuracy of defect detection, reduces the interference of vibration on image processing, and enhances the adaptability and robustness of the system in complex industrial environments.

[0083] The present application further proposes that adjusting the direction angle according to the fracture length and the total edge length also includes:

[0084] Based on the updated direction angle , main vibration frequency Reprocess the next frame of visible light image;

[0085] Extract edge continuity scores of processed images , judge the edge continuity score Whether it is greater than the preset adjustment coefficient;

[0086] If the judgment result is yes, the current parameters are frozen;

[0087] If the judgment result is no, press Increase , Reduce Where 、 For pre-made increasing learning rates.

[0088] When processing visible light images, it is necessary to combine the updated direction angle with the dynamic changes of the main vibration frequency to ensure that the deconvolution processing adapts to the vibration state; the next frame of visible light image is processed in an iterative manner, and the adjustment results of the previous frame are used to optimize the current frame; edge continuity scoring The preset adjustment coefficient is pre-calibrated by experimental data to evaluate the effectiveness of parameter adjustment; the pre-made weight coefficient 、 The adjustment is based on the learning rate 、 and edge continuity score The product relationship of is used to achieve adaptive optimization of parameters.

[0089] Specifically, when processing visible light images, the updated direction angle is used to correct the fuzzy direction deviation caused by vibration, and the dynamic change of the main vibration frequency is combined to improve the deconvolution accuracy. When processing the next frame of image, the reconstructed matrix is ​​used for deconvolution to extract the edge continuity score of the processed image. If the score meets the standard, the current parameters are frozen to ensure the stability of the processing; if it does not meet the standard, the pre-set weight coefficient is 、 Adjustments are made according to pre-set rules, and the effects of angular corrections are balanced by dynamically adjusting parameters. This process gradually approaches the optimal parameter combination through iterative optimization, ensuring the reliability and efficiency of edge continuity improvement.

[0090] Through the above technical solution, the present application can achieve dynamic adjustment of the direction angle and weight coefficient. As a result, the parameters can be adaptively optimized according to the image processing effect, improving the accuracy and robustness of the deblurring process. Furthermore, by introducing the edge continuity score as a feedback indicator, the deblurring effect can be better evaluated and image distortion caused by excessive adjustment can be avoided. In addition, the freezing mechanism can be used to fix the parameters in time when the best effect is achieved, reducing unnecessary computational overhead.

[0091] This application further proposes that the extraction rules of sub-vibration frequencies include:

[0092] When there are multiple peaks in the spectrum, the peak with an energy share exceeding 10% is selected as the sub-vibration frequency;

[0093] If there is no sub-vibration frequency that meets the conditions, set the main and sub-frequency energy The ratio tends to infinity, that is The value is 0.

[0094] The minimum energy percentage is set to 10% to eliminate low-energy noise interference and ensure the validity of secondary vibration frequencies. When the spectrum contains multiple peaks, only candidate frequencies with energy percentages exceeding this threshold are screened to prevent excessive influence of secondary vibration components on the azimuth adjustment. If no secondary vibration frequencies meet the requirements, the energy ratio of the primary and secondary frequencies is set to infinity to eliminate interference from invalid secondary frequencies on the adjustment calculation.

[0095] Specifically, during spectrum analysis, all detected peaks are sorted by energy intensity. When multiple peaks are present, only the sub-vibration frequencies with an energy contribution exceeding 10% are retained as valid input.

[0096] For example, when the energy contribution of a secondary vibration frequency is 12%, this frequency is included in the calculation of the primary-secondary frequency energy ratio; if the energy contribution of a secondary vibration frequency is 8%, it is deemed invalid. When all secondary vibration frequencies do not reach the energy threshold, the energy of the secondary vibration frequency is forced to zero. At this time, the denominator in the primary-secondary frequency energy ratio formula approaches zero, and the calculated result approaches infinity. This processing method not only ensures the mathematical stability of the azimuth adjustment calculation, but also avoids the interference of low-energy noise on the vibration compensation process, effectively improving the calculation accuracy of the edge continuity score of the defect candidate area.

[0097] Through the above technical solution, the present application can accurately extract the secondary vibration frequency, providing a reliable basis for subsequent azimuth adjustment. When there are significant secondary vibration frequencies, this information can be effectively utilized to optimize the reconstruction process of the point spread function matrix. In the absence of significant secondary vibration frequencies, by setting the primary and secondary frequency energy ratio to infinity, unnecessary interference can be avoided, ensuring the stability and reliability of the azimuth adjustment. This flexible processing method can adapt to different vibration scenarios and improve the system's adaptability to complex vibration environments.

[0098] This application further proposes a process for obtaining a normalized temperature difference map based on the reference temperature data and the deblurred image:

[0099] Constructing a theoretical temperature field model based on the reference temperature data through a heat conduction equation, and calculating a normalized temperature difference map between the infrared thermal image and the theoretical temperature field model;

[0100] Calculate the optical flow displacement vector based on consecutive frames of the deblurred image;

[0101] Pixel displacement compensation is performed on the normalized temperature difference map according to the optical flow displacement vector to obtain a corrected normalized temperature difference map.

[0102] When constructing a theoretical temperature field model based on the heat conduction equation, a three-dimensional thermal field distribution model must be established in combination with the thermal conductivity and specific heat capacity parameters of the target device material. The steady-state temperature field is then solved using the finite element method. The optical flow displacement vector is calculated using the Horn-Schunck or Lucas-Kanade algorithm, which calculates the displacement vector field by taking the pixel brightness gradients of two adjacent frames in a deblurred image sequence. Pixel displacement compensation is achieved using bilinear interpolation, which reverses the coordinates of each pixel in the normalized temperature difference map according to the optical flow displacement vector to compensate for image misalignment caused by vibration.

[0103] Specifically, when constructing the theoretical temperature field model based on the heat conduction equation, the reference temperature data is input as the boundary condition, and the theoretical temperature value of each point on the surface of the device is obtained through iterative calculation to generate a two-dimensional matrix of the theoretical temperature field. The measured infrared thermal image is subtracted from the theoretical temperature field matrix pixel by pixel, and then divided by the maximum value of the theoretical temperature field matrix to obtain the initial normalized temperature difference map. When calculating the optical flow displacement vector based on continuous frames of the deblurred image, the pyramid layered optical flow method is used to reduce the calculation error in the case of large displacement through multi-scale image decomposition. During the pixel displacement compensation process, a coordinate mapping relationship is established based on the optical flow displacement vector, and an affine transformation is performed on the normalized temperature difference map to eliminate the spatial offset between the thermal image and the visible light image caused by equipment vibration.

[0104] For example, when the device vibrates laterally, the optical flow displacement vector shows periodic changes in the X-axis direction. The compensation algorithm adjusts the pixel coordinates of the temperature difference map according to the mean value of the displacement vectors of consecutive frames, so that the temperature difference calculation is spatially aligned with the edge features of the visible light image.

[0105] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0106] Based on the baseline temperature data, a theoretical temperature field model was constructed using the heat conduction equation. The normalized temperature difference between the infrared thermal image and the theoretical temperature field model was calculated. The heat conduction equation uses Fourier's law, taking into account parameters such as the thermal conductivity, specific heat capacity, and density of the device surface material. The theoretical temperature field model was solved using the finite element method, using a hexahedral mesh with a 5mm edge length.

[0107] The optical flow displacement vectors are calculated based on consecutive frames of the deblurred image. The Horn-Schunck optical flow algorithm is used with a smoothing constraint parameter of 0.5 and 100 iterations. The optical flow calculation window size is 15×15 pixels, with a step size of 5 pixels.

[0108] The normalized temperature difference map is compensated for pixel displacement based on the optical flow displacement vector to obtain a corrected normalized temperature difference map. Displacement compensation uses bilinear interpolation with a 3×3 interpolation kernel. The compensated temperature difference map is then registered with the deblurred image, with the registration error kept within 1 pixel.

[0109] Through the above technical solution, this application effectively corrects pixel displacement and blurring caused by vibration in thermal images. The temperature field model constructed based on heat conduction theory provides an accurate benchmark, which, combined with the displacement vector calculated by the optical flow algorithm, can accurately compensate for pixel shifts caused by vibration. The corrected normalized temperature difference image retains the thermal characteristics of the actual defect while effectively suppressing interference from non-target heat sources caused by vibration, improving the accuracy and reliability of subsequent defect detection.

[0110] This application further proposes a weighted fusion process of dual-channel fusion feature maps as follows:

[0111] Calculate the signal-to-noise ratio of the edge gradient map The signal-to-noise ratio of the normalized temperature difference map after correction ;

[0112] According to the weight distribution formula 、 Perform weighted superposition on the edge gradient map and the corrected normalized temperature difference map.

[0113] The signal-to-noise ratio of the edge gradient image is obtained by calculating the ratio of the mean value to the standard deviation of the gradient amplitude, and the signal-to-noise ratio of the normalized temperature difference image is obtained by calculating the ratio of the mean value to the standard deviation of the temperature difference.

[0114] Weight distribution formula By dynamically quantifying the difference in the signal-to-noise ratio of the two channel data, the fusion weight is adaptively adjusted; for example, when the signal-to-noise ratio of the edge gradient map is The signal-to-noise ratio of the normalized temperature difference map is 0.8 When it is 0.5, the weight coefficient The calculation is 0.8 / (0.8+0.5)=0.615, It is 1-0.615=0.385.

[0115] Specifically, the edge gradient map is extracted after deconvolution of the visible light image, and correction is performed based on the normalized temperature difference map after optical flow displacement compensation. The signal-to-noise ratio of the two channels is calculated by statistically analyzing the signal mean and noise variance of the local area through a sliding window. For example, the mean and standard deviation of the gradient amplitude are calculated within a 32×32 pixel window to obtain the signal-to-noise ratio of the edge gradient map. is 0.75; calculate the mean and standard deviation of the temperature difference in the same window to obtain the signal-to-noise ratio of the normalized temperature difference map is 0.65. Calculated according to the weight distribution formula The calculation is 0.75 / (0.75+0.65)=0.536, It is 1-0.536=0.464.

[0116] A dual-channel fused feature map is generated through linear weighted superposition. The high-frequency details of the edge gradient map and the temperature anomaly information of the normalized temperature difference map are fused at a ratio of 0.536 and 0.464. For example, a weight of 0.536 on the edge gradient map preserves crack outlines, while a weight of 0.464 on the normalized temperature difference map suppresses non-defect thermal noise. This process objectively quantifies the quality of the two-channel data, avoids the subjective bias of manually set weights, and improves the fused feature map's ability to distinguish defects during the local signal-to-noise ratio analysis phase.

[0117] Through the above technical solution, this application achieves the adaptive fusion of edge gradient images and normalized temperature difference images. This fully utilizes the complementary information of the two images, improving the accuracy and robustness of defect detection. Specifically, when the signal-to-noise ratio of the edge gradient image is high, the fusion result will retain more edge information; when the signal-to-noise ratio of the normalized temperature difference image is high, the fusion result will retain more temperature anomaly information. This adaptive fusion method can effectively cope with changes in image quality under different working conditions and improve the overall performance of defect detection.

[0118] This application further proposes to perform local signal-to-noise ratio analysis on the dual-channel fusion feature map, including:

[0119] Sliding a detection window of a preset size on the dual-channel fusion feature map;

[0120] Calculate the local signal-to-noise ratio of the dual-channel data within the window;

[0121] It is determined whether the local signal-to-noise ratio is greater than a preset defect threshold. If the determination result is yes, the area is marked as a defect candidate area.

[0122] The size of the inspection window is determined based on the relationship between the minimum physical size of surface defects on the target device and the image resolution, for example, a 32×32 pixel square window is used. The calculation of the local signal-to-noise ratio (SNR) involves independent SNR evaluation of each channel in the dual-channel data. The SNR of the visible light edge gradient image is calculated as the ratio of the mean gradient amplitude to the standard deviation, while the SNR of the normalized temperature difference image is calculated as the ratio of the mean temperature difference to the standard deviation of the background noise. The preset defect threshold is set based on statistical analysis of historical defect samples. For example, a local SNR exceeding 5.0 is considered a valid defect signal.

[0123] Specifically, the detection window slides along the row and column directions of the image with a step size of 16 pixels, covering the entire dual-channel fusion feature map. At each window position, the sub-region data of the edge gradient map and the normalized temperature difference map are extracted respectively, and the independent signal-to-noise ratios of the two channels are calculated and the weighted average is taken as the comprehensive signal-to-noise ratio. When the comprehensive signal-to-noise ratio exceeds the preset threshold, the coordinate range covered by the current window is recorded as the defect candidate area. Through the sliding window mechanism, it is possible to effectively identify tiny defect areas while avoiding the problem of insufficient sensitivity caused by the global threshold setting. For example, when detecting cracks on the surface of high-voltage electrical equipment, this method can mark windows with a signal-to-noise ratio higher than 6.0 as candidate areas. After confirmation by subsequent verification steps, the crack detection rate is increased to 98.2%.

[0124] For example, the preset defect threshold may be set to 3. When the local signal-to-noise ratio of a certain detection window is greater than 3, the window area is marked as a defect candidate area.

[0125] Through the above-mentioned technical solution, the present application can effectively screen out potential defect areas from the dual-channel fusion feature map. Due to the use of a sliding window approach, the entire image can be fully scanned without missing any suspicious areas. By calculating the local signal-to-noise ratio and comparing it with a preset threshold, it can effectively distinguish abnormal areas with strong signals from normal background areas, thereby improving the accuracy of defect detection. Furthermore, because the analysis is performed on the dual-channel fusion feature map, it integrates visible light and infrared information, and has greater robustness than single-channel analysis, and can better cope with complex industrial environments. Furthermore, this method is computationally simple and easy to implement, meeting the needs of real-time processing.

[0126] This application further proposes that fractal dimension verification includes:

[0127] Performing binary segmentation on the defect candidate area and calculating the fractal dimension of the segmented area;

[0128] It is determined whether the fractal dimension is greater than a preset fractal dimension threshold value. If the determination result is yes, it is confirmed as a crack feature.

[0129] Binary segmentation uses an adaptive threshold algorithm based on local grayscale histograms to classify pixels in the candidate area into crack pixels and background pixels. Fractal dimension calculation uses a box counting method, which covers the crack area with grids of varying scales and calculates the logarithmic slope of the number of covered grids versus grid scale. The preset fractal dimension threshold can be set to an empirical value between 1.2 and 1.5, which is dynamically adjusted based on the surface material and texture characteristics of the device.

[0130] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0131] When performing binary segmentation on the candidate defect area, an adaptive threshold segmentation algorithm based on grayscale histogram is used to divide the pixel values ​​of the candidate area into two categories: foreground and background.

[0132] Specifically, by traversing the grayscale value distribution of all pixels in the candidate area, the optimal segmentation threshold that maximizes the inter-class variance is calculated, where the candidate area is converted into a binary image containing only pixel values ​​0 and 255. Furthermore, when performing fractal dimension calculation on the binarized area, the box counting method is used to cover the complexity of the image edge, in which square grids of different side lengths are overlaid on the binary image, and the number of grids required to cover the crack edge at each scale is counted. The relationship between the logarithm of the grid side length and the logarithm of the number of grids is fitted by linear regression, and the resulting slope is defined as the fractal dimension. When the fractal dimension calculation result exceeds the preset threshold of 1.2, the area is judged to conform to the fractal law of crack characteristics.

[0133] Through the above technical solution, this application effectively solves the problem of misjudgment caused by blurred crack edges in vibration environments. By quantifying the fractal characteristics of the crack area, it can distinguish the complex edge morphology of real cracks from the random irregular shapes caused by noise. The objective numerical verification mechanism based on fractal dimension avoids the sensitivity of traditional edge continuity analysis to vibration blur, significantly improving the accuracy and anti-interference ability of crack identification.

[0134] This application further proposes that the verification of heat conduction law includes:

[0135] Extracting a temperature gradient vector of a circular area with a predetermined pixel width around the defect candidate area;

[0136] Determining whether an angle between a gradient direction of a temperature gradient vector in the annular region and a direction predicted by a heat conduction equation exceeds a preset angle threshold;

[0137] If the judgment result is yes, it is confirmed to be a real defect.

[0138] The annular region of a predetermined pixel width is set to a width of 5 pixels around the defect candidate area. This region is unaffected by the abnormal thermal conduction of the defect itself and reflects normal thermal conduction patterns. The temperature gradient vector is obtained by calculating the temperature change rate and direction of adjacent pixels within the annular region. The predicted direction of the heat conduction equation is derived from a theoretical model constructed based on the device's baseline temperature data and the material's thermal conductivity. The preset angle threshold is set to 15 degrees. When the actual gradient direction deviates from the theoretical direction by more than this threshold, it indicates the presence of an abnormal heat source.

[0139] Specifically, after the fractal dimension is verified, a circular reference area is formed by expanding 5 pixels outward with the geometric center of the defect candidate area as the origin. The temperature value of each pixel in the area is extracted using the bicubic interpolation algorithm, and the Sobel operator is used to calculate the temperature gradient components in the X / Y direction to synthesize the temperature gradient vector field. At the same time, based on the thermal conductivity of the equipment material and the reference temperature data, the steady-state heat conduction equation is solved using the finite element method to generate a theoretical temperature gradient direction field. The measured gradient direction of the circular area is compared point by point with the theoretical direction. When the direction deviation of more than 60% of the sampling points is greater than 15 degrees, an abnormality judgment is triggered. This verification mechanism uses the physical law that real defects will inevitably destroy the normal heat conduction path to effectively eliminate false detections caused by vibration thermal noise or image artifacts, and improve the defect recognition accuracy to more than 92%.

[0140] Through the above technical solution, this application effectively solves the problem of false detection caused by interference from non-target heat sources on the equipment surface, and achieves accurate identification of heat conduction anomalies through the constraints of physical laws. The verification mechanism based on the basic thermodynamic equations can eliminate interfering factors such as vibration and frictional heat, making the defect detection results clearly physically interpretable and significantly improving the reliability of infrared thermal imaging defect identification under complex working conditions.

[0141] like Figure 3 As shown, the present application discloses an artificial intelligence-based computer vision image processing system, including a data processing module, a visible light image processing module, a temperature difference map acquisition module, a dual-channel fusion module, a defect recognition module, a mask generation module, and a verification feedback module.

[0142] Among them, the data processing module ensures the time alignment of multi-source data by synchronously collecting visible light images, infrared thermal images, vibration spectrum data and reference temperature data.

[0143] The visible light image processing module uses the main vibration frequency and direction angle to construct a point spread function matrix and eliminates image motion blur through deconvolution operation.

[0144] The temperature difference map acquisition module is used to obtain a normalized temperature difference map based on the reference temperature data and the deblurred image.

[0145] The dual-channel fusion module dynamically weights and superimposes the edge gradient map and the normalized temperature difference map through the signal-to-noise ratio.

[0146] The defect recognition module is used to perform local signal-to-noise ratio analysis on the dual-channel fusion feature map and select areas with a signal-to-noise ratio greater than a preset defect threshold as defect candidate areas.

[0147] The mask generation module calculates the fractal dimension of the candidate area. When the fractal dimension exceeds the preset fractal dimension threshold, it is determined to be a crack feature. At the same time, it detects whether the angle between the temperature gradient direction of the annular area and the theoretically predicted direction exceeds the preset angle threshold, and double verification is used to confirm the real defect.

[0148] The verification feedback module iteratively adjusts the directional angle parameters. When the edge fracture length ratio exceeds the threshold, the adjustment amount is calculated based on the ratio of the fracture length to the total length. After updating the directional angle, the point spread function matrix is ​​regenerated until the mask edge continuity meets the requirements. The verified mask is superimposed on the dual-channel feature map to generate a three-channel result map containing temperature distribution, edge details and defect location for visual diagnosis of the equipment status.

[0149] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0150] The data processing module synchronously triggers the visible light camera, infrared thermal imager, and vibration sensor to collect multimodal data on the device surface at the same timestamp. The visible light image resolution is 1920×1080 pixels, the infrared thermal image temperature resolution is 0.05°C, and the vibration spectrum data sampling frequency is 10kHz.

[0151] The visible light image processing module uses an elliptical point spread function constructed based on the main vibration frequency and iterates the Richardson-Lucy deconvolution algorithm 15 times to eliminate image motion blur.

[0152] The temperature difference map acquisition module solves the three-dimensional heat conduction equation through the finite element analysis method, generates a theoretical temperature field model, and then uses the phase correlation algorithm to calculate the optical flow displacement vector and implement sub-pixel displacement compensation on the normalized temperature difference map.

[0153] The dual-channel fusion module applies the Sobel operator to extract the edge gradient map, combines the signal-to-noise ratio characteristics of the normalized temperature difference map, and adopts an adaptive weighted fusion strategy to generate a 512×512 pixel fusion feature map.

[0154] The defect recognition module adopts a sliding window detection mechanism, calculates the local signal-to-noise ratio with 32×32 pixels as the detection unit, and screens out candidate areas with a signal-to-noise ratio exceeding 2.5.

[0155] The mask generation module calculates the fractal dimension through the box counting method and generates a binary defect confidence mask by combining the verification of the temperature gradient direction in the annular area.

[0156] The verification feedback module monitors the edge fracture ratio in real time. When the fracture length exceeds 15%, it automatically triggers iterative optimization of the directional angle until a continuous and complete defect contour is generated. Finally, it outputs a three-channel visualization result of the superimposed temperature field, edge features and defect area.

[0157] Through the above technical solution, this application effectively overcomes the multimodal data mismatch problem caused by equipment vibration, and significantly improves the spatial consistency of the deblurred image and temperature field data by dynamically adjusting the directional angle to optimize the point spread function model. The dual-channel features of edge gradient and temperature difference are integrated to enhance the sensitivity of identifying tiny defects under complex working conditions. The phased verification mechanism accurately distinguishes real defects from vibration noise through the dual constraints of fractal dimension and heat conduction law, avoiding misjudgment due to abnormal heat conduction. The adaptive feedback adjustment mechanism ensures the continuity of the defect edge, solving the problem of missed detection caused by edge fracture in a vibration environment by traditional methods.

[0158] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A computer vision image processing method based on artificial intelligence, characterized in that: The steps include: Obtain visible light images, infrared thermal images, vibration spectrum data, and reference temperature data of the target device surface at the same timestamp; Extracting the main vibration frequency and direction angle according to the vibration spectrum data and performing deconvolution processing on the visible light image to obtain a deblurred image; Obtaining a normalized temperature difference map based on the reference temperature data and the deblurred image; Extracting an edge gradient map from the deblurred image, and weightedly fusing the edge gradient map with the corrected normalized temperature difference map to generate a dual-channel fusion feature map; Performing local signal-to-noise ratio analysis on the dual-channel fusion feature map, and screening areas with a signal-to-noise ratio greater than a preset defect threshold as defect candidate areas; Perform fractal dimension verification and heat conduction law verification on each defect candidate area to generate a defect confidence mask; Extracting the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determining whether the fracture length ratio exceeds a preset length threshold; If the result of the determination is yes, the direction angle is adjusted according to the fracture length and the total edge length until the fracture length ratio in the defect confidence mask generated according to the adjusted direction angle is lower than the preset length threshold; If the judgment result is no, the defect confidence mask is fused with the dual-channel fusion feature map to generate a three-channel result map including the normalized temperature difference map, the deblurred edge map, and the defect confidence mask.

2. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The adjusting the direction angle according to the fracture length and the total edge length includes: Calculate the edge continuity score of all connected regions in the defect confidence mask : , where is the breaking length, is the total edge length; Perform fast Fourier transform on the vibration spectrum data to extract the main vibration frequency and sub-vibration frequencies , and calculate the energy ratio of the primary and secondary frequencies ; According to edge continuity score Energy ratio of primary and secondary frequencies Generate adjustment amount : ,in, 、 All are prefabricated weight coefficients; Update direction angle : , is the direction angle before updating, is the updated direction angle.

3. The computer vision image processing method based on artificial intelligence according to claim 2, characterized in that: The adjusting the direction angle according to the fracture length and the total edge length also includes: Based on the updated direction angle , main vibration frequency Reprocess the next frame of visible light image; Extract edge continuity scores of processed images , judge the edge continuity score Whether it is greater than the preset adjustment coefficient; If the judgment result is yes, the current parameters are frozen; If the judgment result is no, press Increase , Reduce Where 、 For pre-made increasing learning rates.

4. The computer vision image processing method based on artificial intelligence according to claim 2, characterized in that: The extraction rules of the sub-vibration frequency include: When there are multiple peaks in the spectrum, the peak with an energy share exceeding 10% is selected as the sub-vibration frequency; If there is no sub-vibration frequency that meets the conditions, set the main and sub-frequency energy The ratio tends to infinity, that is The value is 0.

5. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The process of obtaining the normalized temperature difference map based on the reference temperature data and the deblurred image is as follows: Constructing a theoretical temperature field model based on the reference temperature data through a heat conduction equation, and calculating a normalized temperature difference map between the infrared thermal image and the theoretical temperature field model; Calculate the optical flow displacement vector based on consecutive frames of the deblurred image; Pixel displacement compensation is performed on the normalized temperature difference map according to the optical flow displacement vector to obtain a corrected normalized temperature difference map.

6. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The weighted fusion process of the dual-channel fusion feature map is: Calculate the signal-to-noise ratio of the edge gradient map The signal-to-noise ratio of the normalized temperature difference map after correction ; According to the weight distribution formula 、 Perform weighted superposition on the edge gradient map and the corrected normalized temperature difference map.

7. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The performing local signal-to-noise ratio analysis on the dual-channel fusion feature map includes: Sliding a detection window of a preset size on the dual-channel fusion feature map; Calculate the local signal-to-noise ratio of the dual-channel data within the window; It is determined whether the local signal-to-noise ratio is greater than a preset defect threshold. If the determination result is yes, the area is marked as a defect candidate area.

8. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The fractal dimension verification includes: Performing binary segmentation on the defect candidate area and calculating the fractal dimension of the segmented area; It is determined whether the fractal dimension is greater than a preset fractal dimension threshold value. If the determination result is yes, it is confirmed as a crack feature.

9. The computer vision image processing method based on artificial intelligence according to claim 1, characterized in that: The heat conduction law verification includes: Extracting a temperature gradient vector of a circular area with a predetermined pixel width around the defect candidate area; Determining whether an angle between a gradient direction of a temperature gradient vector in the annular region and a direction predicted by a heat conduction equation exceeds a preset angle threshold; If the judgment result is yes, it is confirmed to be a real defect.

10. A computer vision image processing system based on artificial intelligence, characterized in that: The method for computer vision image processing based on artificial intelligence according to any one of claims 1 to 9 is applied, comprising: A data processing module is used to obtain visible light images, infrared thermal images, vibration spectrum data and reference temperature data of the target device surface at the same time stamp; a visible light image processing module, configured to extract the main vibration frequency and direction angle according to the vibration spectrum data and perform deconvolution processing on the visible light image to obtain a deblurred image; A temperature difference map acquisition module, configured to obtain a normalized temperature difference map based on the reference temperature data and the deblurred image; A dual-channel fusion module is used to extract an edge gradient map from the deblurred image, and generate a dual-channel fusion feature map after weighted fusion of the edge gradient map and the corrected normalized temperature difference map; A defect recognition module is used to perform local signal-to-noise ratio analysis on the dual-channel fusion feature map and select areas with a signal-to-noise ratio greater than a preset defect threshold as defect candidate areas; The mask generation module is used to perform fractal dimension verification and heat conduction law verification on each defect candidate area to generate a defect confidence mask; Verification feedback module, which is used to perform the following steps: Extracting the fracture length, total edge length, and fracture length ratio of the edge fracture area in the defect confidence mask, and determining whether the fracture length ratio exceeds a preset length threshold; If the result of the determination is yes, the direction angle is adjusted according to the fracture length and the total edge length until the fracture length ratio in the defect confidence mask generated according to the adjusted direction angle is lower than the preset length threshold; If the judgment result is no, the defect confidence mask is fused with the dual-channel fusion feature map to generate a three-channel result map including the normalized temperature difference map, the deblurred edge map, and the defect confidence mask.

Images