Electromechanical temperature measurement and analysis system and method based on machine vision

Through machine vision technology, the temperature measurement of electromechanical equipment is carried out, and the three-dimensional temperature field is reconstructed by visible light image feature extraction and infrared image tracking, which solves the problems of low accuracy of temperature measurement and poor environmental adaptability in the existing technology, and realizes high-precision temperature monitoring and analysis.

CN118758430BActive Publication Date: 2024-12-03SHAOYANG ZHENGHE HENGXIN INFORMATION TECH DEV CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411088062.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing mechanical and electrical equipment temperature measurement methods are difficult to achieve large-scale and real-time temperature monitoring and analysis in complex industrial environments, and the traditional methods have low accuracy and are susceptible to environmental interference and equipment drift.

Method used

Using a machine vision-based method, by extracting the visible light images of electromechanical equipment and tracking the feature of infrared images, the three-dimensional temperature field of electromechanical equipment is reconstructed, combined with object detection and tracking strategies, a temperature model is established and data analysis is carried out to improve temperature detection. Accuracy.

Benefits of technology

Effectively reduce the influence of ambient light and reflection factors, provide detailed temperature change trends and distribution, improve measurement accuracy, is suitable for large-scale and multi-point temperature monitoring and analysis of complex industrial environments, and enhance equipment health status assessment and prediction and maintenance ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118758430B_ABST
    Figure CN118758430B_ABST
Patent Text Reader

Abstract

This invention discloses a machine vision-based electromechanical temperature measurement and analysis system and method, belonging to the field of electromechanical equipment temperature measurement technology. The method specifically includes: acquiring infrared images of the electromechanical equipment, and simultaneously acquiring visible light images of the electromechanical equipment as an auxiliary; preprocessing the acquired infrared and visible light images of the electromechanical equipment; using a target detection and tracking strategy to detect key parts of the electromechanical equipment in the preprocessed visible light image and track them in the preprocessed infrared image; establishing a temperature model of the electromechanical equipment based on the correspondence between the grayscale values ​​and temperature of the preprocessed infrared image; and mapping the established temperature model onto a three-dimensional model of the electromechanical equipment, reconstructing the three-dimensional temperature field of the electromechanical equipment, providing detailed temperature change trends and distribution, and improving temperature measurement accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of electromechanical equipment temperature measurement, and in particular to an electromechanical temperature measurement and analysis system and method based on machine vision. Background Art

[0002] During the operation of electromechanical equipment, temperature is one of the key parameters that affect its performance and life. Traditional temperature measurement methods mainly rely on devices such as thermocouples and infrared thermometers. These methods may be affected by environmental factors, limited measurement accuracy, and complex installation and maintenance in some cases. In addition, electromechanical equipment usually operates in complex industrial environments, making it difficult to achieve large-scale, real-time temperature monitoring and analysis.

[0003] With the development of machine vision technology, temperature measurement methods based on image processing and analysis have gradually attracted attention. Machine vision technology can capture images of the surface of equipment through cameras, extract temperature information through image processing algorithms, and analyze and process it. However, it is relatively difficult to extract electromechanical devices from infrared images, and the accuracy is low, so the accuracy is low when analyzing the temperature of electromechanical equipment.

[0004] For example, a Chinese patent with authorization announcement number CN108998608B discloses a method and system for measuring the temperature of molten iron at a blast furnace tapping hole based on infrared machine vision. The method obtains an infrared thermal image of the molten iron flow at the blast furnace tapping hole, performs image segmentation on the infrared thermal image based on a preset temperature threshold, obtains the molten iron area and the slag area, obtains the molten iron temperature at the tapping hole based on the temperature data of the slag area, and constructs an infrared temperature measurement result compensation model according to the texture features of the infrared thermal image affected by dust, and compensates for the molten iron temperature at the tapping hole according to the infrared temperature measurement result compensation model, thereby solving the technical problem that the existing method is difficult to continuously and accurately detect the molten iron temperature inside the blast furnace hearth, and can not only perform continuous online detection of the molten iron temperature at the tapping hole, but also compensate for the obtained molten iron temperature through the infrared temperature measurement result compensation model constructed according to the texture features of the infrared thermal image affected by dust, thereby greatly improving the detection accuracy of the molten iron temperature at the tapping hole.

[0005] For example, a Chinese patent with publication number CN117146719A discloses a machine vision measurement system and method, wherein the machine vision measurement system includes a light source unit, an image acquisition unit, a storage unit, a temperature measurement unit, a control unit and a data processing unit, wherein the light source unit is used to emit light to illuminate an external object to be measured, the image acquisition element collects light reflected by the external object and outputs first height data with height information of the object to be measured, wherein the storage unit is provided with a height compensation database, which stores height compensation data at different temperatures, the temperature measurement unit measures the working temperature of the machine vision measurement system, the data processing unit receives the first height data output by the image acquisition unit and obtains measured height data through processing, and then determines the height compensation data at this working temperature through the height compensation database according to the working temperature of the machine vision measurement system measured by the temperature measurement unit, and integrates the height compensation data with the obtained measured height data to output the corrected height data.

[0006] The defects of the above patents are: 1) The image segmentation method based on a preset temperature threshold may not be able to accurately distinguish between the molten iron and slag areas under complex circumstances, resulting in temperature data errors; 2) The long-term use of the infrared thermal imager may cause its sensor drift, requiring frequent calibration, otherwise the accuracy of the temperature measurement results will be affected; 3) The height compensation database performs height correction based on pre-stored compensation data, which may not be able to adapt to all temperature changes in actual operations, resulting in inaccurate compensation data; 4) The light source unit and the image acquisition unit may be affected by external light interference and mechanical vibration, resulting in unstable measurement data. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention proposes an electromechanical temperature measurement and analysis system and method based on machine vision. By extracting features from the visible light image of the electromechanical equipment and then tracking the features in the infrared image, the correspondence between the grayscale value and temperature of the infrared image of the electromechanical equipment is analyzed, the three-dimensional temperature field of the electromechanical equipment is reconstructed, and the overall temperature of the electromechanical equipment is obtained, which greatly improves the detection accuracy of the temperature of the electromechanical equipment.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] Electromechanical temperature measurement and analysis method based on machine vision, including:

[0010] Collect infrared images of electromechanical equipment, and collect visible light images of electromechanical equipment as auxiliary;

[0011] Preprocessing the collected infrared images and visible light images of the electromechanical equipment;

[0012] By using the target detection and tracking strategy, the key parts of the electromechanical equipment are detected in the pre-processed visible light image of the electromechanical equipment, and tracked in the pre-processed infrared image of the electromechanical equipment;

[0013] According to the corresponding relationship between the gray value and temperature of the infrared image of the electromechanical equipment after preprocessing, the temperature model of the electromechanical equipment is established;

[0014] Combined with the detection and tracking results of the target detection and tracking strategy, the established electromechanical equipment temperature model is mapped to the three-dimensional model of the electromechanical equipment to reconstruct the three-dimensional temperature field of the electromechanical equipment;

[0015] Perform data analysis on the reconstructed three-dimensional temperature field of electromechanical equipment to determine whether there is abnormal temperature rise and set the temperature threshold. When the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report.

[0016] Specifically, detecting the key parts of the electromechanical device in the pre-processed visible light image of the electromechanical device includes:

[0017] Collecting images of electromechanical devices in public data sets, and using the images of electromechanical devices in the public data sets as inputs of the neural network model;

[0018] Set the size of the convolution filter to 3*3, set the neural network depth to 20, select the loss function to train the electromechanical equipment images in the public dataset, and evaluate the neural network model until the evaluation value of the detection effect converges and remains unchanged, then stop training;

[0019] The preprocessed visible light image of the electromechanical equipment is input into the trained neural network model, and the initial set B of key parts of the electromechanical equipment is output, including the positions and categories of key components of the electromechanical equipment. The i-th position in the set B of key parts of the electromechanical equipment is tested again, and the score of the i-th position in the set B of key parts of the electromechanical equipment is calculated. The specific formula is:

[0020] ,

[0021] Among them, score(B i ) represents the score of the i-th position in the key position set B of the electromechanical equipment, S(B i ) represents the original confidence score of the i-th position in the key position set B of the electromechanical equipment, Represents the indicator function. When the IoU between the i-th position in the key part position set B of the electromechanical equipment and the j-th position in the key part position set B of the electromechanical equipment is greater than a certain threshold, =1, otherwise =0, Thr represents the threshold for determining the overlap between detection frames;

[0022] The key parts set B' of electromechanical equipment is obtained.

[0023] Specifically, the tracking in the infrared image of the pre-processed electromechanical device includes:

[0024] Using correlation filtering, target tracking is performed on the infrared image of the preprocessed electromechanical equipment to obtain the key parts set in the infrared image of the preprocessed electromechanical equipment. The specific formula for target tracking is:

[0025] ,

[0026] Where H represents the set of key parts in the infrared image of the electromechanical equipment after preprocessing, F represents Fourier transform, which is used to convert the time domain signal into the frequency domain signal, and F -1 represents the inverse Fourier transform, which is used to convert the frequency domain signal into the time domain signal. X represents the representation of the key parts set B' of the electromechanical equipment in the frequency domain. Y represents the representation of the infrared image of the electromechanical equipment after preprocessing in the frequency domain. It means that the elements are multiplied point by point, that is, the elements at corresponding positions are multiplied. represents the complex conjugate of the Fourier transform of X, represents the complex conjugate of the Fourier transform of Y, represents the regularization parameter.

[0027] Specifically, the specific formula of the loss function is:

[0028] ,

[0029] Where l(s,c,d) represents the loss function, s represents the electromechanical device images in the public dataset, c represents the category label of the electromechanical device images in the public dataset, d represents the predicted bounding box position, N represents the number of matched default boxes, and l con (s,c) represents the classification loss, l loc (s,d) represents the position loss, Represents the weight parameter.

[0030] Specifically, the establishment of the electromechanical equipment temperature model includes:

[0031] The gray value of the preprocessed infrared image of the electromechanical equipment is converted into a temperature value, and the temperature model of the electromechanical equipment is established. The specific formula is:

[0032] ,

[0033] Wherein, T represents the temperature model of the electromechanical equipment, that is, the temperature value converted from the gray value of the infrared image of the electromechanical equipment after preprocessing, C1 represents the first radiation constant of the Planck constant, C2 represents the second radiation constant of the Planck constant, represents the detection wavelength of the infrared camera, It represents the radiance corresponding to the gray value, E represents the emissivity of the target, and log represents the logarithmic function.

[0034] Specifically, mapping the established electromechanical equipment temperature model to the three-dimensional model of the electromechanical equipment to reconstruct the three-dimensional temperature field of the electromechanical equipment includes:

[0035] Using the existing 3D modeling method, a 3D model of the electromechanical equipment is established, and a 3D coordinate system is established;

[0036] According to the key parts set in the infrared image of the preprocessed electromechanical equipment and the temperature model of the electromechanical equipment, the three-dimensional temperature field of the electromechanical equipment is established to generate a continuous temperature distribution map. The specific formula of the three-dimensional temperature field of the electromechanical equipment is:

[0037] ,

[0038] Among them, T (x, y, z) represents the three-dimensional temperature field of the electromechanical equipment, that is, the temperature value of the position of the electromechanical equipment to be determined (x, y, z), T j represents the temperature value of the jth point of the electromechanical equipment, d i Represents the difference between the unknown point (x, y, z) and the known point (x j ,y j ,z j ), represents the distance weight index, represents a positive smoothing parameter, and M represents the number of known temperature points of the electromechanical device.

[0039] An electromechanical temperature measurement and analysis system based on machine vision, which is used to implement the electromechanical temperature measurement and analysis method based on machine vision, comprises: an image acquisition module, an image processing module, a target tracking module, a temperature model building module, a three-dimensional temperature field module and a temperature analysis module;

[0040] The image acquisition module is used to acquire infrared images of electromechanical equipment and simultaneously acquire visible light images of electromechanical equipment as an auxiliary;

[0041] The image processing module is used to pre-process the collected infrared image and visible light image of the electromechanical device;

[0042] The target tracking module is used to detect the key parts of the electromechanical device in the pre-processed visible light image of the electromechanical device by using the target detection and tracking strategy, and to track the key parts in the pre-processed infrared image of the electromechanical device;

[0043] The temperature model building module is used to build a temperature model of the electromechanical equipment according to the corresponding relationship between the gray value and the temperature of the preprocessed infrared image of the electromechanical equipment;

[0044] The three-dimensional temperature field module is used to map the established electromechanical equipment temperature model onto the three-dimensional model of the electromechanical equipment in combination with the detection and tracking results of the target detection and tracking strategy, and reconstruct the three-dimensional temperature field of the electromechanical equipment;

[0045] The temperature analysis module is used to perform data analysis on the reconstructed three-dimensional temperature field of the electromechanical equipment, determine whether there is an abnormal temperature rise, set a temperature threshold, and when the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report.

[0046] Specifically, the target tracking module includes: a key part detection unit and a target tracking unit;

[0047] The key part detection unit is used to detect the key part of the electromechanical device in the visible light image of the electromechanical device after preprocessing;

[0048] The target tracking unit is used to track the pre-processed infrared image of the electromechanical device and detect key parts of the electromechanical device in the pre-processed infrared image of the electromechanical device.

[0049] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of an electromechanical temperature measurement and analysis method based on machine vision when executing the computer program.

[0050] A computer-readable storage medium stores computer instructions, which, when executed, execute the steps of an electromechanical temperature measurement and analysis method based on machine vision.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. The present invention proposes an electromechanical temperature measurement and analysis method based on machine vision, which adopts advanced image processing algorithms and calibration techniques, can effectively reduce the influence of factors such as ambient light and reflection on temperature measurement, and then perform target tracking in the infrared image of electromechanical equipment, provide detailed temperature change trends and distribution, and improve measurement accuracy.

[0053] 2. The present invention proposes an electromechanical temperature measurement and analysis method based on machine vision, which is applicable to various complex industrial environments. It can monitor and analyze the temperature in a large range and at multiple points, and is particularly suitable for temperature management of large electromechanical equipment and complex systems.

[0054] 3. The present invention proposes an electromechanical temperature measurement and analysis method based on machine vision. The temperature measurement method based on machine vision can be integrated with other monitoring data to perform multi-dimensional data analysis and provide more comprehensive equipment health status assessment and predictive maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flow chart of the electromechanical temperature measurement and analysis method based on machine vision provided by the present invention;

[0056] Figure 2 This is a diagram of the architecture of the electromechanical temperature measurement and analysis system based on machine vision provided by the present invention;

[0057] Figure 3 A schematic diagram of an electronic device provided by the present invention;

[0058] Figure 4 A schematic diagram of the computer-readable storage medium structure provided by the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, in the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate the orientation or position relationship based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "No. 1", "No. 2" and "No. 3" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The present invention is further explained below in conjunction with specific embodiments.

[0060] Example 1

[0061] See also Figure 1 , an embodiment provided by the present invention: an electromechanical temperature measurement and analysis method based on machine vision, comprising the following specific steps:

[0062] Step S1: using a high-resolution infrared camera to collect an infrared image of the electromechanical device, and at the same time collect a visible light image of the electromechanical device as an auxiliary;

[0063] Step S2: preprocessing the collected infrared image and visible light image of the electromechanical device, including denoising, enhancement, grayscale and other operations;

[0064] Step S3: using a target detection and tracking strategy, detecting key parts of the electromechanical device in the preprocessed visible light image of the electromechanical device, and tracking them in the preprocessed infrared image of the electromechanical device;

[0065] The specific steps of step S3 are:

[0066] Step S301: collecting images of electromechanical devices in a public data set, and using the images of electromechanical devices in the public data set as inputs of a neural network model;

[0067] Step S302: Set the size of the convolution filter to 3*3, set the neural network depth to 20, select the loss function to train the electromechanical equipment images in the public data set, and evaluate the neural network model until the evaluation value of the detection effect converges and remains unchanged, then stop the training;

[0068] Step S303: Input the preprocessed visible light image of the electromechanical device into the trained neural network model, output the initial set B of key parts of the electromechanical device, including the positions and categories of key components of the electromechanical device, perform secondary detection on the i-th position in the set B of key parts of the electromechanical device, and calculate the score of the i-th position in the set B of key parts of the electromechanical device. The specific formula is:

[0069] ,

[0070] Among them, score(B i ) represents the score of the i-th position in the key position set B of the electromechanical equipment, S(B i ) represents the original confidence score of the i-th position in the key position set B of the electromechanical equipment, Represents the indicator function. When the IoU (intersection over union) of the i-th position in the key part position set B of the electromechanical equipment and the j-th position in the key part position set B of the electromechanical equipment is greater than a certain threshold, =1, otherwise =0, Thr represents the threshold for determining the overlap between detection frames,

[0071] Get the key parts set B' of electromechanical equipment;

[0072] In this step, the key position detection of the electromechanical equipment is carried out using the detection frame;

[0073] The purpose of step S303 is to adjust the score of each detection frame by considering the overlap between detection frames during the non-maximum suppression (NMS) process, so as to eliminate detection frames with excessive overlap, thereby retaining the most representative detection results;

[0074] Step S304: Utilize correlation filtering to perform target tracking on the infrared image of the preprocessed electromechanical device to obtain a set of key parts in the infrared image of the preprocessed electromechanical device. The specific formula for target tracking is:

[0075] ,

[0076] Where H represents the set of key parts in the infrared image of the electromechanical equipment after preprocessing, F represents Fourier transform, which is used to convert the time domain signal into the frequency domain signal, and F -1 represents the inverse Fourier transform, which is used to convert the frequency domain signal into the time domain signal. X represents the representation of the key parts set B' of the electromechanical equipment in the frequency domain. Y represents the representation of the infrared image of the electromechanical equipment after preprocessing in the frequency domain. It means that the elements are multiplied point by point, that is, the elements at corresponding positions are multiplied. represents the complex conjugate of the Fourier transform of X, represents the complex conjugate of the Fourier transform of Y, Represents the regularization parameter, which is used to prevent overfitting and improve the generalization ability of the model.

[0077] This formula describes how to calculate the filter response map in the frequency domain, convert the input features and target template features to the frequency domain through Fourier transform, and then convert the result back to the time domain through inverse Fourier transform. The regularization parameter in the formula Used to introduce a certain smoothing into the calculations, preventing the filter from overfitting to a particular noise pattern.

[0078] The loss function in step S302 is specifically formulated as follows:

[0079] ,

[0080] Where l(s,c,d) represents the loss function, s represents the electromechanical device images in the public dataset, c represents the category label of the electromechanical device images in the public dataset, d represents the predicted bounding box position, and N represents the number of matched default boxes, which is used to normalize the loss function to ensure that the loss value is proportional to the number of training samples. con (s,c) represents the classification loss, that is, the difference between the predicted category probability and the true category. Softmax loss is used to calculate the classification loss of each default box. loc(s, d) represents the position loss, which is the difference between the predicted bounding box position and the true bounding box position. Represents the weight parameter, which is used to balance the impact of classification loss and position loss and is adjusted according to the specific task.

[0081] Classification loss con The specific formula of (s,c) is: ,in, represents the predicted probability of the i-th default box, Pos and Neg represent the collection of positive samples and negative samples respectively, and the position loss l loc The specific formula for (s,d) is: ,

[0082] Among them, w i represents the predicted position of the i-th default box, Indicates the true position of the i-th default box, smooth L1 Represents the smooth L1 loss function, which is used to calculate the position loss;

[0083] The neural network model is such as YOLO (You Only Look Once) or Faster R-CNN;

[0084] Step S4: establishing a temperature model of the electromechanical equipment according to the corresponding relationship between the gray value and the temperature of the infrared image of the electromechanical equipment after preprocessing;

[0085] The step S4 of establishing a temperature model of the electromechanical equipment includes:

[0086] The gray value of the preprocessed infrared image of the electromechanical equipment is converted into a temperature value, and the temperature model of the electromechanical equipment is established. The specific formula is:

[0087] ,

[0088] Wherein, T represents the temperature model of the electromechanical equipment, that is, the temperature value converted from the gray value of the infrared image of the electromechanical equipment after preprocessing, C1 represents the first radiation constant of the Planck constant, C2 represents the second radiation constant of the Planck constant, represents the detection wavelength of the infrared camera, It represents the radiance corresponding to the gray value, E represents the emissivity of the target, and log represents the logarithmic function.

[0089] Here T is the temperature value of the electromechanical equipment that needs to be calculated in the end, indicating the actual temperature of the electromechanical equipment, and the radiation brightness It refers to the infrared energy radiated by electromechanical equipment captured by the infrared camera, which is expressed in grayscale values ​​and needs to be converted into actual radiation brightness. The emissivity E of the target is a coefficient between 0 and 1, which represents the radiation capacity of the electromechanical material equipment.

[0090] Step S5: combining the detection and tracking results of the target detection and tracking strategy, mapping the established electromechanical equipment temperature model to the three-dimensional model of the electromechanical equipment, and reconstructing the three-dimensional temperature field of the electromechanical equipment;

[0091] The specific steps of step S5 are:

[0092] Step S501: using a 3D modeling method in the prior art to establish a 3D model of the electromechanical device and a 3D coordinate system;

[0093] Step S502: According to the set of key parts in the infrared image of the preprocessed electromechanical equipment and the temperature model of the electromechanical equipment, a three-dimensional temperature field of the electromechanical equipment is established to generate a continuous temperature distribution map. The specific formula of the three-dimensional temperature field of the electromechanical equipment is:

[0094] ,

[0095] Among them, T (x, y, z) represents the three-dimensional temperature field of the electromechanical equipment, that is, the temperature value of the position of the electromechanical equipment to be determined (x, y, z), T j represents the temperature value of the jth point of the electromechanical equipment, which is obtained according to the electromechanical equipment temperature model, d i Represents the difference between the unknown point (x, y, z) and the known point (x j ,y j ,z j ), Represents the distance weight index, which is used to control the impact of distance on weight. It represents a positive smoothing parameter, which is used to avoid the situation where the score is zero when the distance is zero, and improve the stability of the calculation. M represents the number of known temperature points of the electromechanical equipment.

[0096] The point to be found (x, y, z) and the known point (x j ,y j ,z j The specific formula for the distance between ;

[0097] Step S6: Perform data analysis on the reconstructed three-dimensional temperature field of the electromechanical equipment to determine whether there is an abnormal temperature rise and set a temperature threshold. When the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report.

[0098] Example 2

[0099] See also Figure 2, another embodiment provided by the present invention: an electromechanical temperature measurement and analysis system based on machine vision, comprising: an image acquisition module, an image processing module, a target tracking module, a temperature model building module, a three-dimensional temperature field module and a temperature analysis module;

[0100] The image acquisition module is used to acquire infrared images of electromechanical equipment and simultaneously acquire visible light images of electromechanical equipment as an auxiliary;

[0101] The image processing module is used to pre-process the collected infrared image and visible light image of the electromechanical device;

[0102] The target tracking module is used to detect the key parts of the electromechanical device in the pre-processed visible light image of the electromechanical device by using the target detection and tracking strategy, and to track the key parts in the pre-processed infrared image of the electromechanical device;

[0103] The temperature model building module is used to build a temperature model of the electromechanical equipment according to the corresponding relationship between the gray value and the temperature of the preprocessed infrared image of the electromechanical equipment;

[0104] The three-dimensional temperature field module is used to map the established electromechanical equipment temperature model onto the three-dimensional model of the electromechanical equipment in combination with the detection and tracking results of the target detection and tracking strategy, and reconstruct the three-dimensional temperature field of the electromechanical equipment;

[0105] The temperature analysis module is used to perform data analysis on the reconstructed three-dimensional temperature field of the electromechanical equipment, determine whether there is an abnormal temperature rise, set a temperature threshold, and when the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report.

[0106] The target tracking module includes: a key part detection unit and a target tracking unit;

[0107] The key part detection unit is used to detect the key part of the electromechanical device in the visible light image of the electromechanical device after preprocessing;

[0108] The target tracking unit is used to track the pre-processed infrared image of the electromechanical device and detect key parts of the electromechanical device in the pre-processed infrared image of the electromechanical device.

[0109] Example 3

[0110] Figure 3 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the electromechanical temperature measurement and analysis method based on machine vision can be executed.

[0111] The method or system according to the embodiment of the present application can also be used by Figure 3 The electronic device architecture shown in FIG. Figure 3 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a ROM 503, a RAM 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store the electromechanical temperature measurement and analysis method based on machine vision provided in the present application. The electromechanical temperature measurement and analysis method based on machine vision may, for example, include: collecting infrared images of electromechanical equipment, and collecting visible light images of electromechanical equipment as an aid, preprocessing the collected infrared images and visible light images of electromechanical equipment, using target detection and tracking strategies, detecting key parts of the electromechanical equipment in the visible light images of the preprocessed electromechanical equipment, and tracking them in the infrared images of the preprocessed electromechanical equipment, establishing an electromechanical equipment temperature model based on the correspondence between the grayscale value and the temperature of the infrared images of the preprocessed electromechanical equipment, mapping the established electromechanical equipment temperature model to the three-dimensional model of the electromechanical equipment in combination with the detection and tracking results of the target detection and tracking strategy, reconstructing the three-dimensional temperature field of the electromechanical equipment, performing data analysis on the reconstructed three-dimensional temperature field of the electromechanical equipment, determining whether there is an abnormal temperature rise, setting a temperature threshold, and when the temperature of the electromechanical equipment exceeds the threshold, the system will send an alarm signal and automatically generate an alarm report. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 3 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 3 One or more components of an electronic device are shown.

[0112] Example 4

[0113] Figure 4 Schematic diagram of a computer-readable storage medium structure provided by an embodiment of the present application. Figure 4As shown, it is a computer-readable storage medium 600 according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the electromechanical temperature measurement and analysis method based on machine vision according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0114] It should be understood that the methods, apparatuses, and devices of the present application can be implemented in many ways. For example, the methods, apparatuses, and devices of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0115] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0116] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The electromechanical temperature measurement and analysis method based on machine vision is characterized by: include: Collect infrared images of electromechanical equipment, and collect visible light images of electromechanical equipment as auxiliary; Preprocessing the collected infrared images and visible light images of the electromechanical equipment; By using the target detection and tracking strategy, the key parts of the electromechanical equipment are detected in the pre-processed visible light image of the electromechanical equipment, and tracked in the pre-processed infrared image of the electromechanical equipment; According to the corresponding relationship between the gray value and temperature of the infrared image of the electromechanical equipment after preprocessing, the temperature model of the electromechanical equipment is established; Combined with the detection and tracking results of the target detection and tracking strategy, the established electromechanical equipment temperature model is mapped onto the three-dimensional model of the electromechanical equipment to reconstruct the three-dimensional temperature field of the electromechanical equipment; Perform data analysis on the reconstructed three-dimensional temperature field of electromechanical equipment to determine whether there is an abnormal temperature rise and set a temperature threshold. When the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report. The detecting of the key parts of the electromechanical device in the pre-processed visible light image of the electromechanical device comprises: Collecting images of electromechanical devices in public data sets, and using the images of electromechanical devices in the public data sets as inputs of the neural network model; Set the size of the convolution filter to 3*3, set the neural network depth to 20, select the loss function to train the electromechanical equipment images in the public dataset, and evaluate the neural network model until the evaluation value of the detection effect converges and remains unchanged, then stop training; The preprocessed visible light image of the electromechanical equipment is input into the trained neural network model, and the initial set B of key parts of the electromechanical equipment is output, including the positions and categories of key components of the electromechanical equipment. The i-th position in the set B of key parts of the electromechanical equipment is tested again, and the score of the i-th position in the set B of key parts of the electromechanical equipment is calculated. The specific formula is: , Among them, score(B i ) represents the score of the i-th position in the key position set B of the electromechanical equipment, S(B i ) represents the original confidence score of the i-th position in the key position set B of the electromechanical equipment, Represents the indicator function. When the IoU between the i-th position in the key part position set B of the electromechanical equipment and the j-th position in the key part position set B of the electromechanical equipment is greater than a certain threshold, =1, otherwise =0, Thr represents the threshold for determining the overlap between detection frames; The key parts set B' of electromechanical equipment is obtained.

2. The electromechanical temperature measurement and analysis method based on machine vision according to claim 1, characterized in that: Tracking in the pre-processed infrared image of the electromechanical device includes: Using correlation filtering, target tracking is performed on the infrared image of the preprocessed electromechanical equipment to obtain the key parts set in the infrared image of the preprocessed electromechanical equipment. The specific formula for target tracking is: , Where H represents the set of key parts in the infrared image of the electromechanical equipment after preprocessing, F represents Fourier transform, which is used to convert the time domain signal into the frequency domain signal, and F -1 represents the inverse Fourier transform, which is used to convert the frequency domain signal into the time domain signal. X represents the representation of the key parts set B' of the electromechanical equipment in the frequency domain. Y represents the representation of the infrared image of the electromechanical equipment after preprocessing in the frequency domain. It means that the elements are multiplied point by point, that is, the elements at corresponding positions are multiplied. represents the complex conjugate of the Fourier transform of X, represents the complex conjugate of the Fourier transform of Y, represents the regularization parameter.

3. The electromechanical temperature measurement and analysis method based on machine vision as claimed in claim 2, characterized in that: The specific formula of the loss function is: , Where l(s,c,d) represents the loss function, s represents the electromechanical device images in the public dataset, c represents the category label of the electromechanical device images in the public dataset, d represents the predicted bounding box position, N represents the number of matched default boxes, and l con (s,c) represents the classification loss, l loc (s,d) represents the position loss, Represents the weight parameter.

4. The electromechanical temperature measurement and analysis method based on machine vision as claimed in claim 3, characterized in that: The step of establishing a temperature model of electromechanical equipment comprises: The gray value of the preprocessed infrared image of the electromechanical equipment is converted into a temperature value, and the temperature model of the electromechanical equipment is established. The specific formula is: , Wherein, T represents the temperature model of the electromechanical equipment, that is, the temperature value converted from the gray value of the infrared image of the electromechanical equipment after preprocessing, C1 represents the first radiation constant of the Planck constant, C2 represents the second radiation constant of the Planck constant, represents the detection wavelength of the infrared camera, It represents the radiance corresponding to the gray value, E represents the emissivity of the target, and log represents the logarithmic function.

5. The electromechanical temperature measurement and analysis method based on machine vision according to claim 4, characterized in that: The step of mapping the established electromechanical equipment temperature model to the three-dimensional model of the electromechanical equipment to reconstruct the three-dimensional temperature field of the electromechanical equipment includes: Using the existing 3D modeling method, a 3D model of the electromechanical equipment is established, and a 3D coordinate system is established; According to the key parts set in the infrared image of the preprocessed electromechanical equipment and the temperature model of the electromechanical equipment, the three-dimensional temperature field of the electromechanical equipment is established to generate a continuous temperature distribution map. The specific formula of the three-dimensional temperature field of the electromechanical equipment is: , Among them, T (x, y, z) represents the three-dimensional temperature field of the electromechanical equipment, that is, the temperature value of the position of the electromechanical equipment to be determined (x, y, z), T j represents the temperature value of the jth point of the electromechanical equipment, d i Represents the difference between the unknown point (x, y, z) and the known point (x j ,y j ,z j ), represents the distance weight index, represents a positive smoothing parameter, and M represents the number of known temperature points of the electromechanical device.

6. An electromechanical temperature measurement and analysis system based on machine vision, which is used to implement the electromechanical temperature measurement and analysis method based on machine vision according to any one of claims 1 to 5, characterized in that: include: Image acquisition module, image processing module, target tracking module, temperature model building module, three-dimensional temperature field module and temperature analysis module; The image acquisition module is used to acquire infrared images of electromechanical equipment and simultaneously acquire visible light images of electromechanical equipment as an auxiliary; The image processing module is used to pre-process the collected infrared image and visible light image of the electromechanical device; The target tracking module is used to detect the key parts of the electromechanical device in the pre-processed visible light image of the electromechanical device by using the target detection and tracking strategy, and to track the key parts in the pre-processed infrared image of the electromechanical device; The temperature model building module is used to build a temperature model of the electromechanical equipment according to the corresponding relationship between the gray value and the temperature of the preprocessed infrared image of the electromechanical equipment; The three-dimensional temperature field module is used to map the established electromechanical equipment temperature model onto the three-dimensional model of the electromechanical equipment in combination with the detection and tracking results of the target detection and tracking strategy, and reconstruct the three-dimensional temperature field of the electromechanical equipment; The temperature analysis module is used to perform data analysis on the reconstructed three-dimensional temperature field of the electromechanical equipment, determine whether there is an abnormal temperature rise, set a temperature threshold, and when the temperature of the electromechanical equipment exceeds the threshold, the system will issue an alarm signal and automatically generate an alarm report.

7. The electromechanical temperature measurement and analysis system based on machine vision according to claim 6, characterized in that: The target tracking module includes: a key part detection unit and a target tracking unit; The key part detection unit is used to detect the key part of the electromechanical device in the visible light image of the electromechanical device after preprocessing; The target tracking unit is used to track the pre-processed infrared image of the electromechanical device and detect key parts of the electromechanical device in the pre-processed infrared image of the electromechanical device.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the electromechanical temperature measurement and analysis method based on machine vision described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the steps of the electromechanical temperature measurement and analysis method based on machine vision described in any one of claims 1 to 5 are executed.