Data fusion method based on multi-source sensor

Abstract

The invention provides a data fusion method based on a multi-source sensor, and relates to the technical field of equipment state monitoring, and the method comprises the steps: obtaining thermal images of monitored equipment, obtaining a calorific value matrix, an area anomaly index and a temperature anomaly index according to the thermal images, forming an anomaly index of each thermal image, obtaining a temperature increase process variable quantity sequence, and obtaining a heat value matrix; further monitoring is started according to judgment, further monitoring time is obtained, a vibration curve and a voltage curve are obtained, further monitoring scores are generated, a thermal image is continuously obtained, and further monitoring indexes are formed according to the further monitoring scores, the abnormal index variation in the further monitoring process and the abnormal index variation in the temperature increasing process; according to the system and the method, data fusion is carried out on the multi-source sensor, so that the equipment is monitored in an omnibearing manner, and the operation state of the equipment is accurately judged.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment status monitoring, and specifically to a data fusion method based on multi-source sensors. Background Art

[0002] During the operation of modern industrial equipment, the stability and safety of the equipment are crucial. Traditional equipment monitoring methods mainly rely on a single sensor, such as a temperature sensor or a vibration sensor, for data collection and anomaly detection. However, these single-sensor methods have many deficiencies. In recent years, with the rapid development of sensor technology and data processing technology, multi-source sensor data fusion methods have gradually become a research hotspot. By integrating multiple sensors to obtain multi-dimensional data of the equipment, the operating state of the equipment can be reflected more comprehensively and accurately.

[0003] In the prior art, the publication number CN111754483A discloses a method and system for identifying abnormal equipment based on a thermal imager. The method includes: S1: Obtain the current location information and environmental information; S2: Generate and collect an infrared thermal image according to the environmental information, and identify the temperatures of all pixel points in the infrared thermal image; S3: Determine whether the temperature of each pixel point exceeds a threshold. If there are pixel points whose temperature exceeds the threshold, jump to step S4. If not, skip this infrared thermal image; S4: Collect a visible light image corresponding to the infrared thermal image; S5: Extract the corresponding equipment according to the visible light image and the infrared thermal image; S6: Identify the type, model, and common temperature anomaly fault types corresponding to the equipment according to the extracted equipment; S7: Transmit the type and model of the equipment, the common temperature anomaly fault types, the current location data, and the alarm information to the data monitoring center.

[0004] Although the disclosed technical document realizes the monitoring of equipment through a thermal imager, it does not achieve the collaborative monitoring of multiple sensors. Judging the state of the monitored equipment solely from the visual level is not comprehensive and accurate enough, and it is easy to have errors in judging the equipment state.

[0005] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the present invention is to provide a data fusion method based on multi-source sensors to solve the problems raised in the above background art.

[0007] To achieve the above purpose, the present invention provides the following technical solutions: A data fusion method based on multi-source sensors, the specific steps include: Obtain the thermal imaging map of the monitored device at a fixed frequency, and obtain the calorific value matrix of each thermal imaging map within the sliding time window. Set the calorific value threshold and obtain the abnormal pixel points in the calorific value matrix. According to the abnormal pixel points, obtain the area abnormality index and the temperature abnormality index respectively, and form the abnormality index of each thermal imaging map. Form an abnormal index sequence from the abnormal indexes of the thermal imaging maps within the sliding time window and obtain the temperature increase process change amount sequence. Set the abnormal index threshold, and start further monitoring according to the relationship between the abnormal index threshold and the abnormal index, and obtain the temperature increase process. Obtain the further monitoring time according to the temperature increase process. Obtain the vibration curve and voltage curve of the monitored device within the further monitoring time, and obtain the further monitoring score. Continuously obtain the thermal imaging map of the monitored device within the further monitoring time, and obtain the change amount of the abnormal index during the further monitoring process. Form the further monitoring index according to the further monitoring score, the change amount of the abnormal index during the further monitoring process, and the change amount of the abnormal index during the temperature increase process. Set threshold Ⅰ and threshold Ⅱ respectively, and make a judgment according to the relationship between the further monitoring index and threshold Ⅰ and threshold Ⅱ.

[0008] Furthermore, obtain the thermal imaging map of the monitored device at a fixed frequency. The thermal imaging is obtained by a thermal imager. Add a timestamp to each obtained thermal imaging map, and process the thermal imaging map by the median filtering method. Set the sliding window time to 1 hour, and obtain the calorific value matrix of each thermal imaging map within the sliding window time according to the pixel points in the thermal imaging map. The formula is as follows: ; Among them, is the calorific value matrix of the th thermal imaging map, is the calorific value of the pixel point in the th row and the th column, , , is the total number of thermal imaging maps, , , is the number of rows of pixel points in the thermal imaging map, is the number of columns of pixel points in the thermal imaging map; Set the calorific value threshold, calibrate the pixel points with calorific values exceeding the calorific value threshold, and obtain the abnormal pixel points. The logic is as follows: Perform binary processing on the thermal imaging map to obtain the binary mask of each thermal imaging map. The pixel points with calorific values not exceeding the calorific value threshold turn black, and the pixel points with calorific values exceeding the calorific value threshold turn white. The white pixel points are the abnormal pixel points.

[0009] Further, perform an area analysis on the abnormal pixel points in the thermal imaging map to obtain the area anomaly index of the abnormal pixel points. The formula is as follows: ; Wherein, is the area anomaly index of the th thermal imaging map, represents the number of abnormal pixel points in the th thermal imaging map, is the total number of pixel points in the thermal imaging map, represents the spatial aggregation degree of the pixel points in the th thermal imaging map, represents the time difference, are all coefficients, , is the retrieval variable of the thermal imaging map number, , ; The logic for obtaining the spatial aggregation degree is as follows: Obtain the positions of each abnormal pixel point in the thermal imaging map and number them respectively. The position is the row and column data of the abnormal pixel point in the thermal imaging map. Obtain the adjacency relationship between any two abnormal pixel points. The formula is as follows: ; Wherein, represents the adjacency relationship value between the th and the th abnormal pixel points, and respectively represent the positions of the th and the th abnormal pixel points; ; Wherein, is the spatial aggregation degree, represents the adjacency relationship value between the th and the th abnormal pixel points, and is the number of abnormal pixel points; The time difference represents the difference between the timestamps of two adjacent thermal imaging maps.

[0010] Further, perform a temperature analysis on the abnormal pixel points in the thermal imaging map to obtain the temperature anomaly index of the abnormal pixel points. The formula is as follows: ; Wherein, is the temperature anomaly index of the th thermal imaging map, is the The sum of the calorific values of all abnormal pixels in a thermal image are all coefficients , , is the retrieval variable of the thermal image number , ; Obtain the anomaly index of each thermal image, and the formula is as follows: ; Among them, The th anomaly index of the thermal image is the area anomaly index of the th thermal image is the temperature anomaly index of the th thermal image is the retrieval variable of the thermal image number , .

[0011] Furthermore, form an anomaly index sequence of the thermal images within the time window in chronological order, and obtain the change amount of the anomaly index sequence. The formula is as follows: ; Among them, is the change amount of the anomaly index The th anomaly index of the thermal image The th anomaly index of the thermal image is the retrieval variable of the thermal image number , ; Set an anomaly index threshold. When the anomaly index exceeds the anomaly index threshold, start further monitoring, and at the same time trace the temperature increase process. The logic for obtaining the temperature increase process is as follows: When the anomaly index exceeds the anomaly index threshold, check the change amount of the anomaly index sequence from back to front, and obtain a subsequence of continuously increasing anomaly indices that includes the last anomaly index. The subsequence is the temperature increase process; Form a sequence of change amounts of the anomaly index during the temperature increase process.

[0012] Furthermore, obtain the further monitoring time according to the sequence of change amounts of the temperature increase process. The formula is as follows: ; Among them, is the further monitoring time is the average value of the sequence of temperature increase process change amounts, is the standard deviation of the sequence of temperature increase process change amounts, is the change rate of the sequence of temperature increase process change amounts, is the average value threshold, is the standard deviation threshold, is the change rate threshold, is the time of the temperature increase process; The formula for the time of the temperature increase process is as follows: ; wherein, represents the timestamp of the th thermal imaging diagram during the temperature increase process, represents the timestamp of the first thermal imaging diagram during the temperature increase process; The formula for the change rate of the anomaly index is as follows: ; wherein, is the change rate, represents the anomaly index of the th thermal imaging diagram during the temperature increase process, represents the anomaly index of the first thermal imaging diagram during the temperature increase process.

[0013] Furthermore, during the further monitoring process, the vibration curve and voltage curve of the device are obtained. The vibration curve is obtained by a vibration sensor, and the voltage curve is obtained by an oscilloscope. The horizontal axes of the vibration curve and voltage curve are time, the starting moment is the moment when the further monitoring is started, the time length is the further monitoring time, the vertical axis of the vibration curve is the amplitude, and the vertical axis of the voltage curve is the voltage. The further monitoring score is obtained, ; wherein, is the further monitoring score, is the initial score value, is the vibration curve, is the initial amplitude, is the voltage curve, is the initial voltage, is the further monitoring time, and are coefficients, ; The initial score value is a preset value, the initial amplitude is the amplitude of the device during normal operation, and the initial voltage is the voltage of the device during normal operation.

[0014] Further, during the further monitoring process, continuously obtain the thermal imaging map of the monitored device and form the change amount of the abnormal index during the further monitoring process, and generate the further monitoring index. The formula is as follows: ; Wherein, is the further monitoring index, is the th change amount of the abnormal index during the further monitoring process, is the number of change amounts of the abnormal index during the further monitoring process, is the th change amount of the abnormal index in the change amount sequence of the temperature increase process, is the number of change amounts of the abnormal index in the change amount sequence of the temperature increase process, is the further monitoring score, is the initial value of the score.

[0015] Further, respectively set the threshold Ⅰ and threshold Ⅱ of the further monitoring index, and make a judgment according to the further monitoring index. The logic is as follows: When the further monitoring index is less than or equal to the threshold Ⅰ, the device continues to work in the current state; When the further monitoring index is greater than the threshold Ⅰ and less than the threshold Ⅱ, the device starts the heat dissipation system and continues to work; When the further monitoring index is greater than or equal to the threshold Ⅱ, the device starts the heat dissipation system and alarms, and the maintenance personnel intervene in the management.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The present invention obtains the abnormal index of the monitored equipment through the thermal imaging map, conducts further detection according to the abnormal index and obtains the temperature increase process. During the further monitoring process, obtain the vibration curve, voltage curve and thermal imaging map during the further monitoring process and conduct analysis to obtain the further monitoring index. Combine the further monitoring index with the threshold Ⅰ and threshold Ⅱ to judge the monitored device. The present invention analyzes the data obtained by multiple sensors to form the further monitoring index, fuses the data of multi-source sensors, monitors the device in all directions, and accurately realizes the judgment of the operating state of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of the overall method flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to specific embodiments.

[0019] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. The terms such as "comprising" or "including" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0020] Embodiment: Please refer to Figure 1 , the present invention provides a technical solution: A data fusion method based on multi-source sensors, the specific steps include: Step 1: Obtain the thermal imaging map of the monitored device at a fixed frequency, and obtain the calorific value matrix of each thermal imaging map within the sliding time window. Set the calorific value threshold and obtain the abnormal pixel points in the calorific value matrix. According to the abnormal pixel points, obtain the area anomaly index and the temperature anomaly index respectively, and form the anomaly index of each thermal imaging map; The said Step 1 includes the following contents: Step 101: Obtain the thermal imaging map of the monitored device at a fixed frequency. The thermal imaging is obtained by a thermal imager. Add a timestamp to each obtained thermal imaging map, and process the thermal imaging map by the median filtering method. Set the sliding window time to 1 hour, and obtain the calorific value matrix of each thermal imaging map within the sliding window time according to the pixel points in the thermal imaging map. The basis formula is as follows: ; Wherein, is the calorific value matrix of the th thermal imaging map, is the calorific value of the pixel point at the th row and the th column, , , is the total number of thermal imaging maps, , , is the number of rows of pixel points in the thermal imaging map, is the number of columns of pixel points in the thermal imaging map; Among them, the calorific values of each pixel in the thermal imaging map are organized by row and column positions to form a two-dimensional calorific value matrix, completely retaining the spatial information of the thermal distribution on the device surface.

[0021] Set a calorific value threshold, calibrate the pixel points whose calorific values exceed the calorific value threshold, and obtain abnormal pixel points. The logic is as follows: Perform binarization processing on the thermal imaging map to obtain a binary mask for each thermal imaging map. The pixel points with calorific values not exceeding the calorific value threshold turn black, and the pixel points with calorific values exceeding the calorific value threshold turn white. The white pixel points are the abnormal pixel points.

[0022] In this step, the thermal imaging map is obtained at a fixed frequency to ensure the continuity and timeliness of data acquisition. The median filtering method is used, and the neighborhood window size is set to 3X3. The calorific values within the neighborhood window are sorted in descending order, and the 5th largest value is taken to replace the calorific value of the pixel at the center of the window. The median filtering method can effectively suppress random noise, avoid the interference of single-point calorific value fluctuations on anomaly detection, and provide a high-quality data basis for subsequent analysis. The design of the sliding time window takes into account both short-term anomaly capture and long-term trend analysis, making the formation of the anomaly index sequence have time coherence and facilitating subsequent dynamic tracking. The binarization processing clearly calibrates the abnormal pixel points as white areas through the calorific value threshold, directly providing structured input for the area anomaly index calculation in step 102 and the temperature anomaly index in step 103, and avoiding redundant processing of the original thermal imaging data.

[0023] Step 102: Analyze the area of the abnormal pixel points in the thermal imaging map to obtain the area anomaly index of the abnormal pixel points. The formula is as follows: ; Among them, is the area anomaly index of the th thermal imaging map, represents the number of abnormal pixel points in the th thermal imaging map, is the total number of pixel points in the thermal imaging map, represents the spatial aggregation degree of the pixel points in the th thermal imaging map, represents the time difference, are all coefficients, , , is the retrieval variable of the thermal imaging map number, , ; The area anomaly index of the thermal image specifically reflects the abnormal conditions of the monitoring device in the thermal state. The area anomaly index is a comprehensive indicator. The larger the area anomaly index, the more serious the thermal anomaly of the device, which may be accompanied by an increased risk of heat accumulation, a decrease in device safety, and a potential threat of device failure. Therefore, this indicator has important reference value in device monitoring and maintenance.

[0024] It is the ratio of the number of abnormal pixel points to the total number of pixel points. By calculating the ratio of the number of abnormal pixels to the total number of pixels, a standardized indicator can be obtained, which is convenient for comparison between different thermal images. Thermal images of different devices or different resolutions may have different total numbers of pixels. Using the ratio can eliminate these effects and make the indicator comparable.

[0025] The spatial aggregation degree is directly introduced. The spatial aggregation degree is an important indicator for judging the concentration degree of abnormal heat. The non-uniformity of heat distribution directly affects the risk level of the device. Therefore, it is introduced as an independent term in the formula. Using the original value of the spatial aggregation degree can directly reflect the current state without additional transformation. It is used to calculate the change rate of the number of abnormal pixel points at the current moment and the previous moment. By calculating the change in the number of abnormal pixel points, the changing trend of the device's thermal state can be captured. This is crucial for timely detecting potential problems of the device. The faster the increase rate of abnormal pixels, the higher the risk of device failure. Therefore, this dynamic change needs to be reflected in the index.

[0026] The time change rate term of the spatial aggregation degree It is used to calculate the change rate of the spatial aggregation degree at the current moment and the previous moment. In addition to the change in the number of abnormal pixel points, the change in the spatial aggregation degree is also important. This can help judge the severity and urgency of the thermal anomaly. By observing the change in the aggregation degree, it can be determined whether the device is experiencing continuous thermal anomalies, which helps formulate corresponding maintenance strategies.

[0027] The area anomaly index of the thermal image is related to multiple internal variables. The number of abnormal pixel points directly reflects the degree of thermal anomaly on the surface of the device. Each abnormal pixel represents a possible heat source or potential failure area. When the device overheats, the increase in surface temperature will cause more pixel points to show an abnormal thermal state. Therefore, the larger the number, the larger the scale of the thermal anomaly, which directly affects the value of the area anomaly index. The total number of pixel points is the base number used to standardize the number of abnormal pixel points, providing a framework for relative comparison.

[0028] Spatial aggregation describes the distribution of abnormal pixels in the image and reflects the concentration degree of thermal anomalies. If the abnormal pixels are highly concentrated, it usually indicates that the heat in a specific area is too high, which may cause equipment damage or pose safety problems. Therefore, the change in aggregation directly affects the area anomaly index, indicating the thermal anomaly risk of the equipment.

[0029] The change rate of the number of abnormal pixels reflects the change rate of the number of abnormal pixels over time, and the change rate of spatial aggregation reflects the change of spatial aggregation over time. If the thermal anomaly increases rapidly in a short period of time, it may mean that the equipment is encountering serious thermal problems and has a high risk. This dynamic characteristic of the change is the key to evaluating the equipment status.

[0030] The number of abnormal pixels is the most direct indicator to measure the thermal anomaly of the equipment. If there are a large number of abnormal pixels on the surface of the equipment, it indicates that the equipment may have serious overheating problems. Therefore, the influence of this variable should be the greatest, so the coefficient is the largest. Spatial aggregation reflects the concentration degree of abnormal heat. A dense abnormal area is more likely to cause equipment damage than a scattered one. Therefore, the coefficient of spatial aggregation is second only to the number of abnormal pixels. The time change rate reflects the change trend of the abnormal situation. Although the change speed can provide important early warning information, its direct influence is usually lower than that of the number of abnormal pixels and spatial aggregation.

[0031] The term of the change rate of spatial aggregation reflects the change of aggregation. Although it is also meaningful in the fault trend analysis, its direct influence is the smallest compared with other factors. Ensure that the sum of all weights is 1, so that the influence of different terms can be compared on the same basis and maintain the relative proportional relationship.

[0032] The logic for obtaining spatial aggregation is as follows: Obtain the positions of each abnormal pixel point in the thermal imaging diagram and number them respectively. The position is the row and column data of the abnormal pixel point in the thermal imaging diagram. Obtain the adjacency relationship between any two abnormal pixel points. The formula is as follows: ; where represents the adjacency relationship value between the th and the th abnormal pixel points. and respectively represent the positions of the th and the th abnormal pixel points. The adjacency relation value reflects the adjacent relationship between two abnormal pixel points. When the adjacency relation value is 1, it means the two abnormal pixel points are adjacent and border on each other. When the adjacency relation value is 0, it means there is no border between the two abnormal pixel points, that is, they are not adjacent. By separately judging the magnitude relationship between the differences in the horizontal and vertical coordinates of the two abnormal pixel points and 1, when the absolute values of the differences in the horizontal coordinates and the differences in the vertical coordinates are both less than or equal to 1, it indicates that the abnormal pixel point with coordinates must be in the nine - grid centered on , which means is adjacent to . When at least one of the absolute values of the differences in the horizontal and vertical coordinates of the two abnormal pixel points is greater than 1, it indicates that the abnormal pixel point with coordinates is not in the nine - grid centered on , then is not adjacent to .

[0033] To obtain the spatial aggregation degree, the formula is as follows: ; where, is the spatial aggregation degree, represents the adjacency relation value between the th and the th abnormal pixel points, is the number of abnormal pixel points; The spatial aggregation degree specifically reflects the aggregation situation of all abnormal pixel points in the thermal imaging map and is an important indicator reflecting the concentration degree of abnormal pixel points. The larger the spatial aggregation degree, the more concentrated the abnormal pixel points in the thermal imaging map, then the temperature abnormal state is more obvious and uncontrollable, increasing the risk and reducing the equipment safety. Therefore, this indicator has important reference value. The numerator calculates the adjacency relation value between each abnormal pixel point and other abnormal pixel points respectively. Those with an adjacency relation are 1 and added to the numerator, while those without an adjacency relation are 0 and not added to the numerator. The value of the numerator represents twice the number of abnormal pixel points with an adjacency relation. The denominator is the non - linear increase of the abnormal pixel points, reflecting the non - linear influence of the number of abnormal pixel points on the spatial aggregation degree. The number of pixel points in the thermal imaging map is fixed. The increase in the number of abnormal pixel points will inevitably lead to a non - linear increase in the number of adjacency relations. If the numerator remains unchanged, the increased abnormal pixel points do not aggregate with other abnormal pixel points, then the value of the spatial aggregation degree decreases non - linearly. Constructing the spatial aggregation degree can unify the dimensions of different thermal imaging degrees, facilitating subsequent processing.

[0034] The time difference represents the difference between the timestamps of two adjacent thermal imaging maps.

[0035] In this step, the area anomaly index formula is used to comprehensively quantify the thermal distribution anomaly from multiple dimensions by considering the number of abnormal pixels, spatial aggregation degree, and its temporal change rate. The spatial aggregation degree is calculated by judging the adjacency relationship between abnormal pixel points to identify whether the abnormal area is scattered noise or concentrated hot spots. Aggregated abnormal pixels may represent local overheating of the device, while scattered points may be interference, enhancing the sensitivity of the algorithm to real faults. The weight allocation emphasizes the dominant position of the area ratio, ensuring that the algorithm responds preferentially to large-area anomalies while taking into account spatial morphology and temporal changes, such as anomalies that are small in scope but rapidly spreading. This index complements the temperature anomaly index in Step 103. The area reflects the scope of the anomaly, and the temperature reflects the energy intensity, jointly constituting the total anomaly index, providing multi-dimensional data support for the threshold determination and trend analysis in Step 2. In addition, the introduction of the time difference enables the quantification of the dynamic change of the anomaly index, preparing for the determination of the subsequent temperature increase process.

[0036] Step 103: Perform temperature analysis on the abnormal pixel points in the thermal imaging map to obtain the temperature anomaly index of the abnormal pixel points, based on the following formula: ; Where is the temperature anomaly index of the th thermal imaging map, is the sum of the heat values of all abnormal pixel points in the th thermal imaging map, are all coefficients, , , is the retrieval variable of the thermal imaging map number, , ; The temperature anomaly index specifically reflects the temperature anomaly state of the current thermal imaging map and also reflects the temperature change process. The higher the temperature anomaly state, the higher the overall temperature faced by this thermal imaging map, and the greater the overall temperature change between two adjacent thermal imaging maps. The heat in the abnormal area of the device accumulates rapidly, which may indicate an impending serious fault. By fusing the current heat and dynamic changes, the index can more sensitively capture the potential fault trend of the device. Directly introduce the sum of the calorific values of all abnormal pixel points in the thermal imaging map. The sum of the overall calorific values directly reflects the temperature condition of the monitored device. Therefore, this formula takes into account the sum of the calorific values of all abnormal pixel points. It is necessary to not only focus on the static temperature condition but also pay attention to the change of the overall calorific value. A rapid change in the sum of the calorific values of all abnormal pixel points means that the temperature change of the monitored device is also accelerating, indicating that the device may have a sharp temperature increase due to a malfunction, predicting that the device may be about to fail. Therefore, it is necessary to reflect the difference in the sum of the calorific values of all abnormal pixel points between different thermal imaging maps. When the sum of the calorific values of all abnormal pixel points is large, it indicates that the temperature of the currently monitored device is high, and the temperature anomaly index is also larger. When the difference in the sum of the calorific values of all abnormal pixel points between two adjacent thermal imaging maps increases, it means that the temperature change of the monitored device is more intense, and the temperature anomaly condition of the device is more obvious, and the temperature anomaly index is also larger. The sum of the calorific values of all abnormal pixel points only represents the overall calorific value condition of the thermal imaging map at that time and cannot reflect whether the device is operating stably or deteriorating rapidly. Obviously, the risk of a rapid deterioration of the temperature condition is more threatening than a stable high temperature. Therefore, the difference in the sum of the calorific values of all abnormal pixel points between two thermal imaging maps is set with a higher weight to reflect the significant risk brought by the rapid deterioration of the temperature condition, ensuring that the sum of all weights is 1, so that the influence of different items can be compared on the same basis and the relative proportional relationship can be maintained.

[0037] Obtain the anomaly index of each thermal imaging map, and the formula is as follows: ; where The anomaly index of the th thermal imaging map, is the area anomaly index of the th thermal imaging map, is the temperature anomaly index of the th

[0038] The anomaly index represents the overall anomaly condition of each thermal imaging map. Numerically, it directly reflects the anomaly state of each thermal imaging map. The larger the value, the greater the risk of the thermal imaging map in terms of abnormal temperature distribution or abnormal temperature value. Summarize the area anomaly index and temperature anomaly index of the thermal imaging map to reflect the anomaly state of the thermal imaging map from two dimensions and provide a direct numerical basis for subsequent calculations.

[0039] This step focuses on capturing the energy accumulation and change trend in the abnormal area through the temperature anomaly index formula. The intention of designing a higher weight for the dynamic temperature increase term in the coefficient is to emphasize that the "temperature rising speed" is more valuable for early warning than the "absolute heat value". For example, the temperature rising from 50°C to 70°C in a certain area may be more dangerous than staying stable at 80°C. The construction of the anomaly index realizes the integration of the spatial range and energy intensity, enabling the anomaly index sequence in step 2 to comprehensively reflect the equipment status. The key role of this step is to organically combine the static area analysis in step 102 with the dynamic temperature analysis in this step to form a more discriminative comprehensive index. At the same time, the anomaly index increment provides data reference for tracing the temperature increase process in step 2. For example, if the anomaly index rises three times in a row, it is determined as a potential fault, providing an accurate start signal for further monitoring in step 3.

[0040] Step 2: Form an anomaly index sequence from the anomaly indices of the thermal imaging maps within the sliding time window and obtain the change amount sequence of the temperature increase process. Set the anomaly index threshold, and start further monitoring according to the relationship between the anomaly index threshold and the anomaly index, and obtain the temperature increase process; The said step 2 includes the following contents: Form an anomaly index sequence of the thermal imaging maps within the time window in chronological order, and obtain the change amount of the anomaly index sequence. The formula is as follows: ; where is the change amount of the anomaly index, the anomaly index of the th thermal imaging map, is the retrieval variable of the thermal imaging map number, , ; The change amount of the anomaly index reflects the change status of each thermal imaging map from the overall situation. The larger the change amount of the anomaly index, the more intense the temperature change status of the monitored equipment, and the more obvious the rapid deterioration situation. Therefore, it is necessary to identify it and conduct the next step of research and judgment.

[0041] Set the anomaly index threshold. When the anomaly index exceeds the anomaly index threshold, start further monitoring. At the same time, trace the temperature increase process. The logic for obtaining the temperature increase process is as follows: When the anomaly index exceeds the anomaly index threshold, check the change amount of the anomaly index sequence from back to front, and obtain the subsequence of continuously increasing anomaly indices including the last anomaly index. The said subsequence is the temperature increase process; Form the change amount sequence of the anomaly index in the temperature increase process into the change amount sequence of the temperature increase process.

[0042] This step converts static detection into dynamic trend analysis through the change amount of the anomaly index, providing a time-series basis for subsequent decision-making. For the traceability of the temperature increase process, starting from the last anomaly index, continuously rising subsequences are searched backward to capture the "persistence of temperature increase". Short-term fluctuations do not trigger an alarm, but consecutive rises are determined to be a serious anomaly, avoiding interference from occasional noise. The sequence of change amounts in the temperature increase process generated in this step provides a direct input for the statistical parameter calculation in step 301, thereby dynamically determining the further monitoring time. The higher the change rate, the shorter the monitoring time. This connection ensures the coherence between step 1 and step 3, enabling the system to adaptively adjust the monitoring strategy according to the severity of the anomaly.

[0043] Step 3: Obtain the further monitoring time based on the temperature increase process, obtain the vibration curve and voltage curve of the monitored device during the further monitoring time, and obtain the further monitoring score; The said step 3 includes the following content: Step 301: Obtain the further monitoring time according to the sequence of change amounts in the temperature increase process, and the formula is as follows: ; Wherein, is the further monitoring time, is the average value of the sequence of change amounts in the temperature increase process, is the standard deviation of the sequence of change amounts in the temperature increase process, is the change rate of the sequence of change amounts in the temperature increase process, is the average value threshold, is the standard deviation threshold, is the change rate threshold, is the time of the temperature increase process; The further monitoring time reflects the time that requires further focused monitoring. The longer the further monitoring time is, it indicates that a longer time is needed to conduct more detailed monitoring and analysis of the equipment, and it also shows that the current state of the equipment is not an emergency state, allowing sufficient time for detailed monitoring and analysis. Therefore, this indicator has important reference value in equipment monitoring and maintenance. The average value of the sequence of temperature increase process change amounts reflects the overall temperature condition during the current temperature increase process. The larger the average value is, it means that the overall abnormal state performance during the temperature increase process is more obvious, indicating that the condition of the monitored equipment deteriorates more significantly, and the further monitoring time is smaller. The larger the standard deviation of the sequence of temperature increase process change amounts is, it shows that the step phenomenon of the abnormal state during the temperature increase process is more obvious. From another aspect, regarding the deterioration condition of the monitored equipment, the further monitoring time is smaller. The larger the change rate of the sequence of temperature increase process change amounts is, it indicates that the abnormal state during the temperature increase process deteriorates more rapidly, and the further monitoring time is smaller. Since the faster the deterioration speed is, the more urgently detailed judgment and analysis are needed, and the further monitoring time needs to be smaller to avoid missing the opportunity to handle abnormal conditions during long-term monitoring. Therefore, by taking the form of negative signs, the ratios of the average value to the average threshold, the standard deviation to the standard deviation threshold, and the change rate to the change rate threshold are respectively constructed to reflect the relationship between the changes in the temperature increase process and the conditions that the equipment can withstand. Taking the mean value of the three ratios and the coefficient 0.5 can prevent calculation problems caused by overly large values. Referencing the time of the temperature increase process reflects the principle based on the actual temperature increase condition. The temperature increase process has already occupied a certain amount of time, and the further monitoring time must be controlled. The time of the temperature increase process already reflects the time characteristics of the temperature increase process. The larger the time of the temperature increase process is, it indicates that the temperature increase is relatively slow in the early stage, providing a time margin for further monitoring in the later stage. Therefore, the further monitoring time is larger.

[0044] The formula for the time of the temperature increase process is as follows: ; Wherein, represents the timestamp of the th thermal imaging diagram during the temperature increase process, represents the timestamp of the first thermal imaging diagram during the temperature increase process; The time of the temperature increase process is the total time of the determined temperature increase process. Subtracting the time of the first thermal imaging diagram of the temperature increase process from the time of the last thermal imaging diagram gives the time of the temperature increase process.

[0045] The formula for the change rate of the abnormal index is as follows: ; Wherein, is the change rate, represents the The anomaly index of a thermal imaging map represents the anomaly index of the 1st thermal imaging map during the temperature increase process.

[0046] The rate of change represents the degree of speed of the temperature increase process. The faster the temperature increases, it indicates that the state of the monitored device is more urgent. Therefore, the rate of change needs to be considered in the process of obtaining the further monitoring time.

[0047] In this step, the further monitoring time is calculated through a dynamic formula to achieve the matching of the monitoring duration and the anomaly risk. For example: if the change amount during the temperature increase process fluctuates greatly, the further monitoring time is shortened to avoid missing the valuable processing time for the abnormal condition of the device in order to monitor more data; if the rate of change is extremely high, it is quickly shortened for a quick response. The time alignment design ensures the synchronization of thermal, mechanical, and electrical data, laying a foundation for multi-source fusion. The weight coefficient and denominator in the formula normalize the three parameters, balancing the influence of different statistics on the monitoring time and preventing a single parameter from dominating the decision-making. The core value of this step is to convert the time-series data in step 2 into an operable monitoring strategy, which not only avoids over-monitoring and missing the opportunity to handle abnormal conditions but also ensures full coverage of high-risk events.

[0048] Step 302: During the further monitoring process, obtain the vibration curve and voltage curve of the device. The vibration curve is obtained through a vibration sensor, and the voltage curve is obtained through an oscilloscope. The horizontal axis of the vibration curve and voltage curve is time, the start time is the moment when the further monitoring is started, the time length is the further monitoring time, the vertical axis of the vibration curve is amplitude, and the vertical axis of the voltage curve is voltage. Obtain the further monitoring score. ; Among them, is the further monitoring score, is the initial value of the score, is the vibration curve, is the initial amplitude, is the voltage curve, is the initial voltage, is the further monitoring time, and are coefficients, ; The further monitoring score specifically reflects the device state of the monitored device in the further monitoring link. It is a comprehensive index. The larger the further monitoring score, the farther the state of the device in the monitoring link deviates from the initial state, which also represents a more serious abnormal state and may potentially hide a greater fault threat. Therefore, this index has important reference value in the device guardianship and maintenance summary.

[0049] It reflects the degree to which the vibration state of the monitored device deviates from the initial vibration state during further monitoring. It converts the two-dimensional data of the vibration curve into a one-dimensional value reflecting the abnormal state, which is convenient for participating in calculations and has an intuitive manifestation. It reflects the degree to which the voltage state of the monitored device deviates from the initial voltage state during further monitoring. It converts the two-dimensional data of the voltage curve into a one-dimensional value reflecting the abnormal state, which is convenient for participating in calculations and has an intuitive manifestation. The square term enhances the manifestation of the deviation of vibration and voltage from the initial state during further monitoring. During the further monitoring time, the deviation degrees of vibration and voltage are integrated to present the deviation degree from an overall perspective and form specific values for the next calculation. When the vibration and voltage during further monitoring deviate from the initial vibration and voltage states, it indicates that the device has potential faults in mechanical and electrical aspects respectively. An excessive deviation degree means an increased risk of failure. Taking the initial value as a parameter term, it increases the value range of the further monitoring score, which is convenient for subsequent numerical processing and facilitates judging the degree of deviation of the further monitoring process from the initial state. Since the hard damage caused by vibration changes is difficult to recover and is more dangerous, and the voltage changes within a certain range will not cause significant damage, it reflects the importance of preventing mechanical damage.

[0050] The initial value of the score is a preset value. The initial amplitude is the amplitude of the device during normal operation, and the initial voltage is the voltage of the device during normal operation.

[0051] In this step, through the synchronous acquisition of vibration and voltage curves, vibration reflects the mechanical structure stability, and voltage reflects the power supply state, constructing an all-round state of the device operation state. The integral form can quantify the cumulative effect of the deviation of vibration and voltage from the normal values. The magnitude relationship between the coefficients reflects that mechanical faults are usually more destructive than voltage fluctuations. For example, bearing wear may directly lead to shutdown, while voltage sags can be alleviated by redundant power supplies. The calculation of the further monitoring score not only quantifies the real-time abnormal degree but also standardizes the score through step 401, which is convenient for comparison with historical data. The key role of this step is to expand the single thermal anomaly detection to multi-physical field joint analysis, providing data input in the vibration and voltage dimensions for the further monitoring index in step 401, enhancing the comprehensiveness and reliability of fault determination.

[0052] Step 4: Continuously obtain the thermal imaging map of the monitored device during the further monitoring time, and obtain the change amount of the abnormal index during the further monitoring process. Form a further monitoring index based on the further monitoring score, the change amount of the abnormal index during the further monitoring process, and the change amount of the abnormal index during the temperature increase process. Respectively set threshold I and threshold II, and make a judgment according to the relationship between the further monitoring index and threshold I and threshold II.

[0053] The said step 4 includes the following contents: Step 401: During further monitoring, continuously obtain the thermal imaging map of the monitored device and form the change amount of the abnormal index during the further monitoring process to generate a further monitoring index. The basis formula is as follows: ; Wherein, is the further monitoring index, is the th change amount of the abnormal index during the further monitoring process, is the number of change amounts of the abnormal index during the further monitoring process, is the th change amount of the abnormal index in the change amount sequence of the temperature increase process, is the number of change amounts of the abnormal index in the change amount sequence of the temperature increase process, is the further monitoring score, is the initial value of the score.

[0054] The further monitoring index reflects the change situation of the abnormal index in the further monitoring process compared with the temperature increase process. By the change situation of the abnormal index, it is judged whether the device continues to deteriorate at the speed of the temperature increase process after entering the further monitoring process. Introducing the ratio of the further monitoring score to the initial score highlights the deviation degree of the abnormal state in the further monitoring process compared with the initial state. By directly judging the deviation degree of the device in the further monitoring process compared with the initial state and the change state compared with the temperature increase process through the further monitoring index, the larger the further monitoring index, the more serious the abnormal state of the device is, especially the abnormal state in the further monitoring process. Therefore, this index has important reference value in device monitoring and maintenance.

[0055] represents the mean value of the change rate of the change amount of the abnormal index in the further monitoring process, reflecting the speed of increase of the abnormal index in the further monitoring process. The faster the mean value of the change rate is, the faster the abnormal index increases, the faster the abnormal state of the device deteriorates, and the higher the further monitoring index is. It represents the mean value of the change rate of the change amount of the anomaly index during the temperature increase process, reflecting the speed at which the anomaly index increases during the temperature increase process. When the change rate of the change amount of the anomaly index during the temperature increase process is greater than the change rate of the change amount of the anomaly index during the further monitoring process, it indicates that the deterioration speed of the equipment state slows down during the further monitoring process, and the further monitoring index will decrease. The ratio of the further monitoring score to the initial score reflects the degree of deviation of the abnormal state from the initial state during the further monitoring process. The higher the ratio of the further monitoring score to the initial score, the higher the degree of deviation of the equipment state from the initial state during the further monitoring process, and the higher the further monitoring index. Multiplying the ratio of the mean values of the change rates of the two processes directly by the ratio of the two scores visually shows the degree of deterioration of the abnormal state of the equipment.

[0056] This step realizes the deep integration of multi-source data through the further monitoring index formula. The numerator is the mean value of the adjacent change rates during the monitoring process, reflecting the dynamic acceleration trend of real-time anomalies. If the anomaly index increment of each thermal imaging map during the monitoring period is greater than that of the previous thermal imaging map, it indicates that the deterioration speed is accelerating; at the same time, by comparing with the change rate of the anomaly index during the temperature increase process, the change rate of the equipment operation status after the further monitoring link is judged. Scoring correction factor Incorporating the influence of vibration / voltage anomalies into the index can break the limitations of single thermal data - for example: even if the thermal anomaly change rates are similar, if the further monitoring score increases significantly, the further monitoring index will increase synchronously, triggering a higher-level alarm. This index, as the only input for the threshold determination in step 402, realizes the unified quantification of thermal, mechanical, and electrical data, ensuring the simplicity and objectivity of the decision-making logic.

[0057] Step 402: Set the threshold Ⅰ and threshold Ⅱ of the further monitoring index respectively, and make a judgment according to the further monitoring index. The logic is as follows: When the further monitoring index is less than or equal to the threshold Ⅰ, the equipment continues to operate in the current state; When the further monitoring index is greater than the threshold Ⅰ and less than the threshold Ⅱ, the equipment starts the cooling system and continues to operate; When the further monitoring index is greater than or equal to the threshold Ⅱ, the equipment starts the cooling system and alarms, and maintenance personnel intervene in the management.

[0058] In this step, through hierarchical threshold determination, complex multi-source data metrics are transformed into clear operation and maintenance instructions. The intention of setting Threshold Ⅰ is to allow the device to continue operating within a controllable range, such as short-term temperature fluctuations, while triggering the cooling system to inhibit the development of abnormalities; the setting of Threshold Ⅱ targets irreversible risks, such as thermal runaway accompanied by severe vibrations, and requires immediate alarm and manual intervention. The key to this logic lies in the dynamic response. This step and the further monitoring index calculation in Step 401 form a closed-loop control, ultimately achieving the self-adaptability and hierarchical processing efficiency of device health management, avoiding production losses caused by excessive downtime, and maximizing the safety of the device.

[0059] The above formulas are all dimensionless and take their numerical calculations. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0060] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented through electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0061] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, and may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered within the protection scope of this application.

Claims

1. A data fusion method based on multi-source sensors, characterized in that: The specific steps include: Obtain the thermal images of the monitored equipment at a fixed frequency, and obtain the thermal value matrix of each thermal image in the sliding time window, set the thermal value threshold and obtain the abnormal pixel points in the thermal value matrix, obtain the area abnormality index and temperature abnormality index respectively according to the abnormal pixel points, and form the abnormality index of each thermal image; The abnormal index of the thermal image in the sliding time window is used to form an abnormal index sequence and obtain a temperature increase process variation sequence, and an abnormal index threshold is set. According to the relationship between the abnormal index threshold and the abnormal index, further monitoring is initiated to obtain the temperature increase process; Obtaining further monitoring time according to the temperature increase process, obtaining a vibration curve and a voltage curve of the monitored equipment within the further monitoring time, and obtaining a further monitoring score; During the further monitoring period, thermal images of the monitored equipment are continuously obtained, and the change in abnormal index during the further monitoring process is obtained. A further monitoring index is formed according to the further monitoring score, the change in abnormal index during the further monitoring process, and the change in abnormal index during the temperature increase process. Thresholds I and II are set respectively, and judgment is made based on the relationship between the further monitoring index and thresholds I and II.

2. The data fusion method based on multi-source sensors according to claim 1 is characterized in that: The thermal images of the monitored equipment are obtained at a fixed frequency. The thermal images are obtained by a thermal imager. A timestamp is added to each acquired thermal image. The thermal image is processed by a median filter method. The sliding window time is set to 1 hour. The thermal value matrix of each thermal image within the sliding window time is obtained according to the pixel points in the thermal image. The formula is as follows: ； in, For the The thermal value matrix of the thermal image, For the Row, No. The heat value of the pixel in the column, , , is the total number of thermal images, , , is the number of rows of pixels in the thermal image, is the number of columns of pixels in the thermal image; Set the thermal value threshold, calibrate the pixels whose thermal values ​​exceed the thermal value threshold, and obtain abnormal pixels. The logic is as follows: The thermal images are binarized to obtain a binary mask for each thermal image. Pixels whose thermal values ​​do not exceed the thermal value threshold are converted to black, and pixels whose thermal values ​​exceed the thermal value threshold are converted to white. White pixels are abnormal pixels.

3. The data fusion method based on multi-source sensors according to claim 2 is characterized in that: Perform area analysis on abnormal pixels in the thermal image to obtain the area anomaly index of the abnormal pixels. The formula is as follows: ； in, For the The area anomaly index of the thermal image, Indicates The number of abnormal pixels in the thermal image. is the total number of pixels in the thermal image, Indicates The spatial concentration of pixels in a thermal image, Indicates the time difference, are coefficients, , , The search variable for the thermal image number, , ; The logic for obtaining spatial aggregation is as follows: The position of each abnormal pixel in the thermal image is obtained and numbered respectively. The position is the row and column data of the abnormal pixel in the thermal image. The adjacency relationship between any two abnormal pixels is obtained according to the following formula: ； in, Indicates The first The adjacency value between abnormal pixels, and Respectively represent The first The location of the abnormal pixel; The spatial aggregation degree is obtained based on the following formula: ； in, is the spatial aggregation degree, Indicates The first The adjacency value between abnormal pixels, is the number of abnormal pixels; The time difference represents the difference between the time stamps of two adjacent thermal imaging images.

4. The data fusion method based on multi-source sensors according to claim 3 is characterized in that: Perform temperature analysis on abnormal pixels in the thermal image to obtain the temperature anomaly index of the abnormal pixels. The formula is as follows: ； in, For the The temperature anomaly index of the thermal image, For the The sum of the thermal values ​​of all abnormal pixels in a thermal image. are coefficients, , , The search variable for the thermal image number, , ; The anomaly index of each thermal image is obtained according to the following formula: ； in, No. The abnormal index of the thermal image, For the The area anomaly index of the thermal image, For the The temperature anomaly index of the thermal image, The search variable for the thermal image number, , .

5. The data fusion method based on multi-source sensors according to claim 4 is characterized in that: The abnormal indexes of the thermal imaging images within the time window are formed into an abnormal index sequence in chronological order, and the change amount of the abnormal index sequence is obtained. The formula is as follows: ； in, is the abnormal index change, No. The abnormal index of the thermal image, No. The abnormal index of the thermal image, The search variable for the thermal image number, , ; Set the abnormal index threshold. When the abnormal index exceeds the abnormal index threshold, start further monitoring and trace the temperature increase process. The logic of obtaining the temperature increase process is: When the abnormal index exceeds the abnormal index threshold, the change amount of the abnormal index sequence is checked from back to front, and a subsequence of continuously increasing abnormal indexes including the last abnormal index is obtained, and the subsequence is the temperature increase process; The abnormal exponential changes in the temperature increase process are converted into a temperature increase process change sequence.

6. The data fusion method based on multi-source sensors according to claim 5 is characterized in that: The further monitoring time is obtained according to the temperature increase process change sequence, and the formula is as follows: ； in, To further monitor the time, is the average value of the temperature increase process variation series, is the standard deviation of the temperature increase process variation series, is the rate of change of the temperature increase process variation sequence, is the mean value threshold, is the standard deviation threshold, is the change rate threshold, The time for the temperature increase process; The time of the temperature increase process is based on the following formula: ； in, Indicates the temperature increase process The timestamp of the thermal image, Indicates the timestamp of the first thermal image during the temperature increase process; The formula for the abnormal index change rate is as follows: ； in, is the rate of change, Indicates the temperature increase process The abnormal index of the thermal image, Indicates the abnormal index of the first thermal image during the temperature increase process.

7. The data fusion method based on multi-source sensors according to claim 6 is characterized in that: In the process of further monitoring, the vibration curve and voltage curve of the equipment are obtained. The vibration curve is obtained by a vibration sensor, and the voltage curve is obtained by an oscilloscope. The horizontal axis of the vibration curve and the voltage curve is time, the start time is the time when further monitoring is started, the time length is the further monitoring time, the vertical axis of the vibration curve is the amplitude, and the vertical axis of the voltage curve is the voltage, and the further monitoring score is obtained. ； in, To further monitor the scores, is the initial score value, is the vibration curve, is the initial amplitude, is the voltage curve, is the initial voltage, To further monitor the time, and is the coefficient, ; The initial score value is a preset value, the initial amplitude is the amplitude of the device during normal operation, and the initial voltage is the voltage of the device during normal operation.

8. The data fusion method based on multi-source sensors according to claim 7 is characterized in that: In the process of further monitoring, the thermal images of the monitored equipment are continuously obtained and the abnormal index change in the further monitoring process is formed to generate the further monitoring index. The formula is as follows: ； in, To further monitor the index, For further monitoring The abnormal index change, To further monitor the number of abnormal index changes in the process, is the first in the sequence of temperature increase process variation The abnormal index change, is the number of abnormal exponential changes in the temperature increase process change sequence, To further monitor the scores, is the initial value of the score.

9. The data fusion method based on multi-source sensors according to claim 8 is characterized in that: The thresholds I and II of the further monitoring index are set respectively, and the judgment is made according to the further monitoring index. The logic is as follows: When the further monitoring index is less than or equal to threshold I, the device continues to work according to the current state; When the further monitoring index is greater than threshold I and less than threshold II, the device starts the cooling system and continues to work; When the further monitoring index is greater than or equal to threshold II, the equipment starts the cooling system and alarms, and maintenance personnel intervene in management.

Images