A device fault diagnosis method and system for multi-sensor data fusion

Abstract

The present invention provides a method and system for equipment fault diagnosis based on multi-sensor data fusion, which relates to the technical field of equipment fault diagnosis. Through the method of multi-sensor data fusion, the present invention significantly improves the accuracy and reliability of equipment fault diagnosis. Specifically, the present invention extracts features from data of multiple sensors such as images and temperatures, and generates an equipment fault feature index through feature fusion, so as to realize the predictive diagnosis of equipment faults. By collecting temperature data and working images of key parts, performing correlation analysis on the working images to generate an image texture feature evaluation index reflecting the equipment working images, performing correlation processing on the temperature data to generate a temperature feature evaluation index reflecting the equipment temperature feature values, and performing correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate an equipment fault feature index, thereby outputting the fault type with the highest possibility.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment fault diagnosis, and particularly to a method and system for equipment fault diagnosis based on multi-sensor data fusion. Background Art

[0002] Industrial equipment failure refers to the situation where the equipment fails to operate normally during production or operation, resulting in reduced production efficiency, downtime, quality problems, and even safety accidents. When the equipment fails, it will show characteristics such as changes in appearance and temperature changes at key parts.

[0003] During the operation of industrial equipment, timely and accurate diagnosis of equipment failures is crucial for ensuring production efficiency and extending the equipment life. Existing equipment fault diagnosis methods mainly rely on data from a single sensor. When the data of a single temperature sensor or the image data collected by an industrial camera changes and the change exceeds the threshold, a fault will be prompted. However, this method often fails to provide sufficient fault information, resulting in limitations in the accuracy and reliability of diagnosis.

[0004] The above information disclosed in the background art section is only used to enhance the understanding of the background of the present disclosure. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for equipment fault diagnosis based on multi-sensor data fusion to solve the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions:

[0007] A method for equipment fault diagnosis based on multi-sensor data fusion, the specific steps include:

[0008] S1. Collect the working image of the equipment and the equipment temperature data, and the pixel resolution of the working image is 1920x1080;

[0009] S2. Preprocess the working image to generate image grayscale, and preprocess the grayscale to generate a grayscale coefficient. The grayscale is used to reflect the color information of the working image using black tones;

[0010] S3. Perform correlation processing on the grayscale coefficient to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index , and the image texture feature evaluation index is used to reflect the texture features of the equipment working image;

[0011] S4. Perform correlation processing on the temperature data to generate the feature mean, feature root mean square value, and peak value. Combine the feature mean and temperature data for processing to generate the feature standard deviation;

[0012] S5. Perform preprocessing by combining the feature mean, feature root mean square value, peak value, and feature standard deviation to generate a temperature feature evaluation index , and the temperature feature evaluation index is used to reflect the device temperature feature value;

[0013] S6. Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features;

[0014] S7. Set the feature threshold of the historical fault device, and correspond the feature threshold with the fault type one by one. Compare the device fault feature index with the feature threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0015] Furthermore, collect the device working image through an industrial camera, number the pixel points on the working image. The red, green, and blue components of the pixel point at the x-th column and y-th row in the working image are respectively represented by , , . Collect the temperature data of the key parts on the device through a temperature sensor. Among them, collect the temperature data of a total of N key parts, and the temperature of the k-th key part is represented by , and the unit is degree Celsius.

[0016] Furthermore, perform correlation processing on the red component , green component , and blue component of the pixel point at the x-th column and y-th row in the working image to generate the gray scale of the pixel point at the x-th column and y-th row. The formula is: ;

[0017] The gray scale is used to reflect the color information of the working image using a black tone;

[0018] Perform normalization processing on the gray scale to generate the gray scale coefficient of the pixel point at the x-th column and y-th row in the working image. The formula is: ;

[0019] Among them, is the gray scale of the pixel point in the working image is the maximum value of the gray scale of the pixel point in the working image;

[0020] Perform processing on the gray scale coefficient Perform correlation processing to generate an image texture index , and the formula is as follows: ;

[0021] where, , , , , and .

[0022] Furthermore, perform correlation analysis on the image texture index to generate an image texture feature evaluation index , and the formula is as follows: ;

[0023] where, is the weight factor of the pixel at the x-th column and y-th row, and the image texture feature evaluation index is used to reflect the texture feature of the device working image.

[0024] Furthermore, perform correlation processing on the temperature to generate a feature mean value , a feature root mean square value and a peak value , and the formula is as follows: ;

[0025] where, the feature mean value is used to reflect the overall level of the temperature data, the feature root mean square value is used to reflect the square root of the mean of the sum of squares of all temperature data points, and is used to measure the amplitude statistic of the temperature data, and the peak value is used to reflect the maximum value in the temperature data;

[0026] Perform correlation analysis on the temperature and the feature mean value to generate a feature standard deviation , and the formula is as follows: ;

[0027] where, the feature standard deviation is used to reflect the dispersion degree of the temperature data.

[0028] Furthermore, perform correlation analysis on the feature mean value , the feature root mean square value , the peak value and the feature standard deviation to generate a temperature feature evaluation index , and the formula is as follows: ;

[0029] where, is the mean weight factor, is the root mean square value weight factor, is the peak weight factor, is the standard deviation weight factor, and the temperature feature evaluation index is used to reflect the device temperature feature value.

[0030] Further, for the temperature feature evaluation index and the image texture feature evaluation index perform a correlation analysis to generate the device fault feature index SGT. The formula is as follows: ;

[0031] The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features.

[0032] Further, set the feature threshold of the historical fault device, and correspond it one by one with the fault type. Calculate the similarity between the device fault feature index SGT and the feature threshold , and output the fault type with the highest similarity as the prediction result. The formula is as follows: ; ;

[0033] Among them, is the feature threshold the maximum value between the device fault feature index SGTs, indicates that the similarity corresponding to the j-th fault type is represented by , and the subscript j is used to index the fault type.

[0034] The present invention also provides a device fault diagnosis system for multi-sensor data fusion, which is used to execute a device fault diagnosis method for multi-sensor data fusion, including:

[0035] An acquisition module, which is used to acquire the device working image and device temperature data;

[0036] An image analysis module, which is used to preprocess the working image to generate the image grayscale, preprocess the grayscale to generate the grayscale coefficient, perform a correlation process on the grayscale coefficient to generate the image texture index, and perform a correlation analysis on the image texture index to generate the image texture feature evaluation index;

[0037] A temperature analysis module, which is used to perform a correlation process on the temperature data to generate the feature mean value, feature root mean square value and peak value, combine the feature mean value and the temperature data for processing to generate the feature standard deviation, and combine the feature mean value, feature root mean square value, peak value and feature standard deviation for preprocessing to generate the temperature feature evaluation index;

[0038] A comprehensive analysis module, which is used to perform a correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate the device fault feature index;

[0039] A fault output module is used to set the characteristic thresholds of historical fault devices, correspond the characteristic thresholds to the fault types one by one, compare the device fault characteristic index with the characteristic thresholds, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] Through the method of multi-sensor data fusion, the present invention significantly improves the accuracy and reliability of device fault diagnosis. Specifically, the present invention extracts features from multi-sensor data such as images and temperatures, and generates a device fault characteristic index through feature fusion, so as to realize the predictive diagnosis of device faults. By collecting temperature data and working images of key parts, performing correlation analysis on the working images to generate an image texture feature evaluation index reflecting the device working image, performing correlation processing on the temperature data to generate a temperature feature evaluation index reflecting the device temperature characteristic value, and performing correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault characteristic index, and comparing it with the characteristic threshold, so as to output the fault type with the highest possibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0043] Figure 2 It is a schematic diagram of the overall module flow of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to specific embodiments.

[0045] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those of ordinary skill in the field to which the present invention belongs. The "first", "second" and similar terms used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0046] Embodiment:

[0047] Please refer to Figure 1 , the present invention provides a technical solution:

[0048] A device fault diagnosis method for multi-sensor data fusion. When a device fails in industry, it often causes changes in temperature or shape. Therefore, the type of fault generated by the device can be predicted in reverse through the changes in temperature and shape. By setting up an image sensor and a temperature sensor to monitor the operation of industrial devices in real time, when the device fails, the monitoring data changes and is compared with historical faults, so as to output the most likely fault type. The specific steps include:

[0049] Step 1: Collect the working image of the device and the temperature data of the device. The pixel resolution of the working image is 1920x1080; collect the working image of the device through an industrial camera, number the pixel points on the working image, and the red component, green component, and blue component of the pixel point at the x-th column and y-th row in the working image are respectively represented by , , . Collect the temperature data of the key parts on the device through a temperature sensor. The key parts include the points with higher temperature or more frequent temperature changes on the device, and can also be the points where temperature anomalies occur when the device fails. Among them, collect the temperature data of a total of N key parts, and the temperature of the k-th key part is represented by . The subscript k is used for indexing, and the unit is degrees Celsius. Both k and N are positive integers. Among them, the key parts can be easily damaged parts such as the motor, bearing, and gear meshing part of the device.

[0050] Step 2: Preprocess the working image to generate image grayscale, and preprocess the grayscale to generate a grayscale coefficient. The grayscale is used to reflect the color information of the working image using black tones;

[0051] For the red component , green component , and blue component of the pixel point at the x-th column and y-th row in the working image, perform correlation processing to generate the grayscale of the pixel point at the x-th column and y-th row. The formula is: ;

[0052] The grayscale is used to reflect the color information of the working image using black tones;

[0053] Normalize the grayscale to generate the grayscale coefficient of the pixel point at the x-th column and y-th row in the working image. The formula is: ;

[0054] is the grayscale of the pixel point in the working image is the maximum value of the gray level of the pixel points in the working image;

[0055] Step 3: Perform correlation processing on the gray level coefficients to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index , and the image texture feature evaluation index is used to reflect the texture features of the device working image;

[0056] Perform correlation processing on the gray level coefficients to generate an image texture index , and the formula is: ;

[0057] where , , , , and , the image texture index is used to reflect the texture features between the target pixel point and its surrounding pixel points in the working image;

[0058] Perform correlation analysis on the image texture index to generate an image texture feature evaluation index , and the formula is: ;

[0059] where is the weight factor of the pixel point in the x-th column and y-th row, and the image texture feature evaluation index is used to reflect the texture features of the device working image, The value of is determined by the historical data of the working image of the device failure state and expert experience.

[0060] Step 4: Perform correlation processing on the temperature data to generate a feature mean value, a feature root mean square value, and a peak value, and combine the feature mean value and the temperature data for processing to generate a feature standard deviation;

[0061] Perform correlation processing on the temperature to generate a feature mean value , a feature root mean square value and a peak value , and the formula is: ;

[0062] where the feature mean value is used to reflect the overall level of the temperature data, the feature root mean square value is used to reflect the square root of the mean of the sum of the squares of all temperature data points, and is used to measure the amplitude statistic of the temperature data, and the peak value is used to reflect the maximum value in the temperature data;

[0063] Perform correlation processing on the temperature and the feature mean value Perform correlation analysis to generate the standard deviation of features , and the formula is as follows: ;

[0064] Among them, the standard deviation of features is used to reflect the degree of dispersion of temperature data.

[0065] Perform correlation analysis on the feature mean , the root mean square value of features , the peak value and the standard deviation of features to generate the temperature feature evaluation index , and the formula is as follows: ;

[0066] Among them, is the mean weight factor, is the root mean square value weight factor, is the peak value weight factor, is the standard deviation weight factor. The temperature feature evaluation index is used to reflect the device temperature feature value. The peak value in the temperature can directly indicate the instantaneous overheating or abnormal increase of the device; for fault diagnosis, the peak value weight factor has a relatively high proportion and should be set larger; the standard deviation represents the degree of dispersion of temperature data, and a larger standard deviation may indicate unstable device operation or abnormal temperature fluctuations. Therefore, the standard deviation weight factor should also be set larger, but second to the peak value weight factor, to sensitively reflect the amplitude and frequency of temperature changes and timely identify abnormal states of device operation; the mean value of temperature usually reflects the overall thermal state of the device. In most cases, the temperature mean is a basic indicator of whether the device is in a normal working state. Since the mean is not sensitive to short-term temperature fluctuations, it is not appropriate to set too high a weight. The root mean square value is used to measure the fluctuation amplitude of the overall temperature data and is not sensitive to short-term temperature fluctuations. Therefore, the root mean square value weight factor should be set to a smaller value. Thus .

[0067] Step 5. Combine the feature mean, the root mean square value of features, the peak value, and the standard deviation of features for preprocessing to generate the temperature feature evaluation index , and the temperature feature evaluation index is used to reflect the device temperature feature value;

[0068] Step 6. Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate the device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features;

[0069] Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index Perform a correlation analysis to generate the device fault feature index SGT. The formula is as follows: ;

[0070] The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features, and is used to comprehensively evaluate the device fault based on temperature and image features.

[0071] Step 7: Set the feature threshold for historical fault devices, and correspond the feature threshold to the fault type one by one. Compare the device fault feature index with the feature threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0072] Set the feature threshold for historical fault devices, and correspond it to the fault type one by one. Compare the device fault feature index SGT with the feature threshold Calculate the similarity , and output the fault type with the highest similarity as the prediction result. The formula is as follows: ;

[0073] Wherein, is the feature threshold the maximum value between the device fault feature indices SGT, indicates that the similarity corresponding to the j-th fault type is represented by , and the subscript j is used as an index for the fault type.

[0074] Referring to Figure 2 , the present invention also provides a device fault diagnosis system for multi-sensor data fusion, which is used to execute a device fault diagnosis method for multi-sensor data fusion, including:

[0075] An acquisition module, which is used to acquire the device working image and device temperature data;

[0076] An image analysis module, which is used to preprocess the working image to generate the image grayscale, preprocess the grayscale to generate the grayscale coefficient, perform a correlation process on the grayscale coefficient to generate the image texture index, and perform a correlation analysis on the image texture index to generate the image texture feature evaluation index;

[0077] A temperature analysis module, which is used to perform a correlation process on the temperature data to generate the feature mean value, feature root mean square value, and peak value, combine the feature mean value with the temperature data for processing to generate the feature standard deviation, and combine the feature mean value, feature root mean square value, peak value, and feature standard deviation for preprocessing to generate the temperature feature evaluation index;

[0078] A comprehensive analysis module, which is used to perform a correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate the device fault feature index;

[0079] A fault output module is configured to set characteristic thresholds of historical faulty devices, correspond the characteristic thresholds to fault types one by one, compare a device fault characteristic index with the characteristic thresholds, calculate a similarity, and output the fault type with the highest similarity as a prediction result.

[0080] All the above formulas are dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0081] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. They may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0083] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered by the protection scope of this application.

Claims

1. A method for diagnosing equipment faults through multi-sensor data fusion, characterized in that, The specific steps include: S1. Collect the working image of the device and the device temperature data, where the pixel resolution of the working image is 1920x1080; S2. Preprocess the working image to generate image grayscale, and preprocess the grayscale to generate a grayscale coefficient. The grayscale is used to reflect the color information of the working image using a black tone; S3. Perform correlation processing on the gray coefficient to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index , and the image texture feature evaluation index is used to reflect the texture features of the device working image; S4. Perform correlation processing on the temperature data to generate a feature mean value, a feature root mean square value, and a peak value. Combine the feature mean value with the temperature data for processing to generate a feature standard deviation; S5. Preprocess by combining the feature mean value, feature root mean square value, peak value, and feature standard deviation to generate a temperature feature evaluation index , and the temperature feature evaluation index is used to reflect the device temperature feature value; S6. Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features; S7. Set the characteristic threshold of the historical faulty device, correspond the characteristic threshold to the fault type one by one, compare the device fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result. The formula is as follows: Among them, is the feature threshold and the maximum value between the equipment fault feature index SGT, number the fault types, where the feature threshold of the j-th fault type is represented by denoted, the similarity corresponding to the j-th fault type is represented by denoted, and the subscript j is used to index the fault type; Collect the working image through an industrial camera, number the pixel points on the working image, and the red, green, and blue components of the pixel point at the x-th column and y-th row in the working image are respectively represented by , , . Collect the temperature data of the key parts on the device through a temperature sensor. Among them, the temperature data of a total of N key parts are collected, and the temperature of the k-th key part is represented by , and the unit is degree Celsius; The red component of the pixel at the x-th column and y-th row in the working image , the green component , and the blue component are processed for correlation to generate the grayscale of the pixel at the x-th column and y-th row , and the formula used is: The grayscale is used to reflect the color information of the working image using a black tone; Normalize the grayscale to generate the grayscale coefficient of the pixel at the x-th column and y-th row in the working image , and the formula is as follows: Among them, is the minimum gray value of the pixel points in the working image, is the maximum gray value of the pixel points in the working image; Perform correlation processing on the gray coefficient to generate an image texture index , and the formula is as follows: Among them, , , , , and ; Perform correlation processing on temperature to generate the feature mean value , the root mean square value of the feature and the peak value , and the formula is as follows: Among them, The characteristic mean value is used to reflect the overall level of temperature data. The characteristic root mean square value is used to reflect the square root of the mean of the sum of squares of all temperature data points and is used to measure the amplitude statistic of temperature data. The peak value is used to reflect the maximum value in the temperature data; For temperature and the feature mean perform a correlation analysis to generate the feature standard deviation , and the formula used is: Among them, the standard deviation of features is used to reflect the degree of dispersion of temperature data.

2. The device fault diagnosis method for multi-sensor data fusion according to claim 1, characterized in that: Perform a correlation analysis on the image texture index to generate an image texture feature evaluation index , and the formula used is: Among them, is the weight factor of the pixel at the x-th column and y-th row, and the image texture feature evaluation index is used to reflect the texture feature of the device working image.

3. A device fault diagnosis method for multi-sensor data fusion according to claim 1, characterized in that: For the characteristic mean value , the characteristic root mean square value , the peak value and the characteristic standard deviation perform a correlation analysis to generate a temperature characteristic evaluation index , and the formula is as follows: Among them, is the mean weight factor, is the root mean square value weight factor, is the peak weight factor, is the standard deviation weight factor, and the temperature feature evaluation index is used to reflect the device temperature feature value.

4. A device fault diagnosis method for multi-sensor data fusion according to claim 1, characterized in that: Temperature characteristic evaluation index and image texture characteristic evaluation index are subjected to correlation analysis to generate the equipment fault characteristic index SGT, and the formula is as follows: The device fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features.

5. A device fault diagnosis system for multi-sensor data fusion, which is used to execute the device fault diagnosis method for multi-sensor data fusion described in claim 1, and is characterized in that, It includes: A collection module for collecting the working image of the device and the device temperature data; An image analysis module for preprocessing the working image to generate image grayscale, preprocessing the grayscale to generate a grayscale coefficient, performing correlation processing on the grayscale coefficient to generate an image texture index, and performing correlation analysis on the image texture index to generate an image texture feature evaluation index; A temperature analysis module for performing correlation processing on the temperature data to generate a feature mean value, a feature root mean square value, and a peak value, combining the feature mean value with the temperature data for processing to generate a feature standard deviation, and performing preprocessing by combining the feature mean value, the feature root mean square value, the peak value, and the feature standard deviation to generate a temperature feature evaluation index; A comprehensive analysis module for performing correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index; A fault output module for setting the feature threshold of the historical fault device, corresponding the feature threshold to the fault type one by one, comparing the device fault feature index with the feature threshold, calculating the similarity, and outputting the fault type with the highest similarity as the prediction result.

Images