Multi-sensor data fusion equipment fault diagnosis method and system

Abstract

The invention provides a multi-sensor data fusion equipment fault diagnosis method and system, relates to the technical field of equipment fault diagnosis, and remarkably improves the accuracy and reliability of equipment fault diagnosis through a multi-sensor data fusion method. Specifically, features are extracted from image, temperature and multiple sensor data, and an equipment fault feature index is generated through feature fusion, so that prediction and diagnosis of equipment faults are realized, temperature data and working images of key parts are collected, correlation analysis is carried out on the working images, and the accuracy of equipment fault diagnosis is improved. The method comprises the steps of generating an image texture feature evaluation index reflecting an equipment working image, performing correlation processing on temperature data, generating a temperature feature evaluation index reflecting an equipment temperature feature value, performing correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index, and generating an equipment fault feature index. Therefore, the fault type with the highest possibility is output.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment fault diagnosis, and in particular to an equipment fault diagnosis method and system for multi-sensor data fusion. Background Art

[0002] Industrial equipment failure refers to the failure of equipment to function properly during production or operation, resulting in reduced production efficiency, downtime, quality problems, or even safety accidents. When equipment fails, it will reflect characteristics such as changes in appearance and temperature changes in key parts.

[0003] During the operation of industrial equipment, timely and accurate diagnosis of equipment failures is crucial to ensuring production efficiency and extending equipment life. Existing equipment fault diagnosis methods mainly rely on data from a single sensor. When a single temperature sensor data or image data collected by an industrial camera changes and the change exceeds the threshold, a fault will be prompted. However, this method often cannot provide sufficient fault information, resulting in limited accuracy and reliability of diagnosis.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a device fault diagnosis method and system based on multi-sensor data fusion to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions: A device fault diagnosis method based on multi-sensor data fusion, the specific steps comprising: S1, collecting equipment working images and equipment temperature data, wherein the working image pixel resolution is 1920x1080; S2, preprocessing the working image to generate image grayscale, and preprocessing the grayscale to generate a grayscale coefficient, wherein the grayscale is used to reflect the color information of the working image using a black tone; S3, perform correlation processing on the grayscale coefficient to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index , the image texture feature evaluation index is used to reflect the texture features of the device working image; S4, performing correlation processing on the temperature data to generate a characteristic mean, a characteristic root mean square value and a peak value, combining the characteristic mean with the temperature data for processing to generate a characteristic standard deviation; S5. Combine the feature mean, feature RMS value, peak value and feature standard deviation for preprocessing to generate the temperature feature evaluation index ,The temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment; S6. Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the temperature and image features are fused. S7. Set the characteristic threshold of the historical fault equipment, and make a one-to-one correspondence between the characteristic threshold and the fault type, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0007] Furthermore, the working image of the device is collected by an industrial camera, and the pixels on the working image are numbered. The red, green, and blue components of the pixels in the xth column and the yth row in the working image are respectively represented by , , Indicates that the temperature data of key parts of the equipment are collected through temperature sensors, wherein temperature data of a total of N key parts are collected, and the temperature of the kth key part is The unit is Celsius.

[0008] Furthermore, for the red component of the pixel at the xth column and the yth row in the working image, , Green content , blue component Perform correlation processing to generate the grayscale of the pixel at the xth column and the yth row , the formula based on is: ; The grayscale is used to reflect the color information of the working image using a black tone; Grayscale Perform normalization to generate the grayscale coefficient of the pixel in the xth column and yth row of the working image. , the formula based on is: ; in, Grayscale of the pixel in the working image is the maximum value of the grayscale of the pixels in the working image; Gamma Perform correlation processing to generate image texture index , the formula based on is: ; in, , , , ,and .

[0009] Furthermore, the image texture index Perform correlation analysis to generate image texture feature evaluation index , the formula based on is: ; in, is the weight factor of the pixel point in the xth column and the yth row. The image texture feature evaluation index is used to reflect the texture features of the device working image.

[0010] Furthermore, the temperature Perform correlation processing to generate feature mean , characteristic root mean square value and peak , the formula based on is: ; Among them, the characteristic mean is used to reflect the overall level of temperature data, the characteristic root mean square value is used to reflect the square root of the mean of the sum of squares of all temperature data points, which is used to measure the amplitude statistics of temperature data, and the peak value is used to reflect the maximum value in the temperature data; Temperature and the characteristic mean Perform correlation analysis to generate characteristic standard deviation , the formula based on is: ; Among them, the characteristic standard deviation Used to reflect the discreteness of temperature data.

[0011] Furthermore, for the feature mean , characteristic root mean square value , Peak and characteristic standard deviation Perform correlation analysis to generate temperature characteristic evaluation index , the formula based on is: ; in, is the mean weight factor, is the RMS weight factor, is the peak weight factor, is the standard deviation weight factor, and the temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment.

[0012] Furthermore, the temperature characteristic evaluation index and image texture feature evaluation index Correlation analysis is performed to generate the equipment fault characteristic index SGT, based on the formula: ; The equipment fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features.

[0013] Furthermore, the characteristic threshold of the historical fault equipment is set and matched with the fault type one by one, and the equipment fault characteristic index SGT is matched with the characteristic threshold Calculating Similarity , output the fault type with the highest similarity as the prediction result, based on the formula: ; in, is the feature threshold The maximum value between the equipment fault characteristic index SGT, Indicates that the similarity corresponding to the jth fault type is expressed as Indicates that the subscript j is used to index the fault type.

[0014] The present invention also provides a multi-sensor data fusion equipment fault diagnosis system, which is used to execute a multi-sensor data fusion equipment fault diagnosis method, comprising: The acquisition module is used to collect equipment working images and equipment temperature data; An image analysis module is used to preprocess the working image to generate image grayscale, preprocess the grayscale to generate grayscale coefficient, perform correlation processing on the grayscale coefficient, generate image texture index, perform correlation analysis on the image texture index, and generate image texture feature evaluation index; The temperature analysis module is used to perform correlation processing on the temperature data to generate the characteristic mean, characteristic root mean square value and peak value, combine the characteristic mean with the temperature data for processing to generate the characteristic standard deviation, and perform preprocessing in combination with the characteristic mean, characteristic root mean square value, peak value and characteristic standard deviation to generate the temperature characteristic evaluation index; A comprehensive analysis module is used to perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index; The fault output module is used to set the characteristic threshold of the historical fault equipment, and correspond the characteristic threshold to the fault type one by one, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention significantly improves the accuracy and reliability of equipment fault diagnosis through a multi-sensor data fusion method. Specifically, the present invention extracts features from multiple sensor data such as images and temperature, and generates an equipment fault feature index through feature fusion, thereby realizing predictive diagnosis of equipment faults, collecting temperature data of key parts and working images, performing correlation analysis on the working images, generating an image texture feature evaluation index reflecting the equipment working image, performing correlation processing on the temperature data, generating a temperature feature evaluation index reflecting the temperature characteristic value of the equipment, and performing correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index, generating an equipment fault feature index, comparing it with the feature threshold, and outputting the most likely fault type. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the overall method flow of the present invention; Figure 2 It is a schematic diagram of the overall module flow of the present invention. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0018] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] Example: See also Figure 1 , the present invention provides a technical solution: A multi-sensor data fusion equipment fault diagnosis method. When industrial equipment fails, the temperature or shape often changes. Therefore, the change of temperature and shape can reversely predict the type of equipment failure. By setting image sensors and temperature sensors to monitor the operation of industrial equipment in real time, when the equipment fails, the monitoring data changes, and is compared with historical failures to output the most likely fault type. The specific steps include: Step 1: Collect the working image of the equipment and the equipment temperature data. The pixel resolution of the working image is 1920x1080. Collect the working image of the equipment through an industrial camera, and number the pixels on the working image. The red component, green component, and blue component of the pixel point in the xth column and yth row of the working image are respectively , , Indicates that the temperature data of key parts of the equipment are collected through temperature sensors. Key parts include points on the equipment with higher temperatures or more frequent temperature changes, and can also be points where temperature changes occur when the equipment fails. Among them, temperature data is collected for a total of N key parts, and the temperature of the kth key part is The subscript k is used for indexing, and the unit is Celsius. Both k and N are positive integers. Among them, the key parts can be the easily damaged parts of the equipment such as the motor, bearing, gear meshing, etc.

[0020] Step 2: preprocess the working image to generate image grayscale, and preprocess the grayscale to generate grayscale coefficient, where the grayscale is used to reflect the color information of the working image using black tone; For the red component of the pixel at the xth column and the yth row in the working image , Green content , blue component Perform correlation processing to generate the grayscale of the pixel at the xth column and the yth row , the formula based on is: ; Grayscale is used to reflect the color information of the working image using black tones; Grayscale Perform normalization to generate the grayscale coefficient of the pixel in the xth column and yth row of the working image. , the formula based on is: ; Grayscale of the pixel in the working image is the maximum value of the grayscale of the pixels in the working image; Step 3: Perform correlation processing on the grayscale coefficient to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index. , the image texture feature evaluation index is used to reflect the texture features of the device working image; Gamma Perform correlation processing to generate image texture index , the formula based on is: ; in, , , , ,and , image texture index Used to reflect the texture features between the target pixel point and the surrounding pixels of the working image; Texture index for images Perform correlation analysis to generate image texture feature evaluation index , the formula based on is: ; in, is the weight factor of the pixel point in the xth column and the yth row. The image texture feature evaluation index is used to reflect the texture features of the device working image. The value of is determined by historical data of working images of equipment failure states and expert experience.

[0021] Step 4: Perform correlation processing on the temperature data to generate the characteristic mean, characteristic root mean square value and peak value, and combine the characteristic mean with the temperature data to generate the characteristic standard deviation; Temperature Perform correlation processing to generate feature mean , characteristic root mean square value and peak , the formula based on is: ; Among them, the characteristic mean is used to reflect the overall level of temperature data, the characteristic root mean square value is used to reflect the square root of the mean of the sum of squares of all temperature data points, which is used to measure the amplitude statistics of temperature data, and the peak value is used to reflect the maximum value in the temperature data; Temperature and the characteristic mean Perform correlation analysis to generate characteristic standard deviation , the formula based on is: ; Among them, the characteristic standard deviation Used to reflect the discreteness of temperature data.

[0022] The mean of the features , characteristic root mean square value , Peak and characteristic standard deviation Perform correlation analysis to generate temperature characteristic evaluation index , the formula based on is: ; in, is the mean weight factor, is the RMS weight factor, is the peak weight factor, is the standard deviation weight factor. The temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment. The peak value in the temperature can directly indicate the instantaneous overheating or abnormal increase of the equipment. For fault diagnosis, the peak weight factor accounts for a high proportion and should be set larger. The standard deviation represents the discreteness of the temperature data. A larger standard deviation may indicate unstable operation of the equipment or abnormal temperature fluctuation. Therefore, the standard deviation weight factor should also be set larger, but second only to the peak weight factor, so as to sensitively reflect the amplitude and frequency of temperature changes and promptly identify abnormal operation of the equipment. The mean temperature usually reflects the overall thermal state of the equipment. In most cases, the mean temperature is a basic indicator of whether the equipment is in normal working condition. Since the mean is not sensitive to short-term temperature fluctuations, it is not appropriate to set too high a weight. The root mean square value is used to measure the fluctuation amplitude of the overall temperature data and is not sensitive to short-term temperature fluctuations. Therefore, the root mean square value weight factor should be set smaller, so .

[0023] Step 5: Preprocess the feature mean, feature RMS value, peak value and feature standard deviation to generate the temperature feature evaluation index ,The temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment; Step 6: Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the temperature and image features are fused. Temperature characteristic evaluation index and image texture feature evaluation index Correlation analysis is performed to generate the equipment fault characteristic index SGT, based on the formula: ; The equipment fault characteristic index SGT is used to reflect the fault characteristic value after the fusion of temperature and image features, and is used to evaluate equipment faults by integrating temperature and image features.

[0024] Step 7: Set the characteristic threshold of the historical fault equipment, and correspond the characteristic threshold to the fault type one by one, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0025] Set the characteristic threshold of the historical fault equipment and match it with the fault type one by one, and compare the equipment fault characteristic index SGT with the characteristic threshold Calculating Similarity , output the fault type with the highest similarity as the prediction result, based on the formula: ; in, is the feature threshold The maximum value between the equipment fault characteristic index SGT, Indicates that the similarity corresponding to the jth fault type is expressed as Indicates that the subscript j is used to index the fault type.

[0026] See also Figure 2 The present invention also provides a multi-sensor data fusion equipment fault diagnosis system, which is used to perform a multi-sensor data fusion equipment fault diagnosis method, including: The acquisition module is used to collect equipment working images and equipment temperature data; An image analysis module is used to preprocess the working image to generate image grayscale, preprocess the grayscale to generate grayscale coefficient, perform correlation processing on the grayscale coefficient, generate image texture index, perform correlation analysis on the image texture index, and generate image texture feature evaluation index; The temperature analysis module is used to perform correlation processing on the temperature data to generate the characteristic mean, characteristic root mean square value and peak value, combine the characteristic mean with the temperature data for processing to generate the characteristic standard deviation, and perform preprocessing in combination with the characteristic mean, characteristic root mean square value, peak value and characteristic standard deviation to generate the temperature characteristic evaluation index; A comprehensive analysis module is used to perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index; The fault output module is used to set the characteristic threshold of the historical fault equipment, and correspond the characteristic threshold to the fault type one by one, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

[0027] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0028] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0029] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0030] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A device fault diagnosis method based on multi-sensor data fusion, characterized in that: The specific steps include: S1, collecting equipment working images and equipment temperature data, wherein the working image pixel resolution is 1920x1080; S2, preprocessing the working image to generate image grayscale, and preprocessing the grayscale to generate a grayscale coefficient, wherein the grayscale is used to reflect the color information of the working image using a black tone; S3, perform correlation processing on the grayscale coefficient to generate an image texture index, and perform correlation analysis on the image texture index to generate an image texture feature evaluation index , the image texture feature evaluation index is used to reflect the texture features of the device working image; S4, performing correlation processing on the temperature data to generate a characteristic mean, a characteristic root mean square value and a peak value, combining the characteristic mean with the temperature data for processing to generate a characteristic standard deviation; S5. Combine the feature mean, feature RMS value, peak value and feature standard deviation for preprocessing to generate the temperature feature evaluation index ,The temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment; S6. Perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index SGT. The device fault feature index SGT is used to reflect the fault feature value after the temperature and image features are fused. S7. Set the characteristic threshold of the historical fault equipment, and make a one-to-one correspondence between the characteristic threshold and the fault type, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result. The formula is: ; in, is the feature threshold The maximum value between the fault characteristic index SGT and the equipment fault characteristic index SGT is used to number the fault types, where the characteristic threshold of the jth fault type is Indicates that the similarity corresponding to the jth fault type is expressed as Indicates that the subscript j is used to index the fault type.

2. The device fault diagnosis method based on multi-sensor data fusion according to claim 1, characterized in that: The working image of the equipment is collected by an industrial camera, and the pixels on the working image are numbered. The red, green, and blue components of the pixels in the xth column and yth row of the working image are respectively , , Indicates that the temperature data of key parts of the equipment are collected through temperature sensors, wherein temperature data of a total of N key parts are collected, and the temperature of the kth key part is Indicates that the unit is Celsius.

3. The device fault diagnosis method based on multi-sensor data fusion according to claim 2 is characterized by: For the red component of the pixel at the xth column and the yth row in the working image , Green content , blue component Perform correlation processing to generate the grayscale of the pixel at the xth column and the yth row , the formula based on is: ; The grayscale is used to reflect the color information of the working image using a black tone; Grayscale Perform normalization to generate the grayscale coefficient of the pixel in the xth column and yth row of the working image. , the formula based on is: ; in, is the minimum grayscale value of the pixel in the working image, is the maximum value of the grayscale of the pixels in the working image; Gamma Perform correlation processing to generate image texture index , the formula based on is: ; in, , , , ,and .

4. The device fault diagnosis method based on multi-sensor data fusion according to claim 3 is characterized by: Texture index for images Perform correlation analysis to generate image texture feature evaluation index , the formula based on is: ; in, is the weight factor of the pixel point in the xth column and the yth row. The image texture feature evaluation index is used to reflect the texture features of the device working image.

5. The device fault diagnosis method based on multi-sensor data fusion according to claim 2 is characterized in that: Temperature Perform correlation processing to generate feature mean , characteristic root mean square value and peak , the formula based on is: ; Among them, the characteristic mean is used to reflect the overall level of temperature data, the characteristic root mean square value is used to reflect the square root of the mean of the sum of squares of all temperature data points, which is used to measure the amplitude statistics of temperature data, and the peak value is used to reflect the maximum value in the temperature data; Temperature and the characteristic mean Perform correlation analysis to generate characteristic standard deviation , the formula based on is: ; Among them, the characteristic standard deviation Used to reflect the discreteness of temperature data.

6. The device fault diagnosis method based on multi-sensor data fusion according to claim 5 is characterized by: The mean of the features , characteristic root mean square value , Peak and characteristic standard deviation Perform correlation analysis to generate temperature characteristic evaluation index , the formula based on is: ; in, is the mean weight factor, is the RMS weight factor, is the peak weight factor, is the standard deviation weight factor, and the temperature characteristic evaluation index is used to reflect the temperature characteristic value of the equipment.

7. The device fault diagnosis method based on multi-sensor data fusion according to claim 1 is characterized by: Temperature characteristic evaluation index and image texture feature evaluation index Correlation analysis is performed to generate the equipment fault characteristic index SGT, based on the formula: ; The equipment fault feature index SGT is used to reflect the fault feature value after the fusion of temperature and image features.

8. A multi-sensor data fusion equipment fault diagnosis system, used to execute the multi-sensor data fusion equipment fault diagnosis method according to claim 1, characterized in that: include: The acquisition module is used to collect equipment working images and equipment temperature data; An image analysis module is used to preprocess the working image to generate image grayscale, preprocess the grayscale to generate grayscale coefficient, perform correlation processing on the grayscale coefficient, generate image texture index, perform correlation analysis on the image texture index, and generate image texture feature evaluation index; The temperature analysis module is used to perform correlation processing on the temperature data to generate the characteristic mean, characteristic root mean square value and peak value, combine the characteristic mean with the temperature data for processing to generate the characteristic standard deviation, and perform preprocessing in combination with the characteristic mean, characteristic root mean square value, peak value and characteristic standard deviation to generate the temperature characteristic evaluation index; A comprehensive analysis module is used to perform correlation analysis on the temperature feature evaluation index and the image texture feature evaluation index to generate a device fault feature index; The fault output module is used to set the characteristic threshold of the historical fault equipment, and correspond the characteristic threshold to the fault type one by one, compare the equipment fault characteristic index with the characteristic threshold, calculate the similarity, and output the fault type with the highest similarity as the prediction result.

Images