Program, diagnostic method, and diagnostic device
The diagnostic program and device improve abnormality detection in devices by analyzing temperature image regions and correlations, offering enhanced accuracy and cost-effectiveness over existing methods.
Patent Information
- Application Number
- PCT/JP2023/017988
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-21
AI Technical Summary
Existing techniques for diagnosing device abnormalities based on temperature distribution are not sufficiently accurate, necessitating a more precise method to detect abnormalities in devices.
A diagnostic program and device that sets multiple regions in a temperature image, specifies changes in correlations between feature amounts within and between regions, and diagnoses abnormalities based on these changes, utilizing both infrared and visible light images to enhance accuracy.
Enables accurate abnormality diagnosis by focusing on temperature distribution changes between device parts, simplifying the diagnostic process, reducing costs, and improving portability and installation flexibility compared to traditional methods.
Smart Images

Figure JP2023017988_21112024_PF_FP_ABST
Abstract
Description
Program, diagnostic method, and diagnostic device
[0001] The present disclosure relates to a program, a diagnostic method, and a diagnostic device.
[0002] Abnormality diagnosis, which is a diagnosis for detecting abnormalities in various devices, is widely performed. One abnormality diagnosis method focuses on the temperature of the device to determine whether or not there is an abnormality. For example, as disclosed in Patent Document 1, a technology is proposed in which a temperature image showing the temperature distribution of the device is captured by an infrared camera and abnormality diagnosis is performed based on the temperature image.
[0003] Japanese Patent Application Laid-Open No. 2021-181286
[0004] As described above, several techniques for diagnosing abnormalities that focus on the temperature of the device have been proposed, but new proposals for techniques for diagnosing abnormalities in the device with greater accuracy are desired.
[0005] In view of the above-mentioned problems, the present disclosure has an object to provide a program, a diagnostic method, and a diagnostic device that are capable of accurately diagnosing abnormalities in a device.
[0006] In order to solve the above problem, a program according to one aspect of the present disclosure causes a computer to perform a first process of setting multiple regions in a temperature image showing the temperature distribution of the device, a second process of identifying changes in a first correlation, which is the correlation between first temperature-related features between different regions, and a third process of diagnosing an abnormality in the device based on the changes in the first correlation.
[0007] The first feature may include multiple types of feature, and in the second process, in addition to the change in the first correlation, a change in a second correlation may be further identified, which is a correlation between different types of first feature in the same region, or a correlation between different types of first feature corresponding to different regions, and in the third process, an abnormality diagnosis may be performed based on the change in the second correlation in addition to the change in the first correlation.
[0008] In the second process, in addition to the change in the first correlation, a change in a third correlation, which is a correlation between the first feature and a second feature that is independent of the region, is further identified, and in the third process, an abnormality diagnosis may be performed based on the change in the third correlation in addition to the change in the first correlation.
[0009] The second feature amount may include a feature amount relating to the operating state of the device.
[0010] The second feature amount may include a feature amount related to heat input / output to / from the device.
[0011] In the first process, a plurality of regions may be set in the temperature image based on the visible light image in which the device is captured.
[0012] In order to solve the above problem, in a diagnostic method according to one aspect of the present disclosure, a computer performs a first process of setting multiple regions in a temperature image showing the temperature distribution of the device, a second process of identifying changes in a first correlation, which is the correlation between first feature values related to temperature between different regions, and a third process of diagnosing an abnormality in the device based on the changes in the first correlation.
[0013] In order to solve the above problem, a diagnostic device according to one aspect of the present disclosure includes an infrared camera that captures a temperature image showing the temperature distribution of the device, and a processing device that performs a first process of setting multiple regions in the temperature image, a second process of identifying changes in a first correlation, which is the correlation between first feature values related to temperature between different regions, and a third process of diagnosing an abnormality in the device based on the changes in the first correlation.
[0014] According to the program, diagnostic method, and diagnostic device of the present disclosure, it is possible to accurately diagnose abnormalities in a device.
[0015] FIG. 1 is a schematic diagram showing a schematic configuration of a diagnostic device according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing an example of a functional configuration of a processing device according to an embodiment of the present disclosure. FIG. 3 is a diagram showing an example of a visible light image according to an embodiment of the present disclosure. FIG. 4 is a diagram showing an example of a temperature image according to an embodiment of the present disclosure. FIG. 5 is a flowchart showing an example of a processing flow performed by a processing device according to an embodiment of the present disclosure. FIG. 6 is a diagram showing an example of a reference dataset according to an embodiment of the present disclosure. FIG. 7 is a diagram showing an example of a diagnostic target dataset according to an embodiment of the present disclosure. FIG. 8 is a diagram showing an example of a reference dataset according to a first modified example of the present disclosure. FIG. 9 is a diagram showing an example of a reference dataset according to a second modified example of the present disclosure.
[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for ease of understanding and do not limit the present disclosure unless otherwise specified. In this specification and drawings, elements having substantially the same functions and configurations are designated by the same reference numerals to avoid redundant explanation. Elements not directly related to the present disclosure are not shown.
[0017] Fig. 1 is a schematic diagram showing a general configuration of a diagnostic device 1 according to an embodiment of the present disclosure. The diagnostic device 1 performs abnormality diagnosis on a device. In the example of Fig. 1, the diagnostic device 1 performs abnormality diagnosis on an engine 2, which corresponds to an example of a device. However, the device to be diagnosed by the diagnostic device 1 is not limited to the engine 2.
[0018] As shown in FIG. 1, the diagnostic device 1 includes an infrared camera 11 , a visible light camera 12 , and a processing device 13 .
[0019] The infrared camera 11 captures a temperature image. Specifically, the infrared camera 11 has a plurality of imaging elements that detect far-infrared rays emitted by an object. The imaging elements detect the far-infrared rays, thereby capturing a temperature image. The temperature image shows the distribution of the radiation intensity of the far-infrared rays as a temperature distribution. In this embodiment, the infrared camera 11 captures a temperature image IM1 that shows the temperature distribution of the engine 2. The obtained temperature image IM1 is output to the processing device 13.
[0020] The visible light camera 12 captures a visible light image. Specifically, the visible light camera 12 has a plurality of imaging elements that detect visible light. The imaging elements detect visible light, thereby capturing a visible light image. In this embodiment, the visible light camera 12 captures a visible light image IM2 that shows the engine 2. The obtained visible light image IM2 is output to the processing device 13.
[0021] The processing unit 13 performs various processes related to abnormality diagnosis of the engine 2. The processing unit 13 includes, for example, a central processing unit (CPU), a ROM in which programs and the like are stored, and a RAM as a work area.
[0022] 2 is a block diagram showing an example of a functional configuration of the processing device 13 according to an embodiment of the present disclosure. As shown in FIG. 2, the processing device 13 includes, for example, an acquisition unit 13a, a setting unit 13b, an identification unit 13c, a diagnosis unit 13d, and a storage unit 13e. The functions of the processing device 13 described below are realized by a central processing unit executing a program in the processing device 13. The functions of the processing device 13 described below may be realized by a single device or may be shared among multiple devices.
[0023] The acquisition unit 13a acquires various types of information, for example, from the infrared camera 11, the visible light camera 12, and the storage unit 13e.
[0024] The setting unit 13b sets a plurality of regions in the temperature image IM1. As will be described later, in the abnormality diagnosis of the engine 2, a feature amount is calculated for each region in the temperature image IM1. For example, an average value of the temperature can be given as a feature amount. The feature amount will be described in detail later.
[0025] The identification unit 13c identifies a change in the correlation between the feature amounts. Examples of the correlation between the feature amounts include a correlation between feature amounts in different regions. Details of the correlation between the feature amounts will be described later. The identification unit 13c identifies a change in the correlation based on the correlation between the feature amounts when the engine 2 is normal.
[0026] The diagnoser 13d diagnoses an abnormality in the engine 2 based on the identification result of the change in the correlation between the feature amounts by the identifier 13c. For example, when there is a large change in the correlation based on the correlation between the feature amounts when the engine 2 is normal, the diagnoser 13d diagnoses that an abnormality has occurred in the engine 2. Details of the abnormality diagnosis will be described later.
[0027] The storage unit 13e stores various types of information. The information stored in the storage unit 13e is used for various processes performed by the processing device 13.
[0028] FIG. 3 is a diagram illustrating an example of a visible light image IM2 according to an embodiment of the present disclosure. As described above, the visible light image IM2 shows the engine 2. Although not shown in FIG. 3 , the visible light image IM2 indicates brightness, for example, by shading. In the example of FIG. 3 , cylinders 21, 22, 23, 24, pipes 25, 26, 27, 28, and a supercharger 29 are shown as parts of the engine 2. However, the configuration of the engine 2 is not limited to the example of FIG. 3 .
[0029] FIG. 4 is a diagram illustrating an example of a temperature image IM1 according to an embodiment of the present disclosure. As described above, the temperature image IM1 shows the temperature distribution of the engine 2. Although not shown in FIG. 4, in the temperature image IM1, temperatures are indicated by, for example, shading. In the example of FIG. 4, regions R1, R2, R3, R4, R5, R6, R7, R8, and R9 are defined in the temperature image IM1, corresponding to cylinders 21, 22, 23, 24, pipes 25, 26, 27, 28, and turbocharger 29, respectively. Hereinafter, when regions R1 to R9 are not particularly distinguished from one another, they will be simply referred to as regions R. Details of the process for defining regions R will be described later.
[0030] The processing performed by the processing device 13 will be described below with reference to the examples of Figures 3 and 4. However, the visible light image IM2 and the temperature image IM1 are not limited to those shown in the examples of Figures 3 and 4. For example, the parts of the engine 2 shown in each image may be different from the parts shown in each image in the examples of Figures 3 and 4. For example, some of the regions R1 to R9 set in the example of Figure 4 may be omitted, or other regions R may be added.
[0031] 5 is a flowchart illustrating an example of a flow of a process performed by the processing device 13 according to an embodiment of the present disclosure. The process flow illustrated in FIG. 5 is executed, for example, when a predetermined operation is performed on the processing device 13 by a user.
[0032] 5 starts, in step S101, the acquisition unit 13a acquires, from the infrared camera 11, a temperature image IM1 that indicates the temperature distribution of the engine 2. After step S101, in step S102, the acquisition unit 13a acquires, from the visible light camera 12, a visible light image IM2 that shows the engine 2.
[0033] After step S102, in step S103, the setting unit 13b sets a plurality of regions R in the temperature image IM1. Specifically, the setting unit 13b sets a plurality of regions R in the temperature image IM1 based on the visible light image IM2.
[0034] The position of the infrared camera 11 and the position of the visible light camera 12 are different from each other. Therefore, the display positions of the same parts of the engine 2 are different between the temperature image IM1 and the visible light image IM2. The setting unit 13b can estimate the positional relationship between the display positions of the same parts of the engine 2 between the temperature image IM1 and the visible light image IM2 based on the positional relationship between the infrared camera 11 and the visible light camera 12. For example, information indicating the positional relationship between the infrared camera 11 and the visible light camera 12 is stored in advance in the storage unit 13e.
[0035] For example, the setting unit 13b can determine which part of the engine 2 is located at which display position on the visible light camera 12 by performing image processing such as edge detection on the visible light camera 12. Therefore, the setting unit 13b can determine which part of the engine 2 is located at which display position on the temperature image IM1 based on the determination result of which part of the engine 2 is located at which display position on the visible light camera 12 and the estimation result of the positional relationship between the display positions of the same part of the engine 2 between the temperature image IM1 and the visible light image IM2.
[0036] The setting unit 13b can set regions R in the temperature image IM1 in which each part of the engine 2 is located, based on the determination result of which part of the engine 2 is located at which display position in the temperature image IM1. In the example of Fig. 4, the setting unit 13b sets the regions R in the temperature image IM1 in which cylinder 21, cylinder 22, cylinder 23, cylinder 24, pipe 25, pipe 26, pipe 27, pipe 28, and turbocharger 29 are located, respectively, as regions R1, R2, R3, R4, R5, R6, R7, R8, and R9.
[0037] However, the setting unit 13b may set multiple regions R in the temperature image IM1 without using the visible light image IM2. For example, if information indicating which parts of the engine 2 are located at which display positions in the temperature image IM1 is stored in advance in the storage unit 13e, the setting unit 13b can set multiple regions R in the temperature image IM1 without using the visible light image IM2 by using the information. Note that the setting unit 13b may obtain the information indicating which parts of the engine 2 are located at which display positions in the temperature image IM1 based on the detection results of a sensor (e.g., LIDAR) that can detect position information of each part of the engine 2.
[0038] 5, in step S104, the identification unit 13c acquires a reference data set. The reference data set is a data set that summarizes the calculation results of the feature amounts of each region R when the engine 2 is normal. The reference data set is stored in advance in the storage unit 13e.
[0039] 6 is a diagram illustrating an example of a reference data set D1 according to an embodiment of the present disclosure. The reference data set D1 in FIG. 6 includes calculation results of the first feature amount F11 for each region R at each time point T. The first feature amount F11 is a feature amount related to temperature. Specifically, the first feature amount F11 is an average value of the temperature in each region R. The identification unit 13c calculates the first feature amount F11 for each region R at each time point T based on the temperature image IM1 acquired at each time point T.
[0040] The identification unit 13c calculates the first feature amount F11 of each region R at time T1. Next, the identification unit 13c calculates the first feature amount F11 of each region R at time T2. Next, the identification unit 13c calculates the first feature amount F11 of each region R at time T3. In this manner, the identification unit 13c calculates the first feature amount F11 of each region R for each of n time points T from time points T1 to Tn. Note that in FIG. 6, the illustration of the actually calculated value of the first feature amount F11 is omitted. However, the value of the first feature amount F11 may differ for each region R at the same time point T.
[0041] 5, in step S105, the identification unit 13c generates a diagnostic object dataset. The diagnostic object dataset is a dataset obtained by adding the current calculation result of the first feature amount F11 of each region R to the reference dataset D1.
[0042] Fig. 7 is a diagram illustrating an example of a diagnostic object dataset D2 according to an embodiment of the present disclosure. The diagnostic object dataset D2 in Fig. 7 includes the reference dataset D1 in Fig. 6 plus the calculation results of the first feature amount F11 of each region R at the current time Tk. The identification unit 13c calculates the first feature amount F11 of each region R at the current time Tk based on the temperature image IM1 acquired in step S101.
[0043] 5 , in step S106, the identification unit 13c identifies a change in the correlation between the feature amounts. Specifically, the identification unit 13c identifies the change in the correlation based on the correlation between the feature amounts when the engine 2 is normal. More specifically, the identification unit 13c identifies the change in the correlation based on the reference dataset D1 and the diagnosis target dataset D2.
[0044] In the examples of FIGS. 6 and 7, the specifying unit 13c specifies a change in the first correlation, which is the correlation of the first feature amount F11 between the regions R that are different from each other.
[0045] First, the specifying unit 13c calculates a probability density function of the first feature amount F11 in each region R in the reference data set D1.
[0046] For example, the identification unit 13c calculates a probability density function of the first feature quantity F11 in region R1 using a data group G1 consisting of n first feature quantities F11 in region R1. In this case, the probability density function of the first feature quantity F11 in region R1 indicates the probability density of each value of the first feature quantity F11 in the data group G1. Furthermore, the identification unit 13c calculates a probability density function of the first feature quantity F11 in region R2 using a data group G2 consisting of n first feature quantities F11 in region R2. In this case, the probability density function of the first feature quantity F11 in region R2 indicates the probability density of each value of the first feature quantity F11 in the data group G2. Similarly, the identification unit 13c calculates the probability density function of the first feature quantity F11 in the reference data set D1 for each of regions R3 to R9.
[0047] Next, the identification unit 13c calculates a correlation coefficient of the first feature quantity F11 between the region R1 and the region R2 in the reference data set D1 based on the probability density function of the first feature quantity F11 in the region R1 in the reference data set D1 and the probability density function of the first feature quantity F11 in the region R2 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates the correlation coefficient of the first feature quantity F11 between the two regions R in the reference data set D1 for all pairs of two regions R selectable from the regions R1 to R9, except for the pair of region R1 and region R2. The correlation coefficient calculated in this manner corresponds to the first correlation when the engine 2 is normal (i.e., the correlation of the first feature quantity F11 between different regions R).
[0048] Furthermore, the specifying unit 13c calculates a probability density function of the first feature amount F11 in each region R in the diagnostic object data set D2.
[0049] For example, the identification unit 13c calculates a probability density function of the first feature F11 in region R1 using a data group G1k consisting of n+1 first feature amounts F11 in region R1. In this case, the probability density function of the first feature F11 in region R1 indicates the probability density of each value of the first feature amount F11 in data group G1k. Furthermore, the identification unit 13c calculates a probability density function of the first feature amount F11 in region R2 using a data group G2k consisting of n+1 first feature amounts F11 in region R2. In this case, the probability density function of the first feature amount F11 in region R2 indicates the probability density of each value of the first feature amount F11 in data group G2k. Similarly, the identification unit 13c calculates the probability density function of the first feature amount F11 in the diagnostic target dataset D2 for each of regions R3 to R9.
[0050] Next, the identification unit 13c calculates a correlation coefficient of the first feature quantity F11 between the region R1 and the region R2 in the diagnostic object dataset D2 based on the probability density function of the first feature quantity F11 in the region R1 in the diagnostic object dataset D2 and the probability density function of the first feature quantity F11 in the region R2 in the diagnostic object dataset D2. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates the correlation coefficient of the first feature quantity F11 between the two regions R in the diagnostic object dataset D2 for all pairs of two regions R selectable from the regions R1 to R9, except for the pair of region R1 and region R2. The correlation coefficient calculated in this manner corresponds to the first correlation at the current time Tk (i.e., the correlation of the first feature quantity F11 between different regions R).
[0051] The identification unit 13c then calculates, for each pair of two regions R, the difference between the correlation coefficient obtained using the reference dataset D1 as described above and the correlation coefficient obtained using the diagnostic object dataset D2 as described above. For example, for the pair of regions R1 and R2, the identification unit 13c calculates the difference between the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the reference dataset D1 and the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the diagnostic object dataset D2. The identification unit 13c also calculates the difference between the correlation coefficient obtained using the reference dataset D1 and the correlation coefficient obtained using the diagnostic object dataset D2 for all pairs of two regions R selectable from regions R1 to R9, except for the pair of regions R1 and R2. The difference calculated in this manner corresponds to an index indicating a change in the first correlation, which is the correlation of the first feature amount F11 between different regions R.
[0052] After step S106 in FIG. 5, in step S107, the diagnosing unit 13d executes abnormality diagnosis of the engine 2, and the processing flow shown in FIG. 5 ends.
[0053] In step S107, the diagnoser 13d performs an abnormality diagnosis on the engine 2 based on the change in the first correlation identified in step S106. For example, if there is a pair of two regions R selectable from regions R1 to R9 in which the difference between the correlation coefficient obtained using the reference data set D1 and the correlation coefficient obtained using the diagnosis target data set D2 is equal to or greater than a reference value, the diagnoser 13d diagnoses that an abnormality has occurred in the engine 2. In this case, the diagnoser 13d may diagnose what type of abnormality has occurred in the engine 2 or where the abnormality has occurred in the engine 2, based on which pair has the difference equal to or greater than the reference value.
[0054] The diagnosis unit 13d may calculate, as the degree of abnormality, the difference between the correlation coefficient obtained using the reference data set D1 and the correlation coefficient obtained using the diagnosis target data set D2 between two regions R. In this case, the diagnosis unit 13d may store, in the storage unit 13e, information indicating the degree of abnormality for each pair of two regions R selectable from the regions R1 to R9. Note that information indicating the degree of abnormality for each pair of two regions R may be calculated under a condition in which an abnormality in the engine 2 is artificially generated, and the information may be stored in the storage unit 13e. The information indicating the degree of abnormality for each pair of two regions R is stored in the storage unit 13e in association with information, for example, on what type of abnormality has occurred in the engine 2 or on which location of the engine 2 the abnormality has occurred. As a result, the diagnosis unit 13d can appropriately diagnose what kind of abnormality has occurred in the engine 2 or at what location in the engine 2 the abnormality has occurred by comparing information indicating the degree of abnormality for each pair of two regions R obtained at the current time Tk with information indicating the degree of abnormality for each pair of two regions R obtained in the past and stored in the memory unit 13e.
[0055] As described above, in the program, diagnostic method, and diagnostic device 1 according to the embodiments of the present disclosure, a computer (in the above example, the processing device 13) performs a first process of setting multiple regions R in a temperature image IM1 showing the temperature distribution of an apparatus (in the above example, the engine 2), a second process of identifying a change in a first correlation, which is a correlation between first feature amounts related to temperature (in the above example, the first feature amount F11) between different regions R, and a third process of diagnosing an abnormality in the apparatus based on the change in the first correlation. In the above example, the setting unit 13b performs the first process, the identifying unit 13c performs the second process, and the diagnosing unit 13d performs the third process. This enables accurate diagnosis of an abnormality in the apparatus by focusing on the extent to which the temperature relationships between various parts of the apparatus have changed from the normal state of the apparatus.
[0056] Furthermore, according to the program, diagnostic method, and diagnostic device 1 of the embodiments of the present disclosure, by visualizing the change in the first correlation, which is the correlation of the first feature F11 related to temperature between different regions R (in the above example, the difference between the correlation coefficient obtained using the reference dataset D1 and the correlation coefficient obtained using the diagnostic target dataset D2), abnormalities in the device can be diagnosed more easily.
[0057] Furthermore, according to the program, diagnostic method, and diagnostic device 1 according to the embodiments of the present disclosure, the diagnostic device 1 can be simplified compared to, for example, a method of diagnosing an abnormality in a device by providing a temperature sensor in each part of the device. Also, compared to the above-mentioned method, the diagnostic device 1 can be made less expensive. Also, compared to the above-mentioned method, the diagnostic device 1 can be made more portable. Also, compared to the above-mentioned method, the degree of freedom in the installation location of the diagnostic device 1 can be improved.
[0058] In particular, in the program, diagnostic method, and diagnostic device 1 according to the embodiments of the present disclosure, in the first process (specifically, the process of setting multiple regions R in a temperature image IM1 showing the temperature distribution of the device), multiple regions R are set in the temperature image IM1 based on a visible light image IM2 in which the device is displayed. This makes it possible to set the regions R after appropriately determining which part of the device (in the above example, the engine 2) is located at which display position in the temperature image IM1, thereby making it possible to more appropriately set the multiple regions R in the temperature image IM1.
[0059] The configuration and operation of the diagnostic device 1 have been described above with reference to Figures 1 to 7. However, the configuration and operation of the diagnostic device 1 are not limited to the examples described above.
[0060] For example, in the above description, an example has been described in which the diagnostic device 1 has one pair of the infrared camera 11 and the visible light camera 12. However, the diagnostic device 1 may have two or more pairs of the infrared camera 11 and the visible light camera 12. In this case, abnormality diagnosis of the engine 2 can be performed over a wider range.
[0061] For example, in the above example, the region R set in the temperature image IM1 corresponds to the range in which each part of the engine 2 is present. However, the region R set in the temperature image IM1 may include a range in which each part of the engine 2 is present and a range in which each part of the engine 2 is not present. Furthermore, the region R set in the temperature image IM1 may correspond to one pixel in the temperature image IM1. In this case, one region R is set for each pixel in the temperature image IM1.
[0062] For example, in the above example, the average value of the temperature in each region R is used as the first feature value F11. However, a feature value other than the average value (for example, the variance of the temperature in each region R, which will be described later) may be used as the first feature value F11.
[0063] For example, in the above description, an example has been described in which, in step S106, the identifying unit 13c identifies the difference between the correlation coefficient obtained using the reference dataset D1 and the correlation coefficient obtained using the diagnostic object dataset D2 as the change in the first correlation, which is the correlation of the first feature amount F11 between the different regions R. However, in step S106, the identifying unit 13c may also identify the ratio between the correlation coefficient obtained using the reference dataset D1 and the correlation coefficient obtained using the diagnostic object dataset D2 as the change in the first correlation, which is the correlation of the first feature amount F11 between the different regions R.
[0064] For example, in the above description, an example has been described in which step S106 involves calculating a correlation coefficient of the first feature quantity F11 between two regions R in the reference dataset D1 and calculating a correlation coefficient of the first feature quantity F11 between two regions R in the diagnostic target dataset D2. However, these two processes do not necessarily have to be performed. For example, the identification unit 13c may directly identify a change in the first correlation, which is the correlation of the first feature quantity F11 between two regions R, for each pair of two regions R selectable from regions R1 to R9, based on the probability density function of the first feature quantity F11 in each region R in the reference dataset D1 and the probability density function of the first feature quantity F11 in each region R in the diagnostic target dataset D2. Specifically, such processing can be achieved by using a known technique called direct probability density ratio estimation (see, for example, "Anomaly Detection and Change Detection (Machine Learning Professional Series)" by Tsuyoshi Ide and Masashi Sugiyama, published by Kodansha on August 8, 2015). By using direct probability density ratio estimation and omitting the above two processes, the computational load can be reduced.
[0065] For example, in the above example, in step S106, a process of calculating a correlation coefficient of the first feature amount F11 between two regions R in the reference dataset D1 and a process of calculating a correlation coefficient of the first feature amount F11 between two regions R in the diagnostic object dataset D2 are performed. However, in these two processes, the identifying unit 13c may calculate a partial correlation coefficient instead of or in addition to the correlation coefficient. In this case, the identifying unit 13c can identify the difference or ratio between the partial correlation coefficient obtained using the reference dataset D1 and the partial correlation coefficient obtained using the diagnostic object dataset D2 as a change in the first correlation, which is the correlation of the first feature amount F11 between the different regions R.
[0066] In the above, an example has been described in which the first feature quantity F11 is used as a feature quantity in the abnormality diagnosis of the engine 2. However, other feature quantities may be used in addition to the first feature quantity F11 in the abnormality diagnosis of the engine 2. Below, a first modification and a second modification will be described as modification quantities in which other feature quantities are used in addition to the first feature quantity F11.
[0067] FIG. 8 is a diagram illustrating an example of a reference data set D1 according to a first modified example of the present disclosure. The reference data set D1 of FIG. 8 includes not only the calculation results of the first feature quantity F11 for each region R at each time point T, but also the calculation results of the first feature quantity F12 for each region R at each time point T. Like the first feature quantity F11, the first feature quantity F12 is a feature quantity related to temperature. Specifically, the first feature quantity F12 is the variance of temperature in each region R. The identification unit 13c calculates the first feature quantity F12 for each region R at each time point T based on the temperature image IM1 acquired at each time point T. Note that FIG. 8 does not illustrate the actually calculated value of the first feature quantity F12. However, the value of the first feature quantity F12 may differ for each region R at the same time point T.
[0068] In the first modified example, the identification unit 13c generates a data set as a diagnostic object data set D2 by adding the calculation results of the first feature amount F11 and the calculation results of the first feature amount F12 for each region R at the current time Tk to the reference data set D1 in Fig. 8. Then, in the first modified example, the identification unit 13c identifies a change in the correlation between the feature amounts based on the reference data set D1 and the diagnostic object data set D2 obtained as described above.
[0069] First, the specifying unit 13c calculates the probability density function of the first feature amount F11 in each region R in the reference data set D1, as well as the probability density function of the first feature amount F12 in each region R in the reference data set D1.
[0070] For example, the identification unit 13c calculates a probability density function of the first feature F12 in region R1 using a data group G3 consisting of n first feature quantities F12 in region R1. In this case, the probability density function of the first feature quantity F12 in region R1 indicates the probability density of each value of the first feature quantity F12 in data group G3. The identification unit 13c also calculates a probability density function of the first feature quantity F12 in region R2 in the reference dataset D1. Specifically, the identification unit 13c uses a data group G4 consisting of n first feature quantities F12 in region R2 to identify a probability density function of the first feature quantity F12 in region R2. In this case, the probability density function of the first feature quantity F12 in region R2 indicates the probability density of each value of the first feature quantity F12 in data group G4. The identification unit 13c similarly calculates the probability density function of the first feature quantity F12 in the reference dataset D1 for each of regions R3 to R9.
[0071] Next, the identification unit 13c identifies correlations for all combinations of data groups corresponding to each column in the reference data set D1. That is, in the first modified example, the identification unit 13c calculates, for the reference data set D1, in addition to the correlation coefficient of the first feature amount F11 between different regions R, the correlation coefficient of the first feature amount F12 between different regions R, the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R, and the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to each of the different regions R. The method for calculating the correlation coefficient of the first feature amount F12 between different regions R is the same as the method for calculating the correlation coefficient of the first feature amount F11 between different regions R described above.
[0072] For example, the identification unit 13c calculates a correlation coefficient between the first feature quantity F11 and the first feature quantity F12 in the same region R1 in the reference data set D1 based on the probability density function of the first feature quantity F11 in the region R1 in the reference data set D1 and the probability density function of the first feature quantity F12 in the same region R1 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates a correlation coefficient between the first feature quantity F11 and the first feature quantity F12 in the same region R for each of the regions R2 to R9.
[0073] For example, the identification unit 13c calculates a correlation coefficient between the first feature quantity F11 and the first feature quantity F12 corresponding to the different regions R1 and R2 in the reference data set D1 based on the probability density function of the first feature quantity F11 in the region R1 in the reference data set D1 and the probability density function of the first feature quantity F12 in the region R2 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. Furthermore, the identification unit 13c calculates a correlation coefficient between the first feature quantity F11 and the first feature quantity F12 corresponding to the different regions R2 and R1 in the reference data set D1 based on the probability density function of the first feature quantity F11 in the region R2 in the reference data set D1 and the probability density function of the first feature quantity F12 in the region R1 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c calculates the correlation coefficient between the first feature F11 and the first feature F12 corresponding to different regions R for all pairs of two regions R selectable from regions R1 to R9, except for the pair of region R1 and region R2.
[0074] Furthermore, in the diagnostic object dataset D2, the specifying unit 13c specifies correlations for all combinations of data groups corresponding to each column, in the same manner as in the reference dataset D1.
[0075] Then, the identification unit 13c calculates the difference between the various correlation coefficients obtained using the reference dataset D1 as described above and the various correlation coefficients obtained using the diagnostic subject dataset D2 as described above, where the correlation coefficients are the same as the comparison subject.
[0076] For example, the identification unit 13c calculates the difference between the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the reference data set D1 and the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient of the first feature amount F12 between regions R1 and R2 in the reference data set D1 and the correlation coefficient of the first feature amount F12 between regions R1 and R2 in the diagnostic object data set D2. As described above, the difference calculated in this manner corresponds to an index indicating a change in the first correlation, which is the correlation between the first feature amounts (specifically, the first feature amount F11 or the first feature amount F12) between different regions R.
[0077] For example, the identification unit 13c calculates the difference between the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R1 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R1 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to different regions R1 and R2 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to different regions R1 and R2 in the diagnostic object data set D2. These differences correspond to indicators of a change in the correlation between the first feature amount F11 and the first feature amount F12 in the same region R or the second correlation, which is the correlation between the first feature amount F11 and the first feature amount F12 corresponding to different regions R.
[0078] In the first modified example, the diagnoser 13d diagnoses an abnormality of the engine 2 based on a change in the second correlation in addition to a change in the first correlation. For example, in the first modified example, if a region R exists in which the difference between the two data sets D1 and D2 of the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R is equal to or greater than a reference value, the diagnoser 13d diagnoses an abnormality of the engine 2. Also, in the first modified example, if a pair of two regions R exists in which the difference between the two data sets D1 and D2 of the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to different regions R is equal to or greater than a reference value, the diagnoser 13d diagnoses an abnormality of the engine 2. However, the identifying unit 13c may identify a correlation ratio as an index indicating a change in the second correlation instead of a correlation difference.
[0079] As described above, in the first modified example, the first feature quantities F11 and F12 include multiple types of feature quantities, and the second process (specifically, the process of identifying a change in the first correlation, which is the correlation between the first feature quantities related to temperature between different regions R) further identifies, in addition to the change in the first correlation, a change in the correlation between different types of first feature quantities F11 and F12 in the same region R, or a change in the second correlation, which is the correlation between different types of first feature quantities F11 and F12 corresponding to different regions R, and the third process (specifically, the process of diagnosing an abnormality in the device based on the change in the first correlation) performs an abnormality diagnosis based on the change in the second correlation in addition to the change in the first correlation. This enables the abnormality diagnosis to be performed using more information by focusing on the extent to which the temperature relationship between each part of the device (in the above example, engine 2) has changed from a normal state of the device as a reference, thereby enabling a more accurate diagnosis of an abnormality in the device.
[0080] In the above example, a first feature quantity F11, which is the average value of the temperature in each region R, and a first feature quantity F12, which is the variance of the temperature in each region R, are used as the multiple types of first feature quantities. However, the combination and number of types of first feature quantities used for abnormality diagnosis are not limited to the above example. For example, the maximum value of the temperature in each region R may be used as the first feature quantity. For example, three or more types of first feature quantities may be used.
[0081] In the above example, the process of calculating various correlation coefficients between feature quantities is described. However, in the first modified example, direct probability density ratio estimation may be used, and the process of calculating correlation coefficients may be omitted. Also in the first modified example, the identification unit 13c may calculate partial correlation coefficients instead of or in addition to correlation coefficients.
[0082] 9 is a diagram illustrating an example of a reference data set D1 according to a second modification of the present disclosure. The reference data set D1 in FIG. 9 includes calculation results of the first feature amounts F11 and F12 for each region R at each time point T, as well as calculation results of the second feature amounts F21 and F22 for each time point T. The second feature amounts F21 and F22 are feature amounts that do not depend on the region R. In other words, the second feature amounts F21 and F22 are feature amounts that do not depend on the position in the engine 2 and are common to all positions.
[0083] Specifically, the second feature amount F21 is a feature amount related to the operating state of the engine 2. Examples of the second feature amount F21 include the output and rotation speed of the engine 2. The identification unit 13c can acquire the output of the engine 2 at each time point T, for example, based on information acquired from a control device that controls the operation of the engine 2. Furthermore, the identification unit 13c can acquire the rotation speed of the engine 2 at each time point T, for example, based on the detection result of a sensor that detects the rotation speed of the engine 2.
[0084] The second feature quantity F22 is a feature quantity related to the input and output of heat to and from the engine 2. Examples of the second feature quantity F22 include the ambient temperature of the engine 2, the temperature of the coolant, the temperature of the lubricating oil, the temperature of the fuel, the temperature of the intake air, the flow state of the ambient air, and the temperature of the ambient heat source. The identification unit 13c can acquire this information at each time point T, for example, based on the detection results of a sensor that detects this information. However, the identification unit 13c may also calculate each of the above temperatures (e.g., the ambient temperature of the engine 2) at each time point T based on the temperature image IM1 acquired at each time point T.
[0085] In the second modified example, the identification unit 13c generates a data set as a diagnostic object data set D2 by adding the calculation results of the first feature amounts F11 and F12 and the calculation results of the second feature amounts F21 and F22 for each region R at the current time Tk to the reference data set D1 in Fig. 9. Then, in the second modified example, the identification unit 13c identifies a change in the correlation between the feature amounts based on the reference data set D1 and the diagnostic object data set D2 obtained as described above.
[0086] First, the specification unit 13c calculates the probability density functions of the first features F11 and F12 in each region R in the reference data set D1, as well as the probability density functions of the second features F21 and F22 in each region R in the reference data set D1.
[0087] For example, the specification unit 13c calculates a probability density function of the second feature quantity F21 using a data group G5 consisting of n second feature quantities F21. In this case, the probability density function of the second feature quantity F21 indicates the probability density of each value of the second feature quantity F21 in the data group G5. Furthermore, the specification unit 13c calculates a probability density function of the second feature quantity F22 using a data group G6 consisting of n second feature quantities F22. In this case, the probability density function of the second feature quantity F22 indicates the probability density of each value of the second feature quantity F22 in the data group G6.
[0088] Next, the identification unit 13c identifies correlations for all combinations of data groups corresponding to each column in the reference data set D1. That is, in the second modified example, the identification unit 13c identifies correlations between the first feature amounts (specifically, the first feature amount F11 or the first feature amount F12) and the second feature amounts (specifically, the second feature amount F21 or the second feature amount F22) in addition to correlation coefficients between the first feature amounts in the reference data set D1.
[0089] For example, the identification unit 13c calculates a correlation coefficient between the first feature amount F11 and the second feature amount F21 in the region R1 of the reference data set D1 based on the probability density function of the first feature amount F11 in the region R1 of the reference data set D1 and the probability density function of the second feature amount F21 in the reference data set D1. The identification unit 13c also calculates a correlation coefficient between the first feature amount F11 and the second feature amount F21 in each region R of the regions R2 to R9.
[0090] For example, the identification unit 13c calculates a correlation coefficient between the first feature quantity F12 and the second feature quantity F21 in the region R1 of the reference data set D1 based on the probability density function of the first feature quantity F12 in the region R1 of the reference data set D1 and the probability density function of the second feature quantity F21 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates a correlation coefficient between the first feature quantity F12 and the second feature quantity F21 in each region R of the regions R2 to R9.
[0091] For example, the identification unit 13c calculates a correlation coefficient between the first feature quantity F11 and the second feature quantity F22 in the region R1 of the reference data set D1 based on the probability density function of the first feature quantity F11 in the region R1 of the reference data set D1 and the probability density function of the second feature quantity F22 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates a correlation coefficient between the first feature quantity F11 and the second feature quantity F22 in each region R of the regions R2 to R9.
[0092] For example, the identification unit 13c calculates a correlation coefficient between the first feature quantity F12 and the second feature quantity F22 in the region R1 of the reference data set D1 based on the probability density function of the first feature quantity F12 in the region R1 of the reference data set D1 and the probability density function of the second feature quantity F22 in the reference data set D1. The correlation coefficient corresponds to an index indicating the strength of the correlation between the two probability density functions. The identification unit 13c also calculates a correlation coefficient between the first feature quantity F12 and the second feature quantity F22 in each region R of the regions R2 to R9.
[0093] Furthermore, in the diagnostic object dataset D2, the specifying unit 13c specifies correlations for all combinations of data groups corresponding to each column, in the same manner as in the reference dataset D1.
[0094] Then, the identification unit 13c calculates the difference between the various correlation coefficients obtained using the reference dataset D1 as described above and the various correlation coefficients obtained using the diagnostic subject dataset D2 as described above, where the correlation coefficients are the same as the comparison subject.
[0095] For example, the identification unit 13c calculates the difference between the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the reference data set D1 and the correlation coefficient of the first feature amount F11 between regions R1 and R2 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient of the first feature amount F12 between regions R1 and R2 in the reference data set D1 and the correlation coefficient of the first feature amount F12 between regions R1 and R2 in the diagnostic object data set D2. As described above, the difference calculated in this manner corresponds to an index indicating a change in the first correlation, which is the correlation between the first feature amounts (specifically, the first feature amount F11 or the first feature amount F12) between different regions R.
[0096] For example, the identification unit 13c calculates the difference between the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R1 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the first feature amount F12 in the same region R1 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to different regions R1 and R2 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the first feature amount F12 corresponding to different regions R1 and R2 in the diagnostic object data set D2. As described above, the difference calculated in this manner corresponds to an index indicating a change in the correlation between the first feature amount F11 and the first feature amount F12 in the same region R or the second correlation, which is the correlation between the first feature amount F11 and the first feature amount F12 corresponding to different regions R.
[0097] For example, the identification unit 13c calculates the difference between the correlation coefficient between the first feature amount F11 and the second feature amount F21 in the region R1 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the second feature amount F21 in the region R1 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient between the first feature amount F12 and the second feature amount F21 in the region R1 in the reference data set D1 and the correlation coefficient between the first feature amount F12 and the second feature amount F21 in the region R1 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient between the first feature amount F11 and the second feature amount F22 in the region R1 in the reference data set D1 and the correlation coefficient between the first feature amount F11 and the second feature amount F22 in the region R1 in the diagnostic object data set D2. The identification unit 13c also calculates the difference between the correlation coefficient between the first feature F12 and the second feature F22 in the region R1 in the reference data set D1 and the correlation coefficient between the first feature F12 and the second feature F22 in the region R1 in the diagnostic target data set D2. This difference corresponds to an index indicating a change in the third correlation, which is the correlation between the first feature F12 and the second feature F22.
[0098] In the second modified example, the diagnoser 13d diagnoses an abnormality of the engine 2 based on a change in the third correlation in addition to a change in the first correlation and a change in the second correlation. For example, in the second modified example, if there is a region R in which the difference between the two data sets D1 and D2 of the correlation coefficient between the first feature amount (specifically, the first feature amount F11 or the first feature amount F12) and the second feature amount (specifically, the second feature amount F21 or the second feature amount F22) in any region R is equal to or greater than a reference value, the diagnoser 13d diagnoses that an abnormality has occurred in the engine 2. However, the identifier 13c may identify a correlation ratio, instead of a correlation difference, as an index indicating a change in the third correlation.
[0099] As described above, in the second modified example, the second process (specifically, the process of identifying a change in the first correlation, which is the correlation between the first feature amount related to temperature between different regions R) further identifies a change in the third correlation, which is the correlation between the first feature amount (in the above example, the first feature amount F11 or the first feature amount F12) and the second feature amount independent of region R (in the above example, the second feature amount F21 or the second feature amount F22). The third process (specifically, the process of diagnosing an abnormality in the device based on the change in the first correlation) performs an abnormality diagnosis based on the change in the third correlation, in addition to the change in the first correlation. This allows for an abnormality diagnosis to be performed by focusing on the extent to which the temperature relationship between each part of the device (in the above example, the engine 2) has changed from the normal state of the device, and further taking into account the effect of the second feature amount on the temperature of each part of the device. This allows for a more accurate diagnosis of an abnormality in the device.
[0100] In particular, in the second modified example, the second feature amount includes a feature amount (in the above example, the second feature amount F21) related to the operating state of the device (in the above example, the engine 2). This allows abnormality diagnosis to be performed taking into account the effect of the operating state of the device on the temperature of each part of the device, thereby appropriately realizing more accurate diagnosis of abnormalities in the device.
[0101] In particular, in the second modified example, the second feature amount includes a feature amount (in the above example, the second feature amount F22) related to the heat input / output to / from the device (in the above example, the engine 2). This allows for abnormality diagnosis to be performed taking into account the effect of the heat input / output state to / from the device on the temperature of each part of the device, thereby appropriately realizing more accurate diagnosis of abnormalities in the device.
[0102] In the above example, a change in the third correlation is identified in addition to a change in the first correlation and a change in the second correlation, and an abnormality diagnosis is performed based on these changes. However, in the above example, a change in the second correlation does not necessarily have to be identified. For example, one of the first feature amount F11 and the first feature amount F12 may be omitted from the reference data set D1 in FIG. 9, resulting in a single type of first feature amount.
[0103] In the above example, the process of calculating various correlation coefficients between feature quantities is described. However, in the second modified example, direct probability density ratio estimation may be used, and the process of calculating correlation coefficients may be omitted. Also in the second modified example, the identification unit 13c may calculate partial correlation coefficients instead of or in addition to correlation coefficients.
[0104] Although the embodiments have been described above with reference to the accompanying drawings, it goes without saying that the present disclosure is not limited to the above-described embodiments. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.
[0105] 1: Diagnostic device 2: Engine (device) 11: Infrared camera 12: Visible light camera 13: Processing device 13a: Acquisition unit 13b: Setting unit 13c: Identification unit 13d: Diagnosis unit 13e: Storage unit F11: First feature amount F12: First feature amount F21: Second feature amount F22: Second feature amount IM1: Temperature image IM2: Visible light image R1: Region R2: Region R3: Region R4: Region R5: Region R6: Region R7: Region R8: Region R9: Region
Claims
1. A program that causes a computer to perform the following steps: a first process of setting a plurality of regions in a temperature image showing the temperature distribution of an apparatus; a second process of identifying a change in a first correlation, which is a correlation between a first feature value related to temperature between the different regions; and a third process of diagnosing an abnormality in the apparatus based on the change in the first correlation.
2. The program of claim 1, wherein the first feature includes a plurality of types of feature, and the second process further identifies, in addition to the change in the first correlation, a change in a second correlation which is a correlation between the first feature amounts of different types in the same region, or a correlation between the first feature amounts of different types corresponding to the different regions, respectively, and the third process performs the abnormality diagnosis based on the change in the second correlation, in addition to the change in the first correlation.
3. The program according to claim 1 or 2, wherein the second process further identifies, in addition to the change in the first correlation, a change in a third correlation which is a correlation between the first feature and a second feature independent of the region, and the third process performs the abnormality diagnosis based on the change in the third correlation, in addition to the change in the first correlation.
4. The program according to claim 3, wherein the second characteristic amount includes a characteristic amount related to an operating state of the device.
5. The program according to claim 3, wherein the second feature amount includes a feature amount related to heat input / output to / from the device.
6. The program according to claim 1 or 2, wherein in the first process, a plurality of the regions are set in the temperature image based on a visible light image showing the device.
7. A diagnostic method in which a computer performs the following steps: a first process of setting a plurality of regions in a temperature image showing the temperature distribution of an apparatus; a second process of identifying a change in a first correlation, which is a correlation between a first feature amount related to temperature between the different regions; and a third process of diagnosing an abnormality in the apparatus based on the change in the first correlation.
8. A diagnostic device comprising: an infrared camera that captures a temperature image showing the temperature distribution of an apparatus; and a processing device that performs a first process of setting a plurality of regions in the temperature image, a second process of identifying a change in a first correlation that is a correlation between a first feature amount related to temperature between the different regions, and a third process of diagnosing an abnormality in the apparatus based on the change in the first correlation.
Citation Information
Patent Citations
Tire abnormality detection device, tire abnormality detection method and program
JP2021181286A
Method and device for judging overheat fault in GIS device
CN110426613A
Infrared intelligent detection system and intelligent detection method thereof
CN113138024A
Method and apparatus for monitoring abnormality of instrument
JP1994281543A
Method, device and program for extracting image, and method, device and program for detecting abnormality of pole mounted equipment of distribution pole utilizing image extracting method
JP2002366953A
