Information processing system, information processing method, and program

Abstract

To reduce human resources required for improving precision of determination of abnormality while suppressing error detection of determination of abnormality in a facility.SOLUTION: An information processing system monitors a state of a facility. The information processing system comprises a data collection section and a processing section. The data collection section acquires first state data regarding the facility at each first time interval, and acquires second state data regarding the facility at each second time interval that is longer than the first time interval. The processing section determines whether or not the facility is normal on the basis of time-series data of each of the first state data and the second state data. The time-series data of the first state data include first waveform data Wv1. The processing section generates second waveform data Wv2 by interpolation processing to time-series data of the second state data, generates third waveform data Wv3 by addition processing to the first waveform data Wv1 and the second waveform data Wv2, and performs the determination by using the third waveform data Wv3.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Conventionally, a configuration is known in which a model trained by machine learning is used to determine whether a sound emitted by equipment is in a normal state or an abnormal state. For example, Japanese Patent No. 7142530 (Patent Document 1) discloses an information processing system that collects sound emitted from equipment using a sound collection device and determines whether the sound collected by the sound collection device is in a normal state. In this information processing system, sound data that is not determined as normal data by the trained model is output from a speaker, and the sound data can be assigned either a label indicating normal data or a label indicating abnormal data in response to a user operation. This information system makes it possible to easily obtain learning data that enables highly accurate abnormal sound determination. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7142530 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the information processing system disclosed in Patent Document 1, after the start of operation of equipment anomaly detection, overdetection (determining that an actual normal case is an anomaly) by a trained model can be suppressed by additional learning using sound data labeled by a user as training data. However, since creating the training data requires the user to assign labels, a large amount of human resources may be allocated to improving the accuracy of the trained model.

[0005] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to reduce the human resources required to improve the accuracy of abnormality determination while suppressing overdetection in abnormality determination of equipment. [Means for solving the problem]

[0006] An information processing system according to one aspect of the present disclosure monitors the status of equipment. The information processing system includes a data collection unit and a processing unit. The data collection unit acquires first status data related to the equipment at first time intervals and acquires second status data related to the equipment at second time intervals longer than the first time intervals. The processing unit determines whether the equipment is normal or not based on time series data of the first status data and the second status data. The time series data of the first status data includes first waveform data. The processing unit generates second waveform data by interpolating the time series data of the second status data, generates third waveform data by adding the first waveform data and the second waveform data, and makes the above-mentioned determination using the third waveform data.

[0007] An information processing method according to another aspect of the present disclosure is a method for monitoring the status of equipment. The information processing method includes the steps of acquiring first status data related to the equipment at first time intervals and acquiring second status data related to the equipment at second time intervals longer than the first time intervals, and determining whether the equipment is normal or not based on time series data of the first status data and the second status data. The time series data of the first status data includes first waveform data. The determination step generates second waveform data by interpolating the time series data of the second status data, generates third waveform data by adding the first waveform data and the second waveform data, and performs the determination using the third waveform data. [Effects of the Invention]

[0008] According to the information processing system and information processing method disclosed herein, by determining whether or not equipment is normal using third waveform data generated from time series data of first status data and second status data acquired at different time intervals, it is possible to reduce overdetection in equipment abnormality determination while reducing the human resources required to improve the accuracy of the abnormality determination. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the configuration of an information processing system and the equipment in a factory monitored by the information processing system. [Figure 2] 1. FIG. 4 is a diagram for explaining an example of an interpolation process of sensor data performed in the mixing process by the server device of FIG. [Figure 3] 1. FIG. 4 is a diagram for explaining the process of adding waveform data of sound data and waveform data of sensor data, which is performed in the mixing process by the server device of FIG. [Figure 4] FIG. 2 is a diagram showing a setting screen for mixing processing by the server device of FIG. [Figure 5] FIG. 2 is a flowchart illustrating the overall processing flow in the information processing system. [Figure 6] FIG. 10 is a flow diagram for explaining the flow of preparation work for collecting sound data and sensor data in the data collection unit. [Figure 7] FIG. 10 is a flowchart illustrating the flow of a process for generating a reference model that is a normal model. [Figure 8] FIG. 10 is a flowchart illustrating a process flow of equipment monitoring using a reference model. [Figure 9] FIG. 10 is a diagram for explaining processing of data obtained by a data collection unit. [Figure 10] FIG. 10 is a diagram illustrating an example of a display screen displayed on a terminal device. [Figure 11] 10 is a diagram illustrating an example of data displayed on the screen of a terminal device when a tab related to equipment monitoring is selected. FIG. [Figure 12] FIG. 10 is a diagram illustrating another example of data displayed on the screen of the terminal device when a tab related to equipment monitoring is selected. [Figure 13] FIG. 10 is a diagram illustrating yet another example of data displayed on the screen of the terminal device when a tab related to equipment monitoring is selected. [Figure 14] FIG. 10 is a flowchart illustrating the flow of a labeling process executed in the information processing system. [Figure 15] FIG. 10 is a flowchart illustrating the flow of a false positive model generation process. [Figure 16] FIG. 10 is a flowchart illustrating the flow of an anomaly model generation process. [Figure 17] FIG. 10 is a diagram showing a screen displayed on the terminal device. [Figure 18] FIG. 10 is a diagram showing a screen displayed on the terminal device. [Figure 19] 10A is a diagram showing a judgment graph for sound data, a judgment graph for temperature data, and a judgment graph for mixed data of sound data and temperature data. FIG. [Figure 20] FIG. 10 is a diagram showing (a) a judgment graph for sound data, (b) a judgment graph for odor data, and (c) a judgment graph for mixed data of sound data and odor data. [Figure 21] FIG. 10 is a flowchart illustrating a flow of a state monitoring process using a reference model, a false detection model, and an abnormality model. [Figure 22] FIG. 10 is a diagram showing a screen displayed on the terminal device. [Figure 23] FIG. 2 is a diagram illustrating a functional configuration of the information processing system. [Figure 24] FIG. 1 is a diagram illustrating a typical example of a hardware configuration of a server device. [Figure 25] FIG. 1 is a diagram illustrating a typical example of a hardware configuration of a terminal device. [Figure 26] FIG. 10 is a diagram showing a health monitoring screen. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an information processing system according to each embodiment of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0011] <A. System Configuration> FIG. 1 is a diagram showing the configuration of the information processing system 1 and the facilities 900 in the factory 1000 monitored by the information processing system 1. In FIG. 1, as examples of the facilities 900 to be monitored, piping facilities 900A and 900B are shown. An abnormality such as a water leak Dr has occurred in the piping facility 900A. The number of facilities 900 may be one or three or more. Further, the facility 900 is not limited to piping facilities, and may be, for example, mechanical equipment.

[0012] As shown in FIG. 1, the information processing system 1 includes a cloud system 100 that forms the core of facility monitoring, a terminal device 200, and a plurality of data collection units 300 (data collection units). Note that the number of terminal devices 200 is not limited to one. Also, the number of data collection units 300 may be one.

[0013] The information processing system 1 uses the data collection unit 300 and the cloud system 100 to monitor in real time whether an abnormality has occurred in the facility 900. Note that a plurality of state data reflecting the state of one facility 900 is acquired by one data collection unit 300.

[0014] The cloud system 100 is configured, for example, by a server device 100A (processing unit) and a storage device 100B. The server device 100A is communicably connected to a terminal device 200. The cloud system 100 may be configured by a single server device or by multiple server devices. The server device 100A and the storage device 100B are communicably connected to each other. As long as the cloud system 100 includes at least one server device, the configuration of other devices is not particularly limited.

[0015] The data collection unit 300 is disposed near the facility 900. The data collection unit 300 acquires sound data (first status data) at each sampling time (first time interval), and collects sensor data (second status data) at each sampling time (second time interval) longer than the sampling time of the sound data. The data collection unit 300 includes a sound collection device 310 and a measurement sensor 320.

[0016] The sound collection device 310 includes a microphone and a data communication device. The data communication device transmits sound data (packet data) collected by the microphone at every sampling time (for example, 10 ms) to the server device 100A via the network NW. Typically, the data communication device transmits the sound data to the server device 100A in real time. The sound data is sequentially stored in the storage device 100B. The sound collection device 310 may be an integrated device of a microphone and a data communication device.

[0017] The measurement sensor 320 transmits sensor data (packet data) measured at each sampling time (for example, 1 second) to the server device 100A via the network NW. The measurement sensor 320 measures values other than sound (amount or level). The sampling time of the measurement sensor 320 is longer than the sampling time of the sound collection device 310. The sampling time of the measurement sensor 320 is, for example, a time interval that is 100 times or more longer than the sampling time of the sound collection device 310. Values measured by the measurement sensor 320 include, for example, temperature, humidity, odor, current, voltage, rotation speed, pressure, or flow rate. Typically, the measurement sensor 320 transmits sensor data to the server device 100A in real time. The sensor data is sequentially stored in the storage device 100B.

[0018] The server device 100A generates mixed data by performing a mixing process on the time-series data of the sound data (first waveform data) and the time-series data of the sensor data stored in the storage device 100B. The server device 100A sequentially analyzes the mixed data for each piece of equipment 900 to determine whether the equipment 900 is normal or abnormal. Typically, the server device 100A determines whether the equipment 900 is normal or abnormal using a model (trained model) that has been trained by machine learning. The server device 100A stores the analysis results in the storage device 100B. If the server device 100A determines that an abnormality has occurred as a result of the analysis, the server device 100A notifies the terminal device 200 in real time that an abnormality has occurred in the equipment 900. Specifically, the server device 100A notifies the user of the terminal device 200 (maintenance / monitoring personnel 950) of which piece of equipment 900 the abnormality has occurred in.

[0019] The terminal device 200 is configured to be able to display various types of status data collected from the server device 100A. Various types of monitoring screens can be displayed on the terminal device 200. Typically, the terminal device 200 uses a browser to display various types of information provided by the server device 100A.

[0020] In FIG. 1, the case where the waveform data collected by the data collection unit 300 is time-series data of sound data has been described. The waveform data is not limited to time-series data of sound, and may be, for example, time-series data of vibration data or time-series data of acceleration.

[0021] FIG. 2 is a diagram for explaining an example of interpolation processing of sensor data performed in the mixing process by the server device 100A of FIG. 1. FIG. 2(a) is a time chart showing the correspondence between the values of sensor data and the measurement times. FIG. 2(b) shows the waveform data obtained by the interpolation processing for FIG. 2(a).

[0022] In FIG. 2(a), among the plot points, which are combinations of a plurality of measured values measured for each sampling time and a plurality of measurement times included in the sensor data, four plot points P1, P2, P3, and P4 are shown. The plot points P1, P2, P3, and P4 are measured at times t1, t2, t3, and t4 (t1 < t2 < t3 < t4), respectively. Among the times t1, t2, t3, and t4, the time intervals between two adjacent measurement times are equal to the sampling time. The measured values of the plot points P1, P2, P3, and P4 are A3, A4, A2, and A1 (A1 < A2 < A3 < A4), respectively.

[0023] The server device 100A generates waveform data in which the amplitude value is the measurement value at either the start time or the end time of a time interval between two pieces of sensor data adjacent to each other in the time series. The frequency of the waveform data is, for example, 44.1 kHz. Using FIG. 2(a) as an example, the server device 100A generates waveform data of amplitude A3, which is the measurement value at time t1, during the time interval from t1 to t2. The server device 100A generates waveform data of amplitude A4, which is the measurement value at time t2, during the time interval from t2 to t3. The server device 100A generates waveform data of amplitude A2, which is the measurement value at time t3, during the time interval from t3 to t4. The server device 100A generates waveform data of amplitude A1, which is the measurement value at time t4, during the time interval from t4 to t5. Hereinafter, the time-series data of the sensor data generated by the interpolation process shown in FIG. 2 will also be referred to as waveform data of the sensor data (second waveform data).

[0024] 3 is a diagram illustrating the addition process of waveform data Wv1 of sound data (sound waveform data) and waveform data Wv2 of sensor data (sensor waveform data) performed in the mixing process by the server device 100A of FIG. 1. As shown in FIG. 3, the sound waveform data Wv1 and the sensor waveform data Wv2 are held as queues qa and qb, respectively. The queue qa includes at least one packet pa. The at least one packet pa includes at least one packet pb, and the queue qb includes at least one packet pb. In the at least one packet pa and the at least one packet pb, the measurement times of two packets pa_n and pb_n in corresponding order n (n is a natural number) are the same or close to each other. The server device 100A weights and adds the values of the two packets pa_n and pb_n to generate mixed data Wv3 (third waveform data). Note that weighted addition of two packet values means adding the value of the sound data packet multiplied by weight α (first weight) to the value of the sensor data packet corresponding to that packet multiplied by weight β (second weight). The ratio of the sound data weight α to the sensor data weight β is also called the mixing ratio. For example, if the mixing ratio is 0.2, the value of packet pa_n multiplied by 1 is added to the value of packet pb_n multiplied by 0.2.

[0025] Fig. 4 is a diagram showing a setting screen 205 for the mixing process by the server device 100A of Fig. 1. The setting screen 205 is displayed on the terminal device 200. As shown in Fig. 4, the setting screen 205 has list boxes Lst1 and Lst2, add buttons Ba1 and Ba2, delete buttons Bd1 and Bd2, a numerical value setting control Nud, a toggle button Stg, and a reflect button Btn arranged thereon. When the reflect button Btn is pressed, the contents of the setting screen 205 are reflected in the mixing process.

[0026] The list box Lst1 lists waveform data. The user can select waveform data to be subjected to mixing processing from the waveform data listed in the list box Lst1. Fig. 4 shows sound waveform data Wv11, sound waveform data Wv12, vibration waveform data (vibration waveform data) Wv21, sound waveform data Wv31, and acceleration waveform data (acceleration waveform data) Wv31.

[0027] By pressing the Add button Ba1, new waveform data can be added to the list box Lst1, and by pressing the Delete button Bd1, selected waveform data can be deleted from the list box Lst1.

[0028] List box Lst2 lists the time series data of sensor data (sensor time series data) to be mixed together with the waveform data. The user can select sensor data to be referenced in the mixing process from the sensor data listed in list box Lst2. Figure 5 shows time series data of temperature data (temperature time series data) Ts11, temperature time series data Ts12, time series data of odor data (odor time series data) Ts21, time series data of current data (current time series data) Ts31, and time series data of voltage data (voltage time series data) Ts41.

[0029] By pressing the Add button Ba2, new data to be mixed can be added to the list box Lst2, and by pressing the Delete button Bd2, the selected data to be mixed can be deleted from the list box Lst2.

[0030] The mixing ratio can be input to the numerical setting control Nud. This value is reflected in the weighting of the two packet values in the addition process in Figure 3. By adjusting the mixing ratio, it is possible to prevent oversight of abnormal data detection (determining an actual abnormality as normal) in an abnormality determination based on the mixed data. As a result, the detection accuracy of the abnormality determination can be improved.

[0031] The toggle button Stg allows you to switch between enabling and disabling the mixing process. When the mixing process is enabled, mixing is performed based on the contents of the list boxes Lst1 and Lst2 and the numerical setting control Nud. When the mixing process is disabled, mixing is not performed, and anomaly detection is performed based on the waveform data selected in the list box Lst1.

[0032] 5 is a flow diagram illustrating the overall processing flow in the information processing system 1. Note that steps S1 to S11, which will be described later, are not automatically executed in order, but are typically executed by the server device 100A receiving a predetermined instruction based on a user operation (a user instruction to execute each step). Also, a reference model M1, a false positive model M2, and an anomaly model M3, which will be described below, are generated for each piece of equipment 900. Note that, hereinafter, a step will simply be referred to as S.

[0033] 5, in S1, the cloud system 100 generates a reference model M1 (normal model) by machine learning using mixed data of sound data and sensor data acquired from normal equipment 900 as learning data. In S2, the cloud system 100 performs equipment monitoring using the reference model M1. Specifically, the cloud system 100 uses the reference model M1 to determine whether mixed data generated from data collected by the data collection unit 300 is normal data.

[0034] In S3, a human actually listens to the audio data corresponding to the mixed data determined to be abnormal by the reference model M1, and if the human determines that the mixed data is in an abnormal state, the mixed data is labeled as being in an abnormal state. If the human determines that the mixed data is in a normal state (i.e., if the determination by the reference model is a false positive), the mixed data is labeled as being in a normal state.

[0035] In S4, the cloud system 100 generates the false detection model M2 using the sound data labeled as normal. Note that, if there is no mixing data labeled as normal in S3, the process of S4 is not performed.

[0036] In S5, the cloud system 100 generates an abnormal model M3 using sound data labeled as an abnormal state. Note that, if there is no mixing data labeled as an abnormal state in S3, the process of S5 is not performed.

[0037] In S6, the cloud system 100 performs equipment monitoring using the reference model M1, the false positive model M2, and the anomaly model M3. In S7, the cloud system 100 updates the reference model M1 using the false positive model M2. In S8, the cloud system 100 performs equipment monitoring using the updated reference model M1 and the anomaly model M3. Note that the processing in S7 is not essential and may not be performed.

[0038] In S9, a labeling process similar to S3 is performed. In S10, the cloud system 100 updates the false positive model M2 using the mixing data labeled as normal in S9. In S11, the cloud system 100 updates the abnormality model M3 using the mixing data labeled as abnormal in S9.

[0039] FIG. 6 is a flow diagram for explaining the flow of preparation work for collecting sound data and sensor data in the data collection unit 300.

[0040] 6, in S21, the cloud system 100 accepts an operation for synchronizing the clock times of the data communication device of the sound collection device 310, the clock times of the measurement sensor 320, the clock times of the facility 900, and the clock times of the server device 100A. In addition, in S22, the cloud system 100 accepts a user operation for adjusting the volume of the sound collected by the microphone of the sound collection device 310.

[0041] In S23, the cloud system 100 accepts a user operation for setting up transfer of sound data from each of the data communication device of the sound collection device 310 and the measurement sensor 320 to the server device 100A.

[0042] After the above adjustments and settings are completed, in S24, the data collection unit 300 performs data sampling of the collected sound and physical quantities. The sampling period of the sound collection device 310 can be, for example, 44.1 kHz. The sampling period of the measurement sensor 320 can be, for example, 1 second.

[0043] In S25, the sound collection device 310 converts the sound data into a PCM (Pulse Code Modulation) sound source. The data communication device of the sound collection device 310 transmits the PCM sound source sound data (packets) to the server device 100A. The measurement sensor 320 transmits sensor data (packets) to the server device 100A. Thereafter, the processes of S24, S25, and S26 are repeated.

[0044] Below, we will explain the process in the order of "Generation of the reference model M1 (S1)," "Use of the reference model M1 (S2)," "Generation of the false positive model M2 and the abnormality model M3 (S3, S4, S5)," "Use of the false positive model M2 and the abnormality model M3 (S6)," and "Update of the reference model M1 (S7)," following the flow of the flow diagram shown in Figure 5.

[0045] <Generation of the Normal Model M1> FIG. 7 is a flowchart for explaining the flow of the generation process of the normal model, the standard model M1. Typically, the standard model M1 is generated by the processor 151 (see FIG. 19) of the server device 100A using learning data (features described later) and a predetermined learning program.

[0046] As shown in FIG. 7, in S31, the server device 100A uses the data collection unit 300 to acquire sound data and sensor data when the facility 900 is operating normally. In S32, the server device 100A performs mixing processing on the sound data and sensor data. The mixing process in S32 includes the interpolation process S321 shown in FIG. 2 and the addition process S322 shown in FIG. 3.

[0047] In S33, the server device 100A performs preprocessing on the mixed data. Specifically, the server device 100A divides the mixed data into unit data Du per unit time. For example, the server device 100A divides the mixed data into unit data Du of Tu seconds (a predetermined number of seconds less than or equal to 1 second). Further, as preprocessing, noise removal, setting the frequency band to be analyzed, adaptation of a band-pass filter, etc. are performed.

[0048] In S34, the server device 100A extracts features from each unit data Du using a predetermined algorithm. In S35, the server device 100A performs machine learning using the extracted features. By repeating such machine learning, the standard model M1 is generated.

[0049] Note that if the sampling frequency is 44.1 kHz, the measurable frequency is half of the sampling frequency, and the frequency resolution is 10 Hz, the number of dimensions of the feature (the number of dimensions of the input vector) is 2205.

[0050] <Use of the Standard Model M1> 8 is a flow diagram illustrating the processing flow of equipment monitoring using the reference model M1. When the reference model M1 is generated, the false positive model M2 and the abnormality model M3 have not yet been generated, so the information processing system 1 performs equipment monitoring using only the reference model M1.

[0051] 8, in S41, the server device 100A acquires sound data and sensor data when the equipment 900 is operating normally, using the data collection unit 300. In S42, the server device 100A performs a mixing process on the sound data and sensor data, similar to S32 in FIG.

[0052] In S52, the server device 100A stores the data acquired in S41 and the mixed data generated in S42 as data for playback (raw data) in the storage device 100B. Specifically, the server device 100A associates the data with time information and stores it in the storage device 100B.

[0053] In S43, the server device 100A performs pre-processing on the mixed data in the same manner as in S33 of Fig. 7. Specifically, the server device 100A divides the mixed data into unit data Du for each unit time.

[0054] After S43, the server device 100A (more specifically, the processor 151) executes the processes of S44 to S51 for each piece of unit data Du.

[0055] In S44, the processor 151 extracts features from the unit data Du using the above-mentioned predetermined algorithm. In S45, the processor 151 inputs the extracted features to the reference model M1. In S46, the processor 151 obtains an output from the reference model M1. Specifically, an error with respect to the reference model M1 (cluster) is calculated, and the error is output.

[0056] In S47, processor 151 determines whether the calculated error is within a predetermined range R1. If processor 151 determines that the error is within range R1 (YES in S46), processor 151 determines that unit data Du is normal data in S48. Note that the 3σ method may be used, for example, to determine whether the data is within range R1.

[0057] If the processor 151 determines that the error is outside the range R1 (NO in S47), then in S49 the processor 151 provisionally determines that the unit data Du is abnormal data. Next, in S50, the processor 151 executes an abnormality notification process. More specifically, a predetermined warning notification is sent from the server device 100A to the terminal device 200. In this case, the terminal device 200 performs various processes such as displaying a warning. In S51, the processor 151 stores the determination result in the storage device 100B. The processes of S50 and S51 may be performed in parallel, or the process of S51 may be performed before the process of S50.

[0058] FIG. 9 is a diagram for explaining the processing of data obtained by the data collecting unit 300. As shown in FIG. 9, sound waveform data and mixed data are stored in the storage device 100B as storage data Ds1 and Ds. The mixed data is also processed as determination data Dd. That is, the server device 100A stores the mixed data (see (A)) generated from the sound waveform data and sensor time-series data obtained by one data collecting unit 300 as storage data Ds (see (B)), and also stores the mixed data as determination data Dd (see (C)) separately from the storage data Ds. For example, the processor 151 may use the storage data Ds as original data and generate the determination data Dd by copying the original data. The storage data Ds1 is the sound data itself (raw data) collected by the sound collecting device 310, and is data that has not been processed to remove sounds such as noise.

[0059] The processor 151 divides the determination data Dd into unit data Du for each unit time Tu. As described above, the processor 151 extracts a feature amount for each unit data Du.

[0060] 10 is a diagram showing an example of a display screen displayed on the terminal device 200. As shown in FIG.

[0061] The dashboard 270 includes a status monitoring item 271 , a communication log item 272 , a tool item 273 , a report item 274 , and an alert item 275 .

[0062] When the user of the terminal device 200 selects tab 211, a health monitoring screen is displayed. Health monitoring is performed by monitoring "pings" from the data collection unit 300. When the user of the terminal device 200 selects tab 212, a trap monitoring screen is displayed. When the user of the terminal device 200 selects tab 213, a syslog monitoring screen is displayed. When the user of the terminal device 200 selects tab 214, a protocol monitoring screen is displayed. When the user of the terminal device 200 selects tab 215, an equipment monitoring screen is displayed.

[0063] 11 is a diagram illustrating an example of data displayed on the screen of the terminal device 200 when the equipment monitoring tab 215 is selected. As shown in FIG. 11, the terminal device 200 displays a waveform 221 of the mixed data in association with time.

[0064] 12 is a diagram illustrating another example of data displayed on the screen of the terminal device 200 when the equipment monitoring tab 215 is selected. As shown in FIG. 12, the terminal device 200 displays a spectrum 222.

[0065] FIG. 13 is a diagram for explaining still another example of data displayed on the screen of the terminal device 200 when the tab 215 related to facility monitoring is selected. As shown in FIG. 13, the terminal device 200 displays a graph 223 showing the output of the specification model M1 (denoted as "evaluation value") and the normal range of the output.

[0066] The waveform 221, the spectrum 222, and the graph 223 can be utilized in the generation of the false detection model M2 and the abnormal model M3.

[0067] When the processor 151 of the server device 100A determines that the unit data Du is abnormal data, the terminal device 200 performs a warning display on the screen. For example, the terminal device 200 may perform a warning display by superimposing an object image showing a warning display on the screen 201 (see FIG. 10).

[0068] Also, the terminal device 200 or the server device 100A may notify the occurrence of an abnormality to the user (maintenance / supervisor 950) of the terminal device 200 by causing a display lamp (for example, a stacked display lamp), not shown, installed near the terminal device 200 to emit light. Further, the server device 100A may transmit a warning email to a predetermined address.

[0069] <D. Generation of False Detection Model M2 and Abnormal Model M3> First, as a preprocessing for generating the false detection model M2 and the abnormal model M3, the labeling process for data will be described. Next, the generation of the false detection model M2 and the abnormal model M3 will be described.

[0070] 14 is a flow diagram illustrating the flow of the labeling process executed in the information processing system 1. As shown in FIG. 14, in S61, the processor 251 of the terminal device 200 identifies partial data Dp corresponding to unit data Du tentatively determined as abnormal data by the reference model M1 from among the storage data Ds, and plays back partial data Dp1 of the storage data Ds1 corresponding to the partial data Dp. Specifically, the processor 251 outputs the partial data Dp1 of the storage data Ds1 from a speaker. The terminal device 200 acquires the partial data Dp1 from the server device 100A and plays back the partial data Dp1.

[0071] More specifically, based on the time information of the unit data Du determined to be abnormal data, the server device 100A extracts partial data Dp, Dp1 for the time period indicated by the time information from the storage data Ds, Ds1, respectively. The server device 100A transmits the extracted partial data Dp1 to the terminal device 200. The terminal device 200 plays the received partial data Dp1.

[0072] For example, when there are a plurality of unit data Du provisionally determined to be abnormal data consecutively in time, the terminal device 200 may play back the partial data Dp1 corresponding to these plurality of unit data Du. Typically, when the number of unit data Du provisionally determined to be abnormal data is equal to or greater than a threshold within a predetermined time period (for example, within 10 seconds), the terminal device 200 is configured to be able to play back all or a specified part of this period (see FIG. 18). In more detail, the terminal device 200 determines the section (period) to be played back by a user operation, and plays back the partial data Dp1 included in the determined section.

[0073] The user of the terminal device 200 determines whether the sound data used to generate the mixed data determined to be abnormal by the reference model M1 is actually in an abnormal state by listening to the sound output by the playback process of S61. That is, the user determines for himself or herself whether the sound data is an abnormal sound or a false detection (normal sound).

[0074] In S62, the server device 100A determines whether or not the terminal device 200 has received a user operation to select the object 231 (see FIG. 18) that has been indicated to be normal data (falsely detected data). Specifically, the server device 100A determines whether or not a signal indicating that the object 231 has been selected has been received from the terminal device 200.

[0075] If the object 231 is selected (YES in S62), in S63 the server device 100A labels the partial data Dp with a normal state, and stores the partial data Dp labeled with the normal state in the storage device 100B.

[0076] If the object 231 has not been selected (NO in S62), in S64 the server device 100A determines whether or not the terminal device 200 has accepted a user operation to select the object 232 (see FIG. 18) that indicates abnormal data. Specifically, the server device 100A determines whether or not a signal indicating that the object 232 has been selected has been received from the terminal device 200.

[0077] If the object 232 is selected (YES in S64), in S65 the server device 100A labels the partial data Dp as being in an abnormal state, and stores the partial data Dp labeled as being in an abnormal state in the storage device 100B.

[0078] If the object 232 is not selected (NO in S64), the terminal device 200 determines in S66 whether or not an operation for repeatedly playing the partial data Dp1 has been accepted. If the terminal device 200 determines that the operation has been accepted (YES in S66), the process proceeds to S61. If the terminal device 200 determines that the operation has not been accepted (NO in S66), the process proceeds to S62.

[0079] As a result, the labeling of the partial data Dp corresponding to the reproduced partial data Dp1, indicating whether it is in a normal state or an abnormal state, is completed.

[0080] Incidentally, in the processing of FIG. 14, an example has been described in which the user of the terminal device 200 performs labeling only through playback processing, but the present invention is not limited to this.

[0081] If the user cannot determine whether the state is normal or abnormal simply by checking the reproduced sound, the user may display on the screen at least one of the waveform 221 shown in Fig. 11, the spectrum 222 shown in Fig. 12, and the graph 223 shown in Fig. 13. In this case, the user can determine whether the mixed data determined to be abnormal by the reference model M1 is actually abnormal or normal by checking the reproduced sound and the screen information. In this case, the partial data Dp is also labeled based on the determination result.

[0082] 14, a configuration has been described in which a label of normal data or a label of abnormal data is assigned to each of the partial data Dp corresponding to each of all the unit data Du provisionally determined to be abnormal data, but the present invention is not limited to this. Labeling may be performed only on the partial data Dp designated (selected) by the user (see FIGS. 17 and 18). In other words, it is not necessarily required for the user to play back all of the partial data Dp1 corresponding to each of all the unit data Du provisionally determined to be abnormal data and listen to all of the partial data Dp1.

[0083] 15 is a flow diagram illustrating the flow of the process of generating the false positive model M2. As shown in FIG. 15, in S71, the processor 151 of the server device 100A reads out the partial data Dp labeled as normal data from the storage device 100B. In S72, the processor 151 extracts features from the read out partial data Dp using a predetermined algorithm. In S73, the server device 100A performs machine learning using the extracted features.

[0084] In S74, the processor 151 determines whether or not all of the partial data Dp labeled as normal data has been read. Note that the partial data Dp is data relating to the same equipment 900.

[0085] If processor 151 determines that all of the partial data Dp have not been read (NO in S74), it proceeds to S72. If processor 151 determines that all of the partial data Dp have been read (YES in S74), it ends the series of processes.

[0086] The server device 100A repeats such machine learning to generate the false positive model M2.

[0087] 16 is a flow diagram illustrating the flow of the generation process of the abnormal model M3. As shown in FIG. 16, in S81, the processor 151 of the server device 100A reads out the partial data Dp labeled as abnormal data from the storage device 100B. In S82, the processor 151 extracts features from the read partial data Dp using a predetermined algorithm. In S83, the server device 100A performs machine learning using the extracted features.

[0088] In S84, the processor 151 determines whether or not all of the partial data Dp labeled as abnormal data has been read. Note that the partial data Dp is data relating to the same equipment 900.

[0089] If processor 151 determines that all partial data Dp have not been read (NO in S84), it proceeds to S82. If processor 151 determines that all partial data Dp have been read (YES in S84), it ends the series of processes.

[0090] The server device 100A repeats this type of machine learning to generate an anomaly model M3.

[0091] In the following, an example will be described in which partial data Dp (10 seconds of mixed data) corresponding to all consecutive unit data within a predetermined time (hereinafter, 10 seconds) is labeled, rather than each piece of partial data Dp corresponding to one unit data Du. That is, an example will be described in which partial data Dp for 10 seconds is labeled, rather than the time unit (1 second or less) used for division in preprocessing.

[0092] Fig. 17 is a diagram showing a screen 202 displayed on the terminal device 200. As shown in Fig. 17, the dashboard 270 further includes a status monitoring item 276. The item 276 includes an item 2761 for a unit list (a list of data collection units 300), an item 2762 for a determination result, an item 2763 for mixing data learning, and an item 2764 for false positive data learning. By expanding the item 276 based on a user operation, the items 2761 to 2764 are displayed.

[0093] The alert item 275 includes an abnormality history item 2751. When the abnormality history item 2751 is selected by a user operation, a list of abnormality histories is displayed as shown in the figure. The list of abnormality histories includes a plurality of records. Each record includes at least the following fields: device name, model, date and time, and evaluation value. Each record is associated with partial data Dp.

[0094] In this example, in the list of abnormality history, partial data Dp corresponding to all of the unit data Du provisionally determined as abnormal data are associated as records, but partial data Dp corresponding to a period in which the unit data Du provisionally determined as abnormal data is concentrated at a certain rate or more are associated. Specifically, each record is associated with partial data Dp corresponding to a period in which the unit data Du provisionally determined as abnormal data exists at a threshold value or more within a predetermined time (within 10 seconds).

[0095] For example, if the time unit (unit time Tu) for division in preprocessing is 0.5 seconds, 20 unit data Du will be included in 10 seconds. If, for example, 12 or more of these 20 unit data Du are provisionally determined to be abnormal data, the partial data Dp for that 10 seconds will be managed as one record. Also, as described above, the partial data Dp for 10 seconds is labeled.

[0096] Fig. 18 is a diagram showing a screen 203 displayed on the terminal device 200. As shown in Fig. 18, when one record is selected from the list of abnormality histories in Fig. 17 by a user operation, a screen showing the determination result corresponding to the selected record is displayed as shown in the figure.

[0097] The terminal device 200 displays a decision graph corresponding to the data collection unit 300 (unit), model, and date and time corresponding to the selected record (see FIG. 17). The decision graph shows the waveform of the partial data Dp within a predetermined time period (within 10 seconds). The time axis (horizontal axis) is configured so that a section can be selected by user operation. The terminal device 200 can play back the sound of only the selected section (for example, section 235). Note that the section is selected by a pointer device such as a mouse, for example.

[0098] The user uses the button 236 or the like to cause the terminal device 200 to play back the displayed partial data Dp.

[0099] When the false positive object 231 (specifically, a button) is selected by a user operation, the server device 100A labels the displayed partial data Dp (in this example, 10 seconds' worth of data) as normal.

[0100] When the abnormal object 232 (specifically, a button) is selected by a user operation, the server device 100A labels the displayed partial data Dp (10 seconds' worth of data in this example) as being in an abnormal state.

[0101] In addition, when a portion of the partial data Dp is specified in the decision graph and objects 231, 232 are selected, the server device 100A may label the partial data Dp in the specified time domain according to the selected objects 231, 232.

[0102] Furthermore, the information processing system 1 is configured so that when partial data Dp is labeled by selecting the objects 231 and 232, the reason for determining whether the data is normal or abnormal can be written. Specifically, the reason can be written by the user selecting the object 234. The written reason is associated with the labeled partial data Dp and transmitted to the server device 100A.

[0103] According to such a labeling method, it is possible to easily obtain learning data (labeled partial data Dp) that enables highly accurate abnormality determination. Note that the labeling method described above is an example, and is not limited to the configuration using the screen 203 described above. Labeling may also be performed using the user's voice.

[0104] FIG. 19 is a diagram showing together (a) a determination graph of sound data, (b) a determination graph of temperature data, and (c) a determination graph of mixed data of sound data and temperature data. In FIG. 19, it is explained that when the situation where the evaluation value of the normative model M1 exceeds the threshold continues for a reference time (for example, 30 seconds) or more, it is the case where an abnormality has occurred in the facility, and other cases are normal cases. The determination graph of temperature data is a determination graph of waveform data obtained by interpolation processing (see FIG. 2) for the temperature data. The threshold value in the determination graph of sound data is 0.010. The threshold value in the determination graph of temperature data is 0.00018. The threshold value in the determination graph of mixed data is 0.02. The mixing ratio of sound data and temperature data in the mixed data is 0.2. Among the time intervals t11 to t15 (t11 < t12 < t13 < t14 < t15) shown in FIG. 19, it is assumed that the time interval in which an actual abnormality has occurred is the time interval t12 to t13.

[0105] As shown in FIG. 19, in the determination graph of sound data, the time interval t11 to t12 is a normal case. The time interval t12 to t13 is an abnormal case. The time interval t13 to t14 is an abnormal case. The time interval t14 to t15 is a normal case. The detection of abnormality in the time interval t13 to t14 is a false detection.

[0106] In the determination graph of temperature data, the time interval t11 to t12 is a normal case. The time interval t12 to t13 is an abnormal case. The time interval t13 to t14 is a normal case. The time interval t14 to t15 is an abnormal case. The detection of abnormality in the time interval t14 to t15 is a false detection.

[0107] In the decision graph of the mixed data, the time interval t11 to t12 is a normal case. The time interval t12 to t13 is an abnormal case. The time interval t13 to t14 is a normal case. The time interval t14 to t15 is a normal case. In this way, by using mixed data of sound data and temperature data, overdetection of abnormalities can be prevented. As a result, the work of labeling overdetected data as normal data for additional learning is reduced, and the human resources required to improve the accuracy of the normative model M1 can be reduced.

[0108] FIG. 20 shows (a) a judgment graph for sound data, (b) a judgment graph for odor data, and (c) a judgment graph for mixed data of sound data and odor data. In FIG. 20, a case where the evaluation value of the reference model M1 exceeds the threshold value for a reference time (e.g., 30 seconds) or more is considered to be an abnormality in the equipment, and other cases are considered to be normal. The judgment graph for odor data is a judgment graph for waveform data obtained by interpolation processing of odor data (see FIG. 2). The judgment graph for sound data is the same as that in FIG. 19, so the description of the judgment graph for sound data will not be repeated. The threshold in the judgment graph for odor data is 0.0010. The threshold in the judgment graph for mixed data is 0.02. The mixing ratio of sound data and odor data in the mixed data is 0.1. Of the time intervals t11 to t15 shown in FIG. 20, the time interval in which an abnormality actually occurs is assumed to be the time interval t12 to t13.

[0109] As shown in FIG. 20, in the odor data judgment graph, the time interval t11 to t12 is a normal case. The time interval t12 to t13 is an abnormal case. The time interval t13 to t14 is a normal case. The time interval t14 to t15 is an abnormal case. The abnormality detected during the time interval t14 to t15 is an overdetection.

[0110] In the mixing data determination graph, the time intervals t11 to t12 are normal. The time interval t12 to t13 is abnormal. The time interval t13 to t14 is normal. The time interval t14 to t15 is normal. Thus, by using the mixing data of sound data and smell data, it is possible to prevent over-detection of abnormalities without requiring additional learning or the like for the normative model M1. As a result, as described above, it is possible to reduce the human resources required to improve the accuracy of the normative model M1.

[0111] <E. Use of False Detection Model M2 and Abnormal Model M3> FIG. 21 is a flowchart for explaining the flow of the state monitoring process using the normative model M1, the false detection model M2, and the abnormal model M3. As shown in FIG. 21, in S101, the server device 100A uses the data collection unit 300 to acquire sound data and sensor data when the facility 900 is operating normally. In S102, the server device 100A performs a mixing process on the sound data and sensor data in the same manner as S42 in FIG. 8.

[0112] In S116, the server device 100A stores the data acquired in S101 and the mixing data generated in S102 in the storage device 100B as data for playback (raw data). Specifically, the server device 100A stores the data in the storage device 100B in association with time information.

[0113] In S103, the server device 100A performs preprocessing on the acquired sound data in the same manner as S43 in FIG. 8. Specifically, the server device 100A divides the sound data into unit data Du per unit time.

[0114] After S103, the server device 100A (specifically, the processor 151) executes each of the processes of S104 to S115 for each of the unit data Du.

[0115] In S104, the processor 151 extracts features from the unit data Du using the above-mentioned predetermined algorithm. In S105, the processor 151 inputs the extracted features to the reference model M1. In S106, the processor 151 obtains an output from the reference model M1. Specifically, an error with respect to the reference model M1 (cluster) is calculated, and the error is output.

[0116] In S107, the processor 151 determines whether the calculated error is within a predetermined range R1. If the processor 151 determines that the error is within the range R1 (YES in S107), the processor 151 determines that the unit data Du is normal data in S108.

[0117] If the processor 151 determines that the error is outside the range R1 (NO in S107), it determines in S109 whether the extracted feature matches the false positive model M2. For example, if the false positive model M2 is configured as a trained model, the determination result of whether or not there is a match is output from the output layer.

[0118] If processor 151 determines that the feature matches false positive model M2 (YES in S109), it proceeds to S108. If processor 151 determines that the feature does not match false positive model M2 (NO in S109), it determines in S110 whether the extracted feature matches abnormality model M3. For example, if abnormality model M3 is configured as a trained model, the output layer outputs a determination result indicating whether or not there is a match.

[0119] If it is determined that the feature amount matches the abnormality model M3 (YES in S110), the processor 151 determines the unit data Du to be abnormal data in S111. Next, in S112, the processor 151 executes an abnormality notification process.

[0120] When it is determined that the feature amount does not match the abnormal model M3 (NO in S110), in S113, the processor 151 tentatively determines the unit data Du as abnormal data. Next, in S114, the processor 151 executes abnormal notification processing. In S115, the processor 151 stores the determination result in the storage device 100B.

[0121] <F. Update of the Specification Model> FIG. 22 is a diagram showing a screen 204 displayed on the terminal device 200. As shown in FIG. 22, when the item 2764 of misdetection data learning is selected by a user operation, as shown, a misdetection list is displayed. The misdetection list includes at least a unit name, a model, a date and time, an evaluation value, and details. In the details column, an object indicating that labeling has been performed and an object indicating which label of the normal state label and the abnormal state label has been assigned when labeling are displayed for each record.

[0122] According to such a configuration, the user can easily determine whether or not labeling has been performed on the partial data Dp of each record, and which label has been assigned when labeling.

[0123] <G. Functional Configuration> FIG. 23 is a diagram for explaining the functional configuration of the information processing system 1. As shown in FIG. 23, the information processing system 1 includes a cloud system 100, a terminal device 200, and a plurality of data collection units 300. The cloud system 100 typically includes a server device 100A and a storage device 100B.

[0124] Each data collection unit 300 collects sound data and measurement values of facilities in the vicinity where each is installed. Each data collection unit 300 transmits the collected data to the cloud system 100. The server device 100A of the cloud system 100 acquires the data.

[0125] (g1. Server device 100A) The server device 100A includes a control unit 101 and a communication IF (Interface) 103. The control unit 101 includes a data processing unit 111, a model processing unit 112, and a determination unit 113. The model processing unit 112 includes a model generation unit 1121, a label assignment unit 1122, and a model update unit 1123. The determination unit 113 includes a first determination unit 1131, a second determination unit 1132, and a third determination unit 1133.

[0126] The control unit 101 controls the overall operation of the server device 100A. Specifically, the control unit 101 controls the overall operation of the server device 100A by executing an operating system and application programs stored in a storage unit (not shown) or the like. The control unit 101 is typically realized by the processor 151 (see FIG. 24) executing the operating system and application programs.

[0127] The communication IF 103 is an interface for communicating with external devices. The server device 100A communicates with each data collection unit 300 and the terminal device 200 via the communication IF 103. The server device 100A also communicates with the storage device 100B via a communication IF (not shown).

[0128] The data processing unit 111 stores the data collected by the data collection unit 300 as storage data Ds in the storage device 100B and processes the data as determination data Dd to be used for determination. Specifically, the data processing unit 111 divides the determination data Dd into unit data Du for each unit time. The data processing unit 111 sends the unit data Du to the determination unit 113.

[0129] The model generation unit 1121 of the model processing unit 112 acquires unit data Du in the normal state from the data processing unit 111, and generates a reference model M1 based on the unit data Du. The generated reference model M1 is sent to the first determination unit 1131.

[0130] The model generation unit 1121 also generates a false positive model M2 and an abnormality model M3. The generated false positive model M2 is sent to the second determination unit 1132. The generated abnormality model M3 is sent to the third determination unit 1133.

[0131] As described above, the false positive model M2 is generated using the partial data Dp labeled as a normal state. As described above, the abnormality model M3 is generated using the partial data Dp labeled as an abnormal state. Such labeling is performed by the labeling unit 1122. Note that, as described above, the partial data Dp is stored in the storage device 100B.

[0132] The labeling unit 1122 labels the partial data Dp based on an instruction from the terminal device 200 (objects 231, 232 (see FIG. 18) by the user). The labeling unit 1122 assigns a label, depending on a user operation, between a label indicating normal data and a label indicating abnormal data to the partial data Dp (more specifically, the partial data Dp output from the speaker 295 of the terminal device 200).

[0133] Typically, when there is a threshold or more of unit data Du that has not been determined to be normal data within a predetermined time (predetermined unit period), the label assignment unit 1122 assigns a label according to a user operation to the partial data Dp corresponding to the predetermined time.

[0134] The model update unit 1123 updates the reference model M1 using false positive data (i.e., partial data Dp labeled as a normal state). The trigger for the update is typically a user instruction on the terminal device 200. However, the model update unit 1123 may also automatically perform the update process when a predetermined time has elapsed since the previous update. Furthermore, the model update unit 1123 may also automatically perform the update process when the number of false positive data reaches or exceeds a predetermined number. When automatically performing the update, the model update unit 1123 may require user permission from the terminal device 200.

[0135] The determination unit 113 determines whether the collected sound is in a normal state or an abnormal state. The first determination unit 1131 has the reference model M1 described above. More specifically, the first determination unit 1131 has a reference model M1 for each piece of equipment 900. The first determination unit 1131 uses the reference model M1 to determine whether the target equipment 900 is in a normal state. More specifically, the first determination unit 1131 uses the reference model M1 to determine whether each piece of unit data Du is normal data.

[0136] As described above, the second determination unit 1132 has a false positive model M2 generated using the partial data Dp labeled as a normal state. More specifically, the second determination unit 1132 has a false positive model M2 for each piece of equipment 900. As described above, the false positive model M2 is generated after operation. When the unit data Du is not determined to be normal data by the reference model M1, the second determination unit 1132 uses the false positive model M2 to determine whether the unit data Du is normal data.

[0137] As described above, the third determination unit 1133 has an abnormality model M3 generated using partial data Dp labeled with an abnormal state. More specifically, the third determination unit 1133 has an abnormality model M3 for each piece of equipment 900. As described above, the abnormality model M3 is generated after operation. The third determination unit 1133 uses the abnormality model M3 to determine whether or not the unit data Du is abnormal data.

[0138] When the control unit 101 determines (including provisionally determining) that the unit data Du is abnormal data, it issues a predetermined warning notice to the terminal device 200.

[0139] (g2. Terminal device 200) The terminal device 200 includes a control unit 291 , an input device 292 , a communication IF 293 , a display 258 , and a speaker 295 .

[0140] The control unit 291 controls the overall operation of the terminal device 200. Specifically, the control unit 291 controls the overall operation of the terminal device 200 by executing an operating system and application programs stored in a storage unit (not shown) or the like of the terminal device 200. The control unit 291 is typically realized by the processor 251 (see FIG. 25) executing the operating system and application programs.

[0141] The communication IF 293 is an interface for communicating with external devices. The terminal device 200 communicates with the server device 100A via the communication IF 293.

[0142] The input device 292 is a device that accepts user operations. The input device 292 is typically a keyboard, a mouse, or a touch panel. The input device 292 sends the accepted input to the control unit 291. Examples of the input include a selection operation for an item displayed on each screen described above, data input, and the like.

[0143] The display control unit 2911 displays various screens on the display. The display control unit 2911 switches the screen to be displayed on the display 258 based on a user operation using the input device 292. The display control unit 2911 displays, for example, screens 201 to 204 (see FIGS. 10, 17, 18, and 22), a waveform 221 (see FIG. 11), a spectrum 222 (see FIG. 12), and a graph 223 (see FIG. 13) on the display 258. The waveform 221 includes the waveform of unit data Du that was not determined to be normal data.

[0144] The playback processing unit 2912 plays back partial data Dp1 of sound data corresponding to partial data Dp using the speaker based on a user operation via the input device 292. More specifically, when the unit data Du is not determined to be normal data by the reference model M1, the playback processing unit 2912 outputs partial data Dp1 of sound data corresponding to the unit data Du not determined to be normal data from the speaker 295, out of the data to be saved Ds stored in the storage device 100B. Furthermore, on the condition that the unit data Du is not determined to be normal data by the erroneous detection model M2, the playback processing unit 2912 outputs partial data Dp1 of sound data corresponding to the unit data Du not determined to be normal data from the speaker 295.

[0145] When the unit data Du is not determined to be normal data by the first determination unit 1131 of the server device 100A, the control unit 291 performs a predetermined notification based on the warning notification from the server device 100A. Typically, the control unit 291 outputs a predetermined warning sound from the speaker 295. In addition, the control unit 291 displays a predetermined warning on the display 258.

[0146] (g3. Labeling) Next, the labeling process performed by the terminal device 200 and the server device 100A will be described in more detail.

[0147] The display control unit 2911 of the terminal device 200 displays, on the display 258, an operation screen (see FIG. 18) for outputting the partial data Dp1 corresponding to the unit data Du that was not determined to be normal data from the speaker 295. This operation screen has objects 231 and 232.

[0148] After the partial data Dp1 corresponding to the unit data Du is output from the speaker 295 by an operation on the operation screen, when the object 231 is selected, the terminal device 200 transmits a notification indicating that the partial data Dp corresponding to the partial data Dp1 is normal data to the server device 100A. In this case, the labeling unit 1122 of the server device 100A assigns a normal state label to the partial data Dp on the condition that the server device 100A receives the notification.

[0149] After the partial data Dp1 corresponding to the unit data Du is output from the speaker 295 by an operation on the operation screen, when the object 232 is selected, the terminal device 200 transmits a notification indicating that the partial data Dp corresponding to the partial data Dp1 is abnormal data to the server device 100A. In this case, the labeling unit 1122 of the server device 100A assigns an abnormal state label to the partial data Dp on the condition that the server device 100A receives the notification.

[0150] According to such a configuration, it becomes possible to easily obtain learning data (labeled partial data Dp) for enabling highly accurate abnormal sound determination.

[0151] <H. Hardware Configuration> FIG. 24 is a diagram showing a typical example of the hardware configuration of the server device 100A. As shown in FIG. 24, the server device 100A mainly includes a processor 151 that executes a program, a ROM 152 that stores data non-volatilely, a RAM 153 that stores data generated by the execution of the program by the processor 151 or data input via an input device volatilely, an HDD 154 that stores data non-volatilely, a communication IF 155, an operation key 156, a power supply circuit 157, and a display 158. Each component is connected to each other by a data bus. Note that the communication IF 155 is an interface for performing communication with other devices.

[0152] The processing in the server device 100A is realized by the various pieces of hardware and software executed by the processor 151. Such software may be pre-stored in the HDD 154. Alternatively, the software may be stored in other storage media and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet. Such software is read from the storage media by a reading device or downloaded via the communication IF 155 or the like, and then temporarily stored in the HDD 154. The software is read from the HDD 154 by the processor 151 and stored in the RAM 153 in the form of an executable program. The processor 151 executes the program.

[0153] The components constituting server device 100A shown in the figure are common components. Therefore, it can be said that the essential part of this disclosure is RAM 153, HDD 154, software stored in a storage medium, or software downloadable via a network. Note that the operation of each piece of hardware in server device 100A is well known, so detailed description will not be repeated.

[0154] Fig. 25 is a diagram showing a typical example of the hardware configuration of terminal device 200. As shown in Fig. 25, terminal device 200 includes, as main components, a processor 251 that executes a program, a ROM 252 that stores data in a non-volatile manner, a RAM 253 that volatilely stores data generated by execution of the program by processor 251 or data input via an input device, an HDD 254 that stores data in a non-volatile manner, a communication IF 255, operation keys 256, a power supply circuit 257, and a display 258. The components are mutually connected by a data bus. Note that communication IF 255 is an interface for communicating with other devices.

[0155] The processing in the terminal device 200 is realized by software executed by each hardware and the processor 251. Such software may be pre-stored in the HDD 254. Also, the software may be stored in other storage media and distributed as a program product. Alternatively, the software may be provided as a program product downloadable by an information provider connected to the so-called Internet. Such software is read from its storage medium by a reading device or downloaded via a communication IF 255 or the like, and then temporarily stored in the HDD 254. The software is read from the HDD 254 by the processor 251 and stored in the RAM 253 in a form of an executable program. The processor 251 executes the program.

[0156] Each component constituting the terminal device 200 shown in the figure is common. Therefore, it can be said that the essential part of the present disclosure is the RAM 253, the HDD 254, the software stored in the storage medium, or the software downloadable via a network. Note that the operations of each hardware of the terminal device 200 are well-known, so detailed descriptions will not be repeated.

[0157] <I. Other Functions> (1) FIG. 26 is a diagram showing a screen for life / death monitoring. As shown in FIG. 26, when the item 2711 for life / death monitoring is selected, the screen for life / death monitoring is displayed on the display 258 of the terminal device 200. When the ping exceeds the threshold, the terminal device 200 notifies of a life / death monitoring abnormality. For example, the terminal device 200 displays the item 2711 in a manner different from the normal state. For example, the terminal device 200 changes the color of the item.

[0158] Also, when a life / death monitoring abnormality is detected while the terminal device 200 is displaying the screen 201 shown in FIG. 10, the tab 211 may be displayed in a manner different from the normal state.

[0159] (2) The number of input dimensions of the false positive model M2 and the abnormal model M3 is lower than the number of input dimensions of the reference model M1.

[0160] (3) The reference model M1, the false positive model M2, and the anomaly model M3 have different algorithms. For example, the algorithms of the false positive model M2 and the anomaly model M3 can be SVM (support vector machine).

[0161] (4) By integrating not only the false positive model M2 but also the abnormality model M3 into the reference model M1, the server device 100A may determine whether or not the equipment 900 is abnormal.

[0162] (5) The information processing system 1 may be configured such that, if the reference model M1 is not updated using false positive data within a predetermined period after a label of a normal state is assigned, the server device 100A causes the terminal device 200 to issue a predetermined notification.

[0163] (6) The information processing system 1 includes the cloud system 100 and the terminal device 200, but the information processing system 1 may be configured so that the functions of the cloud system 100 and the terminal device 200 are executed by a single information processing device.

[0164] (7) In the above description, the time unit (Tu seconds) for division in preprocessing is one second or less, but this is not limiting and may be one second or more.

[0165] [Note] The above-described embodiment of the present disclosure includes the following configuration.

[0166] [Configuration 1] An information processing system for monitoring a state of equipment, a data collection unit that acquires first status data related to the equipment at first time intervals and acquires second status data related to the equipment at second time intervals that are longer than the first time intervals; a processing unit that determines whether the equipment is normal or not based on time-series data of each of the first status data and the second status data, the time series data of the first status data includes first waveform data, The processing unit generates second waveform data by interpolating the time series data of the second state data, generates third waveform data by adding the first waveform data and the second waveform data, and performs the judgment using the third waveform data.

[0167] [Configuration 2] The information processing system of configuration 1, wherein in the addition process, the processing unit adds a value obtained by multiplying the value of the first status data by a first weight to a value obtained by multiplying the value of the second status data corresponding to the first status data by a second weight different from the first weight.

[0168] [Configuration 3] The information processing system according to configuration 1 or 2, wherein the processing unit, in the interpolation process, interpolates data between two pieces of second status data that are adjacent to each other in a time series so that the value of one of the pieces of second status data becomes the amplitude of a waveform between the two pieces of second status data.

[0169] [Configuration 4] The information processing system according to any one of configurations 1 to 3, wherein the processing unit inputs the third waveform data into a machine learning model for the equipment and makes the judgment based on the output of the machine learning model.

[0170] [Configuration 5] the first status data includes at least one of sound, vibration, and acceleration; 5. The information processing system according to any one of configurations 1 to 4, wherein the second status data includes at least one of temperature, humidity, odor, current, voltage, rotation speed, pressure, and flow rate.

[0171] The embodiments disclosed herein are merely examples and are not limited to the above. The scope of the present disclosure is defined by the claims, and it is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0172] 1 Information processing system, 100 Cloud system, 100A Server device, 100B Storage device, 101, 291 Control unit, 111 Data processing unit, 112 Model processing unit, 113 Judgment unit, 151, 251 Processor, 152, 252 ROM, 153, 253 RAM, 156, 256 Operation keys, 157, 257 Power supply circuit, 158, 258 Display, 200 Terminal device, 201 to 204 Screen, 205 Setting screen, 211, 212, 213, 214, 215 Tab, 221 Waveform, 222 Spectrum, 223 Graph, 231, 232, 234 Object, 235 Section, 270 Dashboard, 271 to 276, 2711, 2751, 2761 to 2764 Item, 292 Input device, 295 Speaker, 300 data collection unit, 310 sound collection device, 320 measurement sensor, 900 equipment, 900A, 900B piping equipment, 950 monitor, 1121 model generation unit, 1122 label assignment unit, 1123 model update unit, 1131 first judgment unit, 1132 second judgment unit, 1133 third judgment unit, 2911 display control unit, 2912 playback processing unit, A1 to A4 amplitude, Ba1, Ba2 add button, Bd1, Bd2 delete button, Btn reflect button, Dd judgment data, Dp, Dp1 partial data, Dr water leakage, Ds, Ds1 storage data, Du unit data, 103, 155, 255, 293 communication IF, Lst1, Lst2 list box, M1 reference model, M2 false detection model, M3 abnormality model, NW Network, Nud numerical setting control, P1~P4 plot points, R1 range, Stg toggle button, Wv1,Wv12,Wv31 sound waveform data, Wv2 sensor waveform data, Wv3 mixing data, pa,pb packets, qa,qb queues.

Claims

1. An information processing system for monitoring a state of equipment, a data collection unit that acquires first status data related to the equipment at first time intervals and acquires second status data related to the equipment at second time intervals that are longer than the first time intervals; a processing unit that determines whether the equipment is normal or not based on time-series data of each of the first status data and the second status data, the time series data of the first status data includes first waveform data, The processing unit generates second waveform data by interpolating the time series data of the second state data, generates third waveform data by adding the first waveform data and the second waveform data, and performs the judgment using the third waveform data.

2. 2. The information processing system of claim 1, wherein the processing unit, in the addition process, adds a value obtained by multiplying the value of the first status data by a first weight to a value obtained by multiplying the value of the second status data corresponding to the first status data by a second weight different from the first weight.

3. 2. The information processing system according to claim 1, wherein the processing unit, in the interpolation process, interpolates data between two pieces of second status data that are adjacent to each other in a time series so that the value of one of the pieces of second status data becomes the amplitude of a waveform between the two pieces of second status data.

4. The information processing system according to claim 1 , wherein the processing unit inputs the third waveform data into a machine learning model for the facility and makes the determination based on an output of the machine learning model.

5. the first status data includes at least one of sound, vibration, and acceleration; 5. The information processing system according to claim 1, wherein the second status data includes at least one of temperature, humidity, odor, current, voltage, rotation speed, pressure, and flow rate.

6. An information processing method for monitoring a state of equipment, comprising: acquiring first status data relating to the equipment at every first time interval, and acquiring second status data relating to the equipment at every second time interval that is longer than the first time interval; determining whether the equipment is normal or not based on time-series data of the first status data and the second status data, the time series data of the first status data includes first waveform data, An information processing method in which the step of making the judgment generates second waveform data by interpolating time series data of the second state data, generates third waveform data by adding the first waveform data and the second waveform data, and makes the judgment using the third waveform data.

7. A program that, when executed by a processor, causes the processor to execute the information processing method according to claim 6.

Images