Control System

Abstract

To provide support information for facilitating determination of a state of a control target.SOLUTION: A control system collects images photographing a target in association with imaging time, periodically collects state values of the target, and acquires representative values of the state values collected during the time for each time of a predetermined length. The control system displays, for a representative value selected from among the acquired representative values for each time through a user operation, time waveforms of the state values collected during the time corresponding to the representative value and moving images of the images associated with the imaging time corresponding to the time side by side.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present technology relates to a control system that controls a control target, and more particularly to a control system that outputs information related to control. [Background technology]

[0002] In various production sites where FA (Factory Automation) is applied, there is a need to improve equipment availability by predictive monitoring of machines and equipment as production facilities. Predictive monitoring is a mechanism for detecting conditions (hereinafter referred to as "predictive conditions") that indicate signs of an abnormality in the production equipment being monitored. When a predictive condition is detected by predictive monitoring, users can perform maintenance work such as servicing or replacement before the equipment reaches a state where it needs to be shut down. To achieve this predictive monitoring, a mechanism has been put into practical use that uses data collected from machines and equipment while they are in operation to help users determine whether any abnormalities have occurred in the machines and equipment.

[0003] For example, in JP 2021-60966 A (Patent Document 1), the system collects vibration data from industrial machinery, compares the frequency analysis result value with a threshold value to determine errors (see paragraph 0058), and displays a camera image taken when a device behaving abnormally on the same screen as a view of the device's output waveform (see Figure 19 and paragraph 0093). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-60966 Summary of the Invention [Problem to be solved by the invention]

[0005] Users at production sites have a need to obtain support information that will enable them to more easily determine the status of controlled objects, such as whether any abnormalities have occurred in the machines or equipment, using data collected from the machines and equipment while they are in operation.

[0006] One object of the present disclosure is to provide a user with support information that allows the user to more easily determine the state of a controlled object. [Means for solving the problem]

[0007] The control system for periodically controlling an object according to the present disclosure includes an imaging unit having an imaging range capable of imaging the object as a subject, an image collection unit that collects images captured by the imaging unit in association with the imaging time, a data collection unit that periodically collects state values ​​of the object, an acquisition unit that acquires a representative value of the state values ​​collected at each predetermined time length from the time series state values ​​periodically collected, a selection operation receiving unit that receives a user operation to select at least one representative value from the representative values ​​acquired by the acquisition unit for each predetermined time length, and a waveform display unit that displays, for each of the at least one selected representative value, a time waveform of the state values ​​periodically collected at the predetermined time length corresponding to the representative value, and a moving image played back based on an image associated with the imaging time corresponding to the time.

[0008] According to this disclosure, when one or more representative values ​​are selected by a user, the time waveform of the representative value corresponding to the selected representative value and the moving image that is played back can be presented to the user as information to assist in determining the state, such as the behavior of the controlled object during the cycle time of the selected representative value.

[0009] In the above disclosure, the control system further includes a representative value display unit that displays a time waveform of the representative value for each predetermined length of time acquired by the acquisition unit.

[0010] According to the above disclosure, by displaying the time waveform of the representative value, information indicating the state, such as the behavior of the controlled object, can be presented to the user as information to assist in determining the state, such as the behavior of the controlled object.

[0011] The selection operation receiving unit receives a user operation to select at least one representative value from the time waveform of the representative values.

[0012] According to the above disclosure, it is possible to provide the user with the time waveform of the representative value as information to assist the user in selecting the representative value.

[0013] In the above disclosure, the control system includes a monitoring unit that monitors the state of the target at predetermined intervals based on a representative value and a threshold value acquired at that interval, and the representative value display unit changes the display mode of each representative value in the time waveform of the representative value depending on the results of monitoring based on the representative value.

[0014] According to the above disclosure, the monitoring results indicated by the display modes of the representative values ​​in the time waveforms of the representative values ​​can be provided to the user as information to assist in the selection of the representative value.

[0015] In the above disclosure, the control system accepts a user operation that specifies a partial waveform in a time waveform, and the moving image includes a moving image that is played back based on an image associated with an imaging time that corresponds to the time of the partial waveform specified by the accepted user operation.

[0016] According to the above disclosure, it is possible to reproduce and provide only a partial image at a time corresponding to a partial waveform designated by the user as information to assist in determining the state, such as the behavior, of a controlled object.

[0017] The above-mentioned control system further includes a monitoring unit that monitors the state of the object every predetermined length of time based on a representative value and a threshold value acquired at that time, and a setting unit that sets the threshold value from the state values ​​collected periodically over a predetermined length of time specified by the user, and the waveform display unit is further configured to display, side by side, the time waveform of the state values ​​collected periodically over a predetermined length of time corresponding to the representative value for each of the representative value of the state values ​​used to set the threshold and at least one selected representative value, and a moving image played back based on an image associated with the imaging time corresponding to that time.

[0018] According to the above disclosure, the time waveform of the representative value corresponding to the representative value and the moving image to be played back can include the time waveform of the state value collected periodically at a time specified by the user to set a threshold, and an image associated with the image capture time corresponding to that time. This makes it possible to provide the user with the time waveform and image used when setting the threshold for state monitoring as information to assist in determining the state, such as the behavior of the controlled object. [Effects of the Invention]

[0019] According to the present disclosure, it is possible to provide the user with support information that allows the user to more easily determine the state of the control target. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a diagram schematically illustrating the overall configuration of a control system 1a according to the present embodiment. [Figure 2] FIG. 1 is a diagram schematically illustrating the overall configuration of a control system 1b according to the present embodiment. [Figure 3] 1 is a block diagram showing an example of a hardware configuration of a PLC 100 according to the present embodiment. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of a server 200 according to the present embodiment. [Figure 5]FIG. 5 is a diagram schematically illustrating a DB manager provided by executing the DB management program 211 of FIG. 4. [Figure 6] FIG. 2 is a block diagram showing an example of a hardware configuration of a terminal 300 according to the present embodiment. [Figure 7] FIG. 1 is a diagram schematically illustrating an example of the configuration of an imaging device 40 according to the present embodiment. [Figure 8] FIG. 2 is a diagram showing an example of a module configuration of a UI tool 340 according to the present embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of a processing sequence according to the present embodiment. [Figure 10] FIG. 2 is a diagram schematically illustrating an example of a time-series DB 250 according to the present embodiment. [Figure 11] 10 is a flowchart illustrating an example of processing in a threshold setting mode according to the present embodiment. [Figure 12] FIG. 10 is a diagram showing an example of a display screen for information to assist a user in a threshold setting mode according to the present embodiment. [Figure 13] FIG. 10 is a diagram showing an example of a display screen for information to assist a user in a threshold setting mode according to the present embodiment. [Figure 14] FIG. 10 is a diagram showing an example of a display screen for information to assist a user in a threshold setting mode according to the present embodiment. [Figure 15] FIG. 10 is a diagram showing an example of a display screen for information to assist a user in a threshold setting mode according to the present embodiment. [Figure 16] FIG. 10 is a diagram showing an example of a display screen for information to assist a user in a threshold setting mode according to the present embodiment. [Figure 17] 10 is a flowchart illustrating an example of processing in an operation mode according to the present embodiment. [Figure 18] FIG. 10 is a diagram showing an example of a display screen for information to assist user operations in an operation mode according to the present embodiment. [Figure 19]This is a diagram showing an example of a display screen of information for assisting user operations in the operation mode according to the present embodiment. [Figure 20] This is a diagram showing an example of a display screen of information for assisting user operations in the operation mode according to the present embodiment. [Figure 21] This is a diagram showing an example of a display screen of information for assisting user operations in the operation mode according to the present embodiment.

Embodiments for Carrying Out the Invention

[0021] Embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and their description will not be repeated.

[0022] <A. Application Example> First, an example of a scene to which the present invention is applied will be described.

[0023] <0​​​​​​​​The PLC 100 is connected to a field device 10. The field device 10 includes any device necessary to control a controlled object. More specifically, the field device 10 includes a device for exchanging information with the controlled object (e.g., manufacturing equipment, manufacturing devices, sensors and actuators included in the manufacturing equipment or manufacturing devices, etc.). In the example shown in FIG. 1, the field device 10 includes a group of relays 14 and a servo motor 18. The field device 10 may further include any device, such as a servo driver 16 and a remote I / O (Input / Output) 12 shown in FIG. 1.

[0026] The PLC 100 acquires data including the status values ​​of the field devices 10, and executes control calculations using the acquired data in accordance with a user program created in advance, thereby generating data including command values ​​to be given to the field devices 10. In the present disclosure, the "status values" include input values ​​collected from the field devices 10 indicating the status of the field devices 10, command values ​​output to the field devices 10, and system status values ​​and internal values ​​managed within the PLC 100.

[0027] In the example shown in FIG. 1, the PLC 100 and the field device 10 are connected via a control network 4, but the present invention is not limited to this and they may be connected by hard wires.

[0028] It is preferable to adopt an industrial communication protocol for the control network 4. Known examples of such communication protocols include EtherCAT (registered trademark), EtherNet / IP (registered trademark), DeviceNet (registered trademark), and CompoNet (registered trademark).

[0029] The PLC system 2 is capable of accessing a cloud-based server 200 on the Internet 8 via an information network 6 and a relay device 20 such as a gateway or a router. The information network 6 may use, for example, the EtherNet / IP (registered trademark) protocol.

[0030] The server 200 communicates with the PLC 100 and the imaging device 40 via the information network 6 and the Internet 8. The server 200 mainly manages information relating to the PLC system 2 and provides various services.

[0031] The server 200 may be placed on a cloud via the Internet 8, or may be placed in a local network where the PLC system 2 exists without going through the Internet 8. The implementation form of the server 200 can be determined arbitrarily depending on the required performance and functions. The server 200 may be configured to include a NAS (Network Attached Storage).

[0032] The terminal 300 is configured, for example, by a general-purpose computer. The terminal 300 can access the server 200 and perform user operations to perform various processes, as will be described later. In this embodiment, this user operation may be realized by a later-described application 312 and UI program 313 of the terminal 300 working together to provide a GUI (Graphical User Interface) for operating the desktop environment of the server 200 connected from the terminal 300 via the Internet 8. The terminal 300 may also exchange information with the PLC 100 via USB (Universal Serial Bus) communication, without using the information network 6 or the Internet 8. If the PLC 100 also has the configuration of a general-purpose computer, the PLC 100 may provide services such as a UI (User Interface) and GUI provided by the terminal 300.

[0033] The PLC 100 controls a control object by calculating a command value through control calculation and outputting the calculated command value to the control object. The PLC 100 periodically controls the control object in accordance with a predetermined control cycle. The PLC 100 includes a data collection unit 118a that collects state values ​​in association with collection times for each control cycle.

[0034] The PLC 100 also includes a monitoring processor 119 that performs monitoring processing to monitor the state of a control target based on a representative value and a threshold value acquired at each predetermined time interval (a cycle time, described below). The monitoring processor 119 includes a representative value acquisition unit 119a, a threshold monitoring unit 119b, and a threshold setting unit 119c. The representative value acquisition unit 119a acquires a representative value of time-series state values ​​periodically collected from the control target at each predetermined time interval by, for example, calculating the representative value of the state values ​​collected at that time interval. The threshold monitoring unit 119b compares the representative value acquired by the representative value acquisition unit 119a with a threshold value stored in a threshold file 400. The threshold setting unit 119c sets a threshold value in the threshold file 400 or changes the threshold value stored in the threshold file 400. The monitoring processor 119 acquires a monitoring result based on the comparison result output by the threshold monitoring unit 119b. The monitoring result indicates whether the condition (representative value>threshold value) or (representative value≦threshold value) is met based on the comparison result.

[0035] "Representative value" is a term that encompasses information contained in the time series data of the object to be processed, and typically may include feature quantities such as maximum value, minimum value, median value, mean value, standard deviation, and variance of the time series state values ​​collected from the object to be controlled.

[0036] "Cycle time" refers to a unit time interval of a predetermined length for determining whether or not some abnormality has occurred in the monitored object in a time series of state values. In other words, the predictive monitoring process for determining whether or not some abnormality has occurred in the monitored object is performed every "cycle time." In this embodiment, the length of the cycle time may be determined based on the type of state value arbitrarily set by the user. In this embodiment, for example, the cycle time indicates the time it takes from the start to the completion of one process for one work (a finished product or part). The start of the cycle time can be defined as the start time of collecting state values.

[0037] The PLC 100 transmits an imaging start command 3 to the imaging device 40 (step S1). Upon receiving the imaging start command 3 from the PLC 100, the imaging device 40 starts imaging in accordance with the received imaging start command 3 (step S5a). The image collection unit 46 starts imaging and collects images captured after the start of imaging, correlating them with the imaging time. The imaging device 40 transfers the collected images to the server 200 (step S5b). The PLC 100 transfers to the server 200 the status values ​​periodically collected by the data collection unit 118a in step S3, representative values ​​for each cycle time acquired from the status values, and monitoring data indicating the results of the monitoring process performed by the monitoring processing unit 119 in step S4 (steps S6, S6a, and S6b). The server 200 stores the images transmitted from the imaging device 40, the status values, representative values, and monitoring data transmitted from the PLC 100 (step S14).

[0038] The UI tool 340 of the terminal 300 provides a selection operation receiving unit that receives a user operation to select at least one representative value from the representative values ​​for each cycle time, and a waveform display unit that displays, for each selected representative value, a time waveform of the state values ​​periodically collected at the cycle time corresponding to the selected representative value, and a moving image played back based on captured images associated with the capture time corresponding to the cycle time. The UI tool 340 may be configured to have a web browser that displays on a display a screen based on screen information such as a web page received from the web server, with the server 200 serving as a web server.

[0039] Therefore, when one or more representative values ​​are selected by the user, the control system 1a can present to the user the time waveform and reproduced image (including moving image) of the representative value corresponding to the selected representative value as information representing the state, such as the behavior of the controlled object, during the cycle time of the selected representative value.

[0040] In this embodiment, the threshold value included in the threshold file 400 is changeable. More specifically, the UI tool 340 has a threshold operation tool 311 that accepts a user operation for setting or changing the threshold value. The threshold operation tool 311 inputs a threshold value based on the accepted user operation and transfers the input threshold value to the server 200. The server 200 performs a prognosis monitoring setting process (step S17). In the prognosis monitoring setting process, the server 200 transfers the threshold value received from the threshold operation tool 311 to the PLC 100 (step S7). The threshold setting unit 119c of the PLC 100 sets the threshold value received from the server 200 in the threshold file 400 by overwriting or the like. The threshold setting unit 119c may update or set the threshold value of the threshold file 400 based on the tendency of the change in the value indicated by at least one of the time waveform of the state value and the time waveform of the representative value. Or, the threshold setting unit 119c may update or set the threshold value of the threshold file 400 based on the comparison result output by the threshold monitoring unit 119b or the monitoring result output by the monitoring processing unit 119.

[0041] The user can set or change the threshold value used for prognosis monitoring by the threshold setting unit 119c of the monitoring processing unit 119 by operating the threshold operation tool 311. When one or more representative values are selected by the user, the control system 1a can present to the user the time waveform of the state value and the reproduction image (including a moving image) for each cycle time corresponding to the representative value as information for supporting the user in setting or changing the threshold value.

[0042] Hereinafter, more specific application examples of the present invention will be described. <B. System Configuration> FIG. 2 is a diagram schematically showing the overall configuration of the control system 1b according to the present embodiment. The control system 1b in FIG. 2 includes a plurality of the PLC systems 2 and imaging devices 40 shown in FIG. 1 respectively. The control system 1b in FIG. 2 includes PLC systems 2-1, 2-2, and 2-3 (hereinafter also collectively referred to as "PLC system 2") provided in different manufacturing processes, and imaging devices 40-1 and 40-2 provided in these manufacturing processes. The control system 1b realizes distributed control for a series of processes by, for example, providing the PLC system 2-1 in the parts process, the PLC system 2-2 in the assembly process, and the PLC system 2-3 in the inspection process. Since other configurations of the control system 1b are the same as those of the control system 1a in FIG. 1, the description will not be repeated. The imaging devices 40-1 and 40-2 capture images with different orientations (imaging directions) of the camera for the same control target (subject) by, for example, varying the angles of the cameras.

[0043] In the following description, the PLC systems 2-1, 2-2, and 2-3 have the same configuration as the PLC system 2 shown in FIG. 1. Also, the imaging devices 40-1 and 40-2 have the same configuration as the imaging device 40 in FIG. 1. Here, three PLC systems 2 and two imaging devices 40 are shown, but the number of the PLC systems 2 and the imaging devices 40 is not limited to these. In the following, in the description of the functions or configurations common to the PLC systems 2-! 2-2, and 2-3, they are collectively referred to as "PLC system 2", and in the description of the functions or configurations common to the imaging devices 40-1 and 40-2, they are collectively referred to as "imaging device 40". Note that in the present embodiment, the imaging device 40 may be connected to the control network 4. The PLC 100 receives an image from the imaging device 40 via the control network 4 and transfers the received image to the server 200. In this case, the image collection unit 46 is provided in the imaging device 40 or the PLC 100.

[0044] <C. Configuration of Each Device> Next, an example of the configuration of the devices constituting the control system according to the present embodiment will be described.

[0045] (c1:PLC100) Fig. 3 is a block diagram showing an example of a hardware configuration of a PLC 100 according to this embodiment. Referring to Fig. 4, the PLC 100 includes a processor 102 such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), a chipset 104, a main memory 106, a storage 109, an information network controller 127 connected to a connector 121 configured as a physical circuit, a control network controller 122 connected to a connector 123 configured as a physical circuit, a USB (Universal Serial Bus) controller 124, a timer 125 configured including a counter circuit, and a memory card interface 126. The connector 121 connects to an information network 6 to which a server 200, the PLC 100, and an imaging device 40 belong, and the connector 123 connects to a control network 4, which is a network different from the information network 6 and to which a field device 10 to be controlled belongs.

[0046] The processor 102 reads out various programs stored in the storage 109, expands them in the main memory 106, and executes them to perform control calculations for controlling the control target and communication processing for communicating with external devices (the server 200, the terminal 300, the imaging device 40, and the field device 10). The chipset 104 controls data transmission between the processor 102 and each component.

[0047] The storage 109 stores an OS (Operating System) 111, a system program 113, a user program 114, and a threshold file 400. The system program 113 includes communication firmware for communicating with external devices including the server 200 via the information network 6, and communication firmware for communicating with the field devices 10 via the control network 4.

[0048] The user program 114 includes a control program 115, a communication program 116, an IO (Input / Output) refresh 117, a collection program 118, and a camera communication program 120. The IO refresh 117 collects status values ​​of the field device 10, which is the control target, via the control network 4 and stores them in the IO refresh area 112. The IO refresh 117 also transmits command values ​​stored in the IO refresh area 112 to the control target via the control network 4. The control program 115 performs control calculations for the control target using the status values ​​stored in the IO refresh area 112 to calculate the above-mentioned command values. When executed, the collection program 118 realizes the module of the data collection unit 118a described above. When executed, the camera communication program 120 communicates with the image capture device 40. The storage 109 also stores a monitoring program 110, which when executed realizes the monitoring processing unit 119. The monitoring program 110 includes a representative value acquisition program 110a, a threshold monitoring program 110b, and a threshold setting program 110c, which, when executed, realize a representative value acquisition unit 119a, a threshold monitoring unit 119b, and a threshold setting unit 119c. The user program 114 may include various other processing programs, including a time synchronization program.

[0049] The user program 114 according to this embodiment is a so-called variable program. More specifically, the user program 114 is configured so that the state value data and internal calculation data referenced by the program during execution can be used by using input variables, output variables, temporary variables, etc. for each data type.

[0050] The PLC 100 of each PLC system 2 performs time synchronization processing according to a time synchronization program to synchronize the time measured by the timer of the other PLCs 100 and the imaging devices 40 connected to the information network 6 with the time measured by a master timer (not shown). Also, according to the time synchronization program, the PLC 100 of each PLC system 2 performs time synchronization processing to synchronize the time measured by the timer of the field device 10 connected to the control network 4 to which the PLC 100 is connected. The master timer is provided on the information network 6, for example, but is not limited to this.

[0051] The information network controller 127 controls data exchange with external devices via the information network 6 connected to the connector 121 .

[0052] The control network controller 122 controls data exchange with the field device 10 via the control network 4 connected to the connector 123 .

[0053] The USB controller 124 controls the exchange of data with an external device (eg, a support device, the terminal 300, etc.) via a USB connection.

[0054] The memory card interface 126 is configured so that a memory card 128 can be attached and detached, and it is possible to write data to the memory card 128 and read various data (such as the user program 114 and data) from the memory card 128.

[0055] The PLC 100 may include an optical drive. The optical drive reads a computer-readable program from a recording medium that non-transiently stores the program (for example, an optical recording medium such as a DVD (Digital Versatile Disc)) and stores the program in the storage 109 or the like.

[0056] The various programs executed by the PLC 100 may be installed via a computer-readable recording medium or memory card 128, or may be installed by downloading from any device such as a computer on a network.

[0057] FIG. 3 shows an example of a configuration in which the processor 102 executes a program to provide the necessary processing, but some or all of the provided processing may be implemented using dedicated hardware circuits (e.g., an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array)). Alternatively, the main part of the PLC 100 may be realized using hardware that conforms to a general-purpose architecture (e.g., an industrial PC based on a general-purpose PC). In this case, virtualization technology may be used to run multiple operating systems with different purposes in parallel, and necessary applications may be executed on each operating system.

[0058] (c2: Server 200) Fig. 4 is a block diagram showing an example of the hardware configuration of server 200 according to this embodiment. Referring to Fig. 4, server 200 includes a processor 202, a main memory 204, an input unit 206, an output unit 208, a storage 210, an optical drive 215, a USB controller 220 for communicating with external devices, and a network controller 413 for connecting to a network including the Internet 8. These components are connected via a processor bus 218.

[0059] The processor 202 is composed of a CPU, a GPU, etc., and realizes various processes as described below by reading out a program stored in the storage 210, expanding it in the main memory 204, and executing it.

[0060] The main memory 204 is configured from a volatile storage device such as a DRAM or SRAM, etc. The storage 210 is configured from a non-volatile storage device such as an HDD or SSD, for example.

[0061] The storage 210 stores an OS 212 for realizing basic functions, as well as various programs for providing the functions of the server 200. The various programs include a DB (database) management program 211, a threshold management program 213 for managing thresholds, a setting mode program 216 for setting thresholds, and an operation mode program 217 for operating an early warning monitoring process based on the set thresholds. The storage 210 also has an area for storing a time-series DB 250. In the setting mode, the threshold management program 213 and the setting mode program 216 are executed to perform an early warning monitoring setting process (step S17 in FIG. 1 ).

[0062] The input unit 206 is configured with a keyboard, a mouse, etc., and accepts user operations on the server 200. The output unit 208 is configured with a display, various indicators, a printer, etc., and outputs processing results from the processor 202, etc.

[0063] The server 200 has an optical drive 215, and a program stored in a recording medium 214 (e.g., an optical recording medium such as a DVD (Digital Versatile Disc)) that non-transiently stores a computer-readable program is read and installed in storage 210, etc.

[0064] FIG. 4 shows an example of a configuration in which the processor 202 executes a program to provide the functions required by the server 200, but some or all of these provided functions may be implemented using dedicated hardware circuits (e.g., ASIC or FPGA).

[0065] Fig. 5 is a diagram schematically illustrating a DB manager provided by executing the DB management program 211 in Fig. 4. Referring to Fig. 5, the DB manager 253 provided by executing the DB management program 211 by the processor 202 manages a time-series DB 250 (e.g., generating, aggregating, editing, analyzing, outputting, etc. a DB) in accordance with database operation commands conforming to, for example, SQL (Structured Query Language). Details of the time-series DB 250 will be described later.

[0066] (c3: Terminal 300) Fig. 6 is a block diagram showing an example of the hardware configuration of a terminal 300 according to this embodiment. Referring to Fig. 6, the terminal 300 includes a processor 302 such as a CPU or an MPU, an optical drive 304, a main memory 306, a storage 310, a network controller 320, a USB controller 324, an input unit 326, and a display unit 328. These components are connected via a bus 308.

[0067] The processor 302 reads out various programs stored in the storage 310, expands them in the main memory 306, and executes them to realize the processing required by the terminal 300.

[0068] The storage 310 is configured with, for example, an HDD or SSD. The storage 310 stores an OS 319, an application 312, and a UI (User Interface) program 313. The UI program 313 provides a UI tool 340 by executing in cooperation with the application 312. The UI program 313 includes a selection program 318, a waveform conversion program 314, an image playback program 315, a threshold operation program 316, an image adjustment program 317, and the selection program 318. When executed, the selection program 318 selects at least one representative value in accordance with a user operation from among the representative values ​​for each cycle time acquired by the representative value acquisition unit. The waveform conversion program 314 converts the time-series representative value collected at the cycle time corresponding to the selected representative value into a time waveform and displays the converted time waveform on a display. The image playback program 315 plays back images collected by the image acquisition unit 46 at the cycle time corresponding to the selected representative value as a moving image (video) on a display according to the imaging time associated with the image. When executed, the threshold manipulation program 316 realizes the threshold manipulation tool 311. When executed, the image adjustment program 317 adjusts the playback of the image displayed by the image playback program 315 in accordance with user operations. The playback adjustments performed by the image adjustment program 317 include adjustments of the image playback direction, such as frame-by-frame, fast-forward, and rewind, and the playback speed. Note that the storage 310 may store necessary programs other than the programs shown in FIG. 5.

[0069] The network controller 320 controls the exchange of data with devices such as the PLCs 100 and the server 200 via the information network 6 .

[0070] The USB controller 324 controls the transfer of data to and from external devices via a USB connection.

[0071] The input unit 326 is configured with a mouse, keyboard, touch panel, etc., and receives instructions from the user. The display unit 328 is configured with a display, various indicators, etc., and outputs processing results from the processor 302. The input unit 326 and the display unit 328 may form a touch screen integrated with the display.

[0072] The terminal 300 may have an optical drive 304. The optical drive 304 reads a computer-readable program from a recording medium 305 (for example, an optical recording medium such as a DVD (Digital Versatile Disc)) that non-transiently stores the program, and stores the program in the storage 310 or the like.

[0073] The various programs executed on the terminal 300 may be installed via a computer-readable recording medium 305, or may be installed by downloading from an arbitrary server on the network.

[0074] FIG. 6 shows an example of a configuration in which the necessary processing is provided by the processor 302 executing a program, but some or all of the provided processing may be implemented using dedicated hardware circuits (e.g., ASIC or FPGA).

[0075] (c4: Imaging device) Fig. 7 is a diagram schematically illustrating an example of the configuration of an imaging device 40 according to the present embodiment. Referring to Fig. 7, imaging device 40 includes, as basic components, an optical system 41 including a lens, an imaging element 42, a DSP (Digital Signal Processor) 43 constituting a processor circuit having an image processing engine 44 and an image memory 47, and a communication I / F (Interface) 48 including a communication circuit for connecting to information network 6. Imaging device 40 integrally includes a processing system of DSP 43 and communication I / F 48 and an imaging system of optical system 41 and imaging element 42, but the processing system may be configured separately from the imaging system.

[0076] Light reflected from a subject within the imaging range passes through the lens of the optical system 41 and enters a photoelectric conversion element, such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor, of the imaging element 42, where it is converted into an image signal by the image sensor. The image processing engine 44 inputs the image signal from the imaging element 42, processes the image signal based on the time output by the timer 45, and converts it into a digital data image in units of frame images. For example, the image processing engine 44 converts the image into N (N>1) frame images per second. The image processing engine 44 stores the frame images in image memory 47 as still images or moving images in association with the imaging time measured by the timer 45. The image memory 47 cooperates with the image processing engine 44 to form an image collection unit 46 that collects images in association with the imaging time measured by the timer 45.

[0077] The communication I / F 48 receives the imaging start command 3 transferred from the PLC 100 via the information network 6, and outputs the received imaging start command 3 to the DSP 43. In response to the imaging start command 3, the DSP 43 starts image processing of the image signal output from the imaging system.

[0078] For example, in this embodiment, imaging device 40 starts imaging in response to imaging start command 3, and continues imaging for a predetermined period after imaging starts. The length of this period can be set. Frame images captured after imaging starts are stored in image memory 47 as, for example, still image files or video files.

[0079] (c5: UI tool configuration) FIG. 8 is a diagram showing an example of the module configuration of the UI tool 340 according to the present embodiment. The UI tool 340 includes a waveform display module 341 that constitutes a "waveform display section", an image reproduction module 342, a threshold operation module 343 that constitutes a "threshold setting section" for setting a threshold, an image adjustment module 344, and a selection module 345, which are realized by the processor 302 executing a waveform conversion program 314, an image reproduction program 315, a threshold operation program 316, an image adjustment program 317, and a selection program 318.

[0080] <D. Outline of the collection processing sequence> FIG. 9 is a diagram schematically showing an example of the sequence of the processing according to the present embodiment. The sequence in FIG. 9 includes the communication between devices including the PLC 100, the imaging device 40, and the server 200, and the sequence of the processing performed by each device.

[0081] The PLC 100 transmits an imaging start command 3 to the imaging device 40 (step S1). The imaging start command 3 is transmitted, for example, when starting the omen monitoring process of the control system 1.

[0082] When the imaging device 40 receives the imaging start command 3 from the PLC 100 (step S1a), it starts imaging according to the received imaging start command 3 (step S5a). The image processing engine 44 performs image processing on the image obtained by imaging.

[0083] The PLC 100 periodically collects state values as the data collection unit 118a (step S3) and performs a monitoring process as the monitoring processing unit 119 (step S4). In the monitoring process, the PLC 100 obtains a representative value indicating the representative value for each cycle as the representative value acquisition unit 119a, and compares the representative value with the threshold for each cycle as the threshold monitoring unit 119b (steps S4a, step S4b). The PLC 100 obtains the monitoring result as the monitoring processing unit 119.

[0084] The imaging device 40 transmits the captured images, which have been collected in the image memory 47 and associated with the image capture times through image processing, to the server 200 (step S5b). For example, the captured images may be transferred to the server 200 on a file-by-file basis. The PLC 100 also transmits to the server 200 control data associated with the collection times, representative values ​​acquired for each cycle time, and monitoring results for each cycle time (step S6). The PLC 100 determines whether to end the processing (step S70). For example, when the user inputs an instruction to stop the symptom monitoring processing to the control system 1, the PLC 100 determines to end the processing based on the instruction (YES in step S70). If it is determined not to end the processing (NO in step S70), the process returns to step S3.

[0085] The server 200 receives data transferred from the PLC 100 and images transferred from the imaging device 40 (steps S11 and S12). The server 200, as the DB manager 253, performs a process of storing the data received from the PLC 100 and the images received from the imaging device 40 in the time-series DB 250, associating the collection time with the image capture time (step S14). The time-series DB 250 stores periodically collected status values, representative values ​​for each cycle time, monitoring results for each cycle time, and image data associated with the image capture time as time-series data. If the server 200 determines to end the process based on the above instruction (YES in step S50), the server 200 ends the process. If the server 200 determines not to end the process (NO in step S50), the server 200 returns to step S11. The imaging device 40 may also determine whether to end the process based on the above instruction.

[0086] In this embodiment, the time synchronization process described above causes the times measured by the timers of the PLC 100 and the imaging device 40 to match, so that the time series DB 250 can manage the periodically collected status values, the representative values ​​for each cycle time, the monitoring results for each cycle time, and the images captured by the imaging device 40 in chronological order, eliminating any time lag between them.

[0087] <Example of Time-Series Database> In this embodiment, in the time-series database 250, data is managed based on, for example, cycle time.

[0088] FIG. 10 is a diagram schematically showing an example of the time-series database 250 according to this embodiment. Referring to FIG. 10, the time-series database 250 includes a time-series database 251, a time-series database 252, and reference data 700 showing collected data of cycle time referred to when setting a threshold value. In the time-series database 251, data is stored in a setting mode in which a threshold value is set in the threshold value file 400 by a prognostic monitoring setting process (step S17). In the time-series database 252, data is stored in an operation mode in which a prognostic monitoring process is performed based on the threshold value set in the threshold value file 400. Since the time-series database 251 and the time-series database 252 have the same configuration, the time-series database 252 will be described and the details of the time-series database 251 will not be repeated.

[0089] In the operation mode, the processor 202 of the server 200 stores, as a DB manager 251, data transferred from the PLC 100 and captured images transferred from the imaging device 40 in the time-series database 252. More specifically, the processor 202 stores, for each cycle time, the data collected at the cycle time and the captured images in file units collected in the image memory 47 as collection control data 430, monitoring data 600, and collected image data 500.

[0090] The collection control data 430 includes an ID 410 for identifying the cycle time, a collection start time 411 at which the state values of one or more variables to be collected are started to be collected, and time-series data 420. The time-series data 420 includes time-series state values 421, 422,... for each variable. The time-series state values 421, 422,... each include a collection time associated with the state value of the variable. The PLC 100 collects the state values at a cycle synchronized with the control cycle, and thus the collection time indicates a time based on the cycle synchronized with the control cycle.

[0091] The collected image data 500 includes imaging timing 510 indicating the time when the imaging start command 3 was received, and time-series image data 550 indicating the imaged images. The image data 550 includes data 515 for each imaging device 40 provided in the control system. The data 515 includes an identifier 520 of the corresponding imaging device 40, a collection start time 525, and a time-series frame 530 representing the time-series frame images. The collection start time 525 indicates the time when the imaging device 40 started imaging in response to the imaging start command 3 and started collecting the captured images. The time-series frame 530 indicates a set of time-series frame images obtained by imaging after the collection start time 525. Each frame image is associated with an imaging time.

[0092] The monitoring data 600 includes an ID 610 for identifying the cycle time, and for each of one or more variables to be collected, data 61 acquired at the cycle time. In FIG. 10, for the monitoring data 600 of a certain variable, it is shown for one cycle time, but actually, the monitoring data 600 is stored for a plurality of cycle times. The data 61 includes, for each variable, a representative value 61a obtained from the state value among the time-series data 420 corresponding to the variable, and a monitoring result 61b based on the representative value 61a. The "T" and "F" of the monitoring result 61b indicate "normal" and "abnormal" to be described later, respectively.

[0093] The reference data 700 includes an identifier ID 710 of the cycle time, time-series data 711, and image data 722 referred to when setting the threshold value. The time-series data 711 includes the time-series state values, representative values, and monitoring results collected at the cycle time. The image data 722 is an image constituting the video of the cycle time and includes a plurality of frame images associated with the imaging time.

[0094] <F. Setting Mode> The threshold setting mode according to this embodiment will be described. FIG. 11 is a flowchart showing an example of processing in the threshold setting mode according to this embodiment. FIGS. 12 to 16 are diagrams showing examples of display screens for information to support user operations in the threshold setting mode according to this embodiment. In the setting mode, the threshold operation tool 311 of the UI tool 340 of the terminal 300 transmits to the server 200, in accordance with user operations, a command to start the setting mode program 216 that searches the time-series DB 251. The server 200 starts the setting mode program 216 in response to the command.

[0095] When the setting mode program 216 is started, it executes an early warning monitoring setting process (step S17). In the early warning monitoring setting process, the server 200 causes the threshold operation tool 311 of the terminal 300 to display the UI screens of Figures 12 to 16 based on screen information (for example, a Web page) on the display of the display unit 328, determines thresholds in accordance with user operations received via the UI screens, and transfers the thresholds determined in accordance with the user operations to the PLC 100 to set them in the threshold file 400.

[0096] 11, terminal 300 displays the screen of FIG. 12 to accept a designation operation of a representative value of a variable (state value) for which a threshold is to be set (step T1). On the screen of FIG. 12, in accordance with a user operation, "welding current" indicated by a bold frame among the state values ​​of the control target is designated, and the type of representative value (e.g., "average value") is also designated. The designated information is transmitted to server 200.

[0097] The server 200 searches the time-series DB 251 for time-series data 612 indicating the "average value" corresponding to the variable "welding current" based on the information received from the terminal 300. The terminal 300 transmits screen information based on the searched time-series data 612 to the terminal 300.

[0098] Based on the screen information from the server 200, the terminal 300 displays, in time series, representative values ​​for each cycle time of the state values ​​indicated by the variables designated by the user (step T3). For example, in region E2 of the screen shown in FIG. 13A, a time waveform graph is displayed in which representative values ​​61a of the state values ​​of the variables designated by the user are plotted in time series on a graph with time on the horizontal axis and value on the vertical axis. In this graph, each plotted representative value 61a is displayed in a different display manner, such as the shape and color of the plotted points, based on the corresponding monitoring result 61b. In region E1 of the screen, thresholds 49a and 49b applied to the monitoring result 61b shown in region E2 are displayed numerically, and in region E2, the thresholds are displayed as bars parallel to the time axis on the graph. By operating button 131 on the screen, the user can switch the graph in region E2 to a histogram shown in FIG. 13B. Before thresholds are set, the displayed thresholds 49a and 49b include, for example, default values.

[0099] The graph of the time waveform of the representative value shown in (A) of FIG. 13 provides the user with information to assist him / her in understanding the trend of change in the representative value over time, and the histogram provides the user with information to assist him / her in understanding the number of representative values ​​for which the monitoring result 61b indicates "normal" and the number for which it indicates "abnormal," as well as the distribution of "normal" or "abnormal" values ​​for each value class.

[0100] The screen in FIG. 14 accepts a user operation to adjust the number of displayed feature quantities. The user can narrow down the total number of representative values ​​61a displayed in FIG. 13 (A) or (B) in FIG. 14 based on the specified attribute by specifying an attribute 14b of the representative value 61a. Attributes include, for example, corresponding monitoring results 61b or whether the value is close to a threshold value. Alternatively, the user can narrow down the total number of displayed representative values ​​61a by specifying a percentage 14A of the number of displayed values ​​or by specifying a period 14C, which is the length of the period during which the cycle time corresponding to the displayed feature quantity is extracted.

[0101] When the user selects one or more representative values ​​(points) plotted in area E2 of FIG. 13A, the terminal 300 receives the selection operation via the selection module 345 (step T5). The terminal 300 transmits information about the selection operation to the server 200. The information about the selection operation includes information about the time corresponding to the plot position on the time waveform of the selected representative value (hereinafter referred to as the plot time). Based on the plot time indicated by the information about the selection operation, the server 200 determines the cycle time to which the plot time corresponds, extracts state values ​​for the determined cycle time from the time-series data 420, and extracts multiple frame images for the cycle time from the image data 550. The server 200 transmits image information including the state values ​​extracted for the cycle time and multiple frame images to the terminal 300.

[0102] The terminal 300 displays the screen of Fig. 15 based on the image information received from the server 200. More specifically, the terminal 300 displays the state values ​​of the cycle time received from the server 200 as a time waveform image in area E3 of the screen of Fig. 15 using the waveform display module 341, and also plays back the frame images captured at the cycle time received from the server 200 as a moving image in area E4 of the screen of Fig. 15 using the image playback module 342 (steps T7 and T9).

[0103] In step T5, the user selects, for example, a representative value of one cycle plotted as "abnormal" and a representative value of another cycle plotted as "normal." In area E3 of the screen in FIG. 15, a graph with the status value on the vertical axis and the passage of time on the horizontal axis displays a time waveform based on the status value of the representative value selected as "abnormal" and a time waveform based on the status value of the representative value selected as "normal." In this embodiment, in addition to the time waveform of the "welding current" selected by the user, as shown in FIG. 15, time waveforms of status values ​​of other variables related to the welding current (such as a voltage related to the welding current or the temperature of the workpiece being welded) may also be displayed.

[0104] In area E3, for the status value of "welding current" selected by the user, both the time waveform determined to be "abnormal" and the time waveform determined to be "normal" are displayed in an overlapping manner. In the overlapping time waveforms, a portion where there is a deviation, for example, a partial waveform at point 150 where the deviation is greater than a predetermined deviation amount, can be enlarged and displayed in window 151. For example, when the user specifies point 150, the specified partial waveform is enlarged and displayed in window 151. Note that the portion to be enlarged and displayed can be any portion of the time waveform and is not limited to point 150. Such a time waveform image provides the user with information to assist in identifying the degree to which the status value (welding current) in an abnormal state deviates from the status value in a normal state and at what point in the cycle time the deviation occurred.

[0105] In area E4 of the same screen as FIG. 15 where the time waveform is displayed, for example, a frame image (hereinafter referred to as an abnormal image) corresponding to the cycle time of the status value selected as "abnormal" is displayed on the right side, and a frame image (hereinafter referred to as a normal image) corresponding to the cycle time of the status value selected as "normal" is displayed on the left side. In FIG. 15, the images played back in area E4 are images captured from the same angle, i.e., images captured by the same imaging device 40, but the played back images are not limited to images captured from the same angle. For example, as shown in FIG. 16, for the normal image or abnormal image for the cycle time of the status value selected as "normal" or "abnormal", images of the control target captured from a different angle, for example, images captured by different imaging devices 40, may also be used.

[0106] In FIG. 15, a bar 152 is displayed in the graph of the time waveform in area E3, which can be moved in the direction of the horizontal axis of the graph, more typically by sliding. The horizontal axis of the graph corresponds to the time axis of the cycle time during which the state values ​​indicated by the waveform were collected. Therefore, the position of bar 152 on the horizontal axis of the graph indicates the time during the cycle time corresponding to that position. When bar 152 is moved along the horizontal axis of the graph by a user operation, a normal image and an abnormal image of frame images captured at the time (capture time) corresponding to the position after movement are displayed side by side in area E4.

[0107] 15 in the direction in which the horizontal axis (time axis) of the graph extends in accordance with the playback speed of the image in area E4 by UI tool 340. Also, when the user manually slides bar 152 in FIG. 15 by operating bar 152, image adjustment module 344 rewinds or fast-forwards the image by UI tool 340 in accordance with the movement direction based on the user operation. Also, image adjustment module 344 can cause UI tool 340 to change the playback speed in accordance with the movement speed of bar 152 based on the user operation.

[0108] 15 shows a point 150 where the time waveform is shifted. The user slides the bar 152 until it reaches the position of the point 150 on the time waveform. When the bar 152 reaches the position of the point 150, the UI tool 340 searches the time-series frames 530 for the frame image at the time corresponding to the point 150, and plays it back.

[0109] In FIG. 15, for the point 150 where the time waveform is shifted, the UI tool 340 plays back the abnormal image and the normal image captured at the time corresponding to the point 150 side by side and simultaneously. Therefore, the behavior of the control target shown in the playback image can be provided as information for supporting the shift of the time waveform, that is, specifying the cause of the abnormality. More specifically, if there is an abnormality in the behavior of the control target shown in the playback image corresponding to the point 150 shown in the region E4, the user can determine that the cause of the abnormality is that the state value (welding current) indicating the behavior of the control target shows an abnormal value. Also, if there is no abnormality in the behavior of the control target shown in the playback image corresponding to the point 150, the user can estimate that the setting of the threshold value used for the precursor monitoring is inappropriate.

[0110] When estimating that the setting of the threshold value used for the precursor monitoring is inappropriate, the user operates and changes the threshold value 49a shown in the region E1 of the screen in FIG. 13, for example. The threshold operation tool 311 receives the user operation and transmits the changed threshold value 49a changed by the received user operation to the server 200 to be set in the threshold file 400 (step T11 in FIG. 11). The server 200 transfers the changed threshold value 49a received from the terminal 300 to the PLC 100 to be set in the threshold file 400.

[0111] Also, the server 200 searches the time-series DB 251 for the representative value of the user-specified cycle time used for the threshold setting, for example, the collected data at the cycle time of the representative value indicating "abnormal" or "normal" in the monitoring result, and stores it as the reference data 700.

[0112] <G. Operation Mode> The operation mode according to this embodiment will be described. FIG. 17 is a flowchart showing an example of processing in the operation mode according to this embodiment. FIGS. 18 to 21 are diagrams showing examples of display screens for information to assist user operations in the operation mode according to this embodiment. These display screens are displayed on the display of the display unit 328 by the UI tool 340. In the operation mode, the UI tool 340 of the terminal 300 transmits to the server 200, in accordance with a user operation, a command to start the operation mode program 217 that searches the time-series DB 252. The server 200 starts the operation mode program 217 in response to the command.

[0113] In the operation mode, the operation mode program 217 monitors the representative value 61a of each variable (state value) received from the PLC 100 and the corresponding monitoring result 61b, and transmits screen information for displaying the monitoring result 61b to the terminal 300.

[0114] 17, terminal 300 displays a screen based on the screen information from server 200. UI tool 340 determines whether or not an abnormality has been detected from the screen information (step R1). If no abnormality is detected (NO in step R1), step R1 is repeated, but if an abnormality is detected (YES in step R1), the screen of FIG. 18 is displayed based on the screen information. The screen of FIG. 18 presents icon 16A notifying that monitoring result 61b of the predictive monitoring based on the status value "welding current" indicates "abnormality."

[0115] When the terminal 300 receives an operation to designate the icon 16A on the screen of FIG. 18 via the UI tool 340, the terminal 300 transfers information about the received user operation to the server 200 (step R3). The server 200 searches the time-series DB 252 for time-series data 612 indicating the "average value" corresponding to the "welding current" variable based on the information received from the terminal 300. The terminal 300 transmits screen information based on the searched time-series data 612 to the terminal 300, and the terminal 300 displays the screen of FIG. 19(A) based on the screen information from the server 200 (step R5). The graph plotting the representative values ​​in FIG. 19(A) is similar to the graph shown in area E2 of FIG. 13, and therefore, a description thereof will not be repeated. The plot graph of FIG. 19(A) can be switched between and displayed as a histogram of FIG. 19(B). The histogram and the mechanism for switching the display are similar to those shown in FIG. 13, and therefore, a description thereof will not be repeated. In order to narrow down the number of plots of representative values ​​displayed in FIG. 19(A), the UI screen of FIG. 14 may be displayed.

[0116] In (A) of Fig. 19, terminal 300 plays back a time waveform image based on the state values ​​of the "abnormal" representative value and the "normal" representative value selected based on a user operation, and a moving image based on the captured image (step R11). The processing of step R11 is similar to the processing of steps T7 and T9, and the description will not be repeated. For example, the screen of Fig. 20 or Fig. 21 is displayed.

[0117] 20, it is determined whether a user operation to move the bar 152 is to be accepted (step R15). If it is determined that this user operation is not accepted (NO in step R15), it is determined whether to end the process based on the user operation (step R19). If it is determined that the process is to be ended (YES in step R19), the process ends, but if it is determined that the process is not to be ended (NO in step R19), the process returns to step R11.

[0118] In step R15, if it is determined that a movement operation of the bar 152 has been accepted based on a user operation (YES in step R15), the image adjustment module 344 plays back the image (fast forward, rewind, frame-by-frame, etc.) based on the accepted amount of operation (step R15). Then, the process proceeds to step R19.

[0119] The flowchart of Fig. 17 is configured to perform the operation mode process (steps R5 to R19) when an abnormality is detected, for example, when icon 16A displayed on the screen of Fig. 18 is operated, but is not limited to this. For example, the operation mode process (steps R5 to R19) may be performed for the status value of any variable when the user specifies the variable name of the variable.

[0120] In the operation mode, the pair of images displayed side by side on the screen in Fig. 20 is a pair of images when the monitoring result is abnormal and when it is normal, but this combination is not limited to this and may be a pair of images captured from different angles when the monitoring result is abnormal or when it is normal, as shown in Fig. 21. Alternatively, the pair may be a pair of an image of image data 712 of reference data 700 used to set a threshold and an image during operation based on the threshold. The time waveform of the representative value corresponding to the representative value and the moving image to be played back may include a time waveform of the periodically collected status value indicated by reference data 700 used to set the threshold in the setting mode, and a moving image of an image associated with the imaging time corresponding to the time.

[0121] This allows the user to determine, for example, from the set of time waveforms of status values ​​and moving images displayed in parallel that the monitoring results of the representative values ​​during operation indicate an "abnormality," but the time waveforms of the status values ​​do not contain any abnormal waveforms (the deviation is not large), and thus infer that the threshold values ​​set in the setting mode are inappropriate.

[0122] In the present embodiment, in the setting mode in region E2 of FIG. 13 and in the operation mode on the screen of (A) in FIG. 19, when the user performs a selection operation on an arbitrary representative value, the UI tool 340 displays a set of the time waveform and the image (video) of the selected representative value. Therefore, in any mode, it is possible to provide the user with support information for more easily determining the state of the control target.

[0123] In region E2 of FIG. 13 or in (A) of FIG. 19, for each of the two representative values selected by the user, a set of the time waveform of the state value and the video is displayed. However, the number of sets to be displayed is not limited to two. That is, if there are three or more representative values selected by the user, three sets of the time waveform of the state value and the video are displayed. Also, if there is one representative value selected by the user, one set of the time waveform of the state value and the video is displayed.

[0124] <H. Supplementary Note> The present embodiment as described above includes the following technical ideas. [Configuration 1] A control system (1a, 1b) that periodically controls a target (10), An imaging unit (40) having an imaging range capable of imaging the target as a subject, An image collection unit (46) that collects the image captured by the imaging unit in association with the imaging time, A data collection unit (118a) that periodically collects the state value of the target, An acquisition unit (119a) that acquires a representative value of the state value collected at a predetermined length of time among the periodically collected time-series state values, A selection operation reception unit (345) that receives a user operation for selecting at least one representative value from the representative values for each of the predetermined lengths of time acquired by the acquisition unit, and a waveform display unit (341, 342) that displays, for each of the at least one selected representative value, a time waveform of the state value collected periodically over the predetermined length of time corresponding to the representative value, and a moving image played back based on the image associated with the imaging time corresponding to the time. [Configuration 2] 2. The control system according to configuration 1, further comprising a representative value display unit (340) that displays a time waveform of the representative value for each of the predetermined lengths of time acquired by the acquisition unit. [Configuration 3] 3. The control system according to configuration 2, wherein the selection operation receiving unit receives a user operation to select the at least one representative value in a time waveform of the representative values. [Configuration 4] the control system includes a monitoring unit (119) that monitors the state of the object for each predetermined length of time based on the representative value and a threshold value acquired during that time; The representative value display unit 4. The control system according to configuration 3, wherein the display mode of each representative value in the time waveform of the representative value is changed depending on the result of the monitoring based on the representative value. [Configuration 5] Accepting a user operation to designate a partial waveform (150) in the time waveform; The control system according to any one of configurations 1 to 4, wherein the moving image includes a moving image played back based on the image associated with the imaging time corresponding to the time of the partial waveform specified by the received user operation. [Configuration 6] The control system includes: a monitoring unit (119) that monitors the state of the target for each predetermined length of time based on the representative value and a threshold value acquired during that time; a setting unit (216) that sets the threshold value from the periodically collected state value during the predetermined length of time designated by a user, The waveform display unit further The control system of any one of configurations 1 to 5 is configured to display, for each of the representative value of the state values ​​used to set the threshold and the selected at least one representative value, a time waveform of the state values ​​collected periodically over the predetermined length of time corresponding to the representative value, and a moving image played back based on the image associated with the imaging time corresponding to the time.

[0125] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0126] 1, 1a, 1b Control system, 3 Imaging start command, 4 Control network, 6 Information network, 61a Representative value, 61b Monitoring result, 8 Internet, 10 Field device, 16A Icon, 20 Relay device, 40 Imaging device, 46 Image acquisition unit, 49a, 49b Threshold, 118a Data acquisition unit, 119 Monitoring processing unit, 119a Representative value acquisition unit, 119b Threshold monitoring unit, 119c Threshold setting unit, 120 Camera communication program, 150 Point, 151 Window, 200 Server, 216 Setting mode program, 217 Operation mode program, 250 Time series DB 250, 314 Waveform conversion program, 315 Image playback program, 316 Threshold operation program, 317 Image adjustment program, 318 Selection program, 328 Display unit, 340 UI tool, 341 Waveform display module, 342 Image reproduction module, 343 threshold operation module, 344 image adjustment module, 345 selection module, 400 threshold file.

Claims

1. A control system for periodically controlling an object, comprising: an imaging unit having an imaging range capable of imaging the target as a subject; an image collecting unit that collects images captured by the imaging unit in association with imaging times; a data collection unit that periodically collects the state values ​​of the object; an acquisition unit that acquires, from the periodically collected time-series state values, a representative value of the state values ​​collected at each predetermined time; a selection operation receiving unit that receives a user operation to select at least one representative value from the representative values ​​for each of the predetermined lengths of time acquired by the acquisition unit; a waveform display unit that displays, for each of the at least one selected representative value, a time waveform of the state value periodically collected during the predetermined length of time corresponding to the selected representative value, and a moving image that is reproduced based on the image associated with the imaging time corresponding to the selected representative value; a monitoring unit that monitors a state of the target for each predetermined length of time based on the representative value and a threshold value acquired during that time; a setting unit that sets the threshold value from the state value periodically collected during the predetermined length of time designated by a user, The waveform display unit a set waveform display unit that displays, for each of the representative value of the state values ​​used to set the threshold and the selected at least one representative value, a time waveform of the state value collected periodically over the predetermined length of time corresponding to the representative value, and a moving image played back based on the image associated with the imaging time corresponding to the time.

2. The control system according to claim 1 , further comprising a representative value display unit that displays a time waveform of the representative value for each of the predetermined lengths of time acquired by the acquisition unit.

3. The control system according to claim 2 , wherein the selection operation receiving unit receives a user operation to select the at least one representative value from the time waveform of the representative values.

4. The representative value display unit The control system according to claim 3 , wherein a display mode of each representative value in the time waveform of the representative value is changed depending on the result of the monitoring based on the representative value.

5. a designation operation receiving unit that receives a user operation to designate a partial waveform in the time waveform; The control system according to claim 1 , wherein the moving image includes a moving image played back based on the image associated with the imaging time corresponding to the time of the partial waveform specified by the received user operation.

Images