Information processing system, information processing method, and program

Abstract

To provide an information processing system capable of appropriately supporting searching for the cause of abnormal operation of a monitoring target.SOLUTION: An information processing system performs information processing using time-series data on actual values of a plurality of variables representing the state of a monitoring target. The information processing system includes a simulator that simulates the behavior of a virtual model corresponding to the monitoring target in virtual space using the actual values of one or more second variables excluding a first variable among the plurality of variables. The information processing system also includes an identification module for identifying abnormality occurrence time at which a difference between a virtual value of the first variable generated in the virtual model and the actual value of the first variable indicated by the time-series data exceeds a reference.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] A device has been developed that supports the investigation of the cause of abnormal behavior of a monitored object such as a production facility. JP 2022-99341 A (Patent Document 1) discloses a technology for displaying on a display unit video data obtained by shooting the monitored object and motion reproduction simulation data that reproduces the virtual monitored object's behavior. The motion reproduction simulation data is generated based on log data that records in chronological order the state of the monitored object and input / output data of control signals between the monitored object and a control device that controls the monitored object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-99341 A [Patent Document 2] JP 2017-97426 A Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology disclosed in Patent Document 1, a user finds the time when an abnormality occurs (hereinafter referred to as "abnormal time") in the real video shown by video data or the virtual video shown by the action reproduction simulation data, and checks the state of the monitored object before and after that time. However, if the shooting time is long, it takes time to visually find the abnormal time in the real video or virtual video. Therefore, it takes time to find the cause of the abnormal action of the monitored object.

[0005] It is possible to identify the time of the abnormality by comparing the abnormal waveform shown by the log data when an abnormality occurs in the monitored object with the normal waveform shown by the log data when the monitored object is normal. However, in cases where the monitored object does not perform regular operations, for example, when the workpieces are transported randomly, it is not possible to prepare a normal waveform. In addition, it is not possible to prepare a normal waveform for a monitored object immediately after installation. Therefore, in these cases, the user cannot identify the time of the abnormality by comparing the abnormal waveform with the normal waveform. Therefore, it takes time to find the cause of the abnormal operation of the monitored object.

[0006] The present disclosure has been made in consideration of the above problems, and has as its objective to provide an information processing system, an information processing method, and a program that can appropriately assist in identifying the cause of abnormal behavior of a monitored object. [Means for solving the problem]

[0007] According to an example of the present disclosure, an information processing system performs information processing using time series data of actual values ​​of multiple variables representing a state of a monitored object. The information processing system includes a simulator that simulates the behavior of a virtual model corresponding to the monitored object in a virtual space using actual values ​​of one or more second variables excluding a first variable among the multiple variables. Furthermore, the information processing system includes an identification module that identifies an abnormal time when a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time series data exceeds a reference value.

[0008] According to this disclosure, a user can easily search for the cause of the abnormal behavior of a monitored object by closely investigating the state of the monitored object at the time of the abnormality identified by a specific module and before and after the time. In this way, the information processing system according to the present disclosure can appropriately support the search for the cause of the abnormal behavior of a monitored object.

[0009] In the above disclosure, the information processing system further includes an output module that outputs data showing a list of anomalous times. According to this disclosure, a user can easily recognize anomalous times by using the list.

[0010] In the above disclosure, the monitored object includes a first sensor that detects the presence of an object. The first variable represents a detection result by the first sensor. The simulator calculates a virtual value of the first variable based on a movement of a virtual object corresponding to the object in a virtual space. According to this disclosure, the user can grasp the time when the first sensor is abnormal.

[0011] In the above disclosure, the monitored object includes a visual sensor. The one or more second variables include a specific variable indicating a real position of the object measured by the visual sensor. The virtual model includes a virtual sensor corresponding to the first sensor. The simulator places the virtual object in a virtual space at a virtual initial position corresponding to the real position represented by the actual value of the specific variable. The simulator moves the virtual object from the virtual initial position in the virtual space according to the behavior of the virtual model. The simulator calculates a virtual value of the first variable based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

[0012] Alternatively, the virtual model includes a virtual sensor corresponding to the first sensor. The simulator identifies a real position of the object based on a real image obtained from a camera that captures the monitored object. The simulator places the virtual object at a virtual initial position corresponding to the real position in the virtual space. The simulator moves the virtual object from the virtual initial position in the virtual space according to the behavior of the virtual model. The simulator calculates a virtual value of the first variable based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

[0013] Alternatively, the first sensor detects the presence of an object at a first location of the monitored object. The monitored object includes a second sensor that detects the presence of an object at a second location upstream of the first location. The one or more second variables include a specific variable that represents a detection result by the second sensor. The virtual model includes a virtual sensor corresponding to the first sensor. The simulator places a virtual object at a virtual initial position corresponding to the second location in the virtual space based on the actual value of the specific variable indicating the presence of the object. The simulator moves the virtual object from the virtual initial position in the virtual space according to the behavior of the virtual model. The simulator calculates a virtual value of the first variable based on the positional relationship between the virtual object and the virtual sensor in the virtual space.

[0014] According to these disclosures, the simulator can set the virtual initial position of the virtual object with high accuracy.

[0015] In the above disclosure, the first variable is a position, a speed, or a torque of a moving part included in the monitoring target. The one or more second variables include a variable indicating a command value for the moving part. The simulator calculates a virtual value of the first variable of a virtual moving part corresponding to the moving part based on the command value. According to this disclosure, a user can grasp the time when an abnormality occurs in the operation of the moving part.

[0016] According to another example of the present disclosure, an information processing method uses time series data of actual values ​​of a plurality of variables representing a state of a monitored object. The information processing method includes one or more processors simulating a behavior of a virtual model corresponding to the monitored object in a virtual space using actual values ​​of one or more second variables excluding a first variable among the plurality of variables. The information processing method further includes one or more processors identifying an abnormal time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time series data exceeds a reference value.

[0017] According to yet another example of the present disclosure, a program causes a computer to execute the above information processing method.

[0018] These disclosures also enable the information processing method and program to appropriately assist in identifying the cause of abnormal behavior of a monitored object. Effect of the Invention

[0019] According to the present disclosure, an information processing system, an information processing method, and a program can appropriately assist in identifying the cause of abnormal behavior of a monitored object. [Brief description of the drawings]

[0020] [Figure 1] 1 is a diagram illustrating an example of a manufacturing system to which an information processing system according to an embodiment is applied; [Diagram 2] 1 is a schematic diagram showing an example of an overall configuration of a manufacturing system according to an embodiment; [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of a control device according to an embodiment. [Figure 4] FIG. 2 illustrates an example of a configuration of a monitoring target. [Diagram 5] FIG. 2 is a diagram for explaining the processing contents of a visual sensor. [Figure 6] FIG. 13 is a diagram illustrating an example of a variable indicating a measurement result of a visual sensor. [Figure 7] FIG. 13 is a diagram illustrating an example of a software configuration of a PC related to a first analysis process of an abnormal time. [Figure 8] FIG. 11 is a diagram illustrating an example of correspondence relationship information. [Figure 9] FIG. 11 is a diagram illustrating an example of initial information. [Figure 10] 13 is a flowchart showing the first half of a first analysis process flow of an anomalous time in a PC. [Figure 11] 13 is a flowchart showing the second half of the flow of a first analysis process of an abnormal time in a PC. [Figure 12] FIG. 11 is a diagram showing an example of a window displayed on the display in step S20. [Figure 13] FIG. 13 is a diagram illustrating yet another setting example of the virtual initial position of the virtual workpiece. [Figure 14]FIG. 13 is a diagram illustrating another example of the configuration of the monitoring target. [Figure 15] FIG. 13 is a diagram illustrating an example of a software configuration of a PC related to a second analysis process of an abnormal time. [Figure 16] 11 is a diagram showing an example of a waveform of a virtual value of an axis variable and a waveform of an actual value of the axis variable; FIG. [Figure 17] 13 is a flowchart showing the first half of a second analysis process flow of an abnormal time in a PC. [Figure 18] 13 is a flowchart showing the second half of the flow of a second analysis process of an abnormal time in a PC. [Figure 19] FIG. 13 is a diagram illustrating an example of a software configuration of a PC related to visualization processing of data for a target period. [Figure 20] FIG. 13 illustrates an example of an operation window. [Figure 21] FIG. 13 is a diagram showing an example of a screen displayed on a display. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and the description thereof will not be repeated. In this specification, unless otherwise specified, the term "or" includes the meaning of "and" in addition to the meaning of "or".

[0022] §1 Examples of application 1 is a diagram illustrating an example of a manufacturing system to which an information processing system according to an embodiment is applied. As shown in FIG. 1, the manufacturing system 1 includes an information processing system 100, a control device 200, and a monitored object 300.

[0023] The control device 200 is typically a programmable logic controller (PLC). The control device 200 is connected to the monitored object 300, and controls the monitored object 300 according to a control program created in advance.

[0024] The monitored object 300 may include an actuator that exerts some physical effect on a production facility or production line (hereinafter referred to as the "field"), various sensors installed in the field, or an input / output device that exchanges information with the field.

[0025] The control device 200 holds actual values ​​of a plurality of variables representing the state of the monitored object, and controls the monitored object 300 by performing calculations using the actual values ​​of one or more selected variables from the plurality of variables. The plurality of variables includes various variables acquired from the monitored object 300 and various variables output to the monitored object 300.

[0026] The actual values ​​of the multiple variables are updated every control cycle. The control device 200 records the actual values ​​of the multiple variables every control cycle and generates time-series data 11 of the actual values ​​of the multiple variables in a target period in which some abnormality occurs. The time-series data 11 is transferred from the control device 200 to the information processing system 100.

[0027] The information processing system 100 performs information processing using time-series data 11. The information processing system 100 is configured with one or more computers. The information processing system 100 includes a database 10, a simulator 20, and a specific module 30.

[0028] The database 10 stores time-series data 11 of actual values ​​of a plurality of variables that represent the state of a monitored object 300 during a target period.

[0029] The simulator 20 simulates the behavior of a virtual model corresponding to a monitored object 300 in a virtual space, using actual values ​​of one or more second variables excluding the first variable among the multiple variables.

[0030] The identifying module 30 identifies an abnormal time t_b at which the difference between the virtual value of the first variable generated in the virtual model and the actual value of the first variable indicated by the time-series data 11 exceeds a reference value.

[0031] According to this embodiment, the user can easily search for the cause of the abnormal operation of the monitored object 300 by investigating in detail the state of the monitored object 300 at the abnormal time t_b identified by the identification module 30 and before and after it.

[0032] In this manner, the information processing system 100 according to this embodiment can appropriately assist in searching for the cause of abnormal operation of the monitored object 300.

[0033] §2 Specific examples <Overall configuration of manufacturing system> Fig. 2 is a schematic diagram showing an example of the overall configuration of a manufacturing system according to an embodiment. As shown in Fig. 2, the manufacturing system 1 includes a personal computer (hereinafter referred to as "PC100A") constituting an information processing system 100, a control device 200, a monitoring target 300, a storage medium 500, and a network camera 600. The PC 100A, the control device 200, and the network camera 600 can communicate with each other using, for example, a general-purpose Ethernet (registered trademark). The PC 100A and the control device 200 may communicate with each other using another communication means (for example, Universal Serial Bus (USB)).

[0034] The network camera 600 captures at least a part of the monitored object 300. Video data generated by the network camera 600 is stored in, for example, a network storage (not shown) or the PC 100A.

[0035] The storage medium 500 includes a volatile storage medium, a non-volatile storage medium, a general-purpose semiconductor storage device such as a Compact Flash (CF) or a Secure Digital (SD), a magnetic storage medium such as a Flexible Disk, or an optical storage medium such as a Compact Disk Read Only Memory (CD-ROM). The storage medium 500 is, for example, an SD memory card.

[0036] The PC 100A is typically a computer having a general-purpose architecture. As shown in Fig. 2, the PC 100A includes a processor 101 such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), a memory 102 configured as a volatile storage device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory), a storage 103, an input interface 104, a display controller 105, a communication interface 106, and a reader / writer 107. These components are connected to each other via a bus so as to be able to communicate data with each other.

[0037] The processor 101 reads out a program stored in the storage 103, and loads the program in the memory 102. The processor 101 executes the loaded program.

[0038] The storage 103 is configured by a non-volatile storage device such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage 103 stores the program executed by the processor 101 and the database 10.

[0039] The programs executed by the processor 101 include a development program 131, a viewer program 132, a data reproduction program 133, and a simulation program 134.

[0040] The development program 131 mainly includes a set of commands for supporting the development of a control program used by the control device 200. Furthermore, the development program 131 provides a setting function related to the generation of the time-series data 11, and a function for monitoring the execution status of the control program by the control device 200 (hereinafter referred to as a "program monitor function").

[0041] A control program developed using the development program 131 is installed in the control device 200. Furthermore, monitoring setting data generated using the setting function of the development program 131 is transferred to the control device 200.

[0042] The viewer program 132 includes a set of commands for visualizing various types of information and providing it to the user. The viewer program 132 provides a function for displaying a graph showing the transition of the values ​​of one or more variables (hereinafter referred to as a "variable graph display function") and a function for playing video.

[0043] The data reproduction program 133 includes a group of commands for reproducing past data. The data reproduction program 133 includes a function for managing time and a function for requesting data to be reproduced.

[0044] The simulation program 134 includes a set of commands for simulating the behavior of a virtual model of the monitored object 300 in a virtual space. The simulation program 134 includes a function for reproducing the behavior of the virtual model (hereinafter referred to as an "action reproduction function"), a script for moving a virtual object corresponding to an object not controlled by the control device 200, a data management function, and an analysis function.

[0045] The script is written in, for example, C# format and includes a start method and a render method. The start method is executed only once when the script starts to be executed. The render method is executed periodically.

[0046] The database 10 stores time-series data 11. The time-series data 11 is transferred from the control device 200, and indicates actual values ​​at each time of a plurality of variables that represent the state of the monitored object 300 during a target period.

[0047] The input interface 104 mediates data transmission between the processor 101 and an input device 110 such as a keyboard, a mouse, or a touch panel. The display controller 105 is connected to a display 120 and displays a screen indicated by screen data generated by the processor 101 on the display 120. The communication interface 106 communicates with the control device 200 via a communication network. The reader / writer 107 mediates data transmission between the processor 101 and an external storage medium (which may include, for example, a storage medium 500).

[0048] Each of the development program 131, the viewer program 132, the data reproduction program 133, and the simulation program 134 may be provided not as a standalone program but as part of an arbitrary program. In this case, the processing according to the present embodiment is realized in cooperation with the arbitrary program. Also, some or all of the functions provided by the development program 131, the viewer program 132, the data reproduction program 133, and the simulation program 134 may be realized by a dedicated hardware circuit (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

[0049] <Control device hardware configuration> Fig. 3 is a schematic diagram showing an example of a hardware configuration of a control device according to an embodiment. As shown in Fig. 3, the control device 200 includes, as main components, a power supply circuit 201 that supplies power PW to each unit of the control device 200, a CPU 202, a chipset 204, a memory 206, a storage 208, a USB controller 250, a field network controller 252, a network controller 254, a timer 260, and a reader / writer 270.

[0050] The CPU 202 reads out a program stored in the storage 208 and loads it in the memory 206. The CPU 202 executes the loaded program.

[0051] The memory 206 is configured with a volatile storage device such as a DRAM or an SRAM. The memory 206 holds the actual values ​​of a plurality of variables. The actual values ​​of the plurality of variables held in the memory 206 are updated according to a control period by the CPU 202 executing a program stored in the storage 208.

[0052] The actual values ​​of a plurality of variables are accumulated in a buffer area 207 of a memory 206 for a certain period of time.

[0053] The storage 208 is configured, for example, by a non-volatile storage device such as an HDD, an SSD, or a flash memory, but a portion of the storage device may be configured by a volatile storage device.

[0054] The chipset 204 mediates data exchange between the CPU 202 and each component, thereby achieving processing of the control device 200 as a whole.

[0055] The storage 208 stores an OS (Operating System) 212 for implementing the basic functions of the control device 200 and a system program 211 having a scheduler program 213 .

[0056] Furthermore, the storage 208 stores a program data set 214 created by the PC 100A. The program data set 214 includes a control program and sampling setting data.

[0057] The control program defines a process for collecting values ​​of various variables from the monitored object 300, and a process for performing calculations based on the collected variable values ​​to generate values ​​of various variables to be output to the monitored object 300. The values ​​of various variables are temporarily stored in memory 206. CPU 202 executes the control program to update the values ​​of the variables stored in memory 206 for each control cycle.

[0058] The sampling setting data defines the generation conditions of the time series data 11. The generation conditions of the time series data 11 indicate the trigger conditions for sampling, the variables to be sampled, the sampling time, and the like.

[0059] The trigger condition includes, for example, a condition that a value of a variable related to an abnormality in the monitored object 300 has changed to a specific value. The CPU 202 reads the time series data 11 indicating values ​​of a plurality of variables in a target period including the timing at which the trigger condition is satisfied from the buffer area 207, and controls the reader / writer 270 to write the time series data 11 to the storage medium 500. The target period includes, for example, a first predetermined period before the timing at which the trigger condition is satisfied, and a second predetermined period after the timing at which the trigger condition is satisfied. The lengths of the first predetermined period and the second predetermined period are defined by the sampling setting data.

[0060] The USB controller 250 is responsible for transferring data to and from external devices (which may include the PC 100A) via a USB connection.

[0061] The field network controller 252 has a connector 252a that connects to the field network, and controls data exchange with the monitored object 300. The field network controller 252 has an internal buffer that stores input data received from the field network and output data output from the control device 200 to the field network.

[0062] The network controller 254 has a connector 254a for connecting to a higher-level network, and controls data exchange between the control device 200 and an external device (which may include the PC 100A).

[0063] The timer 260 includes a clock circuit or a counter circuit that measures time, but is not limited to such circuits and may be configured by a software module executed by the CPU 202.

[0064] The reader / writer 270 mediates data transmission between the CPU 202 and the storage medium 500. The CPU 202 copies the time series data 11 generated according to the generation conditions indicated by the sampling setting data from the buffer area 207 to the storage medium 500. The time series data 11 stored in the storage medium 500 is transferred to the PC 100A via the network controller 254 at a predetermined timing. Alternatively, the user may connect the storage medium 500 to the PC 100A and copy or move the time series data 11 from the storage medium 500 to the PC 100A.

[0065] <First analysis process of abnormal time> The first analysis process of the abnormal time by the PC 100A will be described with reference to FIGS.

[0066] (Monitored) 4 is a diagram showing an example of a configuration of a monitored object 300A. As shown in FIG. 4, the monitored object 300A includes a transport system 310, a robot 320, a visual sensor 330, and an object detection sensor 340.

[0067] The transport system 310 transports a workpiece 3, which is an example of a target object of the present disclosure. The transport system 310 includes an encoder 312. A variable indicating a measurement result of the encoder 312 is collected by the control device 200. An actual value of the variable indicating the measurement result of the encoder 312 is referred to as an "encoder value."

[0068] The robot 320 performs processing on the workpiece 3. The robot 320 operates in accordance with actual values ​​(command values) of various variables received from the control device 200. Actual values ​​(state values) of variables representing the current state of the robot 320 are collected by the control device 200.

[0069] The visual sensor 330 performs image measurement processing such as pattern matching on an image acquired by a camera 332 that captures an image of the monitored object 300A. The camera 332 is installed so as to capture an image of a partial area 313 of the transport system 310.

[0070] FIG. 5 is a diagram for explaining the processing contents of the visual sensor. When the camera 332 captures an image, the visual sensor 330 outputs an on-state image capture flag to the control device 200. The image capture flag is one of the variables collected by the control device 200. The visual sensor 330 acquires an image from the camera 332. The image is obtained by capturing an image of a partial area 313 of the transport system 310. The visual sensor 330 measures the number of workpieces 3 captured in the image and the actual position and actual posture of each workpiece 3. Furthermore, the visual sensor 330 may determine the type of workpiece 3 captured in the image. The actual value of the variable indicating the measurement result of the visual sensor 330 is collected by the control device 200.

[0071] 6 is a diagram showing an example of variables indicating the measurement results of the visual sensor 330. As shown in FIG. 6, the visual sensor 330 collects the actual values ​​of the variable "controller name", the variable "detection flag of work 3", the variable "detection number of work 3", the variable "X coordinate of work 3", the variable "Y coordinate of work 3", the variable "rotation angle of work 3", and the variable "type of work 3". The actual values ​​of the variable "X coordinate of work 3", the variable "Y coordinate of work 3", the variable "rotation angle of work 3", and the variable "type of work 3" are measured for each detected work 3. Therefore, the variable "X coordinate of work 3", the variable "Y coordinate of work 3", the variable "rotation angle of work 3", and the variable "type of work 3" constitute an array variable.

[0072] Returning to Fig. 4, the object detection sensor 340 detects the presence of the workpiece 3 at a location downstream of the area 313 of the transport system 310. The actual value of a variable indicating the detection result by the object detection sensor 340 is collected by the control device 200. The variable output from the object detection sensor 340 takes on either a value of "1" indicating the presence of an object, or a value of "0" indicating the absence of an object. The object detection sensor 340 is an example of a "first sensor" in the present disclosure.

[0073] (Software configuration) 7 is a diagram showing an example of a software configuration of a PC related to a first analysis process of an abnormal time. As shown in FIG. 7, the PC 100A includes a data reproduction module 14, an operation setting module 21, an initial information setting module 22, a variable selection module 23, an operation reproduction module 24, a virtual work generation module 25, a script execution module 26, a virtual space construction module 27, a virtual sensor module 28, an identification module 30, and an output module 32.

[0074] 2 executes a data reproduction program 133. The action setting module 21, the initial information setting module 22, the variable selection module 23, the action reproduction module 24, the virtual work generation module 25, the script execution module 26, the virtual space construction module 27, the virtual sensor module 28, the identification module 30, and the output module 32 are realized by the processor 101 executing a simulation program 134.

[0075] The operation setting module 21, the initial information setting module 22, the variable selection module 23, the operation reproduction module 24, the virtual work generation module 25, the script execution module 26, the virtual space construction module 27, and the virtual sensor module 28 constitute the simulator 20 shown in FIG.

[0076] The data reproduction module 14 reproduces data at a time included in a target period by using the database 10. The data reproduction module 14 includes a time control module 15 and a data request module 16.

[0077] The time control module 15 manages a counter value that indicates the time within a target period. The counter value is the value of a period counter of a control period. Therefore, the counter value can indicate the elapsed time from the start of the target period.

[0078] The time control module 15 updates the counter value. Specifically, the time control module 15 sets the initial counter value to "0" and then adds 1 to the counter value every time a specified time elapses.

[0079] The data request module 16 generates a query according to the specified time represented by the counter value. The specified time is the time that has elapsed from the start time of the target period by the product of a specified time and the counter value. The data request module 16 uses the generated query to obtain the actual value of each variable at the specified time from the time-series data 11 stored in the database 10. The data request module 16 outputs the obtained actual value of each variable to the action reproduction module 24, the virtual work generation module 25, the script execution module 26, and the identification module 30.

[0080] The operation setting module 21 sets correspondence relationship information indicating the correspondence relationship between the value of a related variable and the operation of the movable part for each movable part constituting the monitored object 300A. The operation setting module 21 sets the correspondence relationship information in response to a user input.

[0081] Fig. 8 is a diagram showing an example of the correspondence information. As shown in Fig. 8, the correspondence information 21a indicates the name of a movable part, a variable associated with the operation of the movable part, a vector of a translational operation (translation vector), and information of a rotational operation (rotation information). The translational vector defines the translational operation of the movable part according to the value of the variable. The rotational information defines the rotational operation of the movable part according to the value of the variable. The rotational information includes information defining the rotation axis or information defining the amount of rotation.

[0082] 7, the initial information setting module 22 sets initial information at the start time of the target period for each component of the monitored object 300A and each of the objects (including the workpiece 3) around the monitored object 300A. The objects around the monitored object 300A may include, in addition to the workpiece 3, for example, a tray or a fixed stand. The initial information setting module 22 sets the initial information according to a user input.

[0083] FIG. 9 is a diagram showing an example of initial information. As shown in FIG. 9, the initial information 22a includes an object name, a mesh shape of the outer surface, coordinates defining the position and orientation, and dependency information. The dependency information indicates a dependency destination (including parts of the monitoring target 300A and objects around the monitoring target 300A) and a relative positional relationship with the dependency destination. If there is no dependency destination, the dependency information is set to Null. A first object being subordinate to a second object means that the first object follows the second object while maintaining a relative positional relationship with the second object.

[0084] Returning to FIG. 7, the variable selection module 23 selects a first variable from among the multiple variables. The variable selection module 23 may select the first variable in response to a user input. Alternatively, the variable selection module 23 may select a variable output from the object detection sensor as the first variable in response to the monitoring target 300A including the object detection sensor. In the first analysis process, the variable output from the object detection sensor 340 is selected as the "first variable."

[0085] The operation reproduction module 24 simulates the operation of the virtual model of the monitored object 300A in a target period based on actual values ​​of one or more second variables excluding the first variable among the multiple variables.

[0086] Specifically, the action reproducing module 24 acquires from the data reproducing module 14 the actual values ​​of one or more second variables for each time in the target period in chronological order. The action reproducing module 24 generates data (1) indicating the state of the virtual model based on the actual values ​​of one or more second variables at each time. The data (1) indicates the position and orientation of each part of the virtual model. Hereinafter, the data (1) corresponding to the time tk that is k times the control period after the start time t0 of the target period is referred to as the "kth data (1)."

[0087] When the action reproducing module 24 acquires the actual values ​​of one or more second variables at the time tk, it simulates the action of the part of the virtual model corresponding to each movable part at the time tk based on the acquired actual values ​​and the correspondence information 21a. The action reproducing module 24 generates the kth data (1) indicating the state of the virtual model at the time tk by integrating the actions of each part. The action reproducing module 24 outputs the kth data (1) to the virtual space construction module 27.

[0088] The virtual work generation module 25 sets a virtual initial position of the virtual work corresponding to the work 3. The virtual work generation module 25 acquires, from the data reproduction module 14, actual values ​​of one or more second variables for each time in the target period in chronological order. The virtual work generation module 25 sets a virtual initial position of the virtual work in response to the fact that an actual value of a variable indicating the measurement result of the visual sensor 330 is included in the actual values ​​of the one or more second variables acquired. The virtual work is an example of a "virtual object" in this disclosure.

[0089] Specifically, the virtual work generation module 25 sets a virtual initial position of the virtual work based on the actual value of a specific variable indicating the actual position of the work 3 measured by the visual sensor 330. Furthermore, the virtual work generation module 25 sets a virtual initial orientation of the virtual work based on the actual value of a variable indicating the actual orientation of the work 3 measured by the visual sensor.

[0090] It is preferable that the virtual work generation module 25 corrects the virtual initial position by taking into consideration the time lag from when the camera 332 captures an image to when the visual sensor 330 measures the actual position and actual posture of the work based on the image. For example, the virtual work generation module 25 corrects the virtual initial position by using the actual value (encoder value) of the variable indicating the measurement result of the encoder 312 and the imaging flag output from the visual sensor 330, using the technology described in JP 2017-97426 A (Patent Document 2).

[0091] Specifically, when the virtual work generation module 25 receives a shooting flag indicating an on state from the data reproduction module 14, it records the time tp corresponding to the shooting flag and the encoder value at the time tp. When the actual value of a variable indicating the measurement result of the visual sensor 330 is included in the actual values ​​of one or more second variables corresponding to the time tq, the virtual work generation module 25 extracts the encoder value at time tq from the actual values ​​of the one or more second variables. The time tq is after the time tp. The virtual work generation module 25 calculates the movement amount of the transport system 310 from time tp to time tq based on the difference between the encoder value at time tp and the encoder value at time tq. The virtual work generation module 25 corrects the virtual initial position by the movement amount.

[0092] When the virtual workpiece generating module 25 sets the virtual initial position and the virtual initial posture, it outputs data (2) indicating the virtual initial position and the virtual initial posture to the virtual space construction module 27.

[0093] The script execution module 26 simulates the motion of the virtual workpiece during the target period. The script execution module 26 acquires from the data reproduction module 14 the actual values ​​of one or more second variables for each time during the target period in a chronological order. The script execution module 26 simulates the motion of the part of the virtual model corresponding to the subordinate part of the workpiece 3 based on the actual values ​​of one or more second variables at the time tk. The method of simulation is the same as the method of simulation in the motion reproduction module 24. The script execution module 26 simulates the position and posture of the virtual workpiece at the time tk based on the relative positional relationship indicated by the subordinate information included in the initial information 22a of the workpiece 3 and the position and posture of the part of the virtual model corresponding to the subordinate part. The script execution module 26 generates k-th data (3) indicating the position and posture of the virtual workpiece at the time tk. The motion reproduction module 24 outputs the k-th data (3) to the virtual space construction module 27.

[0094] When a plurality of virtual workpieces exist in the virtual space, the script execution module 26 simulates the position and orientation of each of the plurality of virtual workpieces.

[0095] The script execution module 26 may simulate the position and orientation of the virtual workpiece at time tk based on the k-th data (1) generated by the action reproduction module 24 and the relative positional relationship indicated by the subordinate information.

[0096] The virtual space construction module 27 constructs a virtual space for each time in a target period. When the virtual space construction module 27 acquires the k-th data (1) from the action reproduction module 24, the virtual space construction module 27 places a virtual model in the virtual space so as to assume the position and posture indicated by the k-th data (1).

[0097] When the virtual space construction module 27 acquires the data (2) from the virtual work generation module 25, it places the virtual work in the virtual initial position indicated by the data (2) in the virtual space. At this time, the virtual space construction module 27 determines the posture of the virtual work so that it coincides with the virtual initial posture indicated by the data (2).

[0098] When the virtual space construction module 27 acquires the kth data (3) from the script execution module 26, it moves the virtual workpiece from its initial position and orientation in the virtual space so as to assume the position and orientation indicated by the kth data (3).

[0099] The virtual sensor module 28 simulates the operation of a virtual sensor corresponding to the object detection sensor 340. The virtual sensor is a part of a virtual model, and is arranged in a virtual space according to the position and orientation of the object detection sensor 340 in the monitored object 300A. The virtual sensor module 28 calculates a virtual value of a first variable based on the positional relationship between the virtual workpiece and the virtual sensor in the virtual space. For example, the virtual sensor module 28 sets the virtual value of the first variable to a value indicating the presence of an object in response to the virtual workpiece being located in a target area of ​​object detection by the virtual sensor in the virtual space.

[0100] The identifying module 30 identifies an abnormal time t_b at which the difference between the virtual value of the first variable generated in the virtual model and the actual value of the first variable indicated by the time-series data 11 exceeds a reference value.

[0101] The output module 32 generates data indicating a list of the abnormal times t_b, and outputs the data to the display controller 105. As a result, the list of the abnormal times t_b is displayed on the display 120.

[0102] (First analysis process flow) Fig. 10 is a flow chart showing the first half of the flow of the first analysis processing of an abnormal time in a PC. Fig. 11 is a flow chart showing the second half of the flow of the first analysis processing of an abnormal time in a PC.

[0103] First, in step S1, the processor 101 of the PC 100A sets the start time t0 of the target period as the time t. Specifically, the processor 101 sets the counter value to 0. Furthermore, the processor 101 sets the times t_r and t_v to −1, and sets the ordinal number i to 0.

[0104] In the next step S2, the processor 101 acquires actual values ​​of a plurality of variables at time t from the time-series data 11. The plurality of variables includes a first variable acquired from the object detection sensor 340 and one or more second variables other than the first variable.

[0105] In the next step S3, the processor 101 determines whether or not the performance value of the first variable has changed from "0" to "1".

[0106] If the performance value of the first variable changes from "0" to "1" (YES in step S3), in step S4, processor 101 sets time t_r to the same time as time t. That is, time t_r represents the time when the performance value of the first variable changes from "0" to "1".

[0107] After step S4, or if the actual value of the first variable has not changed from "0" to "1" (NO in step S3), the first analysis process proceeds to step S5. In step S5, the processor 101 simulates the behavior of the virtual model corresponding to the monitored object 300A based on the actual value of one or more second variables. In step S5, if the actual value of one or more second variables includes an actual value of a variable indicating the measurement result of the visual sensor 330, the processor 101 sets a virtual initial position of the virtual workpiece according to the measurement result. Then, the processor 101 places the virtual workpiece at the virtual initial position in the virtual space. Furthermore, in step S5, if a virtual workpiece has already been placed in the virtual space, the processor 101 moves the existing virtual workpiece from the virtual initial position according to the behavior of the virtual model.

[0108] In the next step S6, the processor 101 calculates a virtual value of a first variable from a virtual sensor corresponding to the object detection sensor 340, based on the positional relationship between the virtual workpiece and the virtual sensor in the virtual space.

[0109] In the next step S7, the processor 101 determines whether or not the virtual value of the first variable has changed from "0" to "1".

[0110] If the virtual value of the first variable changes from "0" to "1" (YES in step S7), in step S8, the processor 101 sets the time t_v to the same time as the time t. That is, the time t_v represents the time when the virtual value of the first variable changes from "0" to "1".

[0111] If no virtual workpiece exists in the virtual space, the processor 101 omits the processes of steps S6 to S8.

[0112] In the next step S9, the processor 101 determines whether or not the conditions of time t_r ≠ -1 and time t_v ≠ -1 are met. If the conditions of time t_r ≠ -1 and time t_v ≠ -1 are met (YES in step S9), in step S10, the processor 101 determines whether or not |t_r - t_v| > Th1 is met. The threshold value Th1 is determined in advance. If the condition of |t_r - t_v| > Th1 is met (YES in step S10), in step S11, the processor 101 sets the i-th abnormal time t_b[i] to the same time as the time t_v. Furthermore, the processor 101 adds 1 to the ordinal number i.

[0113] After step S11, or if |t_r-t_v|>Th1 is not satisfied (NO in step S10), the first analysis process proceeds to step S12. In step S12, the processor 101 resets the times t_r and t_v to -1.

[0114] If the conditions of time t_r≠-1 and time t_v≠-1 are not met (NO in step S9), in step S13, the processor 101 determines whether or not the conditions of time t_r≠-1 are met.

[0115] If time t_r ≠ -1 is not satisfied (NO in step S13), in step S14, the processor 101 determines whether or not time t_v ≠ -1 is satisfied. If time t_v ≠ -1 is satisfied (YES in step S14), in step S15, the processor 101 determines whether or not |t_v-t| > Th2 is satisfied. The threshold value Th2 is determined in advance. Note that the threshold value Th2 may be the same as or different from the threshold value Th1. If |t_v-t| > Th2 is satisfied (YES in step S15), the first analysis process proceeds to step S11.

[0116] If time t_r ≠ -1 is satisfied (YES in step S13), in step S16, the processor 101 determines whether or not |t_r-t|>Th2 is satisfied. If |t_r-t|>Th2 is satisfied (YES in step S16), in step S17, the processor 101 sets the i-th abnormal time t_b[1] to the same time as the time t_r. Furthermore, the processor 101 adds 1 to the ordinal number i. After step S17, the first analysis process proceeds to step S12.

[0117] After step S12, or if NO in steps S15 and S16, the first analysis process proceeds to step S18. In step S18, the processor 101 determines whether or not the time t is before the end time T of the target period. If the time t is before the end time T (YES in step S18), in step S19, the processor 101 advances the time t by one control cycle. Specifically, the processor 101 advances the counter value by 1. After step S19, the first analysis process returns to step S2. If the time t is not before the end time T (NO in step S18), in step S20, the processor 101 generates data showing a list of abnormal times t_b and outputs the data to the display controller 105. As a result, the list of abnormal times t_b is displayed on the display 120. After step S20, the first analysis process ends.

[0118] (Screenshot) Fig. 12 is a diagram showing an example of a window displayed on the display in step S20. Data for the window shown in Fig. 12 is generated by the processor 101. As shown in Fig. 12, a window 50 includes display areas 51 and 52. In the display area 51, a waveform of a virtual value of the first variable and a waveform of an actual value of the first variable are displayed. In the display area 52, a list (t_b[1], t_b[2]) of abnormal times t_b at which the difference between the virtual value and the actual value exceeds a reference value is displayed.

[0119] (Modification of the first analysis process) In the above description, the virtual work generation module 25 sets the virtual initial position of the virtual work in response to the fact that the acquired actual values ​​of one or more second variables include an actual value of a variable indicating the measurement result of the visual sensor 330. However, the method of setting the virtual initial position of the virtual work is not limited to this.

[0120] For example, the virtual work generation module 25 may set the virtual initial position of the virtual work based on an image captured by the camera 332. For example, the image captured by the camera 332 is stored in the database 10. When an image corresponding to a specified time is stored in the database 10, the data reproduction module 14 outputs the image to the virtual work generation module 25 together with the actual values ​​of a plurality of variables. When the virtual work generation module 25 receives an image from the data reproduction module 14, the virtual work generation module 25 performs the same processing as the visual sensor 330 on the image to identify the real position and real orientation of the work 3 shown in the image. The virtual work generation module 25 may set the virtual initial position and virtual initial orientation of the virtual work based on the identified real position and real orientation. Then, the virtual space construction module 27 places the virtual work at the virtual initial position corresponding to the real position. At this time, the virtual space construction module 27 matches the orientation of the virtual work with the virtual initial orientation corresponding to the real orientation.

[0121] FIG. 13 is a diagram for explaining yet another setting example of the virtual initial position of the virtual work. As shown in FIG. 13, in the conveying system 310, the work 3 is conveyed along a line. The object detection sensor 345 is disposed upstream of the object detection sensor 340 and detects the presence of the work 3 at the location 314. The object detection sensor 340 detects the presence of the work 3 at the location 315. The detection accuracy of the object detection sensors 340 and 345 depends on the installation environment or specifications. In the example shown in FIG. 13, the detection accuracy of the object detection sensor 345 is superior to the detection accuracy of the object detection sensor 340. In FIG. 13, the object detection sensor 345 is an example of the "second sensor" of the present disclosure. The location 314 is an example of the "second location" of the present disclosure, and the location 315 is an example of the "first location" of the present disclosure.

[0122] The one or more second variables include a specific variable indicating a detection result of the object detection sensor 345. The virtual work generation module 25 may set a position corresponding to the location 314 in the virtual space as a virtual initial position of the virtual work, based on the fact that the actual value of the specific variable indicates the presence of the work 3.

[0123] <Second analysis process of abnormal time> The second analysis process of the abnormal time by the PC 100A will be described with reference to FIGS.

[0124] (Monitored) FIG. 14 is a diagram showing another example of the configuration of the monitored object. As shown in FIG. 14, the monitored object 300B includes an electric cylinder 350. The electric cylinder 350 applies a force to the first member 420 by translating the movable part 352, and repeats the operation of fitting the first member 420 into a recess formed in the second member 430. A variable related to the operation of the movable part 352 is called an "axis variable". The axis variable includes a variable indicating a command value for the movable part 352 and a variable indicating a current state of the movable part 352. The variable indicating the state of the movable part 352 indicates, for example, a state value (actual measurement value) of the position, speed, or torque of the movable part 352. The command value is generated by the control device 200. The state value of the position, speed, or torque is collected from the electric cylinder 350. The axis variable has an analog value.

[0125] When the electric cylinder 350 operates normally, the first member 420 is completely fitted into the recess of the second member 430. When an abnormality (e.g., a reduction gear abnormality) occurs in the electric cylinder 350, the first member 420 is incompletely fitted into the recess of the second member 430.

[0126] (Software configuration) Fig. 15 is a diagram showing an example of a software configuration of a PC related to the second analysis process of an abnormal time. As shown in Fig. 15, the PC 100A includes a data reproduction module 14, an action setting module 21, an initial information setting module 22, a variable selection module 23, an action reproduction module 24, a virtual space construction module 27, an axis variable value calculation module 29, an identification module 30, and an output module 32.

[0127] 2 executes a data reproduction program 133. The action setting module 21, the initial information setting module 22, the variable selection module 23, the action reproduction module 24, the virtual space construction module 27, the axis variable value calculation module 29, the identification module 30, and the output module 32 are realized by the processor 101 executing a simulation program 134.

[0128] The action setting module 21, the initial information setting module 22, the variable selection module 23, the action reproduction module 24, the virtual space construction module 27, and the axis variable value calculation module 29 constitute the simulator 20 shown in FIG.

[0129] The data reproduction module 14 reproduces data at a specified time included in the target period using the database 10, in the same manner as in the first analysis process. The data reproduction module 14 outputs the reproduced performance values ​​of each variable to the action reproduction module 24 and the identification module 30.

[0130] The operation setting module 21 sets, for the movable part 352 constituting the monitoring target 300B, correspondence relationship information 21a (see FIG. 8) indicating the correspondence relationship between the value of a related variable and the operation of the movable part 352. The operation setting module 21 sets the correspondence relationship information 21a in response to a user input.

[0131] The initial information setting module 22 sets initial information 22a (see FIG. 9) at the start time of the target period for each component of the monitored object 300B and each of the objects (including the first member 420 and the second member 430) around the monitored object 300B. The initial information setting module 22 sets the initial information 22a in response to a user input.

[0132] The variable selection module 23 selects a first variable from among the multiple variables. The variable selection module 23 may select the first variable in response to a user input. Alternatively, the variable selection module 23 may select a variable indicating a state (position, speed, or torque) of the movable part 352 from among the axial variables output from the electric cylinder as the first variable in response to the monitoring target 300B including an electric cylinder. In the second analysis process, the variable indicating the state of the movable part 352 from among the axial variables output from the electric cylinder 350 is selected as the "first variable."

[0133] The action reproducing module 24 simulates the action of the virtual model of the monitored object 300B in the target period based on the actual values ​​of one or more second variables excluding the first variable among the multiple variables. Specifically, the action reproducing module 24 acquires the actual values ​​of one or more second variables for each time in the target period in chronological order from the data reproducing module 14. The action reproducing module 24 generates data (1) indicating the state of the parts of the virtual model excluding the parts corresponding to the movable parts 352. The data (1) indicates the position and orientation of each part of the virtual model excluding the parts corresponding to the movable parts 352. Hereinafter, the data (1) corresponding to the time tk at which k times the control period has elapsed from the start time t0 of the target period is referred to as the "kth data (1)".

[0134] The axis variable value calculation module 29 acquires actual values ​​of one or more second variables for each time in the target period in order along a time series. The axis variable value calculation module 29 calculates a virtual value of a variable (first variable) indicating the state of the movable part 352 among the axis variables, based on a command value for the movable part 352 among the actual values ​​of the one or more second variables. The virtual value represents the position, speed, or torque of a virtual movable part corresponding to the movable part 352 among the virtual model. The axis variable value calculation module 29 generates data (2) indicating the state of the virtual movable part corresponding to the movable part 352 among the virtual model. The data (2) indicates the position and orientation of the virtual movable part. Hereinafter, the data (2) corresponding to the time tk at which k times the control period has elapsed from the start time t0 of the target period is referred to as the "kth data (2)".

[0135] The virtual space construction module 27 constructs a virtual space for each time in the target period. When the virtual space construction module 27 acquires the kth data (2) from the axis variable value calculation module 29, it arranges the virtual movable parts in the virtual space so as to assume the position and orientation indicated by the kth data (2). Similarly, when the virtual space construction module 27 acquires the kth data (1) from the action reproduction module 24, it arranges models of parts of the virtual model other than the virtual movable parts so as to assume the position and orientation indicated by the kth data (1).

[0136] The identification module 30 identifies an abnormal time t_b at which the difference between the virtual value of the first variable generated by the axis variable value calculation module 29 and the actual value of the first variable indicated by the time-series data 11 exceeds a reference value.

[0137] FIG. 16 is a diagram showing an example of the waveform of the virtual value of the first variable (variable indicating the state of the movable part) and the waveform of the actual value of the first variable. FIG. 16 shows the waveform of the state value indicating the speed. The virtual movable part operates according to the virtual value of the first variable calculated by the axis variable value calculation module 29. On the other hand, if some abnormality occurs in the monitored object 300B, the electric cylinder 350 cannot operate according to the command value. As a result, as shown in FIG. 16, the waveform of the actual value of the first variable (variable indicating the state of the movable part) may differ from the waveform of the virtual value of the first variable. This allows the identification module 30 to identify the abnormal time t_b based on the difference between the virtual value of the first variable and the actual value of the first variable. For example, the identification module 30 identifies the time when the actual value of the first variable takes the maximum value as the abnormal time t_b in response to the difference between the maximum value of the virtual value of the first variable and the maximum value of the actual value of the first variable exceeding a reference.

[0138] The output module 32 generates data indicating a list of the abnormal times t_b, and outputs the data to the display controller 105. As a result, the list of the abnormal times t_b is displayed on the display 120.

[0139] (Second analysis process flow) Fig. 17 is a flow chart showing the first half of the flow of the second analysis processing of an abnormal time in a PC. Fig. 18 is a flow chart showing the second half of the flow of the second analysis processing of an abnormal time in a PC.

[0140] First, in step S31, the processor 101 of the PC 100A sets the start time t0 of the target period as the time t. Specifically, the processor 101 sets the counter value to 0. Furthermore, the processor 101 sets the times t_r and t_v to −1, the speeds v_r and v_v to −1, and the ordinal number i to 0.

[0141] In the next step S32, the processor 101 acquires actual values ​​of a plurality of variables at time t from the time-series data 11. The plurality of variables include an axis variable (first variable) acquired from the electric cylinder 350 and one or more second variables other than the axis variable. The one or more second variables include a variable related to a command to the electric cylinder 350.

[0142] In the next step S33, the processor 101 determines whether or not the actual value of the axis variable “speed” at time t is greater than the speed v_r.

[0143] If the actual value of the axis variable "speed" is greater than the speed v_r (YES in step S33), in step S34, the processor 101 sets the speed v_r to the same speed as the actual value of the axis variable "speed" at the time t.

[0144] When the actual value of the axis variable "speed" is equal to or less than the speed v_r (NO in step S33), in step S35, the processor 101 sets the time t_r to the same time as the time t.

[0145] After step S34 or step S35, the second analysis process proceeds to step S36. In step S36, the processor 101 simulates the behavior of the virtual model corresponding to the monitored object 300B based on the actual values ​​of one or more second variables. Furthermore, the processor 101 calculates a virtual value of the axis variable "speed", which is the first variable, according to the behavior of the virtual model.

[0146] Next, in step S37, the processor 101 determines whether or not the virtual value of the axis variable "speed" is greater than the speed v_v.

[0147] If the virtual value of the axis variable "speed" is greater than the speed v_v (YES in step S37), in step S38, the processor 101 sets the speed v_v to the same speed as the virtual value of the axis variable "speed" at the time t.

[0148] When the virtual value of the axis variable “speed” is equal to or less than the speed v_v (NO in step S37), in step S39, the processor 101 sets the time t_v to the same time as the time t.

[0149] After step S38 or step S39, the second analysis process proceeds to step S40. In step S40, the processor 101 determines whether or not the conditions of time t_r ≠ -1 and time t_v ≠ -1 are met. If the conditions of time t_r ≠ -1 and time t_v ≠ -1 are met (YES in step S40), in step S41, the processor 101 determines whether or not |v_r-v_v|>Th3 is met. The threshold value Th3 is determined in advance. If the condition of |v_r-v_v|>Th3 is met (YES in step S41), in step S42, the processor 101 sets the i-th abnormal time t_b[i] to the same time as the time t_r. Furthermore, the processor 101 adds 1 to the ordinal number i.

[0150] After step S42, or if |v_r-v_v|>Th3 is not satisfied (NO in step S41), the second analysis process proceeds to step S43. In step S43, the processor 101 resets the times t_r and t_v to -1.

[0151] After step S43, or if the conditions of time t_r ≠ -1 and time t_v ≠ -1 are not met (NO in step S40), the second analysis process proceeds to step S44. In step S44, the processor 101 determines whether the time t is before the end time T of the target period. If the time t is before the end time T (YES in step S44), in step S45, the processor 101 advances the time t by one control cycle. Specifically, the processor 101 advances the counter value by 1. After step S45, the second analysis process returns to step S32.

[0152] If the time t is not earlier than the end time T (NO in step S44), in step S46, the processor 101 generates data indicating a list of the abnormal times t_b and outputs the data to the display controller 105. As a result, the list of the abnormal times t_b is displayed on the display 120. After step S46, the second analysis process ends.

[0153] According to the flowcharts shown in FIG. 17 and FIG. 18, the speed v_r continues to be updated until the actual value of the axis variable "speed" becomes equal to or less than the speed v_r. As a result, the speed v_r represents the maximum value of the actual value of the axis variable "speed". The time t_r represents the time t when the actual value of the axis variable "speed" becomes equal to or less than the speed v_r, that is, the time when the actual value of the axis variable "speed" becomes the maximum. Similarly, the speed v_v continues to be updated until the virtual value of the axis variable "speed" becomes equal to or less than the speed v_v. As a result, the speed v_v represents the maximum value of the virtual value of the axis variable "speed". The time t_v represents the time t when the virtual value of the axis variable "speed" becomes equal to or less than the speed v_v, that is, the time when the virtual value of the axis variable "speed" becomes the maximum.

[0154] In steps S40 to S42, if the absolute value of the difference between the maximum value of the virtual value of the axis variable "speed" and the maximum value of the actual value of the axis variable "speed" exceeds threshold value Th3, the time t_r when the actual value of the axis variable "speed" becomes maximum is identified as the abnormal time t_b.

[0155] (Modification of the second analysis process) The axis variable value calculation module 29 may calculate a virtual value of an axis variable (axis variable "position") indicating the position of a virtual moving part corresponding to the moving part 352. In this case, the identification module 30 may identify, as the abnormal time t_b, the time at which the difference between the virtual value of the axis variable "position" and the actual value of the axis variable "position" exceeds a reference value.

[0156] <Visualization of data for the target period> Fig. 19 is a diagram showing an example of a software configuration of a PC related to visualization processing of data for a target period. As shown in Fig. 19, PC 100A includes a time designation module 13, a data reproduction module 14, a waveform display module 41, a program display module 42, and a video playback module 43. The waveform display module 41 and the video playback module 43 are realized by the processor 101 executing a viewer program 132. The program display module 42 is realized by the processor 101 executing a development program 131.

[0157] In addition to time-series data 11, video data 12 captured by a network camera 600 is stored in the database 10.

[0158] The time designation module 13 sets a playback start counter value and playback conditions, and outputs the set playback start counter value and playback conditions to the data reproduction module 14. The time designation module 13 sets the playback start counter value and playback conditions in response to a user input to an operation window.

[0159] Fig. 20 is a diagram showing an example of an operation window. The operation window 60 shown in Fig. 20 is generated by the time designation module 13, and is displayed on the display 120 shown in Fig. 2. The operation window 60 includes a cursor 61, input fields 62 and 66, a play button 63, a frame rewind button 64, and a frame forward button 65.

[0160] The cursor 61 indicates a time within the target period. The time designation module 13 accepts a user's operation to move the cursor 61, and moves the position of the cursor 61 in response to the movement operation. Furthermore, when the counter value is updated by the data reproduction module 14, the time designation module 13 moves the cursor 61 to a position corresponding to the counter value.

[0161] A counter value corresponding to the position of the cursor 61 is displayed in the input field 62. The counter value represents the ordinal number of the control period in the target period. The user may input a counter value into the input field 62. When the counter value is input into the input field 62, the time designation module 13 moves the cursor 61 to a position corresponding to the input counter value. Furthermore, when the counter value is updated by the data reproduction module 14, the time designation module 13 updates the numerical value displayed in the input field 62.

[0162] The input field 66 is used for inputting the playback speed. The time designation module 13 displays a list of multiple playback speeds in response to the operation of the pull-down button in the input field 66. The time designation module 13 displays the playback speed selected by the user from the list in the input field 66. In the example shown in FIG. 20, the playback speed is represented by a magnification. Therefore, hereinafter, the "playback speed" will be expressed as a "playback magnification."

[0163] The play button 63, the frame rewind button 64, and the frame forward button 65 are used to input operations related to data reproduction by the data reproduction module 14. In response to clicking any of the play button 63, the frame rewind button 64, and the frame forward button 65, the time designation module 13 sets a playback start counter value and playback conditions, and outputs the set playback start counter value and playback conditions to the data reproduction module 14.

[0164] The play button 63 is used to instruct the data reproduction module 14 to start data reproduction. When the play button 63 is clicked, the time designation module 13 sets a playback start counter value and playback conditions, and outputs the set playback start counter value and playback conditions to the data reproduction module 14. Specifically, the time designation module 13 sets the counter value corresponding to the position of the cursor 61 as the playback start counter value. Furthermore, the time designation module 13 sets the playback conditions according to the playback magnification entered in the input field 66.

[0165] The frame rewind button 64 is used to instruct rewinding of time. When the frame rewind button 64 is clicked, the time designation module 13 sets the counter value corresponding to the position of the cursor 61 minus 1 as the playback start counter value. Furthermore, the time designation module 13 sets a playback condition indicating a playback magnification of "0".

[0166] The frame-by-frame button 65 is used to instruct fast-forwarding of time. When the frame-by-frame button 65 is clicked, the time designation module 13 adds 1 to the counter value corresponding to the position of the cursor 61 and sets the counter value as the playback start counter value. Furthermore, the time designation module 13 sets a playback condition indicating a playback magnification of "0".

[0167] Returning to FIG. 19, the data reproduction module 14 reproduces data based on the reproduction start counter value and the reproduction condition. The time control module 15 of the data reproduction module 14 sets the reproduction start counter value as an initial counter value, and thereafter adds a value obtained by multiplying a specified value (e.g., 50 or 100) by the reproduction magnification to the counter value each time a specified time has elapsed. The specified time is a time obtained by multiplying a control period by a specified value. When the reproduction magnification "x1.0" is set as the reproduction condition, the time control module 15 adds the specified value to the counter value each time the specified time has elapsed. When the reproduction magnification "x2.0" is set as the reproduction condition, the time control module 15 adds a value twice the specified value to the counter value each time the specified time has elapsed. Note that when the reproduction magnification "0" is set as the reproduction condition, the time control module 15 does not update the count value.

[0168] The data request module 16 generates a query according to the specified time represented by the counter value. The data request module 16 uses the query to acquire performance values ​​of multiple variables at the specified time from the time series data 11. Furthermore, the data request module 16 uses the query to acquire a frame at the specified time from the video data 12. The data request module 16 outputs the acquired performance values ​​of the multiple variables to the waveform display module 41 and the program display module 42. The data request module 16 outputs the acquired frame to the video playback module 43.

[0169] The waveform display module 41 generates data for a window including a graph showing the transition of the variable values ​​in the target period based on the actual values ​​of the multiple variables received from the data reproduction module 14, and outputs the data to the display controller 105. As a result, a window including a graph showing the transition of the variable values ​​in the target period is displayed on the display 120.

[0170] The program display module 42 generates data for a window showing the execution status of the control program in the target period based on the performance values ​​of the multiple variables received from the data reproduction module 14, and outputs the data to the display controller 105. As a result, a window showing the execution status of the control program in the target period is displayed on the display 120.

[0171] The video playback module 43 generates data for a window in which the video captured during the target period is played, based on the frames received from the data reproduction module 14. The video playback module 43 outputs the data to the display controller 105. As a result, the window in which the video captured during the target period is played is displayed on the display 120.

[0172] Fig. 21 is a diagram showing an example of a screen displayed on the display. As shown in Fig. 21, an operation window 60, a window 81 including a graph showing the transition of the value of a variable, a window 82 showing the execution status of a control program, and a window 83 in which a video is played are displayed side by side on the display 120.

[0173] The user can check the transition of the variable values, the execution status of the control program, and videos around the abnormal time t_b identified by the first analysis process or the second analysis process by operating the operation window 60. This makes it easier for the user to search for the cause of the abnormal operation of the monitored object 300.

[0174] <Modification> In the above description, the information processing system 100 is configured by one PC 100A. However, the information processing system 100 may be configured by multiple computers. That is, the information processing system 100 may include multiple processors. Alternatively, the information processing system 100 may include a hardware environment other than a general-purpose PC, such as a virtual machine on a cloud.

[0175] §3 Supplementary Note As described above, the present embodiment includes the following disclosure.

[0176] (Configuration 1) An information processing system (100, 100A) that performs information processing using time series data (11) of actual values ​​of a plurality of variables that represent the state of a monitoring target (300, 300A, 300B), a simulator (20-29, 101) that simulates the behavior of a virtual model corresponding to the monitoring target (300, 300A, 300b) in a virtual space using actual values ​​of one or more second variables excluding a first variable among the plurality of variables; An information processing system (100, 100A) comprising: an identification module (30, 101) that identifies an anomalous time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time series data exceeds a standard.

[0177] (Configuration 2) The information processing system (100, 100A) according to configuration 1, further comprising an output module (32, 101) that outputs data indicating the list of abnormal times.

[0178] (Configuration 3) The monitored object (300, 300A) includes a first sensor (340) that detects the presence of an object (3); The first variable represents a detection result by the first sensor (340); The simulator (20 to 28, 101) 3. The information processing system (100, 100A) according to configuration 1 or 2, which calculates a virtual value of the first variable based on a movement of a virtual object corresponding to the target object (3) in the virtual space.

[0179] (Configuration 4) The monitored object (300A) includes a visual sensor (330), the one or more second variables include a specific variable indicative of an actual position of the object (3) measured by the visual sensor (330); the virtual model includes a virtual sensor corresponding to the first sensor (340); The simulator (20 to 28, 101) placing the virtual object in the virtual space at a virtual initial position corresponding to a real position represented by an actual result value of the specific variable; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system (100, 100A) according to configuration 3, further comprising: a virtual value of the first variable calculated based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

[0180] (Configuration 5) the virtual model includes a virtual sensor corresponding to the first sensor (340); The simulator (20 to 28, 101) Identifying an actual position of the object (3) based on an actual image obtained from a camera (332) that photographs the monitored object (300A); In the virtual space, the virtual object is placed at a virtual initial position corresponding to the real position; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system (100, 100A) according to configuration 3, further comprising: a virtual value of the first variable calculated based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

[0181] (Configuration 6) The first sensor (340) detects the presence of the object (3) at a first location (315) of the monitored object (300A); The monitoring target (300A) is a second sensor (345) configured to detect the presence of the object (3) at a second location (314) upstream of the first location (315); The one or more second variables include a specific variable representing a detection result by the second sensor (345), the virtual model includes a virtual sensor corresponding to the first sensor (340); The simulator (20 to 28, 101) placing the virtual object at a virtual initial position corresponding to the second location in the virtual space based on the actual value of the specific variable indicating the presence of the object; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system (100, 100A) according to configuration 3, further comprising: a virtual value of the first variable calculated based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

[0182] (Configuration 7) The first variable is a position, a velocity, or a torque of a moving part (352) included in the monitored object (300B), The one or more second variables include a variable indicating a command value for the movable part (352), In the information processing system (100, 100A) according to configuration 1 or 2, the simulator (20 to 27, 29) calculates a virtual value of the first variable of a virtual movable part corresponding to the movable part (352) based on the command value.

[0183] (Configuration 8) An information processing method using time series data (11) of actual values ​​of a plurality of variables representing a state of a monitoring target (300, 300A, 300B), One or more processors (101) simulate (S5, S6, S36) the behavior of a virtual model corresponding to the monitoring target (300, 300A, 300B) in a virtual space using actual values ​​of one or more second variables excluding the first variable among the plurality of variables; The one or more processors (101) identify (S11, S17, S42) an anomalous time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time series data (11) exceeds a standard.

[0184] (Configuration 9) A program for causing a computer to execute an information processing method using time series data (11) of actual values ​​of a plurality of variables representing the state of a monitoring target (300, 300A, 300B), The information processing method includes: simulating (S5, S6, S36) the behavior of a virtual model corresponding to the monitoring target (300, 300A, 300B) in a virtual space using actual values ​​of one or more second variables excluding the first variable among the plurality of variables; identifying an abnormal time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time-series data (11) exceeds a reference value (S11, S17, S42).

[0185] Although the embodiment of the present invention has been described, the embodiment disclosed herein should be considered as illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, and it is intended to include all modifications within the scope and meaning equivalent to the claims. [Explanation of symbols]

[0186] 1 manufacturing system, 3 work, 10 database, 11 time series data, 12 video data, 13 time designation module, 14 data reproduction module, 15 time control module, 16 data request module, 20 simulator, 21 operation setting module, 21a correspondence information, 22 initial information setting module, 22a initial information, 23 variable selection module, 24 operation reproduction module, 25 virtual work generation module, 26 script execution module, 27 virtual space construction module, 28 virtual sensor module, 29 axis variable value calculation module, 30 identification module, 32 output module, 41 waveform display module, 42 program display module, 43 video playback module, 50, 81, 82, 83 window, 51, 52 display area, 60 operation window, 61 cursor, 62, 66 input field, 63 play button, 64 frame rewind button, 65 frame forward button, 100 Information processing system, 100A PC, 101 processor, 102, 206 memory, 103, 208 storage, 104 input interface, 105 display controller, 106 communication interface, 107, 270 reader / writer, 110 input device, 120 display, 131 development program, 132 viewer program, 133 data reproduction program, 134 simulation program, 200 control device, 201 power supply circuit, 202 CPU, 204 chip set, 207 buffer area, 211 system program, 213 scheduler program, 214 program data set, 250 USB controller, 252 field network controller, 252a, 254a connector, 254 network controller, 260 timer, 300, 300A, 300B monitored object, 310 conveying system, 312 encoder, 313 area, 314, 315 Location, 320 robot, 330 visual sensor, 332 camera, 340, 345 object detection sensor, 350 electric cylinder, 352 moving part, 420 first member, 430 second member, 500 storage medium, 600 network camera.

Claims

1. An information processing system that performs information processing using time series data of actual values ​​of a plurality of variables that represent a state of a monitoring target, a simulator that simulates a behavior of a virtual model corresponding to the monitoring target in a virtual space by using actual values ​​of one or more second variables excluding a first variable among the plurality of variables; an identification module that identifies an anomalous time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time-series data exceeds a standard.

2. The information processing system according to claim 1 , further comprising an output module that outputs data indicating the list of abnormal times.

3. The monitored object includes a first sensor for detecting the presence of an object; the first variable represents a detection result by the first sensor; The simulator comprises: The information processing system according to claim 1 , further comprising: a virtual value calculating unit that calculates the virtual value of the first variable based on a movement of a virtual object corresponding to the target object in the virtual space.

4. The monitored object includes a visual sensor; the one or more second variables include a specific variable indicative of an actual position of the object measured by the vision sensor; the virtual model includes a virtual sensor corresponding to the first sensor; The simulator comprises: placing the virtual object in the virtual space at a virtual initial position corresponding to a real position represented by an actual result value of the specific variable; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system according to claim 3 , further comprising: a virtual value calculating unit that calculates the virtual value of the first variable based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

5. the virtual model includes a virtual sensor corresponding to the first sensor; The simulator comprises: Identifying an actual position of the target based on an actual image obtained from a camera that photographs the target; In the virtual space, the virtual object is placed at a virtual initial position corresponding to the real position; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system according to claim 3 , further comprising: a virtual value calculating unit that calculates the virtual value of the first variable based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

6. the first sensor detects the presence of the object at a first location of the monitored object; the monitored object includes a second sensor configured to detect the presence of the object at a second location upstream from the first location; the one or more second variables include a specific variable representing a detection result by the second sensor; the virtual model includes a virtual sensor corresponding to the first sensor; The simulator comprises: placing the virtual object at a virtual initial position corresponding to the second location in the virtual space based on the actual value of the specific variable indicating the presence of the object; In the virtual space, the virtual object is moved from the virtual initial position in accordance with a behavior of the virtual model; The information processing system according to claim 3 , further comprising: a virtual value calculating unit that calculates the virtual value of the first variable based on a positional relationship between the virtual object and the virtual sensor in the virtual space.

7. the first variable is a position, a velocity, or a torque of a moving part included in the monitored object; the one or more second variables include a variable indicating a command value for the movable part; The information processing system according to claim 1 , wherein the simulator calculates a virtual value of the first variable of a virtual movable part corresponding to the movable part based on the command value.

8. An information processing method using time series data of actual values ​​of a plurality of variables representing a state of a monitored object, comprising: One or more processors simulate a behavior of a virtual model corresponding to the monitoring target in a virtual space using actual values ​​of one or more second variables excluding the first variable among the plurality of variables; and identifying an anomalous time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time series data exceeds a standard.

9. A program for causing a computer to execute an information processing method using time series data of actual values ​​of a plurality of variables representing a state of a monitoring target, The information processing method includes: simulating a behavior of a virtual model corresponding to the monitoring target in a virtual space by using actual values ​​of one or more second variables excluding a first variable among the plurality of variables; identifying an anomalous time at which a difference between a virtual value of the first variable generated in the virtual model and an actual value of the first variable indicated by the time-series data exceeds a reference value.

Images