Problem analysis assistance program storage medium, problem analysis assistance device, and problem analysis assistance method

Abstract

The problem analysis support program causes the computer to execute at least one of the process steps, the log data acquisition step, the action reproduction step, the three-dimensional data display step, and the time synchronization step among the program action display step, the waveform display step, and the dynamic image display step. The log data acquisition step acquires log data that records input and output data of the state and the control signal of the operation portion in chronological order. The action reproduction step generates action reproduction simulation data that reproduces an action of a hypothetical control object based on the log data. The three-dimensional data display step displays the action reproduction simulation data as three-dimensional data on the display portion. The time synchronization step synchronizes the time between the three-dimensional data displayed by the three-dimensional data display step and the data displayed by the process step. In the three-dimensional data display step, the three-dimensional data is displayed on the display portion based on display setting information.

Description

Technical Field

[0001] This invention relates to a problem analysis assistance program storage medium, a problem analysis assistance device, and a problem analysis assistance method that assist in the analysis of problems of a controlled object. Background Technology

[0002] Patent Document 1 discloses a fault analysis aid device that assists in analyzing problems that occur in production equipment, including control devices and devices controlled by those control devices. The fault analysis aid device described in Patent Document 1 includes a control information recording unit and an analog unit. The control information recording unit creates a log file of image information, audio signals, and control information of the manufacturing equipment based on audio signals collected by a microphone, image signals captured by a camera, and control signals output from a programmable controller (PLC). The analog unit reproduces the image information and outputs it to a display, reproduces the audio signals and outputs them to a speaker, simulates the operation of the PLC from the input contacts of the control information, compares the simulation results with the output contacts of the control information, and detects any differences. The analog unit outputs the simulated internal information of the PLC to the display in ladder diagram form. The analog unit keeps the output image information, audio information, and the simulated internal information of the PLC approximately synchronized.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2000-250775 Summary of the Invention

[0004] However, according to the aforementioned prior art, the image information is captured from a specific direction relative to the production equipment. Therefore, the image information output by the analog unit can only confirm the state from that specific direction. Furthermore, no image information is obtained regarding the production equipment outside the camera's field of view. Thus, there is a problem that it is difficult to grasp the entirety of the production equipment. Additionally, there is a problem that multiple cameras must be installed to observe the state from multiple directions.

[0005] The present invention was proposed in view of the above situation, and its purpose is to provide a problem analysis auxiliary program that can confirm the state of the problem area of ​​the production equipment from any direction without setting up multiple cameras.

[0006] To address the aforementioned issues and achieve the objectives, the present invention provides a problem analysis assistance program for analyzing problems involving controlled objects with operating parts. This program enables a computer to execute at least one of the following display steps: a program action display step, a waveform display step, and a dynamic image display step. Specifically, it includes a processing step, a log data acquisition step, an action reproduction step, a three-dimensional data display step, and a time synchronization step. The log data acquisition step acquires log data that records the state of the operating part and the input / output data of control signals in a time sequence. The action reproduction step generates action reproduction simulation data based on the log data, reproducing the actions of a hypothetical controlled object. The three-dimensional data display step displays the action reproduction simulation data as three-dimensional data on the display unit. The program action display step displays the execution status of the control program executed by the control device on the display unit from the log data. The waveform display step displays the state of the operating part and the input / output data of control signals as waveforms relative to time on the display unit from the log data. The dynamic image display step displays dynamic image data obtained by capturing images of the state of the controlled object on the display unit. The time synchronization step synchronizes the time between the three-dimensional data reproduced in the action reproduction step, displayed in the three-dimensional data display step, and the data displayed in the processing step. In the three-dimensional data display step, based on display setting information, the three-dimensional data after changing the viewpoint position is displayed on the display unit. The display setting information indicates the viewpoint position when the imaginary control object is displayed as three-dimensional data on the display unit.

[0007] The effects of the invention

[0008] The problem analysis assistance program involved in this invention has the following effect: without setting up multiple cameras, it is possible to confirm the state of the problem area in the production equipment from any direction. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating an example of the structure of the problem analysis assistance system involved in Implementation 1.

[0010] Figure 2 This is a diagram showing an example of a control object set as the parsing object in the problem parsing assistance system involved in Implementation 1.

[0011] Figure 3 This is a diagram showing an example of a program action display screen according to Embodiment 1.

[0012] Figure 4 This is a diagram showing an example of a waveform display screen according to Embodiment 1.

[0013] Figure 5 This is a diagram showing an example of a dynamic image display screen according to Embodiment 1.

[0014] Figure 6 This is a diagram showing an example of a three-dimensional data display screen according to Embodiment 1.

[0015] Figure 7 This is a flowchart illustrating an example of the sequence of problem-solving assistance methods involved in Implementation Method 1.

[0016] Figure 8 This is a flowchart illustrating an example of the sequence of problem-solving assistance methods for the specified time scenario involved in Implementation 1.

[0017] Figure 9 It means Figure 2 A side view of an example of the tray used by the device shown.

[0018] Figure 10 It means in Figure 2 The side view shows an example of a workpiece being normally placed in a tray used by the equipment.

[0019] Figure 11 This is a diagram representing an example of log data when the device is in a normal state.

[0020] Figure 12 It means based on Figure 11 An example of a waveform display generated from log data.

[0021] Figure 13 This is a diagram illustrating an example of the workpiece's configuration on a pallet in the event of a problem occurring in the equipment.

[0022] Figure 14 This is a diagram representing an example of log data in the event of a problem occurring in the device.

[0023] Figure 15 It means based on Figure 14 An example of a waveform display generated from log data.

[0024] Figure 16 This is a diagram schematically illustrating an example of a waveform display screen and a three-dimensional data display screen in the problem analysis auxiliary device involved in Embodiment 1.

[0025] Figure 17 This is a block diagram illustrating an example of the structure of the problem analysis assistance system involved in Implementation 2.

[0026] Figure 18 This is a block diagram illustrating another example of the structure of the problem analysis assistance system involved in Implementation 2.

[0027] Figure 19 This is a block diagram illustrating an example of the structure of the problem analysis assistance system involved in Implementation 3.

[0028] Figure 20 This is a flowchart illustrating an example of the sequence of problem-solving assistance methods in Implementation 3, where a specified time is specified.

[0029] Figure 21 This is a block diagram illustrating an example of the structure of the problem analysis assistance system involved in Implementation 4.

[0030] Figure 22 This is a flowchart illustrating an example of the sequence of methods for saving log data in the control device of the problem analysis auxiliary system involved in Implementation 4.

[0031] Figure 23 This is a block diagram illustrating an example of the structure of the problem analysis assistance system involved in Implementation 5.

[0032] Figure 24 This is a diagram illustrating an example of the hardware structure in the case where the functions of the problem-solving assistance device involved in embodiments 1 to 6 are implemented by a computer system. Detailed Implementation

[0033] The problem-solving assistance program storage medium, problem-solving assistance device, and problem-solving assistance method involved in the embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0034] Implementation method 1.

[0035] Figure 1 This is a block diagram illustrating an example of the structure of the problem-solving assistance system according to Embodiment 1. The problem-solving assistance system 1 includes a control object 10, a control device 20, and a problem-solving assistance device 30.

[0036] The controlled object 10 operates according to the control implemented by the control device 20 and has a moving part. In Embodiment 1, the controlled object 10 is the object of problem analysis. An example of the controlled object 10 is a factory production line or apparatus. An example of the moving part is a belt conveyor, a robotic arm, or a robotic hand.

[0037] Figure 2 This diagram illustrates an example of a control object designated as the analysis object in the problem analysis assistance system described in Embodiment 1. This control object 10 is a device used to test the action of returning a workpiece 100 to the tray 110 after the size of the workpiece 100 is measured by a robotic arm 133, which grips the workpiece 100 placed on the tray 110.

[0038] The controlled object 10 includes a tray 110, a conveying mechanism 120, a robot arm mechanism 130, a sensor 141, a sensor 142, and an inspection unit 150.

[0039] The pallet 110 places the workpiece 100 to be inspected. The conveying mechanism 120 moves the pallet 110 between a first location 121 and a second location 122. Here, the conveying mechanism 120 has a guide rail 123 connecting the first location 121 and the second location 122 on the support table 101, and a moving mechanism (not shown) that moves the pallet 110 on the guide rail 123.

[0040] The robot arm mechanism 130 includes: a rotatable shaft 131 vertically disposed at the center of the support platform 101; a main body 132 connected to the shaft 131 and functioning as an arm; and a robot arm 133 disposed at one end of the main body 132. Figure 2 In this example, axis 131 rotates to allow robot arm 133 to move between a first location 121 and a second location 122. Furthermore, robot arm 133 can grasp workpiece 100 placed on pallet 110 at the first location 121 and can place the grasped workpiece 100 onto pallet 110 at the second location 122. Figure 2 In the example, the robot hand mechanism 130, which includes the robot hand 133, becomes the operating part.

[0041] Sensor 141 detects the presence of tray 110 at a first location 121. Sensor 142 detects the presence of tray 110 at a second location 122. In one example, sensors 141 and 142 are transmittance-type or split-type optical sensors having light-emitting elements 141A and 142A and light-receiving elements 141B and 142B.

[0042] The inspection unit 150 is positioned between the first location 121 and the second location 122 to inspect the workpiece 100.

[0043] exist Figure 2 In the experimental apparatus shown, the robotic arm mechanism 130 grasps the workpiece 100 from the tray 110 at a first location 121, rotates it to move the workpiece 100 to the inspection unit 150. The inspection unit 150 inspects the workpiece 100. Then, the robotic arm mechanism 130 rotates from the inspection unit 150 to a second location 122, and places the workpiece 100 on the tray 110 at the second location 122. Then, the robotic arm mechanism 130 rotates back to the first location 121. The above actions are repeated in the experimental apparatus.

[0044] Return to Figure 1The control device 20 is electrically connected to the controlled object 10 and controls the operation of the controlled object 10 according to a predetermined control program. Furthermore, in Embodiment 1, the control device 20 is a device that records controlled object information, including the state of the operating unit, input and output data of control signals between the controlled object and the controlled object 10, and the actual operation results of the controlled object 10, in a time sequence. The control device 20 includes an instrument control unit 21, a log recording unit 22, and a dynamic image recording unit 23.

[0045] The instrument control unit 21 stores a control program for controlling the controlled object 10 and controls the controlled object 10 based on the control program. In one example, the instrument control unit 21 has: an input unit that receives information from the controlled object 10; an arithmetic processing unit that performs arithmetic processing according to the control program; and an output unit that outputs the result of the arithmetic processing unit to the controlled object 10. The instrument control unit 21 obtains input data representing the state of the controlled object 10 from the input unit, calculates output data for controlling the controlled object 10 by performing arithmetic processing on the input data according to the control program, and outputs the output data to the controlled object 10 from the output unit. An example of the instrument control unit 21 is a programmable logic controller or a robot controller.

[0046] The log recording unit 22 acquires time-series data, i.e., log data, obtained by observing input and output data, including input and output data exchanged between the instrument control unit 21 and the controlled object 10, and status data indicating the state of the operating part of the controlled object 10 over time. An example of input data is the detection values ​​obtained from sensors 141 and 142 of the controlled object 10 controlled by the instrument control unit 21, or the values ​​obtained after calculation or processing of the detection values ​​of sensors 141 and 142. An example of output data is the execution result obtained by the instrument control unit 21 using the input data to execute the control program, which is a control signal output to the controlled object 10. An example of status data, in the case where the operating part is a robot arm mechanism 130, is the axis value of the robot arm mechanism 130, the rotational speed of the motor, and the position of the workpiece 100 obtained by sensors 141 and 142. Figure 2 The angle of shaft 131 in the experimental equipment is an example of state data, which is stored by the instrument control unit 21 of the control device 20. Hereinafter, input data and output data will also be collectively referred to as input-output data. That is, log data is data that records the input and output data of the operating unit's state, the controlled object 10, and the control device 20 that controls the controlled object 10 in a time sequence.

[0047] The motion image recording unit 23 records the motion image data captured by the shooting unit 71 of the shooting control object 10. The motion image data is an example of motion recording data and contains the time information of the shooting.

[0048] In one example, if the controlled object 10 includes a production line or apparatus for manufacturing products, the controlled object 10 and the control device 20 are combined to form a production equipment.

[0049] The problem analysis assistance device 30 is a device that synchronizes and displays the simulation results of the hypothetical controlled object's actions based on log data with the actual controlled object's action state reproduced using at least one of the log data and dynamic image data, assisting in the analysis of problems when a problem occurs in the controlled object 10. In one example, the problem analysis assistance device 30 can communicate with the control device 20. The problem analysis assistance device 30 includes a display unit 31, a log data acquisition unit 32, a time synchronization unit 33, a program action display processing unit 34, a waveform display processing unit 35, a dynamic image display processing unit 36, an action reproduction unit 37, a display setting information storage unit 38, a three-dimensional data display processing unit 39, and a time acquisition unit 40.

[0050] Display unit 31 visually displays information used by problem analysis assisting device 30 to assist in problem analysis. An example of display unit 31 is a display device such as a liquid crystal display. Problem analysis assisting device 30 may have one display unit 31 or multiple display units 31. In one example, problem analysis assisting device 30 may have multiple display units 31 corresponding to each of the following: program action display processing unit 34, waveform display processing unit 35, dynamic image display processing unit 36, and three-dimensional data display processing unit 39 (described later).

[0051] The log data acquisition unit 32 acquires log data from the log recording unit 22 of the control device 20. In one example, the log data acquisition unit 32 receives an instruction from a user of the problem analysis assistance device 30 and acquires log data from the log recording unit 22. The user may be an operator of the problem analysis assistance device 30, a manager who oversees the actions of the controlled object 10, or the like. The log data acquisition unit 32 can acquire log files centrally from the log recording unit 22 of the control device 20 via a file, or it can acquire log files through real-time communication such as data streams.

[0052] The timing synchronization unit 33 synchronizes the timing of log data acquired by the log data acquisition unit 32 and motion image data acquired by the motion image recording unit 23 of the control device 20. Specifically, the timing synchronization unit 33 synchronizes the timing of the log data transmitted to the motion reproduction unit 37, the program motion display processing unit 34, and the waveform display processing unit 35, and the motion image data transmitted to the motion image display processing unit 36. The timing synchronization unit 33 adjusts the timing of the data transmitted to each processing unit by referring to the timing information contained in the log data and the timing information contained in the motion recording data.

[0053] The program operation display processing unit 34 displays the execution status of the control program executed by the control device 20 on the display unit 31. Specifically, the program operation display processing unit 34 performs the following processing: it displays a program operation display screen on the display unit 31, indicating the execution status of the control program executed by the instrument control unit 21. The control program executed by the instrument control unit 21 is usually a ladder diagram program. This ladder diagram program is described using relay symbol language based on the consideration of relay control circuits. Therefore, the program operation display processing unit 34 generates a program operation display screen that displays one of the control programs, namely the ladder diagram program, in the form of a circuit diagram. The program operation display processing unit 34 generates a program operation display screen that displays the program structure elements such as contacts and coils of the control program executed by the instrument control unit 21 in the form of a circuit program. In addition, the program operation display processing unit 34 obtains the values ​​of input / output data or status data corresponding to each program structure element from the log data at a specified time and displays them near the program structure elements on the program operation display screen. Furthermore, the on / off state of the contacts can be displayed by changing the display method on the program action display screen. The technique of displaying the execution status and the values ​​of input / output data or status data in the ladder diagram program involved in the program action display processing unit 34 can be implemented using known methods. The program action display processing unit 34 displays the generated program action display screen on the display unit 31. In addition, the program action display processing unit 34 saves the same control program as the control program executed by the instrument control unit 21.

[0054] Figure 3 This is a diagram illustrating an example of a program action display screen according to Embodiment 1. The program action display screen 200 has a program action display area 210 for displaying a ladder diagram program and a time operation area 220 for receiving operations related to the time displayed by the program action display area 210.

[0055] In the program action display area 210, the read-in program is displayed in the form of a ladder diagram program, i.e., a circuit form. Usually, it is not possible to display the entire ladder diagram program within the program action display area 210. Therefore, a horizontal scroll bar 211 that changes the displayed area in the horizontal direction and a vertical scroll bar 212 that changes the displayed area in the vertical direction are provided in the program action display area 210.

[0056] The timing operation area 220 is an area that displays the playback time of the program displayed in the program action display area 210 and operation buttons related to program playback. The timing operation area 220 includes: a playback operation button 226, which receives operations related to the playback of circuit operations in the program action display area 210; and a slider 227, which can indicate the approximate position of the playback time and specify any playback time. The state of the time corresponding to the position of the slider 227 is displayed in the program action display area 210. Additionally, the timing operation area 220 has a timing display section 228 that displays the time of the program being played in the program action display area 210. The slider 227 is an example of a timing indicator.

[0057] Return to Figure 1 The waveform display processing unit 35 displays the status of the operating unit and the input / output data of the control signals from the log data on the display unit 31 as a waveform relative to time. Specifically, the waveform display processing unit 35 generates a waveform display screen that plots the values ​​of the input / output data of the controlled object 10 or the status data of the operating unit in the log data relative to time, and displays the generated waveform display screen on the display unit 31.

[0058] Figure 4 This diagram illustrates an example of a waveform display screen according to Embodiment 1. The waveform display screen 230 has a waveform display area 240 for displaying waveform data 231. The waveform display area 240 displays the waveform data 231 of the input / output data of the control object 10 and the status data of the operating unit, which are included in the log data. Generally, it is impossible to display all the waveform data 231 of the input / output data and status data within the waveform display area 240; therefore, a horizontal scroll bar 241 that changes the display area horizontally and a vertical scroll bar 242 that changes the display area vertically are provided in the waveform display area 240. The horizontal axis, or time axis, of the multiple waveform data 231 indicates a shared time period. The waveform display area 240 has a cursor 243 indicating the time. The cursor 243 is provided across the multiple waveform data 231 arranged vertically and can be moved along the time axis. The cursor 243 is an example of a time indicator. The cursor 243 can be specified by the user for any given time.

[0059] Return to Figure 1 The dynamic image display processing unit 36 ​​displays the dynamic image data obtained by capturing the state of the controlled object 10 on the display unit 31. Specifically, the dynamic image display processing unit 36 ​​generates a dynamic image display screen that reproduces the dynamic image data according to the format of the dynamic image data recorded in the dynamic image recording unit 23, and displays the generated dynamic image display screen on the display unit 31.

[0060] Figure 5 This diagram illustrates an example of a dynamic image display screen according to Embodiment 1. The dynamic image display screen 250 has a dynamic image display area 260 and a time operation area 270. Dynamic image data is reproduced in the dynamic image display area 260. The time operation area 270 is an area that receives operations related to the reproduction of the dynamic image data reproduced through the dynamic image display area 260, and is an area that indicates the reproduction time of the dynamic image data reproduced through the dynamic image display area 260. The time operation area 270 includes: an operation button 271, which is related to the reproduction of the dynamic image data in the dynamic image display area 260; and a slider 272, which can indicate the approximate position of the reproduction time and specify any reproduction time. The state of the time corresponding to the position of the slider 272 is displayed in the dynamic image display area 260. Additionally, the time operation area 270 has a time display section 273 that displays the time of the reproduced dynamic image data. The slider 272 is an example of a time indicator.

[0061] Return to Figure 1 The motion reproduction unit 37 generates motion reproduction simulation data that reproduces the actions of a hypothetical control object corresponding to the control object 10 based on log data. Specifically, the motion reproduction unit 37 generates a hypothetical control object with the same structure as the control object 10 on a computer, and generates motion reproduction simulation data that reproduces the actions of the hypothetical control object based on log data. That is, the motion reproduction unit 37 uses log data to simulate the actions of the hypothetical control object. The motion reproduction simulation data is data in the form of a three-dimensional representation of the hypothetical control object, and by specifying angles, it is possible to display the hypothetical control object in any direction. The hypothetical control object only needs to reproduce the main parts of the control object 10. However, when the hypothetical control object has the same structure as the control object 10 down to the details, the simulation result obtained by the motion reproduction unit 37 is closer to the actual result.

[0062] The display setting information storage unit 38 stores setting information indicating the position of the viewpoint when a hypothetical controlled object is displayed as three-dimensional data on the display unit 31. Specifically, the display setting information storage unit 38 stores display setting information related to the position of the viewpoint when the three-dimensional data of a hypothetical controlled object, which is used to display motion reproduction simulation data in three dimensions, is displayed on the display unit 31. In one example, the position of the viewpoint when displaying the three-dimensional data of the hypothetical controlled object is predetermined by default. With the position of the hypothetical controlled object fixed, the viewpoint can be rotated in the horizontal plane at an angle specified by the user, or in the vertical plane at an angle specified by the user. In this specification, the angle at which the viewpoint is rotated in the horizontal plane is called the azimuth angle, and the angle at which the viewpoint is rotated in the vertical plane is called the polar angle. The display angle, including the azimuth angle and polar angle input by the user through an input unit (not shown), is called display setting information and is stored in the display setting information storage unit 38. The display setting information storage unit 38 stores display setting information including predetermined display angles by default.

[0063] Furthermore, the display setting information storage unit 38 can specify not only the display angle, but also the area of ​​the display control object 10. In one example, the display setting information is displayed in a way that the portion specified by the user is magnified, and can include the position and magnification of the hypothetical control object in addition to the display angle.

[0064] The 3D data display processing unit 39 displays the motion reproduction simulation data as 3D data on the display unit 31. Specifically, the 3D data display processing unit 39 generates a 3D data display screen that displays the motion reproduction simulation data generated by the motion reproduction unit 37 in 3D based on display setting information, and displays the generated 3D data display screen on the display unit 31. At this time, the 3D data display processing unit 39 obtains display setting information from the display setting information storage unit 38, and generates a 3D data display screen that displays the motion reproduction simulation data according to the position of the viewpoint when displaying the hypothetical control object set in the display setting information.

[0065] Figure 6This diagram illustrates an example of a three-dimensional data display screen according to Embodiment 1. The three-dimensional data display screen 280 has a three-dimensional data display area 290 and a time operation area 300. In the three-dimensional data display area 290, three-dimensional data, in which motion reproduction simulation data is displayed in three dimensions, is reproduced. The time operation area 300 is an area that receives operations related to the reproduction of the three-dimensional data reproduced by the three-dimensional data display area 290, and is an area that indicates the time of reproduction of the three-dimensional data reproduced by the three-dimensional data display area 290. The time operation area 300 includes: an operation button 301, which is related to the reproduction of the three-dimensional data in the three-dimensional data display area 290; and a slider 302, which can indicate the approximate position of the reproduction time and specify any reproduction time. The state of the time corresponding to the position of the slider 302 is displayed in the three-dimensional data display area 290. Additionally, the time operation area 300 has a time display section 303 that displays the time of the reproduced motion image data. The slider 302 is an example of a time indicator.

[0066] When any of the time indicators on the program action display screen 200, waveform display screen 230, dynamic image display screen 250, and 3D data display screen 280 are operated, the time acquisition unit 40 acquires the time specified by the operated time indicator, i.e., the specified time, and transmits the specified time to the time synchronization unit 33. The acquisition of the specified time through the time indicator, i.e., the cursor 243 and the sliders 227, 272, and 302, is achieved using known image display technology.

[0067] If the time synchronization unit 33 receives a specified time from the time acquisition unit 40, it transmits the log data of the transmitted time to the motion reproduction unit 37, the program motion display processing unit 34, and the waveform display processing unit 35, and transmits the motion image data to the motion image display processing unit 36. As a result, the motion reproduction unit 37 generates motion reproduction simulation data for the transmitted time, and displays the three-dimensional data in the three-dimensional data display processing unit 39. Furthermore, the program motion display processing unit 34 and the waveform display processing unit 35 each display the program and waveform data 231 for the transmitted time, and the motion image display processing unit 36 ​​displays the motion image data for the transmitted time. Consequently, synchronization is achieved between the three-dimensional data displayed by the three-dimensional data display processing unit 39, the circuit displayed by the program motion display processing unit 34, the waveform data 231 displayed by the waveform display processing unit 35, and the motion image data displayed by the motion image display processing unit 36.

[0068] In addition, Figure 1The diagram shows a problem-solving assistance device 30 with a program action display processing unit 34, a waveform display processing unit 35, and a dynamic image display processing unit 36. However, it is sufficient to have at least one of these display processing units. In this case, the time acquisition unit 40 acquires the specified time as the specified time when the time of the three-dimensional data displayed by the three-dimensional data display processing unit 39 or the time of the data displayed by at least one of the display processing units (processors) 34, 35, and 36 is specified, and outputs it to the time synchronization unit 33. The time synchronization unit 33 transmits log data corresponding to the specified time to the action reproduction unit 37, and transmits log data or dynamic image data corresponding to the specified time to at least one of the processing units 34, 35, and 36. That is, the time synchronization unit 33 synchronizes the time between the three-dimensional data reproduced by the motion reproduction unit 37 and displayed by the three-dimensional data display processing unit 39 and the data displayed by at least one of the processing units, namely the program motion display processing unit 34, the waveform display processing unit 35 and the dynamic image display processing unit 36.

[0069] Next, the problem analysis assistance methods in Problem Analysis Assistance System 1 will be explained. Figure 7 This is a flowchart illustrating an example of the sequence of the problem analysis assistance method involved in Implementation Method 1. Here, the user activates the control device 20, the log recording unit 22 records log data, and the dynamic image recording unit 23 records dynamic image data.

[0070] The user of the problem analysis assistance device 30 specifies the execution of a problem analysis assistance method by operating the input unit (not shown). As a result, the log data acquisition unit 32 acquires log data from the log recording unit 22 of the control device 20 (step S11).

[0071] Next, the time synchronization unit 33 obtains dynamic image data from the dynamic image recording unit 23 (step S12), synchronizes the time of the log data and the dynamic image data, and transmits the log data to the motion reproduction unit 37, the program motion display processing unit 34 and the waveform display processing unit 35, and transmits the dynamic image data to the dynamic image display processing unit 36 ​​(step S13).

[0072] The program operation display processing unit 34 has the same control program as the control program pre-executed by the instrument control unit 21 of the control device 20. Using this control program and log data, it displays the actual operation status of the control program. The program operation display processing unit 34 displays the control program in circuit form and obtains values ​​corresponding to the program structure elements of the control program from the log data to generate the program operation display screen 200 to be displayed (step S14). Furthermore, the program operation display processing unit 34 displays the program operation display screen 200 on the display unit 31 (step S15).

[0073] The waveform display processing unit 35 generates a waveform display screen 230 (step S16) that plots the input / output data obtained from the instrument control unit 21 and the status data of the operating unit of the controlled object 10 relative to time, which are included in the log data. The waveform display screen 230 is then displayed on the display unit 31 (step S17). The waveform display screen 230 shows the signal status of the input / output data in the instrument control unit 21 of the log data, or the operation status of the operating unit. Additionally, the cursor 243 is positioned at the location of the transmitted time in the waveform display screen 230.

[0074] The dynamic image display processing unit 36 ​​generates a dynamic image display screen 250 that reproduces the dynamic image data transmitted from the dynamic image recording unit 23 of the control device 20 via the time synchronization unit 33 (step S18), and displays the dynamic image display screen 250 on the display unit 31 (step S19).

[0075] The motion reproduction unit 37 has pre-existing three-dimensional data of a hypothetical control object, which has the same structure as the control object 10. Using this three-dimensional data and log data, it performs simulation processing to reproduce the motion of the hypothetical control object. Furthermore, the motion reproduction unit 37 generates motion reproduction simulation data based on the log data to reproduce the motion of the hypothetical control object (step S20). The motion reproduction unit 37 uses log data with different times transmitted from the time synchronization unit 33 to reproduce the motion of the hypothetical control object.

[0076] The 3D data display processing unit 39 obtains display setting information from the display setting information storage unit 38 (step S21), and generates a 3D data display screen 280 that displays the motion reproduction simulation data according to the display setting information (step S22). The 3D data display processing unit 39 generates a 3D data display screen 280 with the viewpoint position moved by the default setting or a user-defined azimuth and polar angle, based on a direction preset for the control object 10. As described above, the motion reproduction simulation data generated by the motion reproduction unit 37 is 3D data, therefore, when displaying, the viewpoint position of the observation control object 10 can be set to any direction. Furthermore, the 3D data display processing unit 39 displays the 3D data display screen 280 on the display unit 31 (step S23).

[0077] The display times of the program action display screen 200 in step S15, the waveform display screen 230 in step S17, the dynamic image display screen 250 in step S19, and the three-dimensional data display screen 280 in step S23 are synchronized.

[0078] After steps S15, S17, S19, and S23, the process returns to step S13, and the above process is repeated. Thus, as time passes, the program action display screen 200, waveform display screen 230, motion image display screen 250, and 3D data display screen 280 are displayed on the display unit 31. That is, synchronized with the motion image display screen 250 reproduced by the motion image display processing unit 36, the program action display screen 200 shows the program's operation status, and the waveform display processing unit 35 shows the signal status of input / output data and the operation status of the operating unit. Furthermore, the 3D data display processing unit 39 displays the simulation results of reproducing the actions of a hypothetical controlled object on the computer based on log data. By referring to these display screens, the user can analyze the problematic parts of the controlled object 10.

[0079] Next, the processing of any of the program action display screen 200, waveform display screen 230, dynamic image display screen 250, and three-dimensional data display screen 280 when the user specifies the time will be explained. Figure 8 This is a flowchart illustrating an example of the sequence of problem-solving assistance methods for the specified time scenario involved in Implementation Method 1. Here, according to Figure 7 The display unit 31 is configured to display the program action display screen 200, waveform display screen 230, dynamic image display screen 250 and three-dimensional data display screen 280.

[0080] First, the user operates the time indicator of any one of the following screens: program action display screen 200, waveform display screen 230, dynamic image display screen 250, and 3D data display screen 280. The time indicator is a slider 227, 272, 302, or a cursor 243. The time acquisition unit 40 detects the operation of the time indicator, reads the specified time (step S31), and transmits the read specified time to the time synchronization unit 33 (step S32).

[0081] The time synchronization unit 33 transmits the received log data at the specified time to the program action display processing unit 34, the waveform display processing unit 35, and the action reproduction unit 37, and transmits the received dynamic image data at the specified time to the dynamic image display processing unit 36 ​​(step S33).

[0082] The program action display processing unit 34 generates a program action display screen 200 based on the received log data at a specified time (step S34), and displays the program action display screen 200 on the display unit 31 (step S35). The waveform display processing unit 35 generates a waveform display screen 230 based on the received log data at a specified time (step S36), and displays the waveform display screen 230 on the display unit 31 (step S37). The motion image display processing unit 36 ​​generates a motion image display screen 250 using the received motion image data at a specified time (step S38), and displays the motion image display screen 250 on the display unit 31 (step S39).

[0083] The motion reproduction unit 37 generates motion reproduction simulation data based on the received log data at a specified time, which enables the hypothetical controlled object to reproduce the motion (step S40).

[0084] The 3D data display processing unit 39 obtains display setting information from the display setting information storage unit 38 (step S41), and generates a 3D data display screen 280 that displays simulated data of motion reproduction according to the display setting information (step S42). Furthermore, the 3D data display processing unit 39 displays the 3D data display screen 280 on the display unit 31 (step S43).

[0085] Then, the time synchronization unit 33 transmits the log data of the next moment after the specified moment to the program action display processing unit 34, the waveform display processing unit 35, and the action reproduction unit 37, and transmits the motion image data of the next moment after the specified moment to the motion image display processing unit 36 ​​(step S44). Then, the process proceeds to steps S34, S36, S38, and S40. Moreover, the above-described motion state reproduction process is repeated.

[0086] Next, a specific example of the problem-solving assistance method in the problem-solving assistance system 1 according to Implementation Method 1 will be described. Here, an example of... Figure 2 The device shown is used to analyze the problem of the controlled object 10. Furthermore, here, the problem analysis auxiliary device 30 is configured without the program action display processing unit 34 and the dynamic image display processing unit 36.

[0087] Figure 9 It means Figure 2 The diagram shows a side view of an example tray used by the device. In tray 110, the direction of movement of tray 110 is defined as the X direction, the vertical direction as the Z direction, and the direction perpendicular to both the X and Z directions as the Y direction. Figure 2 In the illustrated device, the tray 110 has a recess 111 capable of holding the workpiece 100. For example... Figure 2 As shown, at either location 121 or 122, light-emitting elements 141A and 142A and light-receiving elements 141B and 142B are arranged on both sides of the tray 110 in the Y direction. Sensors 141 and 142 are used to detect the workpiece 100, and recesses 111 are provided such that the light paths of sensors 141 and 142 are not obstructed by components of the tray 110. In this example, the recess 111 is a groove extending in the Y direction. That is, at both locations 121 and 122, the light paths of sensors 141 and 142 are contained within the recess 111.

[0088] Figure 10 It means in Figure 2 The diagram shows a side view of an example of a workpiece normally placed on a tray used in the device. When the tray 110 is located at either the first location 121 or the second location 122, and the workpiece 100 is normally placed on the tray 110, the workpiece 100 is present in the optical path of sensors 141 and 142. That is, the light-receiving elements 141B and 142B of sensors 141 and 142 do not receive light from the light-emitting elements 141A and 142A; therefore, sensors 141 and 142 determine that the workpiece 100 is present at either the first location 121 or the second location 122. On the other hand, during the period when the light-receiving elements 141B and 142B continuously receive light from the light-emitting elements 141A and 142A, it is determined that the workpiece 100 is not present at either the first location 121 or the second location 122.

[0089] exist Figure 2 The following operations, steps 1 through 7, are performed in the equipment shown.

[0090] Step 1: The pallet 110, which carries the workpiece 100, is moved to the first location 121.

[0091] Step 2: If sensor 141 detects workpiece 100, it will activate the signal and output a signal.

[0092] Step 3: If sensor 141 outputs an on signal, the robot arm 133 of robot arm mechanism 130 will grasp the workpiece 100 at point 121.

[0093] Step 4: Move pallet 110 from location 121 to location 122.

[0094] Step 5: The robot arm mechanism 130 rotates, and after the inspection unit 150 performs the inspection of the workpiece 100, the workpiece 100 is further transported to the second location 122, where the workpiece 100 is placed on the tray 110 at the second location 122.

[0095] Step 6: The workpiece 100 is placed on the tray 110. If the sensor 142 detects the workpiece 100, it will turn on the signal and output it.

[0096] Step 7: If sensor 142 outputs an on signal, the robot arm mechanism 130 rotates and the robot arm 133 returns to the first location 121.

[0097] Here, the log recording unit 22 records the input signals from sensors 141 and 142 and the angle of the main axis, i.e., axis 131, of the robot hand mechanism 130 as log data in a time series. However, the value of the angle of axis 131 is not an angle, but a relative value represented by setting the case where the robot hand 133 of the robot hand mechanism 130 is present at the first location 121 to 0 and the case where it is present at the second location 122 to 100. Hereinafter, the angle of axis 131 will be referred to as the axis angle.

[0098] Figure 11 This is a diagram representing an example of log data when the device is in a normal operating state. For example... Figure 11 As shown, the log data in this case includes the time, the value of sensor 141, the value of sensor 142, and the axis angle. However, in sensors 141 and 142, "on" is marked as "1" and "off" is marked as "0".

[0099] Figure 12 It means based on Figure 11 An example of a waveform display generated from log data. Figure 12The waveform display screen 230 shows three waveform data points 231a, 231b, and 231c, along with a cursor 243. Waveform data 231a shows the signal status of sensor 141, waveform data 231b shows the signal status of sensor 142, and waveform data 231c shows the axial angle. The horizontal axis of these waveforms represents time.

[0100] Under normal circumstances Figure 12 The device operates as shown in waveform data 231a, 231b, and 231c. If sensor 141 is activated at point 121, the robot arm mechanism 130 rotates to point 122 and stops rotating there. If sensor 142 is activated at point 122, the robot arm mechanism 130 rotates in the opposite direction to point 121 and stops rotating there.

[0101] Here, in the fifth step, a problem occurs when the robot arm 133 of the robot arm mechanism 130 places the workpiece 100 onto the pallet 110. Figure 13 This is a diagram illustrating an example of the workpiece's configuration on a pallet in the event of a problem occurring in the equipment. Figure 13 In this process, at location 122, the position of tray 110 shifts, and workpiece 100 is placed on tray 110 without being embedded in the recess 111 of tray 110. As a result, the optical path of sensor 142 is not blocked by workpiece 100. That is, sensor 142 does not detect the presence of workpiece 100, and therefore sensor 142 continuously detects the disconnected state.

[0102] Figure 14 This is a diagram representing an example of log data in the event of a problem occurring in the device. Figure 15 It means based on Figure 14 An example of a waveform display generated from log data. Figure 15 The waveform display screen 230 shows three waveform data points 232a, 232b, and 232c, and a cursor 243. Waveform data 232a is a waveform showing the signal status of sensor 141, waveform data 232b is a waveform showing the signal status of sensor 142, and waveform data 232c is a waveform showing the axis angle. The horizontal axis of these waveforms indicates time.

[0103] Under normal circumstances, sensor 142 should be switched on during the time corresponding to step 6, from "11:11:17" to "11:11:18", but... Figure 14 and Figure 15In the middle, sensor 142 becomes disconnected. As a result, the robot hand mechanism 130 then continues in the state where the value of the axis angle at the second location 122 remains unchanged from the position of "100".

[0104] Next, the user confirms the problem. The user inputs log data confirmation instructions, etc., from the input section (not shown) of the problem analysis assistance device 30, thus starting the problem analysis assistance method.

[0105] The log data acquisition unit 32 acquires log data from the log recording unit 22 of the control device 20. At this time, the log data acquisition unit 32 preferably acquires log data from the period surrounding the time the problem occurred. The time synchronization unit 33 transmits the log data to the waveform display processing unit 35 and the motion reproduction unit 37. The waveform display processing unit 35 generates waveform data 232a, 232b, and 232c, including data from the sensors 141 and 142 and the axis angle, based on the log data. Figure 15 The waveform display screen 230 shown is displayed on the display unit 31. The motion reproduction unit 37 generates a waveform based on log data that has the same characteristics as the waveform display screen 230. Figure 2 The device, with the same structure as the one shown, reproduces motion simulation data of a hypothetical controlled object. The three-dimensional data display processing unit 39 generates a three-dimensional data display screen 280 containing three-dimensional data, which is displayed on the display unit 31. This three-dimensional data is obtained by changing the position of the viewpoint when displaying the hypothetical controlled object according to the display setting information specifying the motion reproduction simulation data. At this time, the time of the hypothetical controlled object displayed on the three-dimensional data display screen 280 is synchronized with the time specified by the time setting unit displayed on the waveform display screen 230.

[0106] Figure 16 This diagram schematically illustrates an example of a waveform display screen and a three-dimensional data display screen in the problem analysis auxiliary device involved in Embodiment 1. For example... Figure 16 As shown, the display unit 31 displays a waveform display screen 230 and a three-dimensional data display screen 280. The waveform display screen 230 shows... Figure 15 The waveform data 232a, 232b, and 232c are shown, along with a cursor 243 representing the time in the waveform data 232a, 232b, and 232c. The 3D data display screen 280 displays a 3D graph showing the result of simulating the motion state of a hypothetical controlled object at a certain moment. As described above, a certain moment in the 3D data display screen 280 corresponds to the moment at which the cursor 243 is located in the waveform display screen 230.

[0107] In one example, the user makes Figure 16The cursor 243 on the waveform display screen 230 moves, thereby specifying the moment that is desired to be reproduced. The moment acquisition unit 40 acquires the moment corresponding to the position of the cursor 243 on the waveform display screen 230. Here, it is set to... Figure 14 The log data was acquired at the time indicated by the shading line, i.e., "11:11:15". The time acquisition unit 40 notifies the time synchronization unit 33 of the acquired time, and the time synchronization unit 33 transmits the log data of the acquired time to the motion reproduction unit 37. Furthermore, in the motion reproduction unit 37, motion reproduction simulation data that causes the hypothetical controlled object to move is generated using the acquired time log data, and the three-dimensional data display processing unit 39 displays the motion reproduction simulation data in three dimensions on the display unit 31 via the three-dimensional data display screen 280. Thus, the state of the controlled object 10 at the time specified by the waveform display screen 230 can be confirmed through the three-dimensional data display screen 280.

[0108] If the user specifies the next time step, the synchronization of the time between the waveform display processing unit 35 and the three-dimensional data display processing unit 39 is displayed on the display unit 31. Furthermore, subsequent times can also proceed automatically without user specification. However, in this case, the aforementioned processing remains unchanged.

[0109] By repeating the above actions, the user can confirm the status at each moment by displaying both the waveform data of the log data and the three-dimensional data of the control object 10, and at the same time confirm the cause of the problem.

[0110] Furthermore, the above example illustrates an instance where the recurrence time is specified via the waveform display processing unit 35, but even from... Figure 16 The slider 302 of the three-dimensional data display screen 280 can specify the reproduction time, and the waveform data and the action of the hypothetical controlled object can also be confirmed through the same process as described above.

[0111] In Embodiment 1, the time synchronization unit 33 synchronizes the time of log data obtained from the control device 20 and dynamic image data obtained from the dynamic image recording unit 23 of the control device 20, and transmits them to at least one of the following processing units: program action display processing unit 34, waveform display processing unit 35, and dynamic image display processing unit 36, and the action playback unit 37. In the program action display processing unit 34, the control program executed by the instrument control unit 21 of the control device 20 is displayed in circuit form, and the values ​​of input / output data or status data corresponding to program structure elements are obtained from the log data at the time specified by the time synchronization unit 33, and displayed on the program action display screen 200 located near the program structure elements. The waveform display processing unit 35 displays the values ​​of input / output data or status data relative to the time, and displays the waveform display screen 230 with a cursor 243 at the time specified by the time synchronization unit 33. The dynamic image display processing unit 36 ​​displays the dynamic image data at the time specified by the time synchronization unit 33 in the dynamic image data on the dynamic image display screen 250. The motion reproduction unit 37 uses log data at the time specified by the time synchronization unit 33 to generate motion reproduction simulation data that reproduces the motion of a hypothetical control object having the same structure as the control object 10. The three-dimensional data display processing unit 39 displays a three-dimensional data display screen 280 that displays the motion reproduction simulation data as three-dimensional data. Thus, the three-dimensional data display screen 280, which reproduces the motion of the hypothetical control object synchronized with at least one of the program motion display screen 200, waveform display screen 230, and motion image display screen 250 based on the log data, is displayed together with at least one of the program motion display screen 200, waveform display screen 230, and motion image display screen 250. Furthermore, by providing these screens to the user, it is possible to assist in the analysis of the problem state of the control object 10 implemented by the user.

[0112] The 3D data display processing unit 39 can change the position of the viewpoint on which the hypothetical controlled object is displayed on the display unit 31 according to the display setting information stored in the display setting information storage unit 38. Therefore, without using multiple camera units 71 to photograph the controlled object 10, the operation of the hypothetical controlled object can be confirmed from any direction. In other words, regarding the controlled object 10, i.e., the production equipment and the devices constituting the production equipment, an environment can be provided to the user that allows for a comprehensive understanding of the status and operation of the instruments from multiple perspectives. Therefore, without increasing costs by installing multiple camera units 71, it becomes easier for the user to determine the cause of problems, enabling faster recovery and improving the operating rate of production equipment, devices, and instruments.

[0113] Implementation method 2.

[0114] Patent Document 1 discloses a method for roughly synchronizing the output of audio information and internal information of a programmable controller obtained from a ladder diagram simulation. That is, during fault analysis, the analyzer hears the reproduced audio signal, thereby detecting whether a fault has occurred. However, this method has a problem: inexperienced analyzers may miss sounds related to the occurrence of a fault. Therefore, Embodiment 2 was proposed in view of the above situation, with the aim of providing a problem analysis assistance device that uses audio data to assist in detecting the occurrence of problems in production equipment, regardless of the analyzer's skill level.

[0115] In Embodiment 1, a problem analysis aid 30 is shown, which synchronously displays at least one of the program action display data, waveform data 231, and motion image data, and three-dimensional data that displays three-dimensional data in a simulation of action reproduction based on log data, which reproduces the actions of a hypothetical controlled object. In Embodiment 2, a problem analysis aid is described, which synchronously displays at least one of the program action display data, waveform data 231, and motion image data, and audio analysis data obtained by analyzing audio data obtained from recording sounds emitted by the controlled object 10.

[0116] Figure 17 This is a block diagram illustrating an example of the structure of the problem-solving assistance system according to Embodiment 2. Furthermore, structural elements identical to those in Embodiment 1 are labeled with the same reference numerals, and their descriptions are omitted. The problem-solving assistance system 1A according to Embodiment 2 includes a control object 10, a control device 20A, and a problem-solving assistance device 30A.

[0117] Based on the structure of Embodiment 1, the control device 20A also includes an audio recording unit 24. Audio data is recorded, which is the data from the recording of sounds emitted by the audio recording unit 24 and the controlled object 10 via the microphone 72. The audio data includes recording time information.

[0118] The problem analysis auxiliary device 30A, unlike Embodiment 1, removes the motion reproduction unit 37, the display setting information storage unit 38, and the three-dimensional data display processing unit 39, and adds an audio analysis display processing unit 41. The audio analysis display processing unit 41 acquires audio data recorded from the sound emitted by the controlled object 10, and displays the analysis results obtained from analyzing the audio data on the display unit 31. Specifically, the audio analysis display processing unit 41 performs time-frequency analysis on the audio data acquired from the audio recording unit 24 of the control device 20A, generates an audio analysis display screen including audio analysis results such as three-dimensional graphics, and displays the audio analysis display screen on the display unit 31. An example of time-frequency analysis is wavelet transform. Furthermore, although the illustration is omitted, the audio analysis display screen, like the three-dimensional data display screen 280, has an audio analysis result display area and a time operation area for displaying the audio analysis results. The time operation area has the function of specifying the time in the audio analysis results displayed in the audio analysis result display area. The technique of performing time-frequency analysis on the audio data processed by the audio analysis and display processing unit 41 and displaying the results can be implemented using known methods.

[0119] The time synchronization unit 33 synchronizes the time of log data from the log data acquisition unit 32, motion image data from the motion image recording unit 23, and audio data from the audio recording unit 24, and transmits them to the program action display processing unit 34, waveform display processing unit 35, motion image display processing unit 36, and audio analysis display processing unit 41. Thus, similar to Embodiment 1, time synchronization is achieved between the program action display screen 200, waveform display screen 230, motion image display screen 250, and audio analysis display screen.

[0120] Furthermore, similar to Embodiment 1, the problem analysis assistance device 30A only needs to have one or more of the following: a program action display processing unit 34, a waveform display processing unit 35, and a dynamic image display processing unit 36. Additionally, the problem analysis assistance method in Embodiment 2 is the same as described in Embodiment 1, therefore its description is omitted.

[0121] Figure 18 This is a block diagram illustrating another example of the structure of the problem analysis assistance system involved in Embodiment 2. Furthermore, regarding Embodiment 1 and... Figure 17 The same structural elements are labeled with the same number, and their descriptions are omitted. Figure 18 The problem analysis assistance system 1B shown has a control object 10, a control device 20A, and a problem analysis assistance device 30B.

[0122] In the structure of the problem analysis assistance device 30 of Embodiment 1, the problem analysis assistance device 30B also has Figure 17The audio analysis display processing unit 41 described herein. Through the structure described above, the time is synchronized between the program action display screen 200, the waveform display screen 230, the dynamic image display screen 250, the three-dimensional data display screen 280, and the audio analysis display screen.

[0123] Furthermore, similar to the case of Embodiment 1, the problem analysis assistance device 30B only needs to have one or more of the following: program action display processing unit 34, waveform display processing unit 35, and dynamic image display processing unit 36.

[0124] In Embodiment 2, the sound emitted from the controlled object 10 is recorded as audio data by the audio recording unit 24 of the control device 20A. Furthermore, the audio analysis display processing unit 41 of the problem analysis assistance devices 30A and 30B displays an audio analysis display screen containing the analysis results obtained from analyzing the audio data. Additionally, the audio data input to the audio analysis display processing unit 41, the log data input to the program action display processing unit 34 and the waveform display processing unit 35, and the dynamic image data input to the dynamic image display processing unit 36 ​​are synchronized by the timing synchronization unit 33. As described above, the audio data is visualized, thus assisting in the detection of problems occurring in the production equipment regardless of the analyst's skill level. That is, the audio analysis display screen synchronized with at least one of the program action display screen 200, the waveform display screen 230, and the dynamic image display screen 250—the result of the actual action of the controlled object 10—is displayed together with at least one of these screens. Furthermore, providing these images to the user can assist in analyzing the problem state of the user-controlled object 10.

[0125] For example, if a user hears reproduced audio data and makes a judgment, a user unfamiliar with troubleshooting might miss the problematic audio. However, by referring to an audio analysis display screen that includes the analysis results obtained through methods such as time-frequency analysis, the user can easily extract anomalies from the analysis results. In other words, even a user unfamiliar with troubleshooting can extract the parts they believe to be problematic from the audio analysis display screen.

[0126] Implementation method 3.

[0127] In Embodiment 1, when the action in the hypothetical controlled object is reproduced after the time acquisition unit 40 acquires a specified time, the action reproduction unit 37 reproduces the action in the hypothetical controlled object based on log data. However, it is not possible to reproduce the same action in the hypothetical controlled object as the actual controlled object 10 using only log data. This is because the initial state in the hypothetical controlled object is not defined. For example, in Embodiment 1, such as... Figure 13 As shown, an example illustrates a situation where a problem occurs because the workpiece 100 is not embedded in the recess 111 of the tray 110. In this case, when the motion reproduction unit 37 reproduces the motion of the hypothetical controlled object, it is necessary to use the hypothetical controlled object to... Figure 14 The actual position of workpiece 100 relative to tray 110 at the second location 122 at the time of "11:11:17" is defined. If no definition is made, the motion reproduction unit 37 will reproduce the motion of the hypothetical controlled object by setting the workpiece 100 to not properly embedding into the recess 111 of tray 110. That is, the 3D data display screen 280 will display the normal state of workpiece 100 embedding into the recess 111 of tray 110 at the time of "11:11:17". As a result, the user who observes this situation may not find the reason why sensor 142 is not turned on. Therefore, in embodiment 3, a problem analysis assistance system that can reproduce the motion of the hypothetical controlled object in a way that is the same as the actual controlled object 10 will be described.

[0128] Figure 19 This is a block diagram illustrating an example of the structure of the problem-solving assistance system according to Embodiment 3. The problem-solving assistance system 1C according to Embodiment 3 includes a control object 10, a control device 20, and a problem-solving assistance device 30C. Furthermore, structural elements identical to those in Embodiment 1 are labeled with the same reference numerals, and their descriptions are omitted. Additionally, in Embodiment 3, the dynamic image display processing unit 36 ​​is provided as an essential structural component.

[0129] The problem analysis assistance device 30C, in the structure of Embodiment 1, also includes an initial state generation unit 42. The initial state generation unit 42 generates initial state information representing the state of the components constituting the controlled object 10 based on an image of dynamic image data at a specified time. Specifically, if the initial state generation unit 42 obtains a specified time from the time acquisition unit 40, it generates initial state information constituting the controlled object 10 by referring to the dynamic image data at that specified time. In one example, a screenshot image of the specified time is obtained from the dynamic image data, and the position and configuration state of the operating part and workpiece 100 of the controlled object 10 are obtained from the screenshot image to generate initial state information. Furthermore, the initial state generation unit 42 transmits the initial state information to the motion reproduction unit 37. In one example, the initial state generation unit 42 determines and reproduces the position of the workpiece 100 at a specified time, or determines and reproduces the state of the robot arm mechanism 130 at a specified time. The technique of obtaining the configuration relationship of objects contained in an image from an image can be implemented using known methods. Furthermore, here, a screenshot of dynamic image data at a specified moment is used when generating the initial state information, but other data can also be used if it is possible to obtain the state of the components constituting the control object 10 at a specified moment.

[0130] The motion reproduction unit 37 sets the state of the components of the hypothetical controlled object at a specified time based on the initial state information, and generates motion reproduction simulation data that reproduces the motion of the hypothetical controlled object with the initial state information set based on log data. Specifically, when reproducing the motion of the hypothetical controlled object at a specified time, the motion reproduction unit 37 sets the position and configuration state of the operating part of the hypothetical controlled object and the hypothetical workpiece 100 based on the initial state information. Moreover, the motion reproduction unit 37 reproduces the motion of the hypothetical controlled object based on the set initial state information and log data.

[0131] Next, the method for displaying three-dimensional data at a specified time in the problem analysis assistance method involved in Implementation Method 3 will be described. Figure 20 This is a flowchart illustrating an example of the sequence of problem-solving assistance methods according to Embodiment 3 when a specified time is specified. Here, according to Embodiment 1... Figure 7 It is set to a state where the display unit 31 displays the program action display screen 200, the waveform display screen 230, the dynamic image display screen 250, and the three-dimensional data display screen 280.

[0132] First, the user operates any of the time indicators among the program action display screen 200, waveform display screen 230, dynamic image display screen 250, and 3D data display screen 280. The time indicator is a slider 227, 272, 302, or a cursor 243. The time acquisition unit 40 detects the operation of the time indicator, reads the specified time (step S51), and transmits the read specified time to the time synchronization unit 33 and the initial state generation unit 42 (step S52).

[0133] The initial state generation unit 42 acquires dynamic image data at a specified time via the time synchronization unit 33 (step S53), obtains the position and configuration state of the operating part of the controlled object 10 and the workpiece 100 from the dynamic image data at the specified time, and generates initial state information including the position and configuration state of the operating part of the controlled object 10 and the workpiece 100 at the specified time (step S54). Furthermore, the initial state generation unit 42 transmits the initial state information to the motion reproduction unit 37 (step S55).

[0134] The time synchronization unit 33 transmits the received log data at the specified time to the program action display processing unit 34, the waveform display processing unit 35, and the action reproduction unit 37, and transmits the received dynamic image data at the specified time to the dynamic image display processing unit 36 ​​(step S56).

[0135] Then, the program action display processing unit 34, the waveform display processing unit 35, and the dynamic image display processing unit 36 ​​perform operations in conjunction with... Figure 8 The same processing as steps S34 to S39 (steps S57 to S62) is performed. Additionally, the motion reproduction unit 37 generates a hypothetical control object with initial state information set at a specified time, and reproduces motion reproduction simulation data based on log data at the specified time (step S63). Then, the process is performed similarly to... Figure 8 The same process applies to steps S41 to S43 (steps S64 to S66).

[0136] Additionally, after steps S58, S60, S62, and S66, the following steps are performed: Figure 8 The same process as step S44 (step S67) is followed, and the process returns to steps S57, S59, S61, and S63. That is, the program action display screen 200, waveform display screen 230, dynamic image display screen 250, and three-dimensional data display screen 280 for the next specified moment are displayed on the display unit 31 in the same order.

[0137] In embodiment 3, the initial state generation unit 42 obtains the position and configuration state of the operating part and workpiece 100 of the controlled object 10 at a specified time, and generates initial state information including the position and configuration state of the operating part and workpiece 100. The action reproduction unit 37 reproduces the action of the hypothetical controlled object based on the initial state information and the log data at the specified time. As a result, the position and configuration state of the operating part and workpiece 100 in the hypothetical controlled object at the specified time becomes the same as the position and configuration state of the operating part and workpiece 100 of the actual controlled object 10, and the action reproduction based on the log data is performed from this state. That is, in the three-dimensional data display processing unit 39, the action that is the same as the action of the actual controlled object 10 can be reproduced. In addition, the specified time is changed by the time indication unit, thereby changing the initial state by the specified time. The user changes the specified time by repeated experimentation, thereby displaying the state in which the problem has occurred on the program action display screen 200 and the waveform display screen 230. In this case, the occurrence of the problem is displayed on the three-dimensional data display screen 280, and the cause of the problem can be provided to the user.

[0138] Furthermore, a known method involves synchronously displaying motion image data recording the actual actions of the controlled object 10 and three-dimensional data that is three-dimensionally displayed as motion reproduction simulation data reproducing the actions of a hypothetical controlled object through the motion reproduction unit 37. The method then resets based on the motion image data and the three-dimensional data, for example, returning to the initial moment of the motion image data. In this method, to reproduce log data from a specified moment (not the initial moment) through the motion reproduction unit 37, it is necessary to reproduce the data from the initial moment of the motion image data up to the specified moment, which is inefficient. That is, to observe the three-dimensional data at the specified moment, one must wait for the simulation time required from the initial moment of the motion image data up to the specified moment. Additionally, as mentioned above, there is sometimes a deviation in the actions between the three-dimensional data representing the hypothetical actions of the controlled object being reproduced and the motion image data representing the actual actions of the controlled object 10. This is because, even if the state of the three-dimensional data is aligned with the motion image data at the start of the simulation, the state at the specified moment may differ between the three-dimensional data and the motion image data.

[0139] On the other hand, in Embodiment 3, the initial state of the hypothetical controlled object at the start time of motion reproduction, i.e., the specified time, is faithfully reproduced based on the image of the motion image data at the specified time. Therefore, the past motion state of the hypothetical controlled object using the log data in the motion reproduction unit 37 can be faithfully reproduced. Furthermore, the motion of the hypothetical controlled object reproduced as described above is the same as the motion of the actual controlled object 10 displayed by the motion image display processing unit 36. Moreover, since the motion of the hypothetical controlled object is reproduced from the specified time, simulation is not required from the initial time of the motion image data up to the specified time, providing the hypothetical motion of the controlled object at the specified time without making the user wait.

[0140] Furthermore, the above description focuses on the case where the initial state generation unit 42 is provided in the structure of Embodiment 1, but it is also possible to include it in Embodiment 2. Figure 18 An initial state generation unit 42 is set in the structure. In this case, the same effect as described above can also be obtained.

[0141] Implementation method 4.

[0142] In the structure of Embodiment 1, since it is unknown when a problem will occur, it is desirable to record log data and dynamic image data for as long a period of time as possible in the log recording unit 22 and dynamic image recording unit 23 of the control devices 20 and 20A. Therefore, the recording capacity of the log recording unit 22 and dynamic image recording unit 23 must be increased, resulting in increased manufacturing costs for the problem analysis assistance systems 1, 1A, 1B, and 1C. In Embodiment 4, a problem analysis assistance system that can reduce the recording capacity of the log recording unit 22 and dynamic image recording unit 23 will be described.

[0143] Figure 21 This is a block diagram illustrating an example of the structure of the problem-solving assistance system according to Embodiment 4. The problem-solving assistance system 1D according to Embodiment 4 includes a control object 10, a control device 20D, and a problem-solving assistance device 30. Furthermore, structural elements identical to those in Embodiment 1 are labeled with the same reference numerals, and their descriptions are omitted.

[0144] The control device 20D, based on the structure of Embodiment 1, also includes a problem detection unit 25. The problem detection unit 25 detects problems that occur when the control program is executed by the instrument control unit 21. Specifically, the problem detection unit 25 continuously observes the state of the controlled object 10 and detects the occurrence of problems in the controlled object 10. If the problem detection unit 25 detects a problem, it notifies the log recording unit 22 of a signal indicating that a problem has occurred. In one example, the detection information includes the time when the problem was detected. An indicator set in the control program to activate when a foreseeable problem occurs is an example of the problem detection unit 25. An example of a problem is when a moving part, such as the robot hand mechanism 130, stops midway. In this case, Figure 2 The axis angle in the robot hand mechanism 130 of the device is raised for a predetermined period of time, in one example 3 seconds, and under the same circumstances, the indicator is raised, thereby detecting the problem.

[0145] Furthermore, in Embodiment 4, the structure of the log recording unit 22 of the control device 20D differs from that in Embodiment 1. The log recording unit 22 includes a first recording unit 221, a log recording processing unit 222, a log saving processing unit 223, and a second recording unit 224.

[0146] The first recording unit 221 has the capacity to record log data for a predetermined first period. In one example, the first period can be set to 10 minutes.

[0147] In one example, the log recording processing unit 222 uses the first recording unit 221 as a circular buffer to record log data. Specifically, the log recording processing unit 222 records input and output data, including input and output data exchanged between the instrument control unit 21 and the controlled object 10, and status data indicating the status of the operating parts of the controlled object 10, as log data in a time sequence in the first recording unit 221. For example, the log recording processing unit 222 records log data in the first recording unit 221, but the capacity of the first recording unit 221 becomes saturated after 10 minutes from the start of recording. At this time, the log recording processing unit 222 records log data after 10 minutes, sequentially overwriting the log data from minute 0 in the first recording unit 221. Thus, past log data is overwritten, and log data before 10 minutes becomes non-existent. Moreover, the same overwriting process is performed on all recorded log data within the recordable capacity of the first recording unit 221.

[0148] The log saving processing unit 223, upon receiving the moment when the problem is detected by the problem detection unit 25, saves the log data in the first recording unit 221 for a second period shorter than the first period to the second recording unit 224. In one example, the second period includes a predetermined time before and after the moment the problem occurred, and is set shorter than the first period. For example, a four-minute period including two minutes before and two minutes after the moment the problem occurred can be set as the second period. In this case, if the log saving processing unit 223 receives a detection of a problem from the problem detection unit 25, it retrieves the log data for the four-minute period including the two minutes before and after the moment the problem occurred from the first recording unit 221 after two minutes from the moment the problem occurred, and saves it to the second recording unit 224. That is, the log data for the second period, including the moment the problem was detected, is saved as a file in the second recording unit 224.

[0149] The second recording unit 224 saves a file containing log data for the second period, including the time when the problem was detected. The second recording unit 224 is a non-volatile recording unit. The second recording unit 224 only needs to have a capacity capable of saving a file containing one or more log data entries.

[0150] The problem analysis assistance method in the problem analysis assistance system 1D of Embodiment 4 is the same as that described in Embodiment 1, so its description is omitted. However, if the log data acquisition unit 32 of the problem analysis assistance device 30 is input with an instruction such as log data confirmation, it will read the log data stored in the second recording unit 224 of the log recording unit 22 of the control device 20D.

[0151] Here, the method for recording log data in the log recording section 22 will be explained. Figure 22 This is a flowchart illustrating an example of the sequence of methods for saving log data in the control device of the problem analysis auxiliary system involved in Implementation 4.

[0152] First, the log recording processing unit 222 of the log recording unit 22 records log data in the first recording unit 221. This log data records the input / output data and status data of the controlled object 10 along with the time (step S71). Next, the log recording processing unit 222 determines whether detection information indicating the occurrence of a problem has been obtained (step S72). If no detection information is obtained (if step S72 is No), the processing returns to step S71, and log data is recorded in the first recording unit 221.

[0153] If the detection information is obtained (if step S72 is Yes), the log recording processing unit 222 obtains the time when the problem occurred contained in the detection information (step S73) and records the log data from the time when the problem occurred to the first recording unit 221 over a predetermined period (step S74).

[0154] Then, the log saving processing unit 223 determines whether a predetermined period has elapsed since the time the problem occurred (step S75). If no predetermined period has elapsed since the time the problem occurred (if step S75 is No), the process returns to step S74. On the other hand, if a predetermined period has elapsed since the time the problem occurred (if step S75 is Yes), the log saving processing unit 223 retrieves log data containing a predetermined second period from the first recording unit 221, including the time the problem occurred, and saves it as a file in the second recording unit 224 (step S76). Then, the process returns to step S71.

[0155] Furthermore, the dynamic image recording unit 23 or the audio recording unit 24 in Embodiment 2 may have the same structure as the log recording unit 22 described in Embodiment 4.

[0156] In embodiment 4, the log recording unit 22 includes: a first recording unit 221, which has the capacity to record log data for a first period; and a second recording unit 224, which can save log data for a second period, which is shorter than the first period, as a file. The log recording unit 22 also includes a log saving processing unit 223, which, if the problem detection unit 25 detects a problem, saves log data for a predetermined period containing the time of the problem occurrence from the first recording unit 221 to the second recording unit 224. With the structure described above, the recording capacity of the log recording unit 22 can be reduced compared to embodiments 1 to 3. Furthermore, since the user can obtain the log data for the period containing the time of the problem occurrence from the second recording unit 224 and perform problem analysis, the workload of exploring the location of the problem from a large amount of data can be reduced.

[0157] Implementation method 5.

[0158] In embodiments 1 to 4, it is assumed that the cause of the problem lies within the controlled object 10. However, when the controlled object 10 is a production device, during the operation of the production device, there are processes involving operators and others, as well as processes involving unmanned transport vehicles such as AGVs (Automatic Guided Vehicles), forklifts, or unmanned transport robots. Therefore, when people or unmanned transport vehicles come into contact with the production device, problems sometimes occur in the controlled object 10. Therefore, in embodiment 5, a problem analysis assistance system that can analyze the cause of a problem, including people or unmanned transport vehicles, will be described.

[0159] Figure 23 This is a block diagram illustrating an example of the structure of the problem-solving assistance system according to Embodiment 5. The problem-solving assistance system 1E according to Embodiment 5 includes a controlled object 10, a control device 20, a problem-solving assistance device 30E, and a motion information recording device 60 maintained by a person 61 and an unmanned transport 62 existing in the same area as the controlled object 10. Either the person 61 or the unmanned transport 62 need to exist in the same area as the controlled object 10; the following description illustrates the case where both the person 61 and the unmanned transport 62 exist in the same area as the controlled object 10. Furthermore, structural elements identical to those in Embodiment 1 are labeled with the same reference numerals, and their descriptions are omitted.

[0160] The motion information recording device 60 is a device for recording motion information that records the actions of the person 61 and the unmanned transport 62, which are maintained by the motion information recording device 60, along with time. In one example, the motion information is information that records the actions of the person 61 and the unmanned transport 62 at predetermined time intervals. The motion information includes at least one of the position and state of the person 61 and the unmanned transport 62. Here, the example is the case where the motion information is information that records position along with time. The position can be obtained using known techniques. For example, techniques for determining position by receiving radio waves from multiple GPS (Global Positioning System) satellites, techniques for estimating movement trajectories on a map by using angular velocity sensors and accelerometers, etc. In the latter case, at the entrance of the area where the control object 10 is set up, for example, a signal transmitting device that transmits beacon signals with known positions is set up, and the movement trajectory is estimated based on the position of the entrance, thereby enabling the estimation of the positions of the person 61 and the unmanned transport 62 within the area where the control object 10 is set up.

[0161] The problem analysis assistance device 30E, in the structure of embodiment 1, also includes a motion information acquisition unit 43. When the user issues an instruction to acquire log data, the motion information acquisition unit 43 acquires motion information from the motion information recording device 60 of the person 61 and the unmanned transport 62. As described above, the motion information here is information recording the positions of the person 61 and the unmanned transport 62 in a time sequence.

[0162] In embodiment 5, the motion reproduction unit 37 of the problem analysis assistance device 30E generates motion reproduction simulation data that not only reproduces the motion of the hypothetical controlled object based on log data, but also reproduces the trajectories of the hypothetical person corresponding to person 61 and the hypothetical unmanned transport 62 based on motion information. The motion reproduction simulation data simulates the relationship between the position of the hypothetical controlled object and the positions of the hypothetical person and the hypothetical unmanned transport in a way that makes the relationship between the actual position of the controlled object 10 and the positions of person 61 and unmanned transport 62 the same.

[0163] The 3D data display processing unit 39 displays 3D data on the display unit 31, which reproduces not only the hypothetical controlled object but also the movements of the hypothetical person and the hypothetical unmanned transport. In one example, when the hypothetical person or the hypothetical unmanned transport passes near the hypothetical controlled object, the hypothetical person or the hypothetical unmanned transport is displayed in 3D together with the hypothetical controlled object.

[0164] The problem analysis assistance method in the problem analysis assistance system 1E of Embodiment 5 is the same as described in Embodiment 1, except that the motion reproduction unit 37 reproduces the motion of the hypothetical controlled object based on log data, and reproduces the motion of the hypothetical person and the hypothetical unmanned transport based on motion information. Therefore, its description is omitted. In addition, the above description describes the case where Embodiment 5 is applied in the structure of Embodiment 1, but Embodiment 5 can also be applied in the structures of Embodiments 2 to 4.

[0165] In embodiment 5, a motion information recording device 60 is installed in the same area as the controlled object 10, for a person 61 or an unmanned transport 62, and the motion information of the person 61 or the unmanned transport 62 is recorded by the motion information recording device 60. When performing problem analysis, the motion information acquisition unit 43 of the problem analysis assistance device 30E acquires motion information from the motion information recording device 60. The motion reproduction unit 37 generates motion reproduction simulation data that reproduces the hypothetical controlled object's motion using log data and also reproduces the motion of the person 61 or the unmanned transport 62 using motion information. The three-dimensional data display processing unit 39 displays the motion reproduction simulation data in three dimensions. Therefore, by checking the three-dimensional data display screen 280, it is possible to determine whether the cause of the problem is due to contact between the person 61 or the unmanned transport 62 and the controlled object 10. That is, even if the cause of the problem with the controlled object 10 is an external factor, it can be addressed by the problem analysis assistance device 30E.

[0166] Implementation method 6.

[0167] In embodiments 1 to 5, the environment surrounding the controlled object 10, such as temperature and humidity, was not considered. In embodiment 6, a problem analysis assistance system that considers the environment surrounding the controlled object 10 and reproduces the operation of the controlled object 10 is described.

[0168] The structure of the problem analysis assistance system 1 involved in Implementation 6 is the same as that in Implementation 1. Figure 1 The structure is the same as shown. However, the log recording unit 22 records not only log data, but also environmental data obtained from the environmental data measurement unit that measures the environment of the area where the control object 10 is configured. Examples of the environment of the area where the control object 10 is configured include temperature and humidity. In this case, the environmental data measurement unit becomes a temperature measurement unit and a humidity measurement unit. The environmental data measurement unit can measure environmental data representing the area where the control object 10 is configured, or it can measure environmental data of any part of the control object 10. In the latter case, the environmental data measurement unit is provided in the part where the measurement is performed.

[0169] Furthermore, the motion reproduction unit 37 of the problem analysis assistance device 30 not only reproduces the motion of the hypothetical controlled object based on log data, but also generates motion reproduction simulation data that uses information including temperature information measured by the temperature measurement unit and the material of the components constituting the controlled object 10 to simulate changes in the shape of the hypothetical controlled object. That is, the motion reproduction unit 37 uses temperature information to simulate the expansion of each component constituting the controlled object 10. Thus, the state of components made of materials that expand and contract due to temperature changes can also be reproduced.

[0170] When displaying motion reproduction simulation data as three-dimensional data, the three-dimensional data display processing unit 39 changes the state of the object constituting the three-dimensional data in accordance with the level of the acquired environmental data. In one example, as a change in the state of the object, there is a method that changes the color based on the value of the environmental data. If the environmental data is temperature, the three-dimensional data display processing unit 39 displays the object by changing its color, similar to a thermal imager. In this case, the three-dimensional data display processing unit 39 has pre-stored setting information that associates the value of the environmental data with the displayed color, thereby enabling it to change the color of the object for display.

[0171] Furthermore, while the above description exemplifies measuring temperature or humidity as environmental data, measurements of air pressure, dust density, etc., can also be used as environmental data. In Embodiment 5, the body temperature of the person 61 (i.e., the operator) and the temperature of the unmanned transport object 62, which are located in the same area as the controlled object 10, can also be included. In this case, the motion information recording device 60 measures the body temperature of the person 61 or the temperature of the unmanned transport object 62, and records the result as motion information.

[0172] The problem-solving assistance method in the problem-solving assistance system 1 of Embodiment 6 is the same as that described in Embodiment 1, so its description is omitted. Furthermore, while the above description explains the application of Embodiment 6 in the structure of Embodiment 1, Embodiment 6 can also be applied in the structures of Embodiments 2 to 5.

[0173] In embodiment 6, the log recording unit 22 of the control device 20 records log data including environmental data of the area where the controlled object 10 is configured. When the operation reproduction unit 37 of the problem analysis assistance device 30 reproduces the operation of the hypothetical controlled object using the log data, it performs a physical simulation including changes in the components constituting the hypothetical controlled object using the environmental data. Therefore, for example, when the pallet 110 expands due to a temperature rise in the factory, causing the workpiece 100 to shift position, this can also be represented using three-dimensional data. As a result, when the problem is caused by the environment surrounding the controlled object 10, the cause of the problem becomes easier to find than before. Furthermore, the state of the device or the entire factory, i.e., the controlled object 10, is closer to the actual environment, thus further improving the accuracy of problem analysis processing to determine the cause of the problem.

[0174] Furthermore, in the structures of embodiments 1 to 6 described above, dynamic image data is transmitted from the dynamic image recording unit 23 to the problem analysis assistance devices 30, 30A, 30B, 30C, and 30E, and audio data is transmitted from the audio recording unit 24 to the problem analysis assistance device 30. However, the dynamic image data and audio data, like the log data, can be acquired by the log data acquisition unit 32. That is, the log data acquisition unit 32 can acquire log data from the log recording unit 22, dynamic image data from the dynamic image recording unit 23, and audio data from the audio recording unit 24. Moreover, in the time synchronization unit 33, only the data used for problem analysis among the log data, dynamic image data, and audio data needs to be synchronized.

[0175] Here, the hardware structure of the problem-solving assistance devices 30, 30A, 30B, 30C, and 30E will be described. The problem-solving assistance devices 30, 30A, 30B, 30C, and 30E involved in embodiments 1 to 6 are implemented using a computer system such as a personal computer or a general-purpose computer.

[0176] Figure 24 This diagram illustrates an example of the hardware structure when the functions of the problem-solving assistance devices according to embodiments 1 to 6 are implemented using a computer system. When the functions of the problem-solving assistance devices 30, 30A, 30B, 30C, and 30E are implemented using a computer system, the functions of the problem-solving assistance devices 30, 30A, 30B, 30C, and 30E are as follows: Figure 24 As shown, the device includes a processor 351, a memory 352, a storage device 353, an input device 354, a display device 355, and a communication device 356. The processor 351 performs computational processing. The memory 352 provides a working area for the processor 351. The storage device 353 stores programs used as problem-solving aids 30, 30A, 30B, 30C, and 30E. The input device 354 is the input interface with the user. The display device 355 displays information to the user. The communication device 356 has communication capabilities with control devices 20, 20A, 20D, or other various devices. The processor 351, memory 352, storage device 353, input device 354, display device 355, and communication device 356 are connected via a data bus 357.

[0177] Here, processor 351 can be a processing device, a computing device, a microprocessor, a microcomputer, a CPU (Central Processing Unit), or a DSP (Digital Signal Processor), etc. Additionally, memory 352 can be non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), or EEPROM (Electrically EPROM), as well as disks, floppy disks, optical disks, compact disks, mini-disks, or DVDs (Digital Versatile Discs), etc.

[0178] The log data acquisition unit 32, time synchronization unit 33, program action display processing unit 34, waveform display processing unit 35, dynamic image display processing unit 36, motion reproduction unit 37, 3D data display processing unit 39, time acquisition unit 40, audio analysis display processing unit 41, initial state generation unit 42, and motion information acquisition unit 43 are executed, for example, by processor 351. Figure 24 The program stored in the memory 352 shown is used as a log data acquisition unit, a time synchronization unit, a program action display unit, a waveform display unit, a dynamic image display unit, an action reproduction unit, a 3D data display unit, a time acquisition unit, an audio analysis display unit, an initial state generation unit, and an action information acquisition unit, respectively. This program becomes a problem analysis auxiliary program. Furthermore, multiple processors 351 and multiple memories 352 can work together to achieve the above functions. Alternatively, a portion of the functions of the log data acquisition unit 32, the time synchronization unit 33, the program action display processing unit 34, the waveform display processing unit 35, the dynamic image display processing unit 36, the action reproduction unit 37, the 3D data display processing unit 39, the time acquisition unit 40, the audio analysis display processing unit 41, the initial state generation unit 42, and the action information acquisition unit 43 can be installed as electronic circuitry, while other portions can be implemented using the processor 351 and the memory 352. Furthermore, for example, the processor 351 executes... Figure 24 The program stored in the memory 352 shown therein, which enables the motion reproduction unit and the three-dimensional data display unit, becomes a three-dimensional data display program.

[0179] The problem analysis assistance method executed by the problem analysis assistance devices 30, 30A, 30B, 30C, and 30E is implemented through software, firmware, or a combination of software and firmware. The software or firmware is stored as a program description in the storage device 353. The processor 351 reads the software or firmware stored in the storage device 353 into the memory 352 and executes it, thereby realizing various functions of the log data acquisition unit 32, the time synchronization unit 33, the program action display processing unit 34, the waveform display processing unit 35, the dynamic image display processing unit 36, the action reproduction unit 37, the three-dimensional data display processing unit 39, the time acquisition unit 40, the sound analysis display processing unit 41, the initial state generation unit 42, and the action information acquisition unit 43. That is, the computer system has a storage device 353, which stores the problem analysis assistance program that ultimately executes the steps of the problem analysis assistance method involved in embodiments 1 to 6 when the various functions of the log data acquisition unit 32, time synchronization unit 33, program action display processing unit 34, waveform display processing unit 35, dynamic image display processing unit 36, action reproduction unit 37, three-dimensional data display processing unit 39, time acquisition unit 40, sound analysis display processing unit 41, initial state generation unit 42 and action information acquisition unit 43 are executed by the processor 351.

[0180] Furthermore, the program containing the 3D data display program can also be provided via a communication medium or in the form of a storage medium on which the program is recorded. This 3D data display program includes a problem-solving assistance program and a portion thereof that implements the problem-solving assistance methods executed by the problem-solving assistance devices 30, 30A, 30B, 30C, and 30E. Both the storage medium containing the program implementing the problem-solving assistance methods executed by the problem-solving assistance devices 30, 30A, 30B, 30C, and 30E and the storage medium containing the 3D data display program are computer-readable storage media storing a program executable by a computer.

[0181] Furthermore, these programs can be said to enable the processing of various functions of the computer execution log data acquisition unit 32, time synchronization unit 33, program action display processing unit 34, waveform display processing unit 35, dynamic image display processing unit 36, action reproduction unit 37, three-dimensional data display processing unit 39, time acquisition unit 40, sound analysis display processing unit 41, initial state generation unit 42, and action information acquisition unit 43.

[0182] Specific examples of display device 355 are monitors and displays. Specific examples of input device 354 are keyboards, mice, and touch panels.

[0183] As described above, the problem analysis assistance devices 30, 30A, 30B, 30C, and 30E according to embodiments 1 to 6 can confirm the status of the problem location in the control object 10 such as the production equipment from any direction without increasing the cost compared to the past.

[0184] The structure shown in the above embodiments is an example, and it can also be combined with other known technologies, and the embodiments can be combined with each other. Without departing from the spirit of the subject, some parts of the structure can be omitted or changed.

[0185] Explanation of the label

[0186] 1. Problem Analysis Auxiliary System (1A, 1B, 1C, 1D, 1E); 10. Controlled Object; 20. Control Devices (20A, 20D); 21. Instrument Control Unit; 22. Log Recording Unit; 23. Dynamic Image Recording Unit; 24. Audio Recording Unit; 25. Problem Detection Unit; 30. Problem Analysis Auxiliary Devices (30A, 30B, 30C, 30E); 31. Display Unit; 32. Log Data Acquisition Unit; 33. Time Synchronization Unit; 34. Program Action Display and Processing Unit; 35. Waveform Display and Processing Unit; 36. Dynamic Image Display and Processing Unit; 37. Action Reproduction Unit; 38. Display setting information storage unit, 39 Three-dimensional data display processing unit, 40 Time acquisition unit, 41 Audio analysis and display processing unit, 42 Initial state generation unit, 43 Motion information acquisition unit, 60 Motion information recording device, 61 Person, 62 Unmanned transport, 71 Imaging unit, 72 Microphone, 100 Workpiece, 101 Support platform, 110 Pallet, 111 Recess, 120 Conveying mechanism, 121 First location, 122 Second location, 123 Guide rail, 130 Robot arm mechanism, 131 Axis, 132 Main body, 133 Robot arm, 141 142 Sensor, 141A, 142A Light-emitting elements, 141B, 142B Light-receiving elements, 150 Inspection unit, 200 Program operation display screen, 210 Program operation display area, 211, 241 Horizontal scroll bars, 212, 242 Vertical scroll bars, 220, 270, 300 Time operation area, 221 First recording unit, 222 Log recording processing unit, 223 Log saving processing unit, 224 Second recording unit, 226 Replay operation button, 227, 272, 302 Slider, 228, 273, 30 3. Time display unit; 230. Waveform display screen; 231, 231a, 231b, 231c, 232a, 232b, 232c waveform data; 240. Waveform display area; 243. Cursor; 250. Dynamic image display screen; 260. Dynamic image display area; 271, 301. Operation buttons; 280. Three-dimensional data display screen; 290. Three-dimensional data display area; 351. Processor; 352. Memory; 353. Storage device; 354. Input device; 355. Display device; 356. Communication device; 357. Data bus.

Claims

1. A problem analysis auxiliary program storage medium, which stores a problem analysis auxiliary program for assisting in the analysis of problems involving control objects with operating parts. The characteristics of the storage medium used by the problem-solving auxiliary program are as follows: The program causes the computer to perform the following steps: The log data acquisition step involves acquiring log data that records the input and output data of the control signals between the operating unit, the controlled object, and the control device controlling the controlled object in a time sequence. The dynamic image display step involves displaying dynamic image data obtained by capturing the state of the controlled object on the display unit. The action reproduction step generates action reproduction simulation data, which is obtained from the result of a simulation based on the log data, in which a hypothetical control object corresponding to the control object reproduces the action. The three-dimensional data display step involves displaying the motion reproduction simulation data as three-dimensional data on the display unit. The time synchronization step synchronizes the three-dimensional data reproduced through the motion reproduction step and displayed through the three-dimensional data display step with the dynamic image data displayed through the dynamic image display step. In the time acquisition step, if the time of the 3D data displayed in the 3D data display step or the time of the dynamic image data displayed in the dynamic image display step is specified, the specified time is acquired and transmitted as the specified time to the time synchronization step. This specified time is the start time of motion reproduction. The initial state generation step involves generating initial state information, including the position and configuration status of the operating parts constituting the controlled object and the components of the controlled object, based on the log data and the dynamic image data at the specified time. In the motion reproduction step, the state of the component of the hypothetical controlled object at the specified time is set based on the initial state information, and motion reproduction simulation data is generated based on the log data, which reproduces the motion of the hypothetical controlled object with the initial state information set. In the three-dimensional data display step, based on display setting information, the three-dimensional data after changing the viewpoint position is displayed on the display unit. This display setting information indicates the viewpoint position when the hypothetical control object is displayed on the display unit as the three-dimensional data. The program causes the computer to further execute an audio analysis and display step, namely, acquiring audio data from recordings of sounds emitted by the controlled object, and displaying the analysis results obtained from analyzing the audio data on the display unit. In the time synchronization step, time synchronization is achieved between the three-dimensional data used in the motion reproduction step and displayed in the three-dimensional data display step, the dynamic image data displayed in the motion display step, and the analysis results displayed in the sound analysis display step.

2. The problem-solving auxiliary program storage medium according to claim 1, characterized in that, The program also causes the computer to perform a motion information acquisition step, that is, to acquire motion information that records the actions of people or unmanned transported objects existing in the same area as the controlled object along with time. In the motion reproduction step, based on the log data and the motion information, motion reproduction simulation data is generated that reproduces the motions of the hypothetical controlled object and the hypothetical person corresponding to the person or the hypothetical unmanned transport corresponding to the unmanned transport. In the three-dimensional data display step, the three-dimensional data, including the hypothetical controlled object and the hypothetical person or the hypothetical unmanned transport, is displayed on the display unit.

3. The problem-solving auxiliary program storage medium according to claim 1 or 2, characterized in that, The log data also includes environmental data related to the environment in which the controlled object is configured. In the motion reproduction step, the motion of the hypothetical controlled object is reproduced based on the log data, and motion reproduction simulation data is generated by simulating the deformation of the components constituting the controlled object based on the environmental data.

4. The problem-solving auxiliary program storage medium according to claim 3, characterized in that, In the three-dimensional data display step, the state of the object in the three-dimensional data changes accordingly to the value of the environmental data.

Images