Condition monitoring devices and methods for industrial machinery

Abstract

The robot state monitoring device (2) comprises: a camera that captures images of the robot (3) under the control of a controller; a dynamic image generation unit that associates the image data of the robot (3) acquired by the camera with the input and output signals (DO[1], AO[1], DI[1], AI[1]) of the controller along a time axis (830) to generate a dynamic image representing the state changes of the robot (3) and the input and output signals (DO[1], AO[1], DI[1], AI[1]); and a dynamic image regeneration device that regenerates the dynamic image generated by the dynamic image generation unit. The dynamic image generation unit acquires the values ​​of the input and output signals (DO[1], AO[1], DI[1], AI[1]) at the recording time of each frame at a period equal to the frame rate of the image data.

Description

Technical Field

[0001] This invention relates to a condition monitoring device and a condition monitoring method for industrial machinery. Background Technology

[0002] In a robot system where the robot is controlled by a robot control unit, various peripheral devices are connected to the robot control unit. In such a system, when expanding the system or addressing communication failures between the robot control unit and peripheral devices, it is necessary to verify the status of the robot control unit's output and input signals.

[0003] In the robot monitoring system shown in Patent Document 1, an imaging device captures images of the robot moving under the control of a robot control device. The image information captured by the imaging device is then saved in association with time information and the robot's task information. The saved image information, along with the time information and task information, is displayed on a display device. This robot monitoring system allows operators to identify the location of any malfunctions early and easily.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2015-168016 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] As mentioned above, the technique of monitoring a robot's state by recording its movements using a camera device is known. However, in conventional robot state monitoring devices, it is impossible to clearly define the correlation between the robot's movement state and the state of the output and input signals of the robot control device.

[0009] This disclosure was made in view of the aforementioned issues, and provides a state monitoring device and method for industrial machinery that can easily determine the correlation between the operating state of industrial machinery and the state of the signals of its controller along a time axis.

[0010] Methods for solving problems

[0011] One aspect of this disclosure is a state monitoring device for industrial machinery, comprising: an imaging device that captures the industrial machinery under the control of a controller; a dynamic image generation unit that correlates image data acquired by the imaging device with at least one of the input and output signals of the controller along a time axis to generate a dynamic image representing the state changes of the industrial machinery and the input and output signals; and a dynamic image regeneration device that regenerates the dynamic image generated by the dynamic image generation unit.

[0012] One aspect of this disclosure is a method for monitoring the state of industrial machinery. This method involves capturing images of the industrial machinery under the control of a controller using a camera device, establishing a correlation between the image data acquired by the camera device and at least one of the input and output signals of the controller along a time axis, thereby generating a dynamic image representing the state changes of the industrial machinery and the input / output signals. The dynamic image is then regenerated using a dynamic image regeneration device.

[0013] Effects of the Invention

[0014] According to one aspect of this disclosure, the correlation between the operational state of industrial machinery and the state of the input / output signals of its controller can be readily determined along a time axis. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a robot system equipped with a robot status monitoring device according to an embodiment of the present disclosure.

[0016] Figure 2A It is a diagram that schematically illustrates the relationship between the frame rate of image data captured by a camera and the update cycle of the input and output signals.

[0017] Figure 2B It is a diagram that schematically illustrates the relationship between the frame rate of image data captured by a camera and the update cycle of the input and output signals.

[0018] Figure 3 This is a diagram showing an example of a motion image generated by the motion image generation unit.

[0019] Figure 4A This is an example of a program editing image (before the operator performs editing operations) in the program editing department.

[0020] Figure 4B This is an example of a program editing image (after editing operations performed by the operator) in the program editing department. Detailed Implementation

[0021] Hereinafter, with reference to the accompanying drawings, a robot system 1 having a robot state monitoring device 2 according to an embodiment of the present disclosure will be described.

[0022] Figure 1 This is a schematic diagram of a robot system 1 equipped with the robot status monitoring device 2 of this embodiment.

[0023] The robot system 1 includes a robot 3 as an industrial machine, a first peripheral device 41, a second peripheral device 42, and a robot status monitoring device 2 for monitoring the status of the robot 3 and the peripheral devices 41 and 42.

[0024] In this embodiment, the robot 3 is described as, for example, a transport robot that holds a workpiece at a predetermined position and performs a series of transport actions to transport the held workpiece to the predetermined position according to control signals sent from the controller 51 (described later). However, this disclosure is not limited to this. The robot 3 can be any robot that operates according to control signals sent from the controller 51, such as a welding robot or a painting robot.

[0025] The robot status monitoring device 2 includes: a robot control device 5, which controls the robot 3; a camera 6, which acts as a camera to capture the conveying motion of the robot 3; and a dynamic image regeneration device 7, which regenerates dynamic images.

[0026] Camera 6 is positioned near robot 3. Camera 6 captures the conveying motion of robot 3 under the control of robot control device 5 at a predetermined frame rate, and sends the acquired image data of robot 3 to robot control device 5.

[0027] The robot control device 5 is a computer composed of hardware including a communication unit (not shown) that communicates with peripheral devices 41 and 42 via a communication line 43, a processing unit such as a CPU (Central Processing Unit) (not shown), auxiliary storage units such as HDDs (Hard Disk Drives) and SSDs (Solid State Drives) that store various programs, and a main storage unit such as RAM (Random Access Memory) that temporarily stores the data required when the processing unit executes programs (not shown). Figure 1 As shown, in the robot control device 5, various functions such as controller 51, program storage unit 52, dynamic image generation unit 53, program editing unit 54, and dynamic image saving unit 55 are realized through the hardware structure.

[0028] The controller 51 exchanges digital and analog signals with peripheral devices 41 and 42 via communication line 43, and controls the robot 3 according to the program stored in the program storage unit 52.

[0029] The program storage unit 52 stores multiple programs for specifying the values ​​of output signals output from the controller 51 to the robot 3 or peripheral devices 41, 42. Furthermore, at least a portion of the multiple programs stored in the program storage unit 52 can be edited via the program editing unit 54, described later.

[0030] The motion image generation unit 53 acquires image data of the robot 3 transmitted from the camera 6 and input / output signals of the controller 51, and establishes a correlation between the acquired image data and the input / output signals along a common time axis, thereby generating a motion image representing the state changes of the robot 3 and the input / output signals (see below). Figure 3 The input / output signals of the controller 51 obtained by the dynamic image generation unit 53 are at least one of the following: digital input signals (DI[1], DI[2], ...) and analog input signals (AI[1], AI[2], ...) input from the robot 3 or peripheral devices 41, 42 to the controller 51; and digital output signals (DO[1], DO[2], ...) and analog output signals (AO[1], AO[2], ...) output from the controller 51 to the robot 3 or peripheral devices 41, 42. In addition, in this embodiment, a digital signal is a signal that can take only two values ​​consisting of ON and OFF, and an analog signal is a signal that can take integer values ​​within a predetermined range.

[0031] The motion image regeneration device 7 is, for example, a portable communication terminal such as a tablet terminal that can communicate with the robot control device 5. The motion image regeneration device 7 regenerates motion images generated by the motion image generation unit 53 and stored in the motion image storage unit 55 according to the operator's operation.

[0032] Next, refer to Figure 2A , Figure 2B and Figure 3 The steps for generating a dynamic image in the dynamic image generation unit 53 are explained.

[0033] Figure 2A as well as Figure 2B This is a diagram schematically illustrating the relationship between the frame rate [fps] of the image data captured by camera 6 and the update period [seconds] of the input and output signals. Figure 2A as well as Figure 2B In this diagram, only the digital input / output signal DI[1] is illustrated as the input / output signal acquired in the dynamic image generation unit 53. Figure 2A as well as Figure 2B The example shown illustrates a case where the frame rate of the image data is set to 60 fps, meaning that each frame constituting the image data is captured every 1 / 60th of a second, but this disclosure is not limited to this. Furthermore, in Figure 2A The example shown illustrates the case where the update period of the digital input signal DI[1] is set to 1 / 20[second]. Figure 2B The example shown illustrates the case where the update period of the digital input signal DI[1] is set to 1 / 40[second], but this disclosure is not limited thereto.

[0034] like Figure 2A as well as Figure 2B As shown, the image data captured by camera 6 consists of multiple frames F1, F2, F3, F4, ... captured at a period corresponding to the frame rate, i.e., 1 / 60 [second]. More specifically, frame F1 is captured at time t1, frame F2 is captured at time t2, 1 / 60 [second] after time t1, frame F3 is captured at time t3, 2 / 60 [second] after time t1, and frame F4 is captured at time t4, 3 / 60 [second] after time t1.

[0035] The motion image generation unit 53 acquires the values ​​of the digital input signals DI[1] at recording times t1 to t4 for each frame F1 to F4 at the same period as the frame rate of the image data of the camera 6, and correlates the acquired image data of the camera 6 and the digital input signals DI[1] along a common time axis, thereby generating a representation of the state changes of the robot 3 and the digital input signals DI[1], as described later. Figure 3 The dynamic image shown.

[0036] As mentioned above, in Figure 2A In the example shown, the digital input signal DI[1] is updated every 1 / 20 of a second. Therefore, the value of the digital input signal DI[1], which is "OFF" at time t1, is switched to "ON" at time t4. Therefore, in this case, the motion picture generation unit 53 obtains the value of "OFF" as the digital input signal DI[1] at the recording time t1 of frame F1, obtains the value of "OFF" as the digital input signal DI[1] at the recording time t2 of frame F2, obtains the value of "OFF" as the digital input signal DI[1] at the recording time t3 of frame F3, and obtains the value of "ON" as the digital input signal DI[1] at the recording time t4 of frame F4.

[0037] In addition, as mentioned above, in Figure 2BIn the example shown, the digital input signal DI[1] is updated every 1 / 40 of a second. Therefore, the value of the digital input signal DI[1], which is "off" at time t1, switches to "on" at time t5 between time t2 and time t3, and then switches to "off" at time t4. Therefore, in this case, the motion picture generation unit 53 obtains the value of "off" as the digital input signal DI[1] at the recording time t1 of frame F1, obtains the value of "off" as the digital input signal DI[1] at the recording time t2 of frame F2, obtains the value of "on" as the digital input signal DI[1] at the recording time t3 of frame F3, and obtains the value of "off" as the digital input signal DI[1] at the recording time t4 of frame F4.

[0038] As described above, in the motion picture generation unit 53, the input / output signal values ​​of the controller 51 at the recording time of each frame are acquired at the same period as the frame rate of the image data of the camera 6. Therefore, in order to acquire all the state changes of the input / output signal of the controller 51 in the motion picture generation unit 53, the reciprocal of the frame rate of the camera 6 (the shooting period of each frame) is preferably less than or equal to the update period of the input / output signal of the controller 51.

[0039] Figure 3 This is a diagram showing an example of a motion image generated by the motion image generation unit 53.

[0040] The motion image generation unit 53 generates a motion image by establishing a correlation between the image data of the robot 3 obtained above and the input / output signals of the controller 51 along a common time axis. Figure 3 The example includes a dynamic image display bar 81 (see reference). Figure 3 (Upper left of the middle section), signal status display bar 82 (refer to) Figure 3 (upper right side of the middle) and timeline column 83 (see reference) Figure 3 The animated image (the lower part of the text).

[0041] The signal status display bar 82 displays the value of time t (hereinafter referred to as "current time t") determined by the time bar 833 (described later), and the input / output signals selected by the operator (in... Figure 3 In the example, represents the value of the current time t when the digital output signal DO[1], analog output signal AO[1], digital input signal DI[1], and analog input signal AI[1] are selected.

[0042] The dynamic image display bar 81 displays dynamic images of robot 3 (i.e., images obtained by continuously reproducing still images captured by camera 6 at a predetermined frame rate). Additionally, in Figure 3 The image shown is a still image of robot 3 at the current time t.

[0043] Displayed in timeline bar 83 Figure 3 The time axis 830 extends horizontally, a strip-shaped frame bar 831 extends parallel to the time axis 830 below it, a timing diagram bar 832 for input and output signals extends parallel to the time axis 830 below the frame bar 831, and... Figure 3 The time bar 833 extends vertically across the time axis 830, frame bar 831, and timing graph bar 832. Here, the current time t corresponds to the coordinate value of the time bar 833 on the time axis 830.

[0044] On frame bar 831, multiple frames are arranged along time axis 830 (in... Figure 3 In the example, there are 6 still images of robot 3 in each frame. That is, the dynamic image of robot 3 displayed in the dynamic image display bar 81 is an image obtained by continuously reproducing multiple still images arranged along the time axis 830 in the frame bar 831.

[0045] In timing diagram column 832, a timing diagram image is displayed showing the changes in the values ​​of multiple input / output signals DO[1], AO[1], DI[1] and AI[1] selected by the operator along the time axis 830.

[0046] By operating the dynamic image regeneration device 7, the operator can fix the position of the time bar 833 while scrolling the time axis 830, frame bar 831 and timing graph bar 832 together in the left and right directions, or fix the position of the time axis 830, frame bar 831 and timing graph bar 832 while only scrolling the time bar 833 in the left and right directions.

[0047] Next, refer to Figure 4A as well as Figure 4B The order in which the program stored in the program storage unit 52 is edited by the program editing unit 54 will be explained. In addition, the following will explain the case in which the program of the multiple programs stored in the program storage unit 52 is edited by the program editing unit 54, and the program that specifies the value of the digital output signal DO[1] of the controller 51 will be edited.

[0048] If, during the reproduction of a motion picture generated by the motion picture reproduction device 7, the program editing unit 54 performs a predetermined program editing start operation, it stops the reproduction of the motion picture and... Figure 4A The program editing image shown in the example is displayed on the dynamic image regeneration device 7, and is transferred to the program editing mode for editing the program of the digital output signal DO[1].

[0049] like Figure 4AAs shown, the program editing unit 54 displays at least the program editing image containing the timeline column 83 on the dynamic image regeneration device 7. The timeline column 83 in the program editing image includes at least: a frame bar 831 displaying still images of the robot 3 in each frame; and a timing graph column 832 of the digital output signal DO[1] generated by the program that can be edited by the program editing unit 54.

[0050] Program Editorial Department 54 Based on Figure 4A The operator in the program editing image shows a predetermined editing operation on the motion image regeneration device 7, which edits the program for the specified value of the digital output signal DO[1]. The operator can directly change the value of the digital output signal DO[1] in the program editing image. When the program editing unit 54 receives the editing operation performed by the operator in the program editing image, it associates the changed value of the digital output signal DO[1] based on the editing operation with the state of the robot 3 displayed on the frame bar 831, and edits the program of the digital output signal DO[1].

[0051] Reference Figure 4B An example of program editing in Program Editing Department 54 will be explained. Figure 4A and Figure 4B The diagram shows the robot 3's hand opening between time th and ti, and closing after time ti. Additionally, in... Figure 4B In the diagram, the solid line represents the value of the digital output signal DO[1] generated according to the program before editing by the program editing unit 54, and the dashed line represents the value of the digital output signal DO[1] after being changed by the editing operation performed by the operator. Additionally, Figure 4B This indicates that an editing operation was performed to change the value of the digital output signal DO[1] after time ti from "0" to "1". The program editing unit 54 is operated by the operator. Figure 4B During the editing operation shown, the value of the modified digital output signal DO[1] is associated with the state of the robot 3, and the program of the digital output signal DO[1] is edited. More specifically, the program editing unit 54 edits the program of the digital output signal DO[1] so that it outputs "0" when the robot 3's hand is open and "1" when the robot 3's hand is closed. Therefore, according to the program edited by the program editing unit 54, the controller 51 sets the value of the digital output signal DO[1] to "0" when the robot 3's hand is open and sets the value of the digital output signal DO[1] to "1" when the robot 3's hand is closed.

[0052] In the robot state monitoring device 2 described above, the states of the robot 3 and peripheral devices 41 and 42 are monitored in the following order. First, the robot 3 under the control of the controller 51 is captured by the camera 6. Next, the motion image generation unit 53 correlates the image data acquired by the camera 6 and the input / output signals of the controller 51 along the time axis to generate a representation of the state changes of the robot 3 and the input / output signals. Figure 3 The animated image shown is saved to the animated image storage unit 55. Then, the animated image saved in the animated image storage unit 55 is reproduced by using the animated image playback device 7.

[0053] According to this embodiment, the following effects are achieved.

[0054] The robot state monitoring device 2 according to this embodiment includes: a camera 6 that captures images of the robot 3 under the control of the controller 51; a motion image generation unit 53 that correlates the image data acquired by the camera 6 with the input and output signals of the controller 51 along a common time axis 830 to generate a motion image representing the state changes of the robot 3 and the input and output signals; and a motion image playback device 7 that plays back the motion image generated by the motion image generation unit 53 and stored in the motion image storage unit 55. According to this embodiment, by generating such a motion image, the operator can simultaneously confirm the action state of the robot 3 and the state of the input and output signals of the controller 51 by simply viewing the motion image played back by the motion image playback device 7, and thus easily clarify the correlation between the action state of the robot 3 and the state of the input and output signals of the controller 51 along the time axis 830.

[0055] Furthermore, in the robot state monitoring device 2 of this embodiment, the dynamic image generation unit 53 acquires the values ​​of the input / output signals of the controller 51 at the recording time of each frame at a period that is the same as the frame rate of the image data captured by the camera 6. According to this embodiment, by setting the shooting period of each frame to be less than or equal to the update period of the input / output signals, all state changes of the input / output signals of the controller 51 can be acquired.

[0056] Furthermore, in the robot state monitoring device 2 of this embodiment, the dynamic image generation unit 53 generates a dynamic image, which includes a dynamic image display bar 81 displaying a dynamic image of the robot 3, a frame bar 831 displaying still images of the robot 3 in each frame arranged along the time axis 830, and a timeline bar 83 displaying a timing diagram image of the input and output signals along the time axis 830. According to this embodiment, by generating such a dynamic image including the dynamic image display bar 81 and the timeline bar 83, the operator can simultaneously confirm the motion state of the robot 3 and the state of the input and output signals of the controller 51 by simply viewing the dynamic image regenerated by the dynamic image reproduction device 7, and thus easily clarify the correlation between the motion state of the robot 3 and the state of the input and output signals of the controller 51 along the time axis 830.

[0057] Furthermore, in the robot status monitoring device 2 of this embodiment, the program editing unit 54 edits the program stored in the program storage unit 52 based on the editing operations performed by the operator in the program editing image including the timeline bar 83. Therefore, the operator can easily edit the program visually in conjunction with the robot 3's status displayed in the timeline bar 83, which is convenient.

[0058] This disclosure is not limited to the described embodiments, and various changes and modifications are possible.

[0059] For example, in the described embodiment, the condition monitoring device for industrial machinery is applied to the robot condition monitoring device 2, but it is not limited thereto. In addition to the robot 3, it can also be applied to condition monitoring devices for various machine tools and other industrial machinery.

[0060] Furthermore, while the above embodiment describes a case where the dynamic image playback device 7 is a portable communication terminal such as a tablet computer with both communication and dynamic image display functions, it is not limited to this. The dynamic image playback device 7 can also be a portable teaching device equipped with operation keys operable by an operator and a display capable of showing dynamic images for teaching the robot 3 predetermined actions. This reduces the burden of changing equipment when the operator wants to use both the teaching device and the dynamic image playback device simultaneously.

[0061] Explanation of reference numerals in the attached figures

[0062] 1…robot system,

[0063] 2… Robot condition monitoring device (industrial machinery condition monitoring device)

[0064] 3… Robots (industrial machinery)

[0065] 41…First peripheral equipment,

[0066] 42…Second peripheral equipment,

[0067] 43…communication line,

[0068] 5… Robot control device,

[0069] 51… controller,

[0070] 53…Motion Image Generation Department

[0071] 54…Programming Department

[0072] 55…Dynamic Image Storage Department

[0073] 6…camera (video recording device),

[0074] 7…Dynamic image regeneration device (dynamic image regeneration device, teaching device for industrial machinery),

[0075] 81…Dynamic Image Display Bar

[0076] 82… Signal status display bar

[0077] 83…Timeline column.

Claims

1. A condition monitoring device for industrial machinery, characterized in that, The condition monitoring device for this industrial machinery includes: Filming device, industrial machinery controlled by its filming controller; The dynamic image generation unit establishes a correlation along a time axis between the image data acquired by the imaging device and at least one of the input and output signals of the controller, thereby generating a dynamic image representing the state changes of the industrial machinery and the input and output signals. as well as A dynamic image regeneration apparatus that regenerates a dynamic image generated by the dynamic image generation unit. The dynamic image generation unit generates the dynamic image, which includes: A dynamic image display bar that displays dynamic images of the industrial machinery; and The timeline is displayed, along with frame bars extending parallel to the timeline, and a timeline bar in the timing diagram column extending parallel to the timeline below the frame bars. On the frame bar, multiple still images of the industrial machinery from each frame are arranged along the time axis. In the timing diagram section, a timing diagram image is configured to represent the variation of the input and output signals selected by the operator along the time axis. In the timeline column, a time bar is configured that extends across the time axis, the frame bar, and the sequence diagram column.

2. The industrial machinery condition monitoring device according to claim 1, characterized in that, The dynamic image generation unit acquires the values ​​of the input / output signals at the recording time of each frame at a period equal to the frame rate of the image data.

3. The condition monitoring device for industrial machinery according to claim 1, characterized in that, The condition monitoring device for this industrial machinery includes: A program storage unit stores a program for specifying the value of the output signal in the controller; and The program editing department edits the program based on the operator's actions in the editing image containing the timeline bar.

4. The condition monitoring device for industrial machinery according to any one of claims 1 to 3, characterized in that, The dynamic image regeneration device is a teaching device for the industrial machinery.

5. A method for monitoring the condition of industrial machinery, characterized in that, Filming industrial machinery under the control of a controller using a camera device. The image data acquired by the camera device and the input / output signals, which are at least one of the input and output signals of the controller, are correlated along the time axis to generate a dynamic image representing the state changes of the industrial machinery and the input / output signals. The dynamic image is regenerated by a dynamic image regeneration device. The dynamic image is generated, which includes: a dynamic image display bar displaying the dynamic image of the industrial machinery, a timeline displaying the time axis, frame bars extending parallel to the time axis, and a timeline bar below the frame bars extending parallel to the time axis. On the frame bar, multiple still images of the industrial machinery from each frame are arranged along the time axis. In the timing diagram section, a timing diagram image is configured to represent the variation of the input and output signals selected by the operator along the time axis. In the timeline column, a time bar is configured that extends across the time axis, the frame bar, and the sequence diagram column.

Images