Control device, image recording method, and recording medium

Abstract

The control device is connected to the controlled device and the imaging device (22), and controls the controlled device. The control device includes: a first receiving unit (41) that repeatedly receives image information representing an image related to the control of the controlled device captured by the imaging device (22), and the image information includes a first timestamp that shows the elapsed time from a specific moment in association with the image; a second receiving unit (42) that receives communication information related to the communication history of the communication performed by the imaging device (22), and the communication information includes a second timestamp indicating the elapsed time from a specific moment and transmission time information indicating the transmission time when the communication information is transmitted; and a storage unit (43) that stores association information in the storage units (433, 44) based on a comparison between the first timestamp associated with the image shown by the image information and the second timestamp corresponding to the transmission time, the association information showing the corresponding moment corresponding to the first timestamp in association with the image.

Description

Technical Field

[0001] The present invention relates to a control device, an image recording method, and a program. Background Art

[0002] In the field of FA (Factory Automation), a control device controls various devices to implement a production line represented by a manufacturing line, a processing line, and an inspection line. As process management of the production line, there is a technique of recording an image obtained by photographing the production line, and when a failure such as a stop of the production line occurs, inspecting by using the image that records the situation at the time of its occurrence to trace the cause of the failure. For example, by comparing the captured image with the data logged by a PLC (Programmable Logic Controller) as a control device, the detailed situation at the time of the failure is grasped, and the cause of the failure is analyzed (for example, refer to Patent Document 1).

[0003] Patent Document 1 describes that a PLC has a basic unit that executes a ladder diagram program for controlling various devices installed in a factory and an expansion unit that inputs image data from a camera as a field device. The expansion unit transmits the time information and the image data obtained by the camera to the basic unit in an associated manner. According to the PLC of Patent Document 1, the device values and image data stored in the basic unit can be displayed in association with the time, and the user can identify the cause of the failure.

[0004] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2020-13526 Summary of the Invention

[0005] In the expansion unit described in Patent Document 1, for example, if a collection condition such as a relay device being turned on is satisfied, the collection unit outputs a trigger signal. Then, the time management unit of the expansion unit reads the time information from the internal clock according to the trigger signal, and the camera outputs the captured image data according to the trigger signal.

[0006] However, regarding the acquisition of time, since it is performed as an internal process of the expansion unit, it is completed in a short time. In contrast, regarding the acquisition of image data, there is a communication delay between the expansion unit and the camera. Therefore, the time associated with the image data in the expansion unit is different from the actual time when the camera captures the image. As a result, the user who inspects the image afterwards may mistake the actual time when the failure occurred in the captured image, and on top of that, it is impossible to achieve consistency with the data logged by a control device such as a PLC, which is inconvenient as a result. Therefore, there is potential to give a more convenient time to the image data acquired from the outside by the control device.

[0007] An object of the present invention is to give a more convenient time to the data of an image acquired from the outside by a control device.

[0008] To achieve the above object, a control device of the present invention is connected to a controlled device and a photographing device to control the controlled device, and the control device includes: a first receiving unit that repeatedly receives image information representing an image related to the control of the controlled device photographed by the photographing device, and the image information includes a first time stamp that shows the elapsed time from a specific time in association with the image; a second receiving unit that receives communication information related to the communication history of the photographing device, and the communication information includes a second time stamp representing the elapsed time from a specific time and transmission time information representing the transmission time when the communication information is transmitted; and a storage unit that stores association information in a storage unit based on a comparison between the first time stamp associated with the image shown by the image information and the second time stamp corresponding to the transmission time, the association information showing the corresponding time corresponding to the first time stamp in association with the image.

[0009] Effects of the Invention

[0010] According to the present invention, the first receiving unit receives image information including the first time stamp, the second receiving unit receives communication information including the second time stamp, and the storage unit stores association information in the storage unit based on the comparison between the first time stamp and the second time stamp, the association information showing the corresponding time corresponding to the first time stamp in association with the image. Therefore, a corresponding time corresponding to the first time stamp pre-given before the reception by the first receiving unit is given to the image. Thereby, the influence of communication delay can be excluded, and a more convenient time can be given to the data of the image acquired from the outside by the control device. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a diagram showing the structure of a control system according to Embodiment 1.

[0012] Figure 2 It is a diagram for explaining the image information and communication information according to Embodiment 1.

[0013] Figure 3 It is a diagram showing the hardware structure of an execution unit, an input / output unit, and an image recording unit according to Embodiment 1.

[0014] Figure 4 It is a diagram showing the functional structure of an image recording unit according to Embodiment 1.

[0015] Figure 5 It is a diagram for explaining the corresponding time according to Embodiment 1.

[0016] Figure 6 This is a flowchart showing the image recording process related to Embodiment 1.

[0017] Figure 7 This is a diagram showing an example of the calculation of the corresponding time related to Embodiment 1.

[0018] Figure 8 This is a diagram showing the time stamps assigned to the images related to Embodiment 2.

[0019] Figure 9 This is a diagram showing an example of the calculation of the corresponding time related to Embodiment 2.

[0020] Figure 10 This is a diagram showing an example of the calculation of the corresponding time related to Embodiment 3.

[0021] Figure 11 This is the first diagram showing the functional structure of the image recording unit related to the modification example.

[0022] Figure 12 This is the second diagram showing the functional structure of the image recording unit related to the modification example. Detailed Embodiments

[0023] Hereinafter, while referring to the attached Figure 1 the control device 10 related to the embodiments of the present invention will be described in detail.

[0024] Embodiment 1

[0025] In Figure 1 the structure of the control system 100 formed by the control device 10 related to the present embodiment is shown. The control system 100 is a system constructed in a factory to operate a manufacturing line and to photograph and record images for failure inspection. The control system 100 includes a control device 10 that controls the controlled device 21, a controlled device 21 provided on the manufacturing line, and a photographing device 22 that photographs images of the conditions related to the manufacturing line.

[0026] The controlled device 21 and the photographing device 22 are connected to the control device 10 via an industrial network. However, one or both of the controlled device 21 and the photographing device 22 may communicate with the control device 10 by a method different from the communication via the industrial network. As such a method, there are communication via a LAN (Local Area Network) or a dedicated line as an information network and one-way communication that transmits a current signal or a voltage signal via wiring.

[0027] The controlled device 21 is a sensor, actuator, robot, or other FA device installed on the manufacturing line. The controlled device 21 is connected to the input / output unit 12 of the control device 10, and operates according to instructions from the control device 10. For example, the controlled device 21 as a sensor notifies the control device 10 of the sensing result according to the cycle specified by the control device 10. In addition, the controlled device 21 as an actuator moves the workpiece according to the timing and speed specified by the control device 10. In addition, in Figure 1 1 is representatively shown as a controlled device 21, but the number of devices to be controlled by the control device 10 in the control system 100 may be greater than or equal to 2. The manufacturing line operates by the control device 10 controlling one or more controlled devices 21.

[0028] The photographing device 22 has a camera, and the photographing device 22 photographs an image of a situation related to the control of the controlled device 21 by the control device 10. The image is an image of the scene where the controlled device 21 is operating, but is not limited to the image of the controlled device 21 itself. The photographing object of the photographing device 22 may be the controlled device 21, or may be a workpiece or a conveyor belt on a manufacturing line, or may be the environment forming the control system 100. For example, when the control system 100 operates a food manufacturing line, the photographing device 22 may monitor the entrance and exit of the factory where the control system 100 is constructed, and the photographed image may be used to check whether there is an invasion of pests. When the operation of the control system 100 should be stopped when pests invade, the operation is stopped to indicate whether the controlled device 21 should be controlled, and an image related to the operation of the controlled device 21 is photographed. In addition, when the invading pests are eliminated in the inspection process, the operation of the controlled device 21 controlled by the control device 10 in the inspection process will be different depending on the presence or absence of pests, so an image related to the operation of the controlled device 21 is photographed. The example of photographing the entrance and exit of a factory is described, but the present invention is not limited to this. The image photographed by the photographing device 22 can be any image that changes due to the action of the controlled device 21 controlled by the control device 10, changes in conjunction with the action, or is used as a judgment material for determining the action.

[0029] The image captured by the imaging device 22 can be a still image or a moving image, or other images including visible light images or thermal images. Also, the imaging device 22 repeatedly sends the image information representing the captured image to the image recording unit 13 of the control device 10. The transmission cycle of the image information is set, for example, to 1 second, 1 minute, or 1 hour. A time stamp indicating the elapsed time since a specific moment is assigned to the image information repeatedly sent from the imaging device 22. The time stamp is information indicating the timing at which the time stamp is assigned, and the time stamp assigned to the image information represents the elapsed time from a specific moment to the time point at which the time stamp is assigned.

[0030] Here, with reference to Figure 2 , the information sent from the imaging device 22 to the control device 10 will be described. As Figure 2 shown, the imaging device 22 includes an imaging sensor 221 equivalent to a camera, an image processing unit 222 that processes the image captured by the imaging sensor 221, and a communication unit 223 that sends the image information representing the captured image to the control device 10. The imaging sensor 221 sends the image data representing the repeatedly captured images to the image processing unit 222.

[0031] The image processing unit 222 includes an MPU (Micro-Processing Unit), and this image processing unit 222 assigns time stamps to the image data sent from the imaging sensor 221 respectively. The time stamp can be embedded in the image data or attached to the image data as information different from the image data. The image processing unit 222 repeatedly sends the image information representing the image and including the time stamp to the communication unit 223. In Figure 2 , "image information [1]" represents the image information sent for the first time, and "image information [N]" represents the image information sent for the Nth time. In addition, the image processing unit 222 may perform other image processing in addition to the assignment of the time stamp. Other image processing includes, for example, conversion processing or compression processing into a form suitable for communication with the control device 10.

[0032] The time stamp constituting the image information shows the elapsed time from a specific moment in the form processed in the imaging device 22. The form processed in the imaging device 22 is, for example, a form represented by the number of clocks in the imaging device 22, a form in milliseconds, or a form representing the frame number when the imaging device 22 captures a series of images as a video. The specific moment is, for example, the capture moment of the first image in a series of images sequentially captured by the imaging sensor 221, the start moment of the execution program of the capture process, or the moment when the start command for capture is received from the control device 10. The time stamp represents the relative time interval between repeatedly captured images and does not represent an absolute moment such as Coordinated Universal Time. In addition, since the time stamp is given immediately after the image is output from the imaging sensor 221, it usually corresponds to the capture moment.

[0033] The communication unit 223 includes a communication interface circuit for communicating with a device external to the imaging device 22. The communication unit 223 repeatedly transmits the image information sent from the image processing unit 222 to the control device 10. The image information is transmitted according to a specification typified by RTP (Real-time Transport Protocol). When transmitting the image information to the control device 10 via the communication unit 223, a transmission preparation delay and a transmission delay of the imaging device 22 occur. Therefore, as Figure 2 shown, a time difference T10 occurs between the actual capture moment and the moment when the control device 10 receives the image information.

[0034] In addition, when continuously transmitting the image information to the control device 10, the communication unit 223 transmits communication information regarding the communication history with the control device 10. The communication information includes statistical history information indicating the past communication status, a time stamp given in the same form as the image information, and transmission time information indicating the transmission moment when the communication information is transmitted.

[0035] The history information is, for example, information indicating the total number of data packets transmitted to the control device 10 or the total amount of the payload of the data packets. However, the history information is not limited to this, and any statistical information on the communication status is acceptable. Regarding the image information, real-time performance is emphasized. Therefore, for example, even if the image information is lost on the transmission path to the control device 10 due to congestion, the subsequent image information is sequentially transmitted. The history information is used to confirm to what extent such image information is received on the control device 10 side.

[0036] The timestamp included in the communication information indicates the elapsed time from the common time in a format common to the timestamp included in the image information. Hereinafter, the timestamp included in the image information is appropriately recorded as the first timestamp, and the timestamp included in the communication information is appropriately recorded as the second timestamp. The second timestamp is information indicating the elapsed time from a specific time used as the reference time of the first timestamp to the time point to which the second timestamp is assigned, and indicates the relative time interval with the first timestamp. The time point to which the second timestamp is assigned is equal to the sending time indicated by the sending time information.

[0037] The transmission time indicated by the transmission time information is an absolute time common between the imaging device 22 and the control device 10, and is expressed in a microsecond unit in accordance with a time synchronization protocol represented by NTP (Network Time Protocol) and PTP (Precision Time Protocol), for example.

[0038] In order to clearly indicate that the history shown by the history information is the history up to the time point when the communication information was sent, the communication information is given a second time stamp and the sending time information. Figure 2 In the example, the time T20 at which the communication information is transmitted is notified to the control device 10 in two forms: the second time stamp and the transmission time. In addition, the communication delay may be estimated by sending a packet including the transmission time information back and forth between the imaging device 22 and the control device 10.

[0039] In addition, the timing of sending communication information is arbitrarily set. For example, the camera 22 may send communication information in response to a request from the control device 10, or the camera 22 may spontaneously and periodically send communication information. The communication information is transmitted in accordance with a specification represented by RTCP (RTP Control Protocol).

[0040] return Figure 1, the control device 10 is a PLC that controls the controlled device 21 by executing the control process specified by the control program 111 provided by the user, and operates the manufacturing line. The control device 10 has an execution unit 11 that executes the control program 111, an input / output unit 12 that communicates with the controlled device 21, and an image recording unit 13 that records the image information provided by the imaging device 22. The execution unit 11, the input / output unit 12, and the image recording unit 13 are respectively detachable modules with respect to an unillustrated basic unit having a system bus 101. The execution unit 11, the input / output unit 12, and the image recording unit 13 transmit signals to each other via the system bus 101. The execution unit 11 corresponds to a so-called CPU unit, and the input / output unit 12 corresponds to a so-called I / O (Input / Output) unit.

[0041] In Figure 3 the hardware structures of the execution unit 11, the input / output unit 12, and the image recording unit 13 are shown respectively. As Figure 3 shown, the execution unit 11, the input / output unit 12, and the image recording unit 13 each have a processor 31, a main storage unit 32, an auxiliary storage unit 33, an input unit 34, an output unit 35, and a communication unit 36. The main storage unit 32, the auxiliary storage unit 33, the input unit 34, the output unit 35, and the communication unit 36 are all connected to the processor 31 via an internal bus 37.

[0042] The processor 31 includes a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor 31 realizes various functions by executing the program P1 stored in the auxiliary storage unit 33 and executes the processes described later.

[0043] The main storage unit 32 includes a RAM (Random Access Memory). The program P1 is loaded from the auxiliary storage unit 33 to the main storage unit 32. And, the main storage unit 32 is used as the working area of the processor 31.

[0044] The auxiliary storage unit 33 includes non-volatile memories represented by an EEPROM (Electrically Erasable Programmable Read-Only Memory) and an HDD (Hard Disk Drive). In addition to storing the program P1, the auxiliary storage unit 33 also stores various data used in the processing of the processor 31. The auxiliary storage unit 33 supplies the data used by the processor 31 to the processor 31 according to the instruction of the processor 31. In addition, the auxiliary storage unit 33 stores the data supplied from the processor 31.

[0045] The input unit 34 includes input devices typified by hardware switches and input keys. The input unit 34 acquires information input by the user and notifies the acquired information to the processor 31.

[0046] The output unit 35 includes output devices typified by an LED (Light Emitting Diode), an LCD (Liquid Crystal Display), and a speaker. The output unit 35 presents various information to the user according to the instruction of the processor 31.

[0047] The communication unit 36 includes a network interface circuit for communicating with an external device. The communication unit 36 receives a signal from the outside and outputs data represented by the signal to the processor 31. Further, the communication unit 36 transmits a signal representing the data output from the processor 31 to an external device. In addition, Figure 3 one communication unit 36 is representatively shown in

[0048] but is not limited thereto. For example, the image recording unit 13 may additionally have a communication unit 36 for communicating with the execution unit 11 and a communication unit 36 for communicating with the photographing device 22. Figure 3 through the cooperation of the structural elements of the hardware shown by Figure 4 the image recording unit 13 of the control device 10 exhibits the Figure 4 functions shown. Specifically, as its functions, the image recording unit 13 has: a communication unit 40 that communicates with the execution unit 11 and the photographing device 22 for time synchronization; a first receiving unit 41 that receives image information transmitted from the photographing device 22; a second receiving unit 42 that receives communication information transmitted from the photographing device 22; a storage unit 43 that stores association information associating the photographed image with the system time corresponding to the timestamp in the storage unit 44; and a storage unit 44 that stores the association information. In

[0049] The communication unit 40 is mainly realized through the coordinated operation between the processor 31 and the communication unit 36. The communication unit 40 transmits and receives time synchronization information, which is used to synchronize the system time common among the devices constituting the control system 100 according to specifications represented by NTP or PTP among the devices. Specifically, the communication unit 40 synchronizes the system time managed by the execution unit 11 with the system time managed by the image recording unit 13 by receiving the time synchronization information from the execution unit 11 and sending it to the control unit 434. In addition, the communication unit 40 synchronizes the system time managed by the image recording unit 13 with the system time managed by the imaging device 22 by receiving the time synchronization information from the control unit 434 and sending it to the imaging device 22.

[0050] The first receiving unit 41 is mainly realized through the coordinated operation between the processor 31 and the communication unit 36. The first receiving unit 41 receives the image information repeatedly transmitted from the imaging device 22 and sends the received image information to the storage unit 43. The first receiving unit 41 is an example of the first receiving unit in the control device 10. The first receiving unit repeatedly receives the image information representing the image related to the control of the controlled device captured by the imaging device, and the image information includes the first time stamp showing the elapsed time from a specific moment in association with the image.

[0051] The second receiving unit 42 is mainly realized through the coordinated operation between the processor 31 and the communication unit 36. The second receiving unit 42 receives the communication information transmitted from the imaging device 22 and sends the received communication information to the storage unit 43. The second receiving unit 42 is an example of the second receiving unit in the control device 10. The second receiving unit receives the communication information related to the communication history of the imaging device, and the communication information includes the second time stamp showing the elapsed time from a specific moment and the transmission time information showing the transmission time of the communication information.

[0052] The storage unit 43 generates association information associating the corresponding time corresponding to the first timestamp with the image represented by the image information received by the first receiving unit 41, and stores the generated association information in the storage unit 44, where the first timestamp is associated with the image. The corresponding time is a time represented in the same manner as the system time managed by the image recording unit 13. The storage unit 43 is an example of a storage unit in the control device 10. Based on the comparison between the first timestamp associated with the image shown by the image information and the second timestamp corresponding to the transmission time, the storage unit stores the association information in the storage unit, which shows the corresponding time corresponding to the first timestamp associated with the image. The storage unit 43 includes: a time calculation unit 431 that calculates the corresponding time based on the first timestamp and the second timestamp and assigns it to the image; an image processing unit 432 that processes the image; a storage unit 433 that temporarily stores the association information associating the corresponding time with the image; and a control unit 434 that extracts the association information that satisfies the condition from the storage unit 433 and stores it in the storage unit 44.

[0053] The time calculation unit 431 is mainly implemented by the processor 31. The time calculation unit 431 reads the first timestamp from the image information obtained via the first receiving unit 41, reads the second timestamp from the communication information obtained via the second receiving unit 42, and compares the first timestamp with the second timestamp. Specifically, the time calculation unit 431 calculates the difference by subtracting the elapsed time represented by the second timestamp from the elapsed time represented by the first timestamp. The time calculation unit 431 calculates the corresponding time corresponding to the first timestamp by adding the calculated difference to the transmission time read from the communication information. Then, the time calculation unit 431 generates association information by associating the calculated corresponding time with the image shown by the image information, and sends the generated association information to the image processing unit 432.

[0054] However, when the units of the first timestamp and the second timestamp are different from the unit of the transmission time shown by the transmission time information, the time calculation unit 431 unifies the units and calculates the corresponding time. For example, when the first timestamp and the second timestamp represent the number of clocks and the transmission time is the system time in seconds, the time calculation unit 431 converts the unit of the difference by multiplying the difference between the first timestamp and the second timestamp by the clock period or dividing the difference by the clock frequency, and then adds it to the transmission time to calculate the corresponding time.

[0055] In Figure 5 the corresponding time calculated by the time calculation unit 431 is schematically shown. As Figure 5As shown, the second timestamp of the communication information represents the time length TL2 from the reference time RT1 to the transmission time T22, and the first timestamp of the image information represents the time length TL1 from the time RT1. The time calculation unit 431 obtains the corresponding time T21 by adding the difference D obtained by subtracting the second timestamp from the first timestamp to the transmission time T22, and generates associated information.

[0056] In addition, the time calculation unit 431 temporarily retains the sequentially obtained image information until the communication information is obtained. If the communication information is obtained, the time calculation unit 431 calculates the corresponding time for each of the retained image information and the image information obtained thereafter, and generates associated information. Further, in the case where the communication information is obtained multiple times, the time calculation unit 431 only needs to generate the associated information based on the latest communication information.

[0057] The image processing unit 432 is mainly implemented by the processor 31. The image processing unit 432 converts the form of the associated information into a form suitable for post-inspection audio-visual or a form suitable for storage in the storage units 433 and 44. This form conversion may also include data compression.

[0058] The storage unit 433 is mainly implemented by the main storage unit 32. The associated information whose form has been converted by the image processing unit 432 is sequentially stored in the storage unit 433. If the capacity of the stored associated information reaches the allowable amount of the storage unit 433, the earliest associated information is sequentially overwritten by the newly obtained associated information. The storage unit 433 is the first example of the storage unit that stores the associated information in the control device 10.

[0059] The control unit 434 is mainly implemented by the processor 31. The control unit 434 reads out the associated information indicating the corresponding times before and after it from the storage unit 433 and stores it in the storage unit 44 at the timing when the condition that a trigger signal is received from the outside is satisfied. For example, if a trigger signal indicating that an error has occurred during the operation of the production line is received, the control unit 434 copies the associated information including the times from three minutes before to one minute after the time when the trigger signal is received as the corresponding time from the storage unit 433 to the storage unit 44. In addition, the control unit 434 uniformly controls the structural elements of the image recording unit 13 and manages the system time.

[0060] The storage unit 44 is mainly implemented by the auxiliary storage unit 33. The storage unit 433 functions as a circular buffer that temporarily stores the associated information. In contrast, the storage unit 44 stores the images at the time point when a trigger signal is generated and an abnormality is suspected for a long period for post-inspection. The storage unit 44 is the second example of the storage unit that stores the associated information in the control device 10.

[0061] Next, with reference to Figure 6 , the image recording process executed by the control device 10 will be described. Figure 6 The image recording process shown is started by turning on the power to the image recording unit 13. In addition, Figure 6 the flow shown is an example for easy understanding of the description, and the image recording process may also be executed through a Figure 6 different flow.

[0062] In the image recording process, the first receiving unit 41 receives image information from the imaging device 22 (step S1). This step S1 is an example of the first receiving step in which the first receiving unit receives image information in the image recording method executed by the control device 10. Next, the second receiving unit 42 receives communication information from the imaging device 22 (step S2). This step S2 is an example of the second receiving step in which the second receiving unit receives communication information in the image recording method executed by the control device 10. In addition, in the case where the communication information has not been received, the image recording unit 13 may also wait until the communication information is received.

[0063] Next, the time calculation unit 431 of the storage unit 43 calculates the corresponding time corresponding to the first timestamp based on the comparison between the first timestamp of the image information received in step S1 and the second timestamp of the communication information received in step S2, with the transmission time as the reference (step S3). This step S3 is an example of the storage step in which the storage unit stores the association information in the storage unit in the image recording method executed by the control device 10. For example, when the timestamp is represented by the clock count, the time calculation unit 431 derives the corresponding time Tn by the following formula (1).

[0064] Tn = Tbase + [(TSn - TSbase) / fclock] ··· (1)

[0065] where, Tbase represents the transmission time of the communication information, TSbase represents the second timestamp of the communication information, TSn represents the first timestamp included in the image information, and fclock represents the frequency of the clock. The frequency of the clock may be included in at least one of the image information and the communication information, may be notified to the control device 10 from the imaging device 22 as information different from the image information and the communication information, or may be preset in the imaging device 22 and the control device 10 as a common parameter. In Figure 7 the calculation result of the corresponding time in the example where the clock frequency is 90 kHz is shown.

[0066] Next, the storage unit 43 stores the association information showing the corresponding time and the image in association with each other in the storage unit 433 and the storage unit 44 (step S4). Specifically, the image processing unit 432 stores the association information in the storage unit 433 in sequence, and the control unit 434 copies the association information from the storage unit 433 to the storage unit 44 as needed. Then, the image recording unit 13 repeats the process of step S1 and subsequent steps. In addition, after receiving the communication information once, step S2 can be omitted.

[0067] As described above, the first receiving unit 41 receives the image information including the first timestamp, the second receiving unit 42 receives the communication information including the second timestamp, and the storage unit 43 stores the association information showing the corresponding time corresponding to the first timestamp and the image in association with each other in the storage unit 433 and the storage unit 44 based on the comparison between the first timestamp and the second timestamp. Therefore, the corresponding time corresponding to the first timestamp pre-assigned before the reception by the first receiving unit 41 is given to the image. Thereby, the influence of communication delay can be excluded, and a more convenient time can be given to the data of the image acquired by the control device 10 from the outside.

[0068] Specifically, since a corresponding time closer to the actual shooting timing is given to the image, it is expected that the situation where the user mistakes the generation time of the failure reflected in the image will be less. In addition, since the corresponding time is represented by the system time common to the control device 10, it is easy to obtain the consistency between the data logged by the control device 10 and the image. Therefore, the convenience of the image given the corresponding time can be improved.

[0069] In addition, the storage unit 43 obtains a difference value by subtracting the elapsed time represented by the second timestamp from the elapsed time represented by the first timestamp, and adds the difference value to the transmission time shown by the communication information to obtain the corresponding time. Thereby, the corresponding time can be calculated with a small computational load.

[0070] Embodiment 2

[0071] Next, Embodiment 2 will be described centering on the differences from the above-described Embodiment 1. In addition, the same reference numerals are used for the structures that are the same as or equivalent to those in the above-described embodiment, and the description thereof is omitted or simplified. In the above-described Embodiment 1, an example in which the first timestamp is given at the timing when the imaging device 22 captures an image and the first timestamp corresponds to the shooting time has been described. However, depending on the type of the imaging device 22, the first timestamp may be given after the shooting time. Here, it is preferable that the corresponding time given to the image in the control device 10 is close to the shooting time. Hereinafter, an example of giving a corresponding time closer to the shooting time to the image when the first timestamp given by the imaging device 22 corresponds to a time different from the shooting time will be described.

[0072] In Figure 8 an example is shown in which the image processing unit 222 of the imaging device 22 outputs the image information with a timestamp after obtaining an image from the imaging sensor 221 and after the image processing has consumed time. As is known from Figure 8 the time corresponding to the timestamp is delayed by an amount of time T30 with respect to the time of shooting by the imaging sensor 221.

[0073] In order to compensate for such a deviation, the time calculation unit 431 according to the present embodiment adds a preset correction term when calculating the corresponding time. Specifically, the time calculation unit 431 adds the difference between the first timestamp and the second timestamp to the transmission time indicated by the communication information, and then adds the correction term to the result of this addition, thereby obtaining the corresponding time. The correction term can be set by the user, or the control device 10 can determine the type of the imaging device 22 and select a preset correction term according to the type. In addition, the above-described Embodiment 1 corresponds to an example where the correction term is zero.

[0074] For example, when the timestamp is represented by the number of clock counts, the time calculation unit 431 derives the corresponding time Tn by the following formula (2).

[0075] Tn = Tbase + [(TSn - TSbase) / fclock] + α ··· (2)

[0076] where, Tbase represents the transmission time of the communication information, TSbase represents the second timestamp of the communication information, TSn represents the first timestamp included in the image information, fclock represents the frequency of the clock, and α is the correction term. As Figure 8 shown, when the first timestamp is later than the actual shooting time, α is a negative value. In Figure 9 an example of the calculation result of the corresponding time in the case where the clock frequency is 90 kHz is shown.

[0077] As described above, the storage unit 43 obtains the corresponding time by adding the correction term. Thereby, it is possible to assign a corresponding time equal to the actual shooting time to the image.

[0078] Embodiment 3

[0079] Next, Embodiment 3 will be described centering on the differences from the above-described Embodiment 1. In addition, the same reference numerals are used for the structures that are the same as or equivalent to those in the above-described embodiment, and their descriptions are omitted or simplified. In the above-described Embodiment 1, an example in which the first timestamp is represented in a form common to the second timestamp has been described, but an example in which their forms are different can also be considered. Hereinafter, an example in which the unit of the first timestamp is different from the unit of the second timestamp will be described.

[0080] In Figure 10 it is shown that the first timestamp is an incremented shot count value at each shot at regular intervals starting from a specific moment, and the second timestamp is an example of the same clock count as in Embodiment 1. In this example, the moment calculation unit 431 uses the clock count C equal to the shooting period, and derives the corresponding moment Tn by the following formula (3).

[0081] Tn = Tbase + [(TSn × C - TSbase) / fclock] ··· (3)

[0082] In Figure 10 the example of, a calculation example of the corresponding moment in the case where the shooting period corresponds to 3000 clocks is shown.

[0083] In addition, the clock count C in the above formula (3) is an example of a parameter for unifying the units of the first timestamp and the second timestamp. This parameter can be obtained by the image recording unit 13 querying the imaging device 22, or can be preset by the user.

[0084] As described above, the first timestamp can also show the elapsed time information in a form different from the second timestamp.

[0085] Above, the embodiments of the present invention have been described, but the present invention is not limited to the above embodiments.

[0086] For example, the functional structure of the image recording unit 13 is not limited to Figure 4 the structure shown, and can be arbitrarily changed. In Figure 11 it is shown that other structures of the image recording unit 13. It can also be as Figure 11 shown, the image information received by the first receiving unit 41 is stored in the storage unit 433 after being processed by the image processing unit 432, and the moment calculation unit 431 stores the associated information in the storage unit 44 based on the image information read from the storage unit 433 and the communication information received by the second receiving unit 42. In addition, the control unit 434 can also instruct the moment calculation unit 431 to calculate the corresponding moment at the timing of receiving the trigger signal.

[0087] In addition, in the above embodiment, an example of assigning a corresponding moment to each of the sequentially acquired images by the storage unit 43 has been described, but it is not limited thereto. As long as at least one piece of associated information includes the first timestamp and is stored in the storage unit 433 or the storage unit 44, and the other image information is stored in the storage unit 433 or the storage unit 44 as it is, the corresponding moment corresponding to the timestamp of each image information can be obtained as needed.

[0088] In addition, it can also be asFigure 12 As shown, the image recording unit 13 is configured by omitting the storage units 433 and 44. In Figure 12 this example, the associated information is stored in an external server device or a detachable recording medium, namely the storage units 433 and 44. Additionally, the control unit 434 may not read the associated information from the storage unit 433, but instead instruct the storage unit 433 to transfer the associated information to the storage unit 44.

[0089] Furthermore, in the above-described embodiment, the control device 10 is composed of the execution unit 11, the input / output unit 12, and the image recording unit 13, but it may also have other units. Additionally, it may be a structure in which the hardware structure for implementing the functions of multiple units is housed in one housing to realize the above functions of the control device.

[0090] Moreover, the control system 100 is not limited to a system for operating a manufacturing line, and may also be other systems having a processing line or an inspection line, or a system for performing process control in a workshop.

[0091] In addition, the functions of the control device 10 can also be realized by dedicated hardware, and can also be realized by a general computer system.

[0092] For example, by storing the program P1 executed by the processor 31 in a computer-readable non-temporary recording medium and distributing it, and installing the program P1 on the computer, a device for executing the above processing can be constituted. As such a recording medium, for example, a floppy disk, a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), or an MO (Magneto-Optical Disc) comes to mind.

[0093] Alternatively, the program P1 can be pre-stored in a disk device of a server device on a communication network represented by the Internet, and for example, downloaded to the computer by being superimposed on a carrier wave.

[0094] In addition, by starting and executing while transmitting the program P1 via the communication network, the above processing can also be realized.

[0095] Furthermore, by causing all or part of the program P1 to be executed on the server device, and having the computer execute the program while communicating with the information related to the processing via the communication network, the above processing can also be realized.

[0096] In addition, in the case where the above-described functions are implemented by the OS (Operating System) in a shared manner or are implemented through the collaborative operation between the OS and an application, etc., it is also possible to distribute only the part other than the OS by storing it in a medium, and it is also possible to download it to a computer.

[0097] In addition, the unit that implements the functions of the control device 10 is not limited to software, and a part or all of it may be implemented by dedicated hardware including a circuit.

[0098] The present invention can implement various embodiments and modifications without departing from the broad spirit and scope of the present invention. In addition, the above-described embodiments are used to explain the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is shown not by the embodiments but by the claims. And various modifications implemented within the scope of the claims and within the meaning of an invention equivalent thereto are considered to be within the scope of the present invention.

[0099] Industrial Applicability

[0100] The present invention is suitable for a system that controls a device and records an image.

[0101] Explanation of Reference Numerals

[0102] 100 Control system, 10 Control device, 11 Execution unit, 12 Input / output unit, 13 Image recording unit, 101 System bus, 21 Controlled device, 22 Imaging device, 221 Imaging sensor, 222 Image processing unit, 223 Communication unit, 31 Processor, 32 Main storage unit, 33 Auxiliary storage unit, 34 Input unit, 35 Output unit, 36 Communication unit, 37 Internal bus, 40 Communication unit, 41 First receiving unit, 42 Second receiving unit, 43 Storage unit, 431 Time calculation unit, 432 Image processing unit, 433 Storage unit, 434 Control unit, 44 Storage unit, P1 Program.

Claims

1. A control device is connected to a controlled device and a photographing device to control the controlled device, wherein the control device has: a first receiving unit that repeatedly receives image information representing an image related to the control of the controlled device photographed by the photographing device, and the image information includes a first timestamp showing the elapsed time from a specific moment in association with the image; a second receiving unit that receives communication information related to the communication history of the communication performed by the photographing device, and the communication information includes a second timestamp showing the elapsed time from the specific moment and transmission time information showing the transmission time when the communication information is transmitted; and a storage unit that stores association information in a storage unit, where the association information is obtained by subtracting the elapsed time shown by the second timestamp corresponding to the transmission time from the elapsed time shown by the first timestamp associated with the image shown by the image information, adding the difference to the transmission time, taking the resulting time as the corresponding time corresponding to the first timestamp, or taking the time obtained by adding a preset correction term for compensating the deviation between the time of photographing the image by the photographing device and the time corresponding to the first timestamp to the resulting time as the corresponding time, and showing it in association with the image.

2. An image recording method includes the following steps: a first receiving step of receiving, by a first receiving unit, image information representing an image photographed by a photographing device, and the image information includes a first timestamp showing the elapsed time from a specific moment in association with the image; a second receiving step of receiving, by a second receiving unit, communication information related to the communication history of the communication performed by the photographing device, and the communication information includes a second timestamp showing the elapsed time from the specific moment and transmission time information showing the transmission time when the communication information is transmitted; and a storage step of storing, by a storage unit, association information in a storage unit, where the association information is obtained by subtracting the elapsed time shown by the second timestamp corresponding to the transmission time from the elapsed time shown by the first timestamp associated with the image shown by the image information, adding the difference to the transmission time, taking the resulting time as the corresponding time corresponding to the first timestamp, or taking the time obtained by adding a preset correction term for compensating the deviation between the time of photographing the image by the photographing device and the time corresponding to the first timestamp to the resulting time as the corresponding time, and showing it in association with the image.

3. A computer-readable non-transitory recording medium stores a program that causes a computer connected to a photographing device to function as the following units: A first receiving unit that receives image information representing an image captured by the imaging device, and the image information includes a first timestamp that shows the elapsed time from a specific moment in association with the image; A second receiving unit that receives communication information related to the communication history of the communication performed by the imaging device, and the communication information includes a second timestamp indicating the elapsed time from the specific moment and transmission time information indicating the transmission time when the communication information is transmitted; and A storage unit that stores association information in a storage unit, where the association information is obtained by subtracting the elapsed time shown by the second timestamp corresponding to the transmission time from the elapsed time shown by the first timestamp associated with the image shown by the image information, adding the difference to the transmission time, taking the resulting time as the corresponding time corresponding to the first timestamp, or taking the time obtained by adding a preset correction term for compensating the deviation between the imaging time of the image achieved by the imaging device and the time corresponding to the first timestamp to the resulting time as the corresponding time, and showing it in association with the image.

Images