Camera device with verification assistance function, verification assistance device, and verification assistance method for camera device

Abstract

This invention enables the observation, on a time axis having nanosecond accuracy, of observation items corresponding to each of an input / output signal to / from a camera device, a control signal by a control PC, an internal signal of the camera device, etc.　In a camera device, in which a plurality of parts to be controlled that operate in response to a plurality of commands are controlled, log data indicating that each part to be controlled has been controlled is integrated with a nanosecond-precision timestamp and control item check data, and the integrated log data is stored in a log storage circuit disposed corresponding to each part to be controlled.

Description

Camera device having a verification support function, verification support device, and verification support method for a camera device

[0001] Embodiments of the present invention relate to a camera device having a verification support function, a verification support device, and a verification support method for a camera device.

[0002] For example, a video system has been introduced for the automation of factories and the like. The video system monitors the inside of the factory by arranging camera devices at multiple locations inside the factory.

[0003] Here, it is assumed that problems such as malfunction or error may occur in the video system. In such a case, in order to find the root cause, a verification operation for each camera device itself is required.

[0004] Conventionally, it is common to use measuring instruments such as an oscilloscope and a logic analyzer for verifying the operation of the camera device itself. By probing the signal to be observed, parallel input / output signals and internal signals can be observed. However, there are limitations to the signals that can be observed.

[0005] For the video system, a system control unit (usually using a PC (personal computer), hereinafter referred to as a control PC) that controls the video system from the outside is connected to the camera device. In order to perform a detailed verification, it is required that the control PC controls the camera device and observes whether the camera device is correctly responding to this control (so-called control situation). However, since the control PC has a serial interface, it has been difficult to observe the overall control situation with the above-mentioned measuring instruments such as an oscilloscope and a logic analyzer for this part.

[0006] In order to solve this problem, there has been a method of observing a serial interface with a measuring instrument similar to a bus analyzer as a known technique. However, on the other hand, since only the information output to the serial interface can be verified, there is a problem that the input / output signals and internal signals of the camera device described above cannot be directly observed by this method.

[0007] Depending on the nature of the verification, there may be cases where it is necessary to simultaneously observe both the input / output signals and internal signals of the camera device (observation using the first measurement method) and the control status by the control PC (observation using the second measurement method). For example, when exposure start control is initiated internally by the camera device to begin exposure, and exposure time change control is performed from an external control device to change the exposure time, it may be necessary to observe these two types of control on the same time axis. In such cases, it is necessary to perform observation using both the first and second measurement methods described above. In other words, the first measurement method is to use measuring instruments such as the oscilloscope and logic analyzer mentioned above to verify the operation of the camera device, and the second measurement method is to observe the serial interface using measuring instruments similar to a bus analyzer.

[0008] However, the method using the two different measuring instruments described above has a drawback: it is not possible to simultaneously observe multiple control states for the camera device on the same time axis.

[0009] Furthermore, adopting the first and second measurement methods described above would require using measuring equipment separate from the camera system, making it difficult to integrate them seamlessly into a factory environment where the camera system is actually in operation.

[0010] To resolve this issue, a known technique involved using the CPU (Central Processing Unit) built into the control PC or camera device to create verification logs via software control. This approach eliminates the need for dedicated measuring equipment and avoids connection problems. However, it suffers from the limitation that temporal accuracy depends on the performance of the control PC or camera's built-in CPU, resulting in lower accuracy compared to dedicated measuring equipment (which can measure in milliseconds).

[0011] Japanese Patent Publication No. 2012-023644, Japanese Patent Publication No. 2013-211606, Japanese Patent Publication No. Hei 10-93754

[0012] As mentioned above, conventional video systems have the problem that it is difficult to accurately observe the timing of changes in the operating state of each part of the camera device on the same time axis in relation to the timing of input of various control signals to the camera device. Furthermore, in situations where it is difficult to connect measuring equipment, a technique can be considered in which logs are created by a control PC or software that controls the camera device, but this has the problem that log creation is in milliseconds, resulting in a decrease in temporal accuracy.

[0013] Therefore, in order to solve the above problems, the present invention aims to provide a camera device, a verification support device, and a verification support method for a camera device that enable the observation of each observation item corresponding to the control signal from the control PC and the internal signal of the camera device on the same time axis, and furthermore, have a verification support function that enables the storage of each single log data, in which each observation item and a timestamp common to each observation item are integrated, in a storage circuit with an independent hardware configuration.

[0014] According to one embodiment, a camera device is provided that has a verification support function, comprising: a packet transmission / reception circuit that receives serial data from a computer; a camera control circuit that receives control data from the packet transmission / reception circuit; an image sensor control circuit controlled by the camera control circuit and controlling an image sensor; a timestamp control circuit that outputs a timestamp; a first log storage circuit, a second log storage circuit, and a third log storage circuit, each configured in hardware, that store log data in which the timestamp and control item check data for each of the packet transmission / reception circuit, the camera control circuit, and the image sensor control circuit are integrated when each of them is given a control command to operate; and a memory that stores the log data of the first log storage circuit, the second log storage circuit, and the third log storage circuit in chronological order of the timestamp.

[0015] Figure 1 is a block diagram showing the overall configuration of a camera device according to one embodiment. Figure 2 is a block diagram showing the internal configuration of the log memory 101 in Figure 1. Figure 3 is a timing chart showing an example of the operation of the main parts of the camera device according to this embodiment.

[0016] The embodiments will be described below with reference to the drawings. Figure 1 shows the overall configuration when one embodiment is applied to the camera device 1. The camera device of this embodiment uses interfaces such as USB 3.0, Ethernet, CameraLink, and CoaXPress that enable high-speed communication.

[0017] A control PC 3 is connected to the camera device 1 via a serial interface 2. During normal operation, the control PC 3 stores imaging conditions (commands) such as exposure time and white balance in packets and transmits them to the camera device 1 via the serial interface 2.

[0018] The camera device 1 receives packets sent from the control PC 3 via the packet transmission / reception circuit 107, extracts commands, and transmits them to the camera control circuit 108. The camera control circuit 108 provides control to the image sensor control circuit 109 according to the content of the commands, and the image sensor control circuit 109 controls the image sensor 110. For example, the image sensor control circuit 109 controls the exposure time of the image sensor 110, reads the video data captured by the image sensor 110, and converts the video data into a predetermined format. The converted video data is sent to the packet transmission / reception circuit 107 by the camera control circuit 108. The video data is constructed into packets by the packet transmission / reception circuit 107 and transmitted to the control PC 3 via the serial interface 2. The camera control circuit 108 may also send video data to the external input / output control circuit 111.

[0019] The external input / output control circuit 111 can receive camera control signals from outside the camera device 1 and inputs these signals to the camera control circuit 108. These camera control signals are transmitted via a parallel interface.

[0020] The external input / output control circuit 111 described above is often used by on-site users within the factory. In contrast, the control PC 3 is mainly installed by the manufacturer that supplied the video system or by the manager of the factory where the camera equipment is used, and is used to perform pre-configured system control.

[0021] Next, we will describe the system for acquiring log data on the operation and control status of the camera device 1.

[0022] First, 101 is a log memory, 102 is a log data transmission / reception circuit, and 103 to 106 are log storage circuits. The log memory 101 can store log data held by log storage circuits 103, 104, 105, and 106. The log data transmission / reception circuit 102 can read log data stored in the log memory 101 and input it to the packet transmission / reception circuit 107 in response to control from the packet transmission / reception circuit 107. The above log processing block is made up of hardware and may be built on a dedicated circuit board, for example.

[0023] The log saving circuit 103 saves data that pairs some or all of the received command with the timestamp received from the timestamp control circuit 112.

[0024] In this specification, data sent from the control PC 3 is referred to as a "command," and responding to it is referred to as an "acknowledgment." In contrast, internal control by control data generated within the camera device (for example, generated by the camera control unit 108) is referred to as "control." Based on this premise, the following control item check data is defined. Specifically, the "control item check data" may be data consisting of a part or all of a command paired with a timestamp, hereinafter referred to as "single log data." Here, the "control item check data" may also be a code representing the camera's control content, control location, or a combination of these. Furthermore, the "control item check data" may use a code representing identification data (for example, an address, name, or operation name) assigned to an internal function of the camera device (which may also be referred to as a controlled unit) that should be controlled by the command. Moreover, in order to control each part of the camera's functions, an address may be assigned to each camera function (controlled unit), and a so-called memory-mapped I / O method may be incorporated into and used within the "control item check data." In short, the data should clearly show what kind of control was applied to which part of the camera device. The same applies to the log data, which will be explained later.

[0025] The above single log data is written from log storage circuits 103, 104, 105, and 106 to log memory 101 (details of log memory 101 are shown in Figure 2).

[0026] The camera control circuit 108 also interprets commands received from the packet transmission / reception circuit 107, sends back an acknowledgment, and performs control (change, switch, adjust, etc.) of the operating state of the camera device 1. For example, if the control content of the received command relates to the operating state of the image sensor 110, it sends the control data to the image sensor control circuit 109. In this case, the control data may include setting the shutter timing (exposure time), readout control, and selecting / switching the readout area.

[0027] This system allows the image sensor 110 to set the frame angle of view and switch the readout area, and it is also possible to set the frame frequency of the readout video data to a high frequency.

[0028] In the above operation, the log saving circuit 105 saves the change in the operating state of the camera control circuit 108 and the timestamp of the timestamp control circuit 112 at that time as a single log data. Subsequently, this single log data is written to the log memory 101.

[0029] The basic clock for the operation of the camera control circuit 108 is synchronized with the basic clock from the timestamp control circuit 112. The timestamp control circuit 112 is also connected to and synchronizes with timestamp control circuits or camera control circuits of other camera devices (not shown).

[0030] The log storage circuit 105 may also store the internal temperature, power supply voltage, and power saving status of the camera device, as known by the camera control circuit 108, as log data within the control item check data. These conditions can be displayed on the display unit of the control PC 3.

[0031] The image sensor control circuit 109 controls the image sensor 110 based on the control content (such as setting the exposure time and selecting / switching the readout area) provided by the camera control circuit 108.

[0032] At this time, the log storage circuit 106 stores a single log data set consisting of a pair of control item check data (part or all of the control or identification data, etc.) and a timestamp received from the timestamp control circuit 112, when various controls based on the control content are specifically executed on the image sensor 110. This single log data is then written to the log memory 101.

[0033] The video data acquired by the image sensor 110 reaches the packet transmission / reception circuit 107 via the image sensor control circuit 109 and the camera control circuit 108. Even when the video data acquired by the image sensor 110 reaches the packet transmission / reception circuit 107 via the image sensor control circuit 109 and the camera control circuit 108, control item check data and timestamps are stored in the corresponding log storage circuits 106, 105, and 103, respectively.

[0034] The video data is constructed into packets by the packet transmission / reception circuit 107 and transmitted to the control PC 3 via the serial interface 2.

[0035] At this time, the log saving circuit 103 saves the control item check data indicating that video data has been transmitted, along with a timestamp, as a single log data. This single log data is then written to the log memory 101.

[0036] Furthermore, the log storage circuit 103 also receives information such as the occurrence of errors on the communication path and the recovery processing status from the status of the packet transmission / reception circuit 107, and can store these observed items as log data along with a timestamp as a single log data. This single log data is then written to the log memory 101.

[0037] Furthermore, the camera device 1 has an external input / output control circuit 111. This external input / output control circuit 111 has a function to receive control input from the outside. For example, when the external input / output control circuit 111 receives an image capture start control from the outside, it notifies the camera control circuit 108 of this control. Also, for example, if the camera device 1 is set to output an image capture timing pulse, the image capture timing pulse generated by the camera control circuit 108 can be output via the external input / output control circuit 111.

[0038] When the external input / output control circuit 111 receives the external control described above, the log saving circuit 104 saves the control item check data and the timestamp at that time. At this time, the control item check data will of course include identification data that indicates that the external input / output control circuit 111 has received the control.

[0039] External inputs and outputs are connected to devices other than the control PC 3, such as photoelectric sensors. These devices and the control PC 3 are not synchronized, and the commands input to the packet transmission / reception circuit 107 and the controls input to the external input / output circuit 111 are asynchronous. In such cases, it was sometimes difficult to identify the input order of the commands and controls, and the order in which they were propagated to the camera control circuit 108. However, in this embodiment, as will be explained later in Figure 3, such identification becomes possible with nanosecond-level accuracy.

[0040] As described above, based on the timestamp provided by the timestamp control circuit 112, the log storage circuits 103, 104, 105, and 106 can save status information inside the camera device with nanosecond precision, which is finer than millisecond precision on the time axis. This is because the log storage circuits 103, 104, 105, and 106 are configured in hardware with a one-to-one correspondence to the packet transmission / reception circuit 107, the external input / output control circuit 111, the camera control circuit 108, and the image sensor circuit 109, respectively. With this configuration, there is no influence from the software processing steps based on the CPU processing speed.

[0041] The contents of the single log data stored in log storage circuits 103, 104, 105, and 106 can be summarized as follows: Log storage circuit 103... Commands, acknowledgments, errors on the communication line; Log storage circuit 104... Changes in the state of external input / output; Log storage circuit 105... Changes in the internal state of the camera, temperature changes, power supply voltage changes, power saving state; Log storage circuit 106... Image sensor operating state.

[0042] Next, the command packet transmitted and received in the serial interface 2 described above will be explained. The command packet output by the camera control PC 3 during PC control includes, as packet data, access mode (read / write), address (e.g., address in the memory-mapped I / O method), write data (during write access), and other data indicating these. However, it is not limited to the above method, and means for identifying each part of the camera device, such as circuit name, function name, etc., may be used as data for control item check. In the case of the address in the memory-mapped I / O method, when continuously controlling the same address, a mechanism is used where the count value is incremented by 1 for each control at the same address. And the reset of the added value of the address in the memory-mapped I / O method is executed, for example, after the entire log data in the log memory 101 is read out.

[0043] Also, the acknowledgment packet sent from the camera device 1 to the camera control PC 3 may include control status (normal end / error, etc.), read data (for read access), etc. in the data for control item check.

[0044] The entire log data collected from the single log data stored in the log memory 101 as described above can be read by the control PC 3 at an arbitrary timing. When there is a request to read the entire log data from the control PC 3, the log data transmission / reception circuit 102 transmits the entire log data stored in the log memory 101 to the packet transmission / reception circuit 107. The packet transmission / reception circuit 107 generates the entire log data in packet form and transmits it to the control PC 3 via the serial interface 2. By this operation, the control PC 3 can read the entire log data stored in the log memory 101 via the serial interface 2 at an arbitrary timing.

[0045] As the above-mentioned log memory 101, the implementation forms such as FIFO (First-In First-Out) and ring buffer are not restricted. Also, it is not an independent memory, and it is possible to share it with other-purpose memories such as the image memory. Further, in addition to the memory function, it has write arbitration of the log storage circuits 103, 104, 105, 106, and a function of temporarily stopping and restarting the storage operation (or the controlled part).

[0046] FIG. 2 shows an example in which the log memory 101 is composed of a randomly accessible memory.

[0047] 1011 is a read control circuit, 1012 is a memory, 1013 is a write control circuit, and 1014 is a write arbitration circuit.

[0048] The single log data transmitted from the log storage circuits 103, 104, 105, 106 is sequentially selected as a write target by the write arbitration circuit 1014 with the value of the time stamp from the past. At this time, other single log data that has not become a write target is temporarily stored in the log storage circuits 103, 104, 105, 106 and thus is not lost. The single log data selected as a write target is transmitted to the write control circuit 1013. The write control circuit 1013 performs writing so as to add the single log data to the end of the overall log data.

[0049] When there is a transmission request for the overall log data from the log data transmission / reception circuit 102, the read control circuit 1011 reads the overall log data from the memory 1012 and transmits it to the log data transmission / reception circuit 102. At this time, the read control circuit 1011 controls so that the value of the time stamp is transmitted in time series from the past while referring to the write address pointer of the write control circuit 1013 as necessary.

[0050] The write control circuit 1013 also controls the transition between enabled and disabled states of the overall log data writing operation. The write control circuit 1013 receives commands from the control PC 3 and can accept requests for temporary suspension and resumption of saving the overall log data, and can control state transitions such as automatically suspending saving based on pre-set trigger conditions. Commands related to the above transitions are provided directly to the log memory 101 from the packet transmission / reception circuit 107.

[0051] The overall log data stored in the log memory 101 can also include, in addition to the commands, controls, changes in the camera's internal state due to those controls, the image sensor's operating state, and changes in the state of external inputs and outputs, as well as changes in the camera's surrounding conditions, such as errors occurring on the communication line and recovery processing status, temperature, power supply voltage, and power saving status.

[0052] As described above, according to this embodiment, information necessary for verification work, such as commands from the control PC, the operating status in image sensor control, and changes in the state of GPIO (General-purpose input / output), is stored in the log memory with high-precision timestamps that share a common time axis. Therefore, there is an advantage in that verification can be performed by referring to various data on the same time axis.

[0053] Furthermore, logging is performed using the camera's timestamp as the time axis. This has the advantage of not relying on a control PC, and therefore achieving nanosecond or even finer time axis accuracy (higher accuracy than the clock frequency of the camera control circuit being used) even when no measuring equipment is used.

[0054] Figure 3 is an explanatory diagram showing an example of a log data acquisition sequence in chronological order when acquiring log data in the camera device according to this embodiment.

[0055] This is a timing chart showing when events occur in the packet transmission / reception circuit 107, external input / output control circuit 111, camera control circuit 108, and image sensor control circuit 109 that warrant saving as logs, and when the write control circuit 1013 (shown in Figure 2) writes these events to memory 1012 (shown in Figure 2). Log saving circuits 103 to 106 are omitted for simplification.

[0056] When the packet transmission / reception circuit 107 receives a command X1 sent from the control PC 3, the log storage circuit 103 sends the command along with a timestamp T1 from the timestamp control circuit 112 to the log memory 101.

[0057] The write control circuit 1013 (Figure 2) in the log memory 101 immediately writes the received timestamp T1 and command W1 to memory 1012 because there are no other writes (T1, W1).

[0058] Next, if the camera control circuit 108 changes the operating state X2 of the camera device 1 based on the command X1, the log saving circuit 105 sends the change in operating state along with the timestamp T2 of the timestamp control circuit 112 to the log memory 101. The write control circuit 1013 (shown in Figure 2) in the log memory 101 is writing the command at this time, so it writes the timestamp T2 and the change in operating state W2 after the writing operation W1 of this command is completed.

[0059] In the above case, the writing control circuit 1013 will be in a standby state for writing the change in operating state W2, but the timestamp T2 will not shift due to the delay in writing, because the log storage circuit 105 holds the value of the time when the change in operating state X2 occurred.

[0060] Next, we will explain an example where the external input / output change X3 in the external input / output control circuit 111 and the image sensor control X4 in the image sensor control circuit 109 occur almost simultaneously.

[0061] When there is an external input / output change X3 in the external input / output control circuit 111, the log saving circuit 104 transmits the external input / output change X3 along with the timestamp T3 of the timestamp control circuit 112 to the log memory 101. Almost simultaneously with this transmission, or at a later timing, if the image sensor control circuit 109 performs image sensor control X4 based on the change in the operating state X2 of the camera control circuit 108, the log saving circuit 106 transmits the image sensor control X4 along with the timestamp T4 of the timestamp control circuit 112 to the log memory 101.

[0062] At this time, the write control circuit 1013 is writing the operating state change W2, so the external input / output change W3 and the image sensor control W4 are waiting to be written. After the writing of the operating state change W2 is completed, the write control circuit 1013 starts writing the timestamp T3 and the external input / output change W3, and then the timestamp T4 and the image sensor control W4 in the order in which the write operations arrived.

[0063] Furthermore, when the packet transmission / reception circuit 107 sends an acknowledgment X5 for a command while the external input / output change W3 is being written, the log storage circuit 103 sends the acknowledgment along with the timestamp T5 of the timestamp control circuit 112 to the log memory 101. At this time, since the writing of the image sensor control W4 is in a waiting state first, the write control circuit 1013 writes the image sensor control W4, and then writes the timestamp T5 and the acknowledgment W5. In this way, in this system, when the writing process of a single log data that should be saved first to the memory 101 has not been completed, a single log data that should be saved next may be generated in the memory 101. In such cases, the generation order of multiple single log data to be saved can be managed by multiple log storage circuits in hardware configuration using timestamps with a high-speed clock.

[0064] Once all write waiting states are resolved, the write control circuit 1013 returns to the operation of immediately performing writes upon arrival of a write request. When the packet transmission / reception circuit 107 transmits video data information X6, the log storage circuit 103 transmits the video data information along with the timestamp T6 from the timestamp control circuit 112 to the log memory 101. Since there are no writes in the waiting state, the write control circuit 1013 immediately starts writing the timestamp T6 and video data information W6.

[0065] As described above, according to this embodiment, even if the timing of a command being given and the period during which the system operates in response do not coincide (i.e., even if the interval is very short, such as a nanosecond), the timing of the command being given is managed chronologically using a timestamp.

[0066] This allows for precise management of the control timing for the camera device and its corresponding operational status. The control PC 3 also receives log data from the memory 101 that stores the log data and has means for displaying, for example, an identification mark for each log data in chronological order of the timestamp. The identification mark for the log data may be identification data that identifies the data used for checking control items, or it may be a predetermined sequence of colors or patterns set in the controlled unit. Furthermore, since it is possible to identify data that has been transmitted via the external input / output control circuit 111, it is also possible to easily analyze the status of external access. In the figure, one log storage circuit 105 is provided in correspondence with the camera control circuit 108, however, multiple log storage circuits may be provided in parallel.

[0067] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. Furthermore, even if each component of a claim is expressed by dividing it, by combining multiple components, or by combining them, it remains within the scope of the present invention. Multiple embodiments may also be combined, and embodiments composed of such combinations also fall within the scope of the invention.

[0068] Furthermore, the present invention applies to any claim expressed as control logic, as a program containing instructions for a computer to execute, or as a computer-readable recording medium containing such instructions. The use of names and terms is also not limited; other expressions, if substantially the same in content and intent, are included in the present invention.

Claims

1. A camera device having a verification support function, comprising: a packet transmission / reception circuit for receiving serial data from a computer; a camera control circuit for receiving control data from the packet transmission / reception circuit; an image sensor control circuit controlled by the camera control circuit for controlling an image sensor; a timestamp control circuit for outputting a timestamp; a first log storage circuit, a second log storage circuit, and a third log storage circuit, each configured in hardware, for storing log data that integrates the timestamp and control item check data for the packet transmission / reception circuit, the camera control circuit, and the image sensor control circuit, respectively, when a command to instruct the operation of each of them is given; and a memory for storing the log data of the first log storage circuit, the second log storage circuit, and the third log storage circuit in chronological order of the timestamp.

2. A camera device having the verification support function according to claim 1, further comprising a fourth log storage circuit, wherein log data including control item check data and a timestamp from an external input / output control circuit that receives control different from the interface of the serial data received by the packet transmission / reception circuit is input to the fourth log storage circuit.

3. The camera device according to claim 1 or 2, wherein a log data transmission and reception circuit for transmitting the log data to an external source is connected to the memory.

4. A verification support device to which a camera device according to any one of claims 1 to 3 is connected, wherein the computer has means for receiving the log data from a memory that stores the log data and displaying it in the chronological order of the timestamps.

5. A method for supporting the verification of a camera device, comprising: a camera control circuit that receives control data from a packet transmission / reception circuit that receives serial data from a computer; an image sensor control circuit controlled by the camera control circuit and controlling an image sensor; and a timestamp control circuit that outputs a timestamp, wherein when a command to instruct the operation of the packet transmission / reception circuit, the camera control circuit, and the image sensor control circuit is given to each of the first, second, and third log storage circuits, which are configured as hardware, log data is stored in which the timestamp and control item check data for the packet transmission / reception circuit, the camera control circuit, and the image sensor control circuit are integrated, and each of the log data is stored in memory in the chronological order of the timestamp.

6. A method for supporting the verification of a camera device according to claim 5, wherein the stored log data is transmitted to an external source.

Images