Image capture device failure determination system

Abstract

To provide a technique which enables determining the existence or non-existence of a fault in an imaging apparatus substantially instantly.SOLUTION: A fault determination system 1 includes: an imaging apparatus 111 which images a moving image; a radiation device 112 which radiates a test pattern with sufficiently short predetermined time intervals; and a dimming mirror 113 which, while the radiation device 112 radiates the test pattern, becomes a reflective state to reflect the light of the test pattern toward the imaging apparatus 111, whereas becomes a transmissive state while the radiation device 112 does not radiate the test pattern. The fault determination system 1 further includes: fault determination processing which determines the existence or non-existence of a fault in the imaging apparatus 111, using an image in which the test pattern is captured among images captured by the imaging apparatus 111; and a data processing device 12 which, using other images, performs object detection processing which detects an obstacle which hinders the travel of a train 9. On detecting a fault in the imaging apparatus 111 by the fault determination processing, the data processing device 12 notifies a control device 8 of that matter.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for determining a failure in an imaging device. [Background technology]

[0002] There are technologies that recognize objects from images captured by an imaging device and control the device based on the results of the recognition. For example, a technology has been developed that recognizes passersby and other people from images captured by an imaging device mounted on a vehicle and controls the vehicle to stop so as not to collide with the recognized passersby and other people.

[0003] When an imaging device is used for the above-mentioned purposes, if the imaging device malfunctions, errors will occur in the captured image, and objects will not be correctly recognized from the image, which could result in the vehicle failing to avoid pedestrians, etc.

[0004] Therefore, it is necessary to determine whether or not the imaging device has a malfunction, and if it is determined that a malfunction has occurred, it is necessary to take measures such as bringing the vehicle to an emergency stop.

[0005] An example of a document disclosing a technique for determining whether an imaging device has a malfunction is Patent Document 1. Patent Document 1 describes a mechanism for determining that a digital camera has a malfunction if a digital image captured by the digital camera remains uniform over a predetermined period of time. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-36927 Summary of the Invention [Problem to be solved by the invention]

[0007] In the case of the mechanism described in Patent Document 1, after a failure occurs in the imaging device, the imaging device is determined to have failed after a predetermined number of uniform images are captured. Therefore, the mechanism described in Patent Document 1 takes too long to determine whether or not there is a failure in an imaging device used for control that requires real-time performance, such as automatic driving of a vehicle.

[0008] In view of the above circumstances, the present invention provides a technique that enables substantially instantaneous determination of whether or not an imaging device has a malfunction. [Means for solving the problem]

[0009] The present invention relates to a method for capturing a moving image at a predetermined frame rate by an imaging device, and a method for capturing a moving image at a predetermined frame rate by an imaging device. For each of the groups, the frame group Of Different predetermined number As a first aspect, we propose a system for determining a failure of an imaging device, which projects a different test pattern into the imaging range when capturing each of multiple frames, determines whether or not the imaging device is malfunctioning from the frame image of the imaging device in which the test pattern is captured, and performs object detection processing using other frame images.

[0010] According to the failure determination system of the first aspect, it is possible to determine substantially instantaneously whether or not the imaging device has a failure.

[0011] The present invention relates to a method for detecting a moving image by intermittently projecting a test pattern into an imaging range of an imaging device for capturing moving images. A reflecting member that reflects the test pattern toward the imaging device a switchable mirror that is in a reflective state when a test pattern is irradiated and in a transmissive state when the test pattern is not irradiated; Equipped with a failure determination system for an imaging device that determines whether or not the imaging device has a failure based on an image of the imaging device that includes the test pattern; As a second aspect suggest .

[0012] According to the failure determination system of the second aspect, it is possible to determine substantially instantaneously whether or not the imaging device has a failure. Also, According to the failure determination system of the second aspect, the test pattern can be projected onto the reflecting member, causing the imaging device to capture an image of the test pattern.

[0014] Also, No. 2 According to the failure determination system according to the aspect, there is no need to move the reflecting member.

[0015] The present invention provides a reflective member that intermittently projects a test pattern into an imaging range of an imaging device that captures moving images, and reflects the test pattern toward the imaging device, A mirror that is within the imaging range of the imaging device when a test pattern is projected and is outside the imaging range of the imaging device when the test pattern is not projected. and determining whether or not the imaging device has a malfunction based on an image of the imaging device that includes the test pattern. Image capture device failure determination system As the third aspect suggest .

[0016] No. 3 According to the failure determination system of the aspect (3), a clearer image is captured when the test pattern is not projected, compared to when a dimming mirror is used as the reflecting member, for example.

[0019] 1st to 1st 3 In the failure determination system according to any one of the above aspects, the imaging device may be configured to capture images at shorter time intervals when the test pattern is irradiated than when the test pattern is not irradiated. 4 This may be adopted as an embodiment.

[0020] No. 4 According to the failure determination system according to the aspect, the time during which normal photography cannot be performed due to photographing the test pattern is shortened. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram schematically illustrating the overall configuration of a failure determination system according to an embodiment; [Figure 2] FIG. 1 is a diagram schematically illustrating a configuration of an imaging unit according to an embodiment. [Figure 3] FIG. 1 is a diagram showing the configuration of a data processing device according to an embodiment. [Figure 4] FIG. 4 is a diagram showing a processing flow of an image processing unit according to an embodiment. [Figure 5] FIG. 4 is a diagram showing a processing flow of an irradiation instruction signal output unit according to an embodiment. [Figure 6] 1 is a diagram showing a correspondence relationship between an image captured by an imaging device and processing performed by a data processing device according to an embodiment; [Figure 7]FIG. 10 is a diagram schematically illustrating the configuration of an imaging unit according to a modified example. [Figure 8] FIG. 10 is a diagram schematically illustrating the configuration of an imaging unit according to a modified example. [Figure 9] FIG. 10 is a diagram illustrating an example of a test pattern used in a failure determination system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0022] [Embodiment] A fault determination system 1 according to one embodiment of the present invention will be described below. FIG. 1 is a diagram schematically illustrating the overall configuration of the fault determination system 1. The fault determination system 1 in FIG. 1 is an object detection system that recognizes objects captured in an image of the area ahead of a train 9, and notifies a control device 8 that controls the running of the train 9 of the detection of the obstacle if it detects an object that may obstruct the running of the train 9. However, in addition to the functions of a normal object detection system, the fault determination system 1 also has a function of determining whether or not an imaging device has malfunctioned in substantially real time, and if it determines that the imaging device has malfunctioned, notifying the control device 8 of the malfunction of the imaging device.

[0023] The failure determination system 1 includes an imaging unit 11 and a data processing device 12. The imaging unit 11 is disposed in front of the lead car of the train 9, and captures an image of the area ahead of the train 9.

[0024] The data processing device 12 is placed inside the train 9, acquires images captured by the imaging unit 11, recognizes objects from the images, and if an obstacle is recognized, notifies the control device 8 of the detection of the obstacle. For this purpose, the data processing device 12 is connected to communicate with the imaging unit 11 and the control device 8.

[0025] 2 is a diagram schematically illustrating the configuration of the imaging unit 11. The imaging unit 11 includes an imaging device 111 that captures an image, an irradiation device 112 that irradiates a test pattern, and a switchable mirror 113 (an example of a reflective member). Although not shown in FIG. 2, the imaging device 111, the irradiation device 112, and the switchable mirror 113 are each connected to the data processing device 12 for communication.

[0026] The imaging device 111 is a visible light camera that captures moving images. In this embodiment, for example, the time interval at which the imaging device 111 captures images, that is, the frame rate, is assumed to be 60 fps (Frames per Second).

[0027] The imaging device 111 promptly outputs images captured every 1 / 60 seconds to the data processing device 12. The imaging device 111 also outputs a synchronization signal to the data processing device 12 every 1 / 60 seconds at the timing of capturing images.

[0028] The projection device 112 is a color projector. The projection device 112 projects a test pattern in response to a projection instruction signal output from the data processing device 12. In the present embodiment, for example, when the frames captured by the imaging device 111 per second are the first to sixtieth frames, the projection device 112 projects a red test pattern over the entire surface at the timing of capturing the fifty-eighth frame, projects a green test pattern over the entire surface at the timing of capturing the fifty-ninth frame, and projects a blue test pattern over the entire surface at the timing of capturing the sixtieth frame.

[0029] The switchable mirror 113 is a device that switches between a mirror state in which it reflects most of the light and a transmissive state in which it transmits most of the light by electrical control. The switchable mirror 113 switches to the mirror state in response to an irradiation instruction signal output from the data processing device 12, and switches to the transmissive state when the output of the irradiation instruction signal from the data processing device 12 stops.

[0030] 2, the dashed line indicates the imaging range of the imaging device 111, and the dashed line indicates the illumination range of the illumination device 112. As shown in FIG. 2, the positions of the imaging device 111, the illumination device 112, and the switchable mirror 113 are adjusted so that the light reflected from the test pattern illuminated by the illumination device 112 by the switchable mirror 113 in the mirror state is directed toward the imaging device 111.

[0031] The hardware of the data processing device 12 is a general computer equipped with a communication function. A processor included in the computer, which is the hardware of the data processing device 12, performs data processing in accordance with a program stored in a memory, thereby realizing the data processing device 12 having the configuration shown in Fig. 3. Note that the data processing device 12 may also be configured as a so-called dedicated device.

[0032] The following describes each of the components of the data processing device 12 shown in Figure 3. The image acquisition unit 121 acquires the image output by the imaging device 111.

[0033] The image processing unit 122 receives images acquired by the image acquisition unit 121 from the imaging device 111, and performs either object detection processing or fault determination processing according to the frame number of the received image. In this embodiment, the image processing unit 122 performs object detection processing for images with frame numbers 1 to 57 (images of the first to 57th frames), and performs fault detection processing for images with frame numbers 58 to 60 (images of the 58th to 60th frames).

[0034] The object detection process recognizes objects in an image using a known image recognition method, and determines whether the object will be an obstacle to the movement of the train 9 based on the type of object recognized and its position in the image.If the object is determined to be an obstacle, an obstacle detection signal is output to the control device 8 to notify it that an obstacle has been detected.

[0035] The failure detection process determines whether the image of the 58th frame is substantially entirely red, whether the image of the 59th frame is substantially entirely green, and whether the image of the 60th frame is substantially entirely blue, and if it is determined that the image of any of these frames is not substantially entirely the specified color, it outputs a failure detection signal to notify the control device 8 that a failure in the imaging device 111 has been detected.

[0036] Note that "substantially the entire surface is XX (XX is the name of a predetermined color)" means that not only all pixels of the image are of the predetermined color, but also that the image contains only noise that inevitably occurs even when the imaging device 111 is operating normally. The image processing unit 122 determines that the entire surface of the image is substantially the predetermined color when predetermined conditions are met, for example, the ratio of pixels that are not of the predetermined color is equal to or less than a threshold, and the number of consecutive pixels that are not of the predetermined color is equal to or less than a threshold.

[0037] 4 is a diagram showing the flow of processing by the image processing unit 122. When the image processing unit 122 acquires an image from the imaging device 111 via the image acquisition unit 121 (step S101), it determines whether the frame number assigned to the image is 58, 59, 60, or any other number (step S102).

[0038] If the frame number is any one of 58 to 60 (step S102: "1," "2," or "3"), the image processing unit 122 performs processing to determine whether or not there is an error (step S103). Specifically, in step S103, if the frame number is 58 (step S102: "1"), the image processing unit 122 determines that there is no error if the image acquired in step S101 is an image that is substantially entirely red, and that there is an error if the image is not an image that is substantially entirely red.

[0039] Similarly, in step S103, if the frame number is 59 (step S102: "2"), the image processing unit 122 determines that there is no error if the image acquired in step S101 is an image in which the entire image is substantially green, and that there is an error if the image is not an image in which the entire image is substantially green. Also, in step S103, if the frame number is 60 (step S102: "3"), the image processing unit 122 determines that there is no error if the image acquired in step S101 is an image in which the entire image is substantially blue, and that there is an error if the image is not an image in which the entire image is substantially blue.

[0040] If it is determined in step S103 that an error exists (step S103: "Yes"), the image processing unit 122 outputs a failure detection signal to the data processing device 12 notifying that a failure in the imaging device 111 has been detected (step S104).

[0041] When the control device 8 receives a failure detection signal output from the image processing unit 122 of the data processing device 12, it notifies, for example, the driver of the train 9 of the failure of the imaging device 111 and performs predetermined control processing such as deceleration on the train 9.

[0042] If it is determined in step S103 that there is no error (step S103: "No"), or if the processing of step S104 is completed, the image processing unit 122 returns the processing to step S101 and repeats the processing from step S101 onwards for the next image acquired from the imaging device 111 via the image acquisition unit 121.

[0043] The processes in steps S103 and S104 are the above-mentioned failure determination process.

[0044] In step S102, if it is determined that the frame number is not one of 58 to 60 (step S102: "4"), the image processing unit 122 recognizes the object appearing in the image acquired in step S101 using a known image recognition method (step S105).

[0045] Next, the image processing unit 122 determines whether or not the object is an obstacle to the running of the train 9 based on the type of the object recognized in step S105 and the position of the object recognized in the image (step S106).

[0046] In step S106, if it is determined that an object that may be an obstacle to the running of the train 9 has been recognized (step S106: "Yes"), the image processing unit 122 outputs an obstacle detection signal to the data processing device 12 notifying that an obstacle has been detected (step S107).

[0047] When the control device 8 receives an obstacle detection signal output from the image processing unit 122 of the data processing device 12, it notifies, for example, the driver of the train 9 that an obstacle has been detected, and performs predetermined control processing on the train 9, such as stopping the train.

[0048] In step S106, if it is determined that no object that would hinder the movement of the train 9 has been recognized (step S106: "No"), or if the processing of step S107 has been completed, the image processing unit 122 returns the processing to step S101 and repeats the processing from step S101 onwards for the next image acquired from the imaging device 111 via the image acquisition unit 121.

[0049] The processing in steps S105 to S107 is the object detection processing described above.

[0050] 3, the description of the components of the data processing device 12 continues. The synchronization signal acquisition unit 123 acquires the synchronization signal output by the imaging device 111. As described above, the synchronization signal is a signal output by the imaging device 111 at the timing when the imaging device 111 captures an image every 1 / 60 seconds.

[0051] The illumination instruction signal output unit 124 outputs an illumination instruction signal to the illumination device 112 and the switchable mirror 113. Based on the synchronization signal acquired by the synchronization signal acquisition unit 123, the illumination instruction signal output unit 124 starts outputting an illumination instruction signal that instructs the illumination device 112 to illuminate a red test pattern after the imaging device 111 captures the 57th frame image and before the imaging device 111 captures the 58th frame image. Subsequently, the illumination instruction signal output unit 124 starts outputting an illumination instruction signal that instructs the illumination device 112 to illuminate a green test pattern after the imaging device 111 captures the 58th frame image and before the imaging device 111 captures the 59th frame image, in place of the illumination instruction signal that had been output up to that point. Subsequently, the illumination instruction signal output unit 124 starts outputting an illumination instruction signal that instructs the illumination device 112 to illuminate a blue test pattern after the imaging device 111 captures the 59th frame image and before the imaging device 111 captures the 60th frame image, in place of the illumination instruction signal that had been output up to that point. Next, the irradiation instruction signal output unit 124 stops outputting the irradiation instruction signal at a timing after the imaging device 111 captures the image of the 60th frame and before the imaging device 111 captures the image of the first frame.

[0052] The irradiation device 112 irradiates a red, green, or blue test pattern in response to an irradiation instruction signal output from the irradiation instruction signal output unit 124 of the data processing device 12. The switchable mirror 113 is in a reflective state while an irradiation instruction signal for one of the colors is being output from the irradiation instruction signal output unit 124 of the data processing device 12, and is in a transmissive state while no irradiation instruction signal is being output.

[0053] 5 is a diagram showing the flow of processing by the irradiation instruction signal output unit 124. For example, when the data processing device 12 is started up, the irradiation instruction signal output unit 124 first assigns an initial value of 0 to a counter C (step S201).

[0054] Next, when the irradiation instruction signal output unit 124 acquires a synchronization signal from the imaging device 111 via the synchronization signal acquisition unit 123 (step S202), it increments the counter C by 1 (step S203) and then determines whether the counter C is 57, 58, 59, 60, or some other value (step S204).

[0055] If counter C is any one of 57 to 59 (step S204: "1", "2", or "3"), irradiation instruction signal output unit 124 starts outputting irradiation instruction signals to irradiation device 112 and switchable mirror 113 (step S205).

[0056] Specifically, when counter C is 57 (step S204: "1"), the irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing irradiation of a red test pattern in step S205. When counter C is 58 (step S204: "2"), the irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing irradiation of a green test pattern instead of red in step S205. When counter C is 59 (step S204: "3"), the irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing irradiation of a blue test pattern instead of green in step S205.

[0057] If counter C is 60 (step S204: "4"), irradiation instruction signal output unit 124 stops outputting the irradiation instruction signal to irradiation device 112 and switchable mirror 113 (step S206). Note that the irradiation instruction signal whose output is stopped in step S206 is an irradiation instruction signal that instructs irradiation of a blue test pattern.

[0058] If the counter C is not between 57 and 60 (step S204: "5"), or if the processing of step S205 is completed, the irradiation instruction signal output unit 124 returns the processing to step S202 and repeats the processing from step S202 onwards in accordance with the next synchronization signal acquired from the imaging device 111 via the synchronization signal acquisition unit 123.

[0059] On the other hand, when the processing of step S206 is completed, the irradiation instruction signal output unit 124 returns the processing to step S201, and after substituting the initial value 0 into the counter C (step S201), repeats the processing from step S202 onwards in accordance with the next synchronization signal acquired from the imaging device 111 via the synchronization signal acquisition unit 123.

[0060] 6 is a diagram showing the correspondence between images captured by the imaging device 111 and the processing performed by the data processing device 12. Of the 60 images captured per second by the imaging device 111, the first to 57th frames are images of the front of the train 9. The 58th to 60th frames are images of a red, green, or blue test pattern.

[0061] The data processing device 12 performs object detection processing using images from the 1st frame to the 57th frame out of the 60 images captured per second by the imaging device 111. Then, the data processing device 12 performs failure determination processing using images from the 58th frame to the 60th frame.

[0062] According to the above-described failure determination system 1, if a failure occurs in the imaging device 111, the failure is detected in approximately one second at the latest. That is, the presence or absence of a failure in the imaging device 111 is determined substantially instantaneously. As a result, it is possible to avoid accidents such as the train 9 colliding with an obstacle that should be detected but is not detected due to a failure of the imaging device 111.

[0063] [Variations] The above-described embodiment may be modified in various ways within the scope of the technical concept of the present invention. These modifications are shown below. Note that two or more of the modifications shown below may be combined as appropriate.

[0064] [First Modification] Instead of the dimming mirror 113 employed in the above-described embodiment, a movable mirror may be employed so as to be within the imaging range of the imaging device 111 when the test pattern is being irradiated by the irradiation device 112, and to be outside the imaging range of the imaging device 111 when the test pattern is not being irradiated by the irradiation device 112.

[0065] 7 is a diagram schematically illustrating the configuration of an imaging unit 21 that the failure determination system 1 according to the first modification has in place of the imaging unit 11. Note that the same reference numerals as those used in the imaging unit 11 are used for components that the imaging unit 21 has in common with the imaging unit 11.

[0066] The imaging unit 21 includes a movable mirror 213 (an example of a reflective member) instead of the dimming mirror 113. Fig. 7(a) shows a state in which the movable mirror 213 is outside the imaging range of the imaging device 111, and Fig. 7(b) shows a state in which the movable mirror 213 is within the imaging range of the imaging device 111. The movable mirror 213 is moved by a drive mechanism (not shown) to the position shown in Fig. 7(a) when the irradiation device 112 is not irradiating a test pattern, and is moved to the position shown in Fig. 7(b) when the irradiation device 112 is irradiating a test pattern.

[0067] According to the first modification, the presence or absence of a malfunction in the imaging device 111 can also be determined substantially instantaneously.

[0068] [Second Modification] Instead of the switchable mirror 113 employed in the above-described embodiment, a half mirror (an example of a reflecting member) that transmits approximately half of the light and reflects the remaining light may be employed. The configuration of the imaging unit provided in the failure determination system 1 according to this second modified example is such that a half mirror is disposed in place of the switchable mirror 113 (see FIG. 2) provided in the imaging unit 11 according to the above-described embodiment.

[0069] When the illumination device 112 is not illuminating the test pattern, the image captured by the imaging device 111 is an image drawn by light that passes through the half mirror from the front of the train 9 and reaches the imaging device 111, and light that reaches the half mirror from various directions and is reflected by the half mirror and heads toward the imaging device 111. In this case, the amount of light that reaches the half mirror from various directions and is reflected by the half mirror and heads toward the imaging device 111 is sufficiently small compared to the amount of light that passes through the half mirror from the front of the train 9 and reaches the imaging device 111, and can be ignored. Therefore, the image captured by the imaging device 111 is essentially an image of the area ahead of the train 9.

[0070] When the illumination device 112 illuminates the test pattern, the image captured by the imaging device 111 is an image drawn by light that passes through the half mirror from the front of the train 9 and reaches the imaging device 111, light that reaches the half mirror from various directions and is reflected by the half mirror toward the imaging device 111, and light of the test pattern that is illuminated from the illumination device 112, reflected by the half mirror, and toward the imaging device 111. The amount of light of the test pattern that is illuminated from the illumination device 112, reflected by the half mirror, and toward the imaging device 111 increases as the output of the illumination device 112 increases. Therefore, when the output of the illumination device 112 is sufficiently high, the proportion of light of the test pattern in the light toward the imaging device 111 becomes sufficiently large, and the image captured by the imaging device 111 is substantially an image of the test pattern.

[0071] According to the second modification, the presence or absence of a malfunction in the imaging device 111 can also be determined substantially instantaneously.

[0072] [Third Modification] In the case of the failure determination system 1 according to the embodiment described above, during the period of one second when three images from the 58th frame to the 60th frame are captured, that is, during a period of 3 / 60 seconds = 0.05 seconds, no images of the area ahead of the train 9 are captured, and no object detection processing is performed during that time. In order to shorten this period during which no object detection processing is performed, the imaging device 111 may capture images at shorter time intervals when the test pattern is projected than when the test pattern is not projected.

[0073] For example, in this third modified example, the imaging device 111 captures images from the 1st frame to the 59th frame at intervals of 1 / 60 seconds, and from the 60th frame to the 62nd frame at intervals of 1 / 180 seconds.

[0074] Then, the irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing the irradiation of a red test pattern after the imaging device 111 captures the image of the 59th frame and before the imaging of the image of the 60th frame, starts outputting an irradiation instruction signal instructing the irradiation of a green test pattern instead of red after the imaging device 111 captures the image of the 60th frame and before the imaging of the image of the 61st frame, starts outputting an irradiation instruction signal instructing the irradiation of a blue test pattern instead of green after the imaging device 111 captures the image of the 61st frame and before the imaging of the image of the 62nd frame, and stops outputting the irradiation instruction signal after the imaging device 111 captures the image of the 62nd frame and before the imaging of the image of the 1st frame.

[0075] Then, the image processing unit 122 uses the images from the 1st frame to the 59th frame for the object detection process, and the images from the 60th frame to the 62nd frame for the failure determination process.

[0076] According to the failure determination system 1 of the third modified example, the time during which the object detection process is not performed is shorter than in the case of the failure determination system 1 according to the embodiment described above, which is desirable.

[0077] [Fourth Modification] The time during which the object detection process is not performed may be shortened by a method different from the third modified example described above. Specifically, a configuration may be adopted in which the imaging device 111 constantly captures images at a frame rate (e.g., 1 / 180 fps) higher than the frame rate (e.g., 1 / 60 fps) required to avoid the train 9 colliding with an obstacle, and thinned images of images captured ahead of the train 9 are used for the object detection process.

[0078] For example, in this fourth modified example, the imaging device 111 always captures images at 180 fps. The illumination instruction signal output unit 124 starts outputting an illumination instruction signal instructing the imaging device 111 to illuminate a red test pattern after capturing the 177th frame image and before capturing the 178th frame image, starts outputting an illumination instruction signal instructing the imaging device 111 to illuminate a green test pattern instead of red after capturing the 178th frame image and before capturing the 179th frame image, starts outputting an illumination instruction signal instructing the imaging device 111 to illuminate a blue test pattern instead of green after capturing the 179th frame image and before capturing the 180th frame image, and stops outputting the illumination instruction signal after capturing the 180th frame image and before capturing the 1st frame image.

[0079] Then, the image processing unit 122 uses only the images of frames having frame numbers (3n-2) (where n is a natural number satisfying 1≦n≦59) from the images from the first frame to the 177th frame (images captured in front of the train 9) for the object detection process.

[0080] The failure determination system 1 according to the fourth modification also preferably shortens the time during which the object detection process is not performed compared to the failure determination system 1 according to the embodiment described above.

[0081] [Fifth Modification] Instead of using the switchable mirror 113 employed in the above-described embodiment, the irradiation device 112 may directly irradiate the imaging device 111 with a test pattern. Fig. 8 is a diagram schematically showing the configuration of an imaging unit 31 that is provided in the failure determination system 1 according to this fifth modification in place of the imaging unit 11. Note that the same reference numerals as those used in the imaging unit 11 are used for components that the imaging unit 31 has in common with the imaging unit 11.

[0082] As shown in FIG. 8, in the imaging unit 31, the positions of the imaging device 111 and the illumination device 112 are adjusted so that the test pattern illuminated by the illumination device 112 is directed directly toward the objective lens of the imaging device 111.

[0083] According to the fifth modification, the presence or absence of a malfunction in the imaging device 111 can also be determined substantially instantaneously.

[0084] [Sixth Modification] In the above-described embodiment, the three test patterns for red, green, and blue are captured in three consecutive frames. Alternatively, the three test patterns may be captured in three non-consecutive frames.

[0085] In the sixth modified example, for example, the irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing the imaging device 111 to irradiate a red test pattern after capturing a 19th frame image and before capturing a 20th frame image, and stops outputting the irradiation instruction signal after capturing the 20th frame image and before capturing a 21st frame image. The irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing the imaging device 111 to irradiate a green test pattern after capturing a 39th frame image and before capturing a 40th frame image, and stops outputting the irradiation instruction signal after capturing the 40th frame image and before capturing a 41st frame image. The irradiation instruction signal output unit 124 starts outputting an irradiation instruction signal instructing the imaging device 111 to irradiate a blue test pattern after capturing a 59th frame image and before capturing a 60th frame image, and stops outputting the irradiation instruction signal after capturing the 60th frame image and before capturing a 1st frame image.

[0086] As a result of the operation of the irradiation device 112 and the dimming mirror 113 in accordance with the processing of the irradiation instruction signal output unit 124 described above, of the 60 images captured by the imaging device 111 every second, the 20th frame is an image of a red test pattern, the 40th frame is an image of a green test pattern, the 60th frame is an image of a blue test pattern, and the other frames are images of the area ahead of the train 9.

[0087] The image processing unit 122 performs failure determination processing using the images of the 20th, 40th, and 60th frames, and performs object detection processing using the images of the other frames.

[0088] According to the sixth modification, the frequency of periods in which the object detection process is not performed increases compared to the failure determination system 1 according to the above-described embodiment, but the length of these periods is shortened, which is desirable.

[0089] [Seventh Modification] The switchable mirror 113 used in the fault determination system 1 according to the above-described embodiment ideally reflects all light in the reflective state and transmits all light in the transmissive state. However, due to technical limitations, the switchable mirror 113 in the reflective state may transmit a certain proportion of light. Therefore, in a frame capturing an image of a test pattern, an image in which an image of the front of the train 9 is overlaid on the image of the test pattern (hereinafter referred to as a "mixed image") may be captured. This phenomenon can also occur in the above-described second modified example (in which a half mirror is used instead of the switchable mirror 113) and fifth modified example (in which the test pattern is directly irradiated onto the imaging device 111).

[0090] Therefore, a configuration may be adopted in which the image processing unit 122 removes at least some of the components of the image ahead of the train 9 from the mixed image, generates an image similar to an image capturing only the test pattern (hereinafter referred to as a "test pattern separated image"), and performs fault detection processing using the test pattern separated image.

[0091] In this seventh modified example, for example, the irradiation instruction signal output unit 124 outputs an irradiation instruction signal so that test patterns are captured in frames 20, 40, and 60, similarly to the sixth modified example. Then, the image processing unit 122 generates a red test pattern separation image by removing components of the image of frame 19 from the image of frame 20 (combined image), generates a green test pattern separation image by removing components of the image of frame 39 from the image of frame 40 (combined image), and generates a blue test pattern separation image by removing components of the image of frame 59 from the image of frame 60 (combined image).

[0092] The procedure for generating the test pattern separated image will be described below using a red test pattern separated image as an example. The image processing unit 122 calculates a value for each of the red, green, and blue of each pixel in the 20th frame image (combined image) by subtracting from that value the value obtained by multiplying the color value of the corresponding pixel in the 19th frame by a predetermined multiplier. This multiplier is determined as appropriate based on the characteristics of the image capturing device 111, the intensity of the light of the test pattern irradiated by the irradiation device 112, the intensity of the ambient light around the image capturing device 111, and the like, so that the components of the forward image of the train 9 can be removed as much as possible from the image obtained by combining the forward image of the train 9 and the test pattern image.

[0093] The image processing unit 122 generates an image having the values ​​calculated by the above subtraction as pixel values ​​as a test pattern separation image.

[0094] According to the seventh modification, even in a situation where an image of the front of the train 9 is mixed in with an image of a frame capturing a test pattern, it is possible to determine with high accuracy whether or not the imaging device 111 has a malfunction.

[0095] [Other variations] (1) In the above-described embodiment, except when capturing an image of a test pattern, the switchable mirror 113 does not reflect the light emitted from the irradiation device 112 toward the imaging device 111. Similarly, in the above-described first modified example, except when capturing an image of a test pattern, the movable mirror 213 does not reflect the light emitted from the irradiation device 112 toward the imaging device 111. Therefore, in the above-described embodiment or the first modified example, the irradiation device 112 may always emit the test pattern.

[0096] (2) In the above-described embodiment, an image entirely covered with a predetermined color is used as the test pattern. The form of the test pattern is not limited to this. For example, the vertical stripes shown in FIG. 9(a), the horizontal stripes shown in FIG. 9(b), or the houndstooth test pattern shown in FIG. 9(c) may be used. A single test pattern may also include a plurality of different colors. The colors used in the test pattern are not limited to the three primary colors. For example, a white test pattern obtained by mixing the three primary colors of light may be projected, and the presence or absence of a malfunction in the imaging device 111 may be determined by checking the red, green, and blue components of the pixel values ​​of the image captured by the imaging device 111.

[0097] (3) In the above-described embodiment, for the sake of convenience, the test pattern is imaged every second. However, the frequency of image capture of the test pattern is not limited to every second.

[0098] (4) In the above-described embodiment, the image capturing device 111 is a visible light camera. However, other types of image capturing devices, such as an infrared camera, may also be used. [Explanation of symbols]

[0099] 1...fault determination system, 8...control device, 9...train, 11...imaging unit, 12...data processing device, 21...imaging unit, 31...imaging unit, 111...imaging device, 112...illumination device, 113...dimming mirror, 121...image acquisition unit, 122...image processing unit, 123...synchronization signal acquisition unit, 124...illumination instruction signal output unit, 213...movable mirror.

Claims

1. With respect to each of a group of frames captured per second by an imaging device that captures moving images at a predetermined frame rate, a different test pattern is projected within the imaging range when capturing each of a plurality of frames at different predetermined positions in the group of frames, and whether or not the imaging device is faulty is determined from the frame images of the imaging device in which the test pattern is captured, and object detection processing is performed using other frame images. A system for determining failures in an imaging device.

2. A test pattern is intermittently projected into the imaging range of an imaging device that captures moving images, and the test pattern is reflected back toward the imaging device by a reflective member that is a dimming mirror that is in a reflective state when the test pattern is projected and in a transmissive state when the test pattern is not projected. The presence or absence of a malfunction in the imaging device is determined from an image of the imaging device that reflects the test pattern. A system for determining failures in an imaging device.

3. A reflective member that intermittently projects a test pattern into an imaging range of an imaging device that captures video and reflects the test pattern toward the imaging device, the reflective member comprising a mirror that is within the imaging range of the imaging device when the test pattern is projected and that is outside the imaging range of the imaging device when the test pattern is not projected, and that determines whether or not the imaging device is faulty from an image of the imaging device that reflects the test pattern. A system for determining failures in an imaging device.

4. The imaging device captures images at shorter time intervals when the test pattern is illuminated than when the test pattern is not illuminated. The imaging device failure determination system according to claim 1 .

Images