Signal processing device, signal processing method, and imaging device

Abstract

The signal processing device includes: a plurality of first processing units, which are arranged to correspond to a plurality of first signals, and each of the plurality of first processing units performs a predetermined process based on the corresponding first signal among the plurality of first signals, so as to be able to generate a second signal; a control unit, which can generate a selection control signal; a first selection unit, which can select a first signal to be provided to the selected first processing unit from the plurality of first signals based on the selection control signal; a second processing unit, which performs a predetermined process based on the first signal selected by the first selection unit, so as to be able to generate a third signal; a second selection unit, which can select the second signal generated by the selected first processing unit from the plurality of second signals based on the selection control signal; and a first comparison unit, which can compare the third signal with the second signal selected by the second selection unit from the plurality of second signals.

Description

Technical Field

[0001] The present disclosure relates to a signal processing apparatus that performs signal processing, a signal processing method used in such a signal processing apparatus, and an imaging apparatus including such a signal processing apparatus. Background Art

[0002] Signal processing apparatuses generally use a plurality of processors to perform processing in parallel. For example, Patent Document 1 discloses a technique for detecting a failure in two or more paths in a signal processing apparatus that performs processing in parallel using two or more paths using test data.

[0003] Citation List

[0004] Patent Document

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2019-4361 Summary of the Invention

[0006] It is desirable for a signal processing apparatus configured to perform processing in parallel using a plurality of processors to make a determination during normal operation.

[0007] There is a desire to provide a signal processing apparatus, a signal processing method, and an imaging apparatus that enable a determination to be made during normal operation.

[0008] A signal processing apparatus according to an embodiment of the present disclosure includes a plurality of first processors, a controller, a first selector, a second processor, a second selector, and a first comparator. The plurality of first processors are provided in association with a plurality of first signals. Each of the plurality of first processors is configured to perform a predetermined process based on the associated first signal among the plurality of first signals to generate a second signal. The controller is configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result. The first selector is configured to select, based on the selection control signal, the first signal among the plurality of first signals that is to be provided to the selected first processor. The second processor is configured to perform a predetermined process based on the first signal selected by the first selector among the plurality of first signals to generate a third signal. Each of the plurality of second selectors is configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals. The first comparator is configured to compare the third signal with the second signal selected by the second selector among the plurality of second signals.

[0009] A signal processing method according to an embodiment of the present disclosure includes: causing each of a plurality of first processors configured to be associated with a plurality of first signals to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal; selecting any one of the plurality of first processors and generating a selection control signal based on the selection result; selecting, based on the selection control signal, the first signal among the plurality of first signals to be provided to the selected first processor; causing a second processor to perform the predetermined process based on the selected first signal among the plurality of first signals, thereby generating a third signal; selecting, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and comparing the third signal and the selected second signal among the plurality of second signals with each other.

[0010] An imaging device according to an embodiment of the present disclosure includes: an imager, a plurality of first processors, a controller, a first selector, a second processor, a second selector, and a first comparator. The imager is configured to perform an imaging operation to generate an image signal including a plurality of first signals. The plurality of first processors are arranged to be associated with the plurality of first signals. Each of the plurality of first processors is configured to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal. The controller is configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result. The first selector is configured to select, based on the selection control signal, the first signal among the plurality of first signals to be provided to the selected first processor. The second processor is configured to perform a predetermined process based on the first signal selected by the first selector among the plurality of first signals, thereby generating a third signal. The second selector is configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals. The first comparator is configured to compare the third signal and the second signal selected by the second selector among the plurality of second signals with each other.

[0011] A signal processing device, a signal processing method, and an imaging device according to an embodiment of the present disclosure, wherein each of the plurality of first processors performs a predetermined process based on the associated first signal among the plurality of first signals to generate a second signal. Further, any one of the plurality of first processors is selected, and a selection control signal based on the selection result is generated. Further, based on the selection control signal, the first signal among the plurality of first signals to be provided to the selected first processor is selected. A second processor performs a predetermined process based on the selected first signal among the plurality of first signals to generate a third signal. Based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals is selected. Further, the third signal and the selected second signal among the plurality of second signals are compared with each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a block diagram showing a configuration example of an imaging device according to an embodiment of the present disclosure.

[0013] Figure 2 is a diagram showing Figure 1 an explanatory diagram of a configuration example of the pixel array shown.

[0014] Figure 3 is a block diagram showing a specific example of an image processing unit and a determination unit according to a first embodiment.

[0015] Figure 4 is a diagram showing the Figure 3 example of the image data provided to the image processing unit shown.

[0016] Figure 5 is a diagram showing the Figure 3 another explanatory diagram of the example of the image data provided to the image processing unit shown.

[0017] Figure 6 is a diagram showing the Figure 1 example of the transmission format of the transmission data transmitted from the imaging device shown.

[0018] Figure 7 is a diagram showing the Figure 3 example of the timing waveform of the operation of the image processing unit and the determination unit shown.

[0019] Figure 8 is a diagram showing the Figure 3 another timing waveform diagram of the operation example of the image processing unit and the determination unit shown.

[0020] Figure 9 is a diagram showing the Figure 3Another timing waveform diagram of the operation examples of the image processing unit and the determination unit shown.

[0021] Figure 10 It shows Figure 3 Another timing waveform diagram of the operation examples of the image processing unit and the determination unit shown.

[0022] Figure 11 It is a block diagram showing a specific example of the image processing unit and the determination unit according to the second embodiment.

[0023] Figure 12 It shows Figure 11 The timing waveform diagram of the operation examples of the image processing unit and the determination unit shown.

[0024] Figure 13 It shows Figure 11 The explanatory diagram of the example of the operation state of the image processing unit and the determination unit shown.

[0025] Figure 14 It shows Figure 11 Another explanatory diagram of the example of the operation state of the image processing unit and the determination unit shown.

[0026] Figure 15 It shows Figure 11 Another timing waveform diagram of the operation examples of the image processing unit and the determination unit shown.

[0027] Figure 16 It is a block diagram showing a specific example of the image processing unit and the determination unit according to the modified example of the second embodiment.

[0028] Figure 17 It shows Figure 16 The timing waveform diagram of the operation examples of the image processing unit and the determination unit shown.

[0029] Figure 18 It is the explanatory diagram of the usage example of the imaging device.

[0030] Figure 19 It is a block diagram showing an example of the schematic configuration of the vehicle control system.

[0031] Figure 20 It is the explanatory diagram of the example assisting in explaining the installation positions of the outside vehicle information detection unit and the imaging unit. Detailed Description of the Embodiment

[0032] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the description will be given in the following order.

[0033] 1. First Embodiment

[0034] 2. Second Embodiment

[0035] 3. Usage example of imaging device

[0036] 4. Application example of moving body

[0037] <First Embodiment>

[0038] [Configuration example]

[0039] Figure 1 A configuration example of the imaging device 1 including a signal processing device according to the embodiment is shown. Note that the signal processing method and the imaging device according to the embodiment of the present disclosure will be described together because they are implemented by this embodiment. The imaging device 1 includes an imager 10, a sensor interface 21, a determination unit 22, an image processing unit 30, a determination unit 40, a transmission data generation unit 23, a determination unit 24, a transmission unit 25, a determination result collection unit 26, a register 27, and a communication unit 28.

[0040] The imager 10 is configured to perform an imaging operation to generate image data DTA. The imager 10 includes a pixel array 11, an AD converter 12, and a determination unit 13.

[0041] The pixel array 11 includes a plurality of pixels P arranged in a matrix. The pixel P is configured to include a photodiode that performs photoelectric conversion, and is configured to generate a pixel voltage based on the light reception amount.

[0042] Figure 2 A configuration example of the pixel array 11 is shown. The plurality of pixels P are arranged in units (units U) of four pixels P arranged in two rows and two columns. The unit U includes four pixels PR, PGr, PGb, and PB. The pixel PR is configured to receive red light, the pixels PGr and PGb are configured to receive green light, and the pixel PB is configured to receive blue light. In this example, in the unit U, the pixel PR is arranged in the upper left, the pixel PGr is arranged in the upper right, the pixel PGb is arranged in the lower left, and the pixel PB is arranged in the lower right. Therefore, in the pixel array 11, the pixels PR, PGr, PGb, and PB are arranged in a so-called Bayer array.

[0043] AD converter 12( Figure 1 ) is configured to perform AD conversion based on the pixel voltage provided from each pixel P, thereby generating a pixel value as a digital value. In addition, the AD converter 12 sequentially provides the generated pixel values to the sensor interface 21 in units of pixel values of one frame using the image data DTA.

[0044] The determination unit 13 is configured to perform a determination process on the AD converter 12 to detect a failure of the AD converter 12. In addition, the determination unit 13 notifies the determination result collection unit 26 of determination data INF1 indicating the determination result regarding the AD converter 12.

[0045] The sensor interface 21 is an interface between the imager 10 and the image processing unit 30. The sensor interface 21 is configured to provide the image data DTB based on the image data DTA provided from the imager 10 to the image processing unit 30.

[0046] The determination unit 22 is configured to perform a determination process on the sensor interface 21 to detect a failure of the sensor interface 21. In addition, the determination unit 22 notifies the determination result collection unit 26 of determination data INF2 indicating the determination result regarding the sensor interface 21.

[0047] The image processing unit 30 is configured to perform predetermined image processing based on the image data DTB provided from the sensor interface 21 to generate image data DTC. The predetermined image processing includes, for example, at least one of gain adjustment processing, white balance adjustment processing, black level adjustment processing, HDR (high dynamic range) synthesis processing, noise removal processing, and pixel defect correction processing.

[0048] The determination unit 40 is configured to perform a determination process on the image processing unit 30 to detect a failure of the image processing unit 30. In addition, the determination unit 40 notifies the determination result collection unit 26 of determination data INF3 indicating the determination result regarding the image processing unit 30.

[0049] Figure 3 A configuration example of the image processing unit 30 and the determination unit 40 is shown. The image data DTB provided to the image processing unit 30 includes four pieces of image data DB (image data DB0 to DB3). The image data DTC generated by the image processing unit 30 includes four pieces of image data DC (image data DC0 to DC3).

[0050] Figure 4 Examples of the image data DB0, DB1, DB2, and DB3 are shown. Figure 5 An example of the image data DB0 is shown.

[0051] As Figure 4 shown, the image data DB0 includes a plurality of pixel values related to the red pixels PR. Specifically, as Figure 5As shown, the image data DB0 includes line image data such as DB_R0, DB_R2, DB_R4, DB_R6, etc. The line image data DB_R0 includes pixel values of a plurality of pixels PR related to the 0th row L, the line image data DB_R2 includes pixel values of a plurality of pixels PR related to the 2nd row L, the line image data DB_R4 includes pixel values of a plurality of pixels PR related to the 4th row L, and the line image data DB_R6 includes pixel values of a plurality of pixels PR related to the 6th row L.

[0052] In a similar manner, as Figure 4 shown, the image data DB1 includes a plurality of pixel values related to the green pixels PGr. Specifically, the image data DB1 includes line image data such as DB_Gr0, DB_Gr2, DB_Gr4, DB_Gr6, etc. The line image data DB_Gr0 includes pixel values of a plurality of pixels PGr related to the 0th row L, the line image data DB_Gr2 includes pixel values of a plurality of pixels PGr related to the 2nd row L, the line image data DB_Gr4 includes pixel values of a plurality of pixels PGr related to the 4th row L, and the line image data DB_Gr6 includes pixel values of a plurality of pixels PGr related to the 6th row L.

[0053] In a similar manner, as Figure 4 shown, the image data DB2 includes a plurality of pixel values related to the green pixels PGb. Specifically, the image data DB2 includes line image data such as DB_Gb1, DB_Gb3, DB_Gb5, DB_Gb7, etc. The line image data DB_Gb1 includes pixel values of a plurality of pixels PGb related to the 1st row L, the line image data DB_Gb3 includes pixel values of a plurality of pixels PGb related to the 3rd row L, the line image data DB_Gb5 includes pixel values of a plurality of pixels PGb related to the 5th row L, and the line image data DB_Gb7 includes pixel values of a plurality of pixels PGb related to the 7th row L.

[0054] In a similar manner, as Figure 4 shown, the image data DB3 includes a plurality of pixel values related to the blue pixels PB. Specifically, the image data DB3 includes line image data such as DB_B1, DB_B3, DB_B5, DB_B7, etc. The line image data DB_B1 includes pixel values of a plurality of pixels PB related to the 1st row L, the line image data DB_B3 includes pixel values of a plurality of pixels PB related to the 3rd row L, the line image data DB_B5 includes pixel values of a plurality of pixels PB related to the 5th row L, and the line image data DB_B7 includes pixel values of a plurality of pixels PB related to the 7th row L.

[0055] As Figure 3As shown, the image processing unit 30 has four processors 31 (processor 31A, processor 31B, processor 31C, and processor 31D). The four processors 31 each have the same circuit configuration as each other and are configured to perform image processing in a similar manner to each other. Processor 31A is configured to perform predetermined image processing based on the image data DB0 to generate the image data DC0. Processor 31B is configured to perform predetermined image processing based on the image data DB1 to generate the image data DC1. Processor 31C is configured to perform predetermined image processing based on the image data DB2 to generate the image data DC2. Processor 31D is configured to perform predetermined image processing based on the image data DB3 to generate the image data DC3. With this configuration, the image processing unit 30 performs image processing in parallel. Hereinafter, the processors 31A, 31B, 31C, and 31D will also be appropriately referred to as channels CH0, CH1, CH2, and CH3.

[0056] The determination unit 40 includes a selector 41, a processor 42, a selector 43, a comparison unit 44, and a determination processor 45.

[0057] The selector 41 is configured to select any one of the image data DB0 to DB3 based on the selection control signal SEL and supply the selected image data DB to the processor 42.

[0058] The processor 42 has the same circuit configuration as each of the four processors 31 of the image processing unit 30. The processor 42 is configured to perform predetermined image processing based on the image data DB supplied from the selector 41 to generate the image data DS. Hereinafter, the processor 42 will also be appropriately referred to as the channel CH4.

[0059] The selector 43 is configured to select any one of the image data DC0 to DC3 based on the selection control signal SEL and supply the selected image data DC to the comparison unit 44.

[0060] The comparison unit 44 is configured to mutually compare the image data DS generated by the processor 42 with the image data DC supplied from the selector 43 and supply the comparison result to the determination processor 45.

[0061] The determination processor 45 is configured to control the determination process of the image processing unit 30. Specifically, the determination processor 45 generates a selection control signal SEL that gives an instruction to select any one of the processors 31A to 31D (channels CH0 to CH3) of the image processing unit 30. For example, the determination processor 45 generates the selection control signal SEL such that the processors 31A, 31B, 31C, and 31D are sequentially and cyclically selected. In addition, the determination processor 45 determines which one of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed based on the comparison results obtained by the comparison unit 44 when the processor 31A (channel CH0) is selected, the comparison results obtained by the comparison unit 44 when the processor 31B (channel CH1) is selected, the comparison results obtained by the comparison unit 44 when the processor 31C (channel CH2) is selected, and the comparison results obtained by the comparison unit 44 when the processor 31D (channel CH3) is selected. In addition, the determination processor 45 notifies the determination result collection unit 26 of determination data INF3 indicating the determination result regarding the image processing unit 30. The determination data INF3 includes, for example, data regarding whether the image processing unit 30 and the determination unit 40 have failed. In addition, the determination data INF3 includes detailed determination data, such as which one of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed.

[0062] With this configuration, in the image processing unit 30, the four processors 31A to 31D (channels CH0 to CH3) perform image processing based on the image data DB0 to DB3. The determination processor 45 sequentially selects one of the four processors 31A to 31D in a time-division manner, and the selector 41 supplies the image data DB input to the selected processor 31 to the processor 42 (channel CH4) of the determination unit 40. The processor 42 performs image processing based on the image data DB supplied from the selector 41 to generate image data DS. The selector 43 supplies the image data DC generated by the selected processor 31 to the comparison unit 44. The comparison unit 44 mutually compares the image data DS supplied from the processor 42 with the image data DC supplied from the selector 43. In the case where the image processing unit 30 and the determination unit 40 have not failed, these image data should be consistent with each other. In addition, the determination processor 45 detects a failure in the image processing unit 30 based on the comparison result obtained by the comparison unit 44.

[0063] The transmission data generation unit 23 ( Figure 1 ) is configured to generate transmission data DT having a predetermined transmission format based on the image data DTC. In addition, the transmission data generation unit 23 can embed the data supplied from the register 27 into the transmission data DT.

[0064] The determination unit 24 is configured to perform a determination process on the transmission data generation unit 23 to detect a failure of the transmission data generation unit 23. In addition, the determination unit 24 notifies the determination result collection unit 26 of determination data INF4 indicating the determination result regarding the transmission data generation unit 23.

[0065] The transmission unit 25 is configured to transmit the transmission data DT provided from the transmission data generation unit 23 to the host device 100 (not shown) via a high-speed interface such as MIPI (Mobile Industry Processor Interface), for example.

[0066] The determination result collection unit 26 is configured to collect the determination data INF1 to INF4 provided from the determination unit 13, the determination unit 22, the determination unit 40, and the determination unit 24, and store the determination data INF indicating a detailed determination result based on the determination data INF1 to INF4 in the register 27. In addition, the determination result collection unit 26 generates an error signal ERR indicating whether a failure has occurred in the imaging device 1 based on the determination data INF1 to INF4, and outputs the error signal ERR.

[0067] The register 27 is configured to hold the determination data INF. The data stored in the register 27 can be read by an external device such as the host device 100 (not shown) via the communication unit 28. In addition, the register 27 can provide the determination data INF to the transmission data generation unit 23. In this case, the transmission data generation unit 23 embeds the determination data INF into the transmission data DT.

[0068] The communication unit 28 is configured to communicate with an external device such as the host device 100 (not shown) via a low-speed interface such as I2C, for example.

[0069] Figure 6 An example of the transmission format of the transmission data DT is shown. The data from the frame start FS to the frame end FE represents the image data of one frame. The transmission data generation unit 23 generates a plurality of packets PCT based on the image data DTC to generate the transmission data DT of one frame.

[0070] As Figure 6As shown in (A) of FIG. , the imaging device 1 transmits image data in the image area A. The image area A includes an effective pixel area A1, and virtual areas A2 and A3. The effective pixel area A1 is an image area corresponding to an area where so-called effective pixels are arranged in the pixel array 11. The virtual area A2 is provided on the upper side of the effective pixel area A1, and the virtual area A3 is provided on the lower side of the effective pixel area A1. In the case where the determination data INF is embedded in the transmission data DT, the transmission data generation unit 23 can insert the determination data INF into, for example, the virtual area A2 or the virtual area A3.

[0071] As Figure 6 shown in (B) of FIG. , a single packet PCT includes a header PH, a payload PP, and a footer PF. The payload PP includes image data of one row in the image area A. Therefore, the imaging device 1 can transmit the image data in the image area A using the same number of packets PCT as the number of pixels in the image area A in the vertical direction. Therefore, the imaging device 1 transmits the image data of one frame in units of rows.

[0072] Here, the processors 31A to 31D correspond to specific examples of the "multiple first processors" in the present disclosure. The image data DB0 to DB3 correspond to specific examples of the "multiple first signals" in the present disclosure. The image data DC0 to DC3 correspond to specific examples of the "multiple second signals" in the present disclosure. The determination processor 45 corresponds to a specific example of the "controller" in the present disclosure. The selector 41 corresponds to a specific example of the "first selection unit" in the present disclosure. The processor 42 corresponds to a specific example of the "second processor" in the present disclosure. The image data DS corresponds to a specific example of the "third signal" in the present disclosure. The selector 43 corresponds to a specific example of the "second selection unit" in the present disclosure. The comparison unit 44 corresponds to a specific example of the "first comparison unit" in the present disclosure. The determination result collection unit 26 corresponds to a specific example of the "output unit" in the present disclosure. The imager 10 and the sensor interface 21 correspond to specific examples of the "imager" in the present disclosure.

[0073] [Operations and Functions]

[0074] Next, the operations and functions of the imaging device 1 according to the present embodiment will be described.

[0075] (Overview of Overall Operations)

[0076] First, reference will be made to Figure 1Outline the overall operation of the imaging device 1. The imager 10 performs an imaging operation to generate image data DTA. Specifically, each pixel P of the pixel array 11 generates a pixel voltage based on the amount of light received. The AD converter 12 performs AD conversion based on the pixel voltages provided from the respective pixels P to generate pixel values as digital values. The determination unit 13 performs a determination process on the AD converter 12 and notifies the determination result collection unit 26 of determination data INF1 indicating the determination result regarding the AD converter 12. The sensor interface 21 supplies image data DTB based on the image data DTA provided from the imager 10 to the image processing unit 30. The determination unit 22 performs a determination process on the sensor interface 21 and notifies the determination result collection unit 26 of determination data INF2 indicating the determination result regarding the sensor interface 21. The image processing unit 30 performs predetermined image processing based on the image data DTB provided from the sensor interface 21 to generate image data DTC. The determination unit 40 performs a determination process on the image processing unit 30 and notifies the determination result collection unit 26 of determination data INF3 indicating the determination result regarding the image processing unit 30. The transmission data generation unit 23 generates transmission data DT having a predetermined transmission format based on the image data DTC. The determination unit 24 performs a determination process on the transmission data generation unit 23 and notifies the determination result collection unit 26 of determination data INF4 indicating the determination result regarding the transmission data generation unit 23. The transmission unit 25 transmits the transmission data DT provided from the transmission data generation unit 23 to the host device 100 (not shown). The determination result collection unit 26 collects the determination data INF1 to determination data INF4 provided from the determination unit 13, the determination unit 22, the determination unit 40, and the determination unit 24, and stores determination data INF indicating a detailed determination result based on the determination data INF1 to INF4 in the register 27. In addition, the determination result collection unit 26 generates an error signal ERR indicating whether a failure has occurred in the imaging device 1 based on the determination data INF1 to INF4 and outputs the error signal ERR. The register 27 holds the determination data INF. The communication unit 28 communicates with an external device such as the host device 100 (not shown).

[0077] (Detailed operation)

[0078] Figure 7 An operation example of the image processing unit 30 and the determination unit 40 is shown in the case where no failure has occurred in the imaging device 1. Figure 8 Shown in detail Figure 7 Part W of Figure 7(A) shows the waveform of the synchronization signal Vsync. (B) to (E) respectively show the image data DB0 to DB3 input to the processors 31A to 31D of the image processing unit 30. (F) shows the image data DB input to the processor 42 of the determination unit 40. (G) to (J) respectively show the image data DC0 to DC3 generated by the processors 31A to 31D. (K) shows the image data DC output from the selector 43. (L) shows the image data DS generated by the processor 42. (M) shows the waveform of the error signal ERR output from the imaging device 1. Figure 8 (A) shows the selection control signal SEL. Figure 8 (B) to (M) are similar to Figure 7 (B) to (M). In Figure 7 (B) to (E) and Figure 8 (B) to (E), the portions shown by thick lines represent the image data provided to the processor 42. In Figure 7 (G) to (J) and Figure 8 (G) to (J), the portions shown by thick lines represent the image data provided to the comparison unit 44. In this example, the error signal ERR is a signal that is at a low level when the imaging device 1 is not malfunctioning and at a high level when the imaging device 1 is malfunctioning. Note that the error signal ERR is not limited to this, and for example, it can be a signal that is at a high level when the imaging device 1 is not malfunctioning and at a low level when the imaging device 1 is malfunctioning.

[0079] In the image processing unit 30, four processors 31A to 31D (channels CH0 to CH3) perform image processing based on the image data DB0 to DB3, thereby generating the image data DC0 to DC3. The determination processor 45 sequentially selects one of the four processors 31A to 31D in a time-division manner. The selector 41 provides the image data DB input to the selected processor 31 to the processor 42 (channel CH4) of the determination unit 40. The processor 42 performs image processing based on the image data DB provided from the selector 41, thereby generating the image data DS. The selector 43 provides the image data DC generated by the selected processor 31 to the comparison unit 44. The comparison unit 44 mutually compares the image data DS provided from the processor 42 with the image data DC provided from the selector 43. In addition, the determination processor 45 detects a failure in the image processing unit 30 based on the comparison result obtained by the comparison unit 44. The operation will be described in detail below.

[0080] As Figure 7 shown, at time t1, a pulse is generated in the synchronization signal Vsync, and thereby the frame period F ( Figure 7 (A)) starts. AsFigure 8 As shown, in this frame period F, the processor 31A (channel CH0) performs image processing based on the image data DB0 (row image data DB_R0, DB_R2, DB_R4, DB_R6, etc.), thereby generating image data DC0 (row image data DC_R0, DC_R2, DC_R4, DC_R6, etc.) ( Figure 8 of (B) and (G)). In this frame period F, the processor 31B (channel CH1) performs image processing based on the image data DB1 (row image data DB_Gr0, DB_Gr2, DB_Gr4, DB_Gr6, etc.), thereby generating image data DC1 (row image data DC_Gr0, DC_Gr2, DC_Gr4, DC_Gr6, etc.) ( Figure 8 of (C) and (H)). In this frame period F, the processor 31C (channel CH2) performs image processing based on the image data DB2 (row image data DB_Gb1, DB_Gb3, DB_Gb5, DB_Gb7, etc.), thereby generating image data DC2 (row image data DC_Gb1, DC_Gb3, DC_Gb5, DC_Gb7, etc.) ( Figure 8 of (D) and (I)). In this frame period F, the processor 31D (channel CH3) performs image processing based on the image data DB3 (row image data DB_B1, DB_B3, DB_B5, DB_B7, etc.), thereby generating image data DC3 (row image data DC_B1, DC_B3, DC_B5, DC_B7, etc.) ( Figure 8 of (E) and (J)).

[0081] The determination unit 40 performs determination processing on the four processors 31A to 31D in sequence during the period T.

[0082] First, at time t11, the determination processor 45 selects the processor 31A (channel CH0) and generates a selection control signal SEL based on the selection result ( Figure 8 of (A)). The selector 41 selects the image data DB0 related to the channel CH0 based on this selection control signal SEL ( Figure 7 of (B) and Figure 8 of (B)). Therefore, the row image data DB_R0 included in the image data DB0 is provided to the processor 42 (channel CH4) ( Figure 7 of (F) and Figure 8 of (F)). The processor 42 performs predetermined processing based on the row image data DB_R0, thereby generating row image data DC_R0 ( Figure 8 of (L)). In addition, this row image data DC_R0 is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC0 related to the channel CH0 based on the selection control signal SELFigure 7 and (G) of Figure 8 Therefore, the line image data DC_R0 included in the image data DC0 is provided to the comparison unit 44 ( Figure 7 and (K) of Figure 8 and (K) of).

[0083] The comparison unit 44 mutually compares the line image data DC_R0 provided from the processor 42 with the line image data DC_R0 provided from the selector 43. In this example, the selected processor 31A (channel CH0) and the processor 42 (channel CH4) do not malfunction. Therefore, these line image data DC_R0 are identical to each other ( Figure 8 and (K) and (L) of). Therefore, the comparison unit 44 determines that the processor 31A (channel CH0) and the processor 42 (channel CH4) do not malfunction. Therefore, the determination result collection unit 26 maintains the error signal ERR at a low level ( Figure 7 and (M) of Figure 8 and (M) of).

[0084] Next, at time t12, the determination processor 45 selects the processor 31B (channel CH1) and generates a selection control signal SEL based on the selection result ( Figure 8 and (A) of). The selector 41 selects the image data DB1 related to the channel CH1 based on the selection control signal SEL ( Figure 7 and (C) of Figure 8 and (C) of). Therefore, the line image data DB_Gr2 included in the image data DB1 is provided to the processor 42 (channel CH4) ( Figure 7 and (F) of Figure 8 and (F) of). The processor 42 performs a predetermined process based on the line image data DB_Gr2 to generate line image data DC_Gr2 ( Figure 8 and (L) of). In addition, this line image data DC_Gr2 is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC1 related to the channel CH1 based on the selection control signal SEL ( Figure 7 and (H) of Figure 8 and (H) of). Therefore, the line image data DC_Gr2 included in the image data DC1 is provided to the comparison unit 44 ( Figure 7 and (K) of Figure 8 and (K) of).

[0085] The comparison unit 44 mutually compares the line image data DC_Gr2 provided from the processor 42 with the line image data DC_Gr2 provided from the selector 43. In this example, the selected processor 31B (channel CH1) and processor 42 (channel CH4) do not malfunction. Therefore, these line image data DC_Gr2 are identical to each other ( Figure 8 of (K) and (L)). Therefore, the comparison unit 44 determines that the processor 31B (channel CH1) and processor 42 (channel CH4) do not malfunction. Therefore, the determination result collection unit 26 maintains the error signal ERR at a low level ( Figure 7 of (M) and Figure 8 of (M)).

[0086] Next, at time t13, the determination processor 45 selects the processor 31C (channel CH2) and generates a selection control signal SEL based on the selection result ( Figure 8 of (A)). The selector 41 selects the image data DB2 related to the channel CH2 based on the selection control signal SEL ( Figure 7 of (D) and Figure 8 of (D)). Therefore, the line image data DB_Gb5 included in the image data DB2 is provided to the processor 42 (channel CH4) ( Figure 7 of (F) and Figure 8 of (F)). The processor 42 performs a predetermined process based on the line image data DB_Gb5, thereby generating line image data DC_Gb5 ( Figure 8 of (L)). In addition, this line image data DC_Gb5 is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC2 related to the channel CH2 based on the selection control signal SEL ( Figure 7 of (I) and Figure 8 of (I)). Therefore, the line image data DC_Gb5 included in the image data DC2 is provided to the comparison unit 44 ( Figure 7 of (K) and Figure 8 of (K)).

[0087] The comparison unit 44 mutually compares the line image data DC_Gb5 provided from the processor 42 with the line image data DC_Gb5 provided from the selector 43. In this example, the selected processor 31C (channel CH2) and processor 42 (channel CH4) do not malfunction. Therefore, these line image data DC_Gb5 are identical to each other ( Figure 8 of (K) and (L)). Therefore, the comparison unit 44 determines that the processor 31C (channel CH2) and processor 42 (channel CH4) do not malfunction. Therefore, the determination result collection unit 26 maintains the error signal ERR at a low level ( Figure 7 of (M) and Figure 8(M))。

[0088] Next, at time t14, the determination processor 45 selects the processor 31D (channel CH3), and generates a selection control signal SEL based on the selection result ( Figure 8 (A)). The selector 41 selects the image data DB3 related to the channel CH3 based on the selection control signal SEL ( Figure 7 (E) and Figure 8 (E)). Therefore, the line image data DB_B7 included in the image data DB3 is provided to the processor 42 (channel CH4) ( Figure 7 (F) and Figure 8 (F)). The processor 42 performs a predetermined process based on the line image data DB_B7, thereby generating line image data DC_B7 ( Figure 8 (L)). In addition, this line image data DC_B7 is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC3 related to the channel CH3 based on the selection control signal SEL ( Figure 7 (J) and (J)). Therefore, the line image data DC_B7 included in the image data DC3 is provided to the comparison unit 44 ( Figure 7 (K) and Figure 8 (K)).

[0089] The comparison unit 44 mutually compares the line image data DC_B7 provided from the processor 42 with the line image data DC_B7 provided from the selector 43. In this example, the selected processor 31D (channel CH3) and the processor 42 (channel CH4) do not malfunction. Therefore, these line image data DC_B7 are identical to each other ( Figure 8 (K) and (L)). Therefore, the comparison unit 44 determines that the processor 31D (channel CH3) and the processor 42 (channel CH4) do not malfunction. Therefore, the determination result collection unit 26 maintains the error signal ERR at a low level ( Figure 7 (M) and Figure 8 (M)).

[0090] In this way, the determination unit 40 performs a determination process on the four processors 31A to 31D in sequence for each unit of line image data in the period T. In addition, the determination unit 40 repeats this process at a cycle of T. Therefore, the determination unit 40 cyclically selects the four processors 31A to 31D in the frame period F, thereby repeating the determination process on the four processors 31A to 31D.

[0091] Next, the operations of the image processing unit 30 and the determination unit 40 in the case of a malfunction will be described in detail with reference to several examples.

[0092] Figure 9 An operation example of the image processing unit 30 and the determination unit 40 is shown in the case where a failure occurs in the processor 31B (channel CH1) of the image processing unit 30. In Figure 9 (H), the part W1 represents a part of the image data different from the desired image data. In this example, the processor 31B (channel CH1) fails after the time t21. Therefore, after the time t21, the image data DC1 generated by the processor 31B is different from the desired image data ( Figure 9 (H)).

[0093] The determination unit 40 cyclically selects the four processors 31A to 31D, thereby repeating the determination process for the four processors 31A to 31D. In addition, at the time t22, the determination processor 45 selects the processor 31B (channel CH1) and generates a selection control signal SEL based on the selection result. The selector 41 selects the image data DB1 related to the channel CH1 based on the selection control signal SEL ( Figure 9 (C)). Therefore, the line image data included in the image data DB1 is provided to the processor 42 (channel CH4) ( Figure 9 (F)). The processor 42 performs a predetermined process based on the line image data, thereby generating line image data ( Figure 9 (L)). In addition, this line image data is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC1 related to the channel CH1 based on the selection control signal SEL ( Figure 9 (H)). Therefore, the line image data included in the image data DC1 is provided to the comparison unit 44 ( Figure 9 (K)).

[0094] The comparison unit 44 mutually compares the line image data provided from the processor 42 ( Figure 9 (L)) with the line image data provided from the selector 43 ( Figure 9 (K)). In this example, the processor 31B (channel CH1) fails after the time t21. Therefore, the line image data provided from the processor 42 and the line image data provided from the selector 43 are inconsistent with each other. The determination processor 45 provides the determination data INF3 including the comparison result to the determination result collection unit 26. Therefore, the determination result collection unit 26 changes the error signal ERR from the low level to the high level ( Figure 9 (M)).

[0095] Then, the determination processor 45 continues to perform determination processing on the four processors 31A to 31D. In addition, in the period from time t22 to time t23, the determination processor 45 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed based on the result of performing determination processing on each of the four processors 31A to 31D once. In this example, only when the processor 31B (channel CH1) is selected, the comparison result shows "inconsistent", and when the processor 31C (channel CH2) is selected, when the processor 31D (channel CH3) is selected, and when the processor 31A (channel CH0) is selected, the comparison result shows "consistent". Therefore, the determination processor 45 determines that the processor 31B has failed. In addition, the determination processor 45 provides the determination data INF3 including the determination result to the determination result collection unit 26.

[0096] In this example, since the processor 31B (channel CH1) has failed, the image processing unit 30 outputs image data different from the desired image data. Therefore, the imaging device 1 transmits image data different from the desired transmission data to the host device 100 (not shown). In addition, the determination processor 45 notifies the determination result collection unit 26 that the processor 31B (channel CH1) has failed, and the determination result collection unit 26 activates the error signal ERR.

[0097] Figure 10 An operation example of the image processing unit 30 and the determination unit 40 is shown in the case where the processor 42 (channel CH4) of the determination unit 40 has failed. In Figure 10 (L), the part W2 represents the part of the image data different from the desired image data. In this example, the processor 42 (channel CH4) fails after time t26. Therefore, after time t26, the image data DS generated by the processor 42 is different from the desired image data ( Figure 10 (L)).

[0098] The determination unit 40 cyclically selects the four processors 31A to 31D, thereby repeating the determination processing on the four processors 31A to 31D. At time t26, the determination processor 45 selects the processor 31D (channel CH3) and generates a selection control signal SEL based on the selection result. The selector 41 selects the image data DB3 related to the channel CH3 based on the selection control signal SEL ( Figure 10 (E)). Therefore, the line image data included in the image data DB3 is provided to the processor 42 (channel CH4) ( Figure 10 (F)). The processor 42 performs predetermined processing based on the line image data to generate line image data ( Figure 10(L) of it. In addition, the line image data of this row is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC3 related to the channel CH3 based on the selection control signal SEL( Figure 10 (J) of it. Therefore, the line image data included in the image data DC3 is provided to the comparison unit 44( Figure 10 (K) of it.

[0099] The comparison unit 44 mutually compares the line image data provided from the processor 42( Figure 10 (L) of it) with the line image data provided from the selector 43( Figure 10 (K) of it). In this example, the processor 42 (channel CH4) of the determination unit 40 fails after the time t26. Therefore, the line image data provided from the processor 42 and the line image data provided from the selector 43 are inconsistent with each other. The determination processor 45 provides the determination data INF3 including the comparison result to the determination result collection unit 26. Therefore, the determination result collection unit 26 changes the error signal ERR from low level to high level( Figure 10 (M) of it.

[0100] Then, the determination processor 45 continues the determination process for the four processors 31A to 31D. In addition, in the period from the time t26 to the time t27, the determination processor 45 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) fails based on the result of performing one determination process on each of the four processors 31A to 31D. In this example, when any one of the processors 31A to 31D (channels CH0 to CH3) is selected, the comparison result shows "inconsistent". Therefore, the determination processor 45 determines that the processor 42 of the determination unit 40 has failed. In addition, the determination processor 45 provides the determination data INF3 including the determination result to the determination result collection unit 26.

[0101] In this example, since the processor 42 (channel CH4) has failed while the processors 31A to 31D (channels CH0 to CH3) have not failed, the image processing unit 30 outputs the desired image data. Therefore, the imaging device 1 transmits the desired transmission data to the host device 100 (not shown). In addition, the determination processor 45 notifies the determination result collection unit 26 that the processor 42 (channel CH3) has failed, and the determination result collection unit 26 activates the error signal ERR.

[0102] As described above, in the imaging apparatus 1, a processor 42 having the same circuit configuration as each of the four processors 31 of the image processing unit 30 is provided. The determination processor 45 selects one of the four processors 31A to 31D, and the selector 41 supplies the image data DB input to the selected processor 31 to the processor 42. Further, the comparison unit 44 mutually compares the image data DC generated by the selected processor 31 with the image data DS generated by the processor 42, and the determination processor 45 determines the image processing unit 30 based on the comparison result obtained by the comparison unit 44. Therefore, in the imaging apparatus 1, the determination process can be performed in a cycle (so-called operation time) for performing the imaging operation (normal operation).

[0103] That is, for example, when the cycle for determination is set to a cycle different from the cycle for the imaging operation (normal operation) of the imaging apparatus (such as a so-called vertical blanking period), and for example, when the determination process is performed using the data for determination, the time for performing the determination process is limited. Therefore, sufficient determination may not be possible. Further, it is desirable that the data for determination has a data pattern with high precision for detecting a failure. Therefore, it is necessary to provide a generation circuit for generating such data for determination.

[0104] On the other hand, according to the present embodiment, the determination process is allowed to be performed in parallel with the normal image processing in the cycle for performing the imaging operation. Therefore, the time for performing the determination process can be ensured. Further, it is allowed to perform the determination process using the actual image data obtained by the imaging operation. This makes it unnecessary to provide a generation circuit for generating the data for determination. As described above, in the imaging apparatus 1, the time for performing the determination process can be ensured and the determination process can also be performed using the actual image data. Therefore, the determination accuracy in the imaging apparatus 1 can be improved.

[0105] [Effect]

[0106] As described above, according to the present embodiment, a processor 42 having the same circuit configuration as each of the four processors 31 is provided. The determination processor selects one of the four processors 31, and the selector 41 supplies the image data input to the selected processor 31 to the processor 42. Further, the comparison unit mutually compares the image data generated by the selected processor 31 with the image data generated by the processor 42, and the determination processor determines the image processing unit based on the comparison result obtained by the comparison unit. Therefore, in the imaging apparatus, the determination process can be performed in the cycle for performing the imaging operation (normal operation).

[0107] [Modification Example 1-1]

[0108] According to the above-described embodiment, four processors 31 are sequentially selected in a time-division manner for each line image data. However, this is not restrictive. Alternatively, for example, four processors 31 may be sequentially selected in a time-division manner for multiple lines of image data, or four processors 31 may be sequentially selected in a time-division manner for image data less than that of one line.

[0109] [Variant Example 1-2]

[0110] According to the above-described embodiment, four processors 31 are provided in the image processing unit 30. However, this is not restrictive. Alternatively, for example, multiple processors 31 less than three may be provided, or five or more processors 31 may be provided.

[0111] [Variant Example 1-3]

[0112] According to the above-described embodiment, as Figure 9 shown, the determination processor 45 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has a failure based on the result of performing a determination process on each of the four processors 31A to 31D once in the period from time t22 to time t23. However, this is not restrictive. For example, after time t22, if the comparison result in the case of performing a determination process on the processor 31B shows "inconsistent", and then the comparison result in the case of performing a determination process on the processor 31C shows "consistent", it may be determined that the processor 31B has a failure.

[0113] Similarly, according to the above-described embodiment, as Figure 10 shown, the determination processor 45 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has a failure based on the result of performing a determination process on each of the four processors 31A to 31D once in the period from time t26 to time t27. However, this is not restrictive. For example, after time t26, if the comparison result in the case of performing a determination process on the processor 31D shows "inconsistent", and then the comparison result in the case of performing a determination process on the processor 31A shows "inconsistent", it may be determined that the processor 42 has a failure.

[0114] [Other Variant Examples]

[0115] In addition, two or more of these variant examples can be combined.

[0116] <2. Second Embodiment>

[0117] Next, the imaging device 2 according to the second embodiment will be described. This embodiment is configured to perform image processing using the processor 42 of the determination unit in place of the failed processor 31 in the case where any one of the four processors 31 in the image processing device fails. Note that components that are substantially the same as those of the imaging device 1 according to the first embodiment described above are denoted by the same reference numerals, and their description will be appropriately omitted.

[0118] As Figure 1 shown, the imaging device 2 includes an image processing unit 50 and a determination unit 60. The image processing unit 50 is configured to perform predetermined image processing based on the image data DTB provided from the sensor interface 21, thereby generating image data DTD. The determination unit 60 is configured to perform a determination process on the image processing unit 50 to detect a failure of the image processing unit 50.

[0119] Figure 11 A configuration example of the image processing unit 50 and the determination unit 60 is shown. The image data DTD generated by the image processing unit 50 includes four pieces of image data DD (image data DD0 to DD3).

[0120] The image processing unit 50 includes a replacement unit 51. The replacement unit 51 is configured to replace one of the four pieces of image data DC0 to DC3 generated by the four processors 31A to 31D with the image data DS generated by the processor 42 based on the control signal CTL, thereby generating four pieces of image data DD0 to DD3. The replacement unit 51 includes four selectors 52 (selectors 52A, 52B, 52C, and 52D). The selector 52A is configured to select one of the image data DC0 and the image data DS based on the control signal CTLA included in the control signal CTL, and output the selected image data as the image data DD0. The selector 52B is configured to select one of the image data DC1 and the image data DS based on the control signal CTLB included in the control signal CTL, and output the selected image data as the image data DD1. The selector 52C is configured to select one of the image data DC2 and the image data DS based on the control signal CTLC included in the control signal CTL, and output the selected image data as the image data DD2. The selector 52D is configured to select one of the image data DC3 and the image data DS based on the control signal CTLD included in the control signal CTL, and output the selected image data as the image data DD3. With this configuration, any one of the four selectors 52 selects the image data DS based on the control signal CTL, and the replacement unit 51 thereby generates four pieces of image data DD0 to DD3.

[0121] The determination unit 60 includes a determination processor 65. The determination processor 65 is configured to control the determination process of the image processing unit 50. Specifically, similar to the determination processor 45 according to the above-described first embodiment, the determination processor 65 generates a selection control signal SEL that gives an instruction to select any one of the processors 31A to 31D (channels CH0 to CH3) in the image processing unit 50. In addition, the determination processor 65 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed based on the comparison results obtained by the comparison unit 44 when the processor 31A (channel CH0) is selected, the comparison results obtained by the comparison unit 44 when the processor 31B (channel CH1) is selected, the comparison results obtained by the comparison unit 44 when the processor 31C (channel CH2) is selected, and the comparison results obtained by the comparison unit 44 when the processor 31D (channel CH3) is selected. In addition, the determination processor 65 notifies the determination result collection unit 26 of determination data INF3 indicating the determination result regarding the image processing unit 50. In addition, when any one of the processors 31A to 31D (channels CH0 to CH3) fails, the determination processor 65 generates a control signal CTL that gives an instruction to replace the image data DC generated by the failed processor 31 with the image data DS generated by the processor 42.

[0122] Here, the replacement unit 51 corresponds to a specific example of the "replacement unit" of the present disclosure. The image data DD0 to DD3 correspond to specific examples of the "plurality of fourth signals" of the present disclosure.

[0123] Figure 12 An operation example of the image processing unit 50 and the determination unit 60 is shown when the processor 31B (channel CH1) in the image processing unit 50 fails. In Figure 12 (H) thereof, the part W3 represents a part of the image data different from the desired image data. In this example, the processor 31B (channel CH1) fails after the time t33. Therefore, after the time t33, the image data DC1 generated by the processor 31B is different from the desired image data ( Figure 12 (H) thereof).

[0124] The frame period F starts at the time t31, and at the time t32, the image processing unit 50 starts to generate the image data DD0 to DD3 based on the image data DB0 to DB3 ( Figure 12 (B) to (E), (G) to (J), and (N) to (Q) thereof).

[0125] Figure 13Shows the operation states of the image processing unit 50 and the determination unit 60. The thick lines show the paths of the image data. At time t32, the processors 31A, 31B, 31C, and 31D (channels CH0 to CH3) do not have faults. Therefore, the processor 31A (channel CH0) performs image processing based on the image data DB0, thereby generating the image data DC0. The selector 52A selects the image data DC0 based on the control signal CTLA and outputs the image data DC0 as the image data DD0. Similarly, the processor 31B (channel CH1) performs image processing based on the image data DB1, thereby generating the image data DC1. The selector 52B selects the image data DC1 based on the control signal CTLB and outputs the image data DC1 as the image data DD1. The processor 31C (channel CH2) performs image processing based on the image data DB2, thereby generating the image data DC2. The selector 52C selects the image data DC2 based on the control signal CTLC and outputs the image data DC2 as the image data DD2. The processor 31D (channel CH3) performs image processing based on the image data DB3, thereby generating the image data DC3. The selector 52D selects the image data DC3 based on the control signal CTLD and outputs the image data DC3 as the image data DD3.

[0126] Similar to the case of the first embodiment described above, the determination unit 60 cyclically selects the four processors 31A to 31D, thereby repeating the determination process on the four processors 31A to 31D ( Figure 12 from (B) to (M)). In this example, the processor 31B (channel CH1) fails at time t33. Therefore, after time t33, the image data DC1 generated by the processor 31B is different from the expected image data.

[0127] In addition, at time t34, the determination processor 65 selects the processor 31B (channel CH1) and generates a selection control signal SEL based on the selection result. The selector 41 selects the image data DB1 related to the channel CH1 based on the selection control signal SEL ( Figure 12 from (C)). Therefore, the line image data included in the image data DB1 is provided to the processor 42 (channel CH4) ( Figure 12 from (F)). The processor 42 performs a predetermined process based on the line image data, thereby generating line image data ( Figure 12 from (L)). In addition, this line image data is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC1 related to the channel CH1 based on the selection control signal SEL ( Figure 12 from (H)). Therefore, the line image data included in the image data DC1 is provided to the comparison unit 44 ( Figure 12 from (K)).

[0128] The comparison unit 44 mutually compares the line image data ( Figure 12 (L) of) provided from the processor 42 with the line image data ( Figure 12 (K) of) provided from the selector 43. In this example, the processor 31B (channel CH1) fails after the time t33. Therefore, the line image data provided from the processor 42 and the line image data provided from the selector 43 are inconsistent with each other. The determination processor 65 supplies the determination data INF3 including the comparison result to the determination result collection unit 26. Therefore, the determination result collection unit 26 changes the error signal ERR from low level to high level ( Figure 12 (M) of).

[0129] In addition, in the period from the time t34 to the time t35, the determination processor 65 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed based on the result of performing a determination process once for each of the four processors 31A to 31D. In this example, only when the processor 31B (channel CH1) is selected, the comparison result shows "inconsistent", and when the processor 31C (channel CH2) is selected and when the processors 31D (channel CH3) and 31A (channel CH0) are selected, the comparison result shows "consistent". Therefore, the determination processor 65 determines that the processor 31B has failed. In addition, the determination processor 65 supplies the determination data INF3 including the determination result to the determination result collection unit 26.

[0130] In addition, after the time t35, the determination processor 65 selects the failed processor 31B (channel CH1) and generates a selection control signal SEL based on the selection result. In addition, the determination processor 65 generates a control signal CTL that gives an instruction to replace the image data DC generated by the failed processor 31B (channel CH1) with the image data DS generated by the processor 42.

[0131] Figure 14The operation states of the image processing unit 50 and the determination unit 60 are shown. The thick lines indicate the paths of the image data. After time t35, the selector 41 selects the image data DB1 input to the processor 31B (channel CH1) based on the selection control signal SEL, and supplies the image data DB1 to the processor 42 (channel CH4). The processor 42 performs image processing based on the image data DB1, thereby generating image data DS (image data DC1). The selector 52B of the replacement unit 51 selects the image data DS based on the control signal CTL, and outputs the image data DS as image data DD1. In this way, the replacement unit 51 replaces the image data DC1 in the four pieces of image data DC0 to DC3 generated by the four processors 31A to 31D with the image data DS (image data DC1) generated by the processor 42 based on the control signal CTL, thereby generating four pieces of image data DD0 to DD3.

[0132] In this example, even when the processor 31B (channel CH1) fails, the image processing unit 50 outputs the desired image data after time t35. Therefore, the imaging device 1 transmits the desired transmission data to the host device 100 (not shown). In addition, the determination processor 65 notifies the determination result collection unit 26 that the processor 31B (channel CH1) has failed, and the determination result collection unit 26 activates the error signal ERR.

[0133] Figure 15 An operation example of the image processing unit 50 and the determination unit 60 in the case where the processor 42 (channel CH4) of the determination unit 60 fails is shown. In Figure 15 (L) thereof, the portion W4 represents a portion of the image data that is different from the desired image data. In this example, the processor 42 (channel CH4) fails after time t36. Therefore, after time t36, the image data DS generated by the processor 42 is different from the desired image data ( Figure 15 (L) thereof).

[0134] The determination unit 60 cyclically selects the four processors 31A to 31D, thereby repeating the determination process for the four processors 31A to 31D. At time t36, the determination processor 65 selects the processor 31D (channel CH3), and generates a selection control signal SEL based on the selection result. The selector 41 selects the image data DB3 related to the channel CH3 based on the selection control signal SEL ( Figure 15 (E) thereof). Therefore, the line image data included in the image data DB3 is supplied to the processor 42 (channel CH4) ( Figure 15 (F) thereof). The processor 42 performs a predetermined process based on the line image data, thereby generating line image data ( Figure 15(L) of). In addition, the line image data is provided to the comparison unit 44. On the other hand, the selector 43 selects the image data DC3 related to the channel CH3 based on the selection control signal SEL ( Figure 15 (J) of). Therefore, the line image data included in the image data DC3 is provided to the comparison unit 44 ( Figure 15 (K) of).

[0135] The comparison unit 44 mutually compares the line image data provided from the processor 42 ( Figure 15 (L) of) with the line image data provided from the selector 43 ( Figure 15 (K) of). In this example, the processor 42 (channel CH4) of the determination unit 60 fails after the time t36. Therefore, the line image data provided from the processor 42 and the line image data provided from the selector 43 are inconsistent with each other. The determination processor 65 provides the determination data INF3 including the comparison result to the determination result collection unit 26. Therefore, the determination result collection unit 26 changes the error signal ERR from low level to high level ( Figure 15 (M) of).

[0136] Then, the determination processor 65 continues to perform the determination process on the four processors 31A to 31D. In addition, in the period from the time t36 to the time t37, the determination processor 65 determines which of the processors 31A to 31D and the processor 42 (channels CH0 to CH4) has failed based on the result of performing the determination process on each of the four processors 31A to 31D once. In this example, when any one of the processors 31A to 31D (channels CH0 to CH3) is selected, the comparison result shows "inconsistent". Therefore, the determination processor 65 determines that the processor 42 of the determination unit 60 has failed. In addition, the determination processor 65 provides the determination data INF3 including the determination result to the determination result collection unit 26.

[0137] In this example, since the processor 42 (channel CH4) has failed and the processors 31A to 31D (channels CH0 to CH3) have not failed, the image processing unit 50 outputs the desired image data. That is, as Figure 13As shown, the processor 31A (channel CH0) performs image processing based on the image data DB0, thereby generating image data DC0. The selector 52A selects the image data DC0 based on the control signal CTLA, and outputs the image data DC0 as image data DD0. This similarly applies to the processors 31B to 31D (channels CH1 to CH3) and the selectors 52B to 52D. Accordingly, the imaging device 2 transmits the desired transmission data to the host device 100 (not shown). In addition, the determination processor 65 notifies the determination result collection unit 26 that a failure has occurred in the processor 42 (channel CH4), and the determination result collection unit 26 activates the error signal ERR.

[0138] As described above, the imaging device 2 is provided with a replacement unit 51. The replacement unit 51 replaces one of the four pieces of image data DC0 to DC3 generated by the four processors 31A to 31D with the image data DS generated by the processor 42 based on the comparison result obtained by the comparison unit 44. Therefore, in the imaging device 2, when any one of the processors 31A to 31D (channels CH0 to CH3) fails, the processor 42 generates the image data DS based on the image data DB provided to the failed processor 31, and the replacement unit 51 replaces the image data DC output from the failed processor 31 with the image data DS. Thereby, the desired image data can be output. Therefore, in the imaging device 2, even when a failure occurs in the processor 31, the desired image data can be output. Thus, a so-called fail operational configuration can be achieved.

[0139] As described above, according to the present embodiment, based on the comparison result obtained by the comparison unit, one of the four pieces of image data generated by the four processors 31 is replaced with the image data generated by the processor 42. Therefore, even in the case of a failure, the desired image data can be output.

[0140] [Modification Example 2-1]

[0141] According to the above-described embodiment, the determination process is performed based on the actual image data obtained by the imaging operation. However, the determination process can also be performed based on the test data TS. The imaging device 2A according to this modification example will be described in detail below. Similar to the imaging device 2 ( Figure 1 ) according to the above-described second embodiment, the imaging device 2A includes an image processing unit 50A and a determination unit 60A.

[0142] Figure 16 A configuration example of the image processing unit 50A and the determination unit 60A is shown. The image data DTD generated by the image processing unit 50A includes 4 pieces of image data DD (image data DD0 to DD3).

[0143] The image processing unit 50A includes a replacement unit 51A. Similar to the replacement unit 51 according to the second embodiment described above, the replacement unit 51A is configured to replace one of the four pieces of image data DC0 to DC3 generated by the four processors 31A to 31D with the image data DS generated by the processor 42 based on the control signal CTL, thereby generating four pieces of image data DD0 to DD3. In addition, the replacement unit 51A outputs each piece of test data TS generated by a test data generator 62A (described later) as each of the four pieces of image data DD0 to DD3, for example, during the vertical blanking period.

[0144] The determination unit 60A includes a test data generator 62A, a selector 63A, a comparison unit 67A, a comparison unit 68A, an OR circuit 69A, and a determination processor 65A.

[0145] The test data generator 62A is configured to generate test data TS having a predetermined signal pattern, for example, during the vertical blanking period, based on an instruction given from the determination processor 65A.

[0146] The selector 63A is configured to select one of the image data DS provided from the processor 42 (channel CH4) and the test data TS provided from the test data generator 62A based on the selection control signal SEL2, and supply the selected data to the comparison unit 44 and the replacement unit 51A. Specifically, in this example, the selector 63A is configured to select the image data DS provided from the processor 42 (channel CH4) during the period when the image data DTB is provided from the sensor interface 21, and select the test data TS provided from the test data generator 62A during the vertical blanking period.

[0147] The comparison unit 67A is configured to perform a comparison operation of comparing the image data DD0 and the image data DD2 with each other, for example, during the vertical blanking period, based on the control signal CTL2.

[0148] The comparison unit 68A is configured to perform a comparison operation of comparing the image data DD1 and the image data DD3 with each other, for example, during the vertical blanking period, based on the control signal CTL2.

[0149] Alternatively, circuit 69A is configured to determine the logical OR of the comparison results obtained by comparison unit 44, comparison unit 67A, and comparison unit 68A. Here, the comparison result obtained by comparison unit 44 is "1" when "inconsistent" and "0" when "consistent". This similarly applies to the comparison results obtained by comparison unit 67A and comparison unit 68A. That is, OR circuit 69A outputs "0" when all the comparison results obtained by comparison unit 44, comparison unit 67A, and comparison unit 68A show "consistent", and outputs "1" when any one or more of the comparison results obtained by comparison unit 44, comparison unit 67A, and comparison unit 68A show "inconsistent".

[0150] Determination processor 65A is configured to control the determination process for image processing unit 50A. Determination processor 65A controls the determination process for replacement unit 51A, for example, during the vertical blanking period. Specifically, determination processor 65A controls the operation of test data generator 62A such that test data generator 62A generates test data TS during the vertical blanking period. In addition, determination processor 65A generates selection control signal SEL2 during the vertical blanking period, which gives an instruction to select test data TS generated by test data generator 62A. Further, determination processor 65A generates control signal CTL during the vertical blanking period, which gives an instruction to replacement unit 51A to output test data TS as each of the four image data DD0 to DD3. Determination processor 65A generates control signal CTL2 during the vertical blanking period, which gives an instruction for comparison units 67A and 68A to perform a comparison operation. Further, determination processor 65A is configured to detect whether replacement unit 51A has failed based on the output signal from OR circuit 69A during the vertical blanking period.

[0151] Here, test data generator 62A corresponds to a specific example of the "signal generator" of the present disclosure. Comparison units 67A and 68A correspond to specific examples of the "second comparison unit" of the present disclosure.

[0152] Figure 17 An operation example of image processing unit 50A and determination unit 60A is shown.

[0153] Frame period F starts at time t41, and during the period from time t42 to time t43, image processing unit 50A generates image data DD0 to DD3 ( Figure 17 in (B) to (E), (G) to (J), and (N) to (Q) of

[0154] In addition, in the period from time t43 to time t44 (vertical blanking period), the test data generator 62A generates test data TS, and the selector 63A supplies the test data TS to the replacement unit 51. The replacement unit 51A outputs the test data TS as each of the four pieces of image data DD0 to DD3 ( Figure 17 from (N) to (Q)).

[0155] In the period from time t43 to time t44, the comparison unit 67A mutually compares the image data DD0 and the image data DD2, and the comparison unit 68A mutually compares the image data DD1 and the image data DD3. When the replacement unit 51A does not malfunction, it is expected that the image data DD0 and the image data DD2 are the same as each other, and it is expected that the image data DD1 and the image data DD3 are the same as each other. In this case, the OR circuit 69A outputs "0". Therefore, the determination result collection unit 26 maintains the error signal ERR at a low level ( Figure 17 from (M)).

[0156] When the replacement unit 51A malfunctions, for example, the image data DD0 and the image data DD2 are inconsistent with each other, or the image data DD1 and the image data DD3 are inconsistent with each other. In this case, the OR circuit 69A outputs "1". Therefore, the determination processor 65A determines that the replacement unit 51A has malfunctioned. In addition, the determination processor 65A supplies determination data INF3 including the comparison result to the determination result collection unit 26.

[0157] In the imaging device 2A, a test data generator 62A, a comparison unit 67A, and a comparison unit 68A are provided. For example, in the vertical blanking period, the replacement unit 51A outputs the test data TS generated by the test data generator 62A and having a predetermined signal pattern as each of the pieces of image data DD0 to DD3. In addition, the comparison unit 67A mutually compares the image data DD0 with the image data DD2, and the comparison unit 68A mutually compares the image data DD1 with the image data DD3. Therefore, a malfunction of the replacement unit 51A in the imaging device 2A can be detected.

[0158] Note that in this example, the comparison unit 67A mutually compares the image data DD0 and the image data DD2, and the comparison unit 68A mutually compares the image data DD1 and the image data DD3. However, this is not restrictive. Alternatively, for example, the comparison unit 67A may mutually compare the image data DD0 and the image data DD1, and the comparison unit 68A may mutually compare the image data DD2 and the image data DD3.

[0159] In addition, in this example, two comparison units 67A and 68A are provided. However, this is non-limiting. Alternatively, for example, one comparison unit may be provided, and this comparison unit may compare whether four pieces of image data DD0 to DD3 are identical to each other.

[0160] In addition, in this example, the test data generator 62A generates a single piece of test data TS. However, this is non-limiting. Alternatively, for example, the test data generator may generate two pieces of test data TS (test data TS1 and test data TS2) that are identical to each other, and the replacement unit 51A may output test data TS1 as each of the image data DD0 and the image data DD2, and may output test data TS2 as each of the image data DD1 and the image data DD3.

[0161] [Modification Example 2-2]

[0162] Each modification example according to the above first embodiment can be applied to the imaging device 2 according to the above embodiment.

[0163] [Other Modification Examples]

[0164] In addition, two or more of these modification examples can be combined.

[0165] [Usage Examples of Imaging Device]<3. Usage Examples of Imaging Device>

[0166] Figure 18 Usage examples of the imaging device 1 and the imaging device 2 according to the above embodiment are shown. The above imaging device 1 can be used, for example, to sense various situations including light such as visible light, infrared light, ultraviolet light, X-rays, etc.

[0167] · A device for taking images for appreciation, examples of which include digital cameras and mobile devices having a camera function.

[0168] · A device set for traffic use, examples of which include: in-vehicle sensors that take images of the front area, rear area, surrounding area, interior area, etc. of an automobile for safe driving such as automatic parking or for identifying the driver's state; surveillance cameras that monitor moving vehicles, roads, etc.; and ranging sensors that measure the distance between vehicles.

[0169] · A device for taking images of a user's gesture in household appliances such as a television, refrigerator, or air conditioner and operating the device based on the gesture.

[0170] · A device for healthcare use, examples of which include: an endoscope, a device that takes an image of blood vessels by receiving infrared light.

[0171] · Devices for security purposes, examples of which include: surveillance cameras for security purposes and cameras for authentication purposes.

[0172] · Devices for beauty purposes, examples of which include: skin measurement devices for taking skin images and microscopes for taking scalp images.

[0173] · Devices for sports purposes, examples of which include: action cameras for sports purposes and wearable cameras.

[0174] · Devices for agricultural purposes, examples of which include: cameras for monitoring fields and crop conditions.

[0175] <4. Application Examples of Moving Bodies>

[0176] The technology according to the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure can be implemented as a device installed on any type of moving body such as an automobile, an electric vehicle, a hybrid vehicle, a motorcycle, a bicycle, a personal mobility device, an aircraft, a drone, a ship, or a robot.

[0177] Figure 19 is a block diagram showing an example of the schematic configuration of a vehicle control system as an example of a moving body control system to which the technology of the embodiments according to the present disclosure can be applied.

[0178] The vehicle control system 12000 includes a plurality of electronic control units connected to each other via a communication network 12001. In Figure 19 the example shown, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050. In addition, a microcomputer 12051, a sound and image output unit 12052, and a vehicle-mounted network interface (I / F) 12053 are shown as the functional configurations of the integrated control unit 12050.

[0179] The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 serves as a control device for the following devices: a driving force generation device such as an internal combustion engine or a drive motor for generating the driving force of the vehicle; a driving force transmission mechanism for transmitting the driving force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; and a braking device for generating the braking force of the vehicle.

[0180] The vehicle body system control unit 12020 controls the operations of various devices provided to the vehicle body according to various programs. For example, the vehicle body system control unit 12020 serves as a control device for a keyless entry system, a smart key system, an electric window device, or various lights such as a headlight, a taillight, a brake light, a turn signal light, or a fog light. In this case, radio waves transmitted from a portable device as an alternative to a key or signals of various switches can be input to the vehicle body system control unit 12020. The vehicle body system control unit 12020 receives these input radio waves or signals and controls the vehicle door lock device, the electric window device, the lights, etc.

[0181] The vehicle exterior information detection unit 12030 detects information about the exterior of the vehicle having the vehicle control system 12000. For example, the vehicle exterior information detection unit 12030 is connected to the imaging unit 12031. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to image an image of the vehicle exterior and receives the captured image. Based on the received image, the vehicle exterior information detection unit 12030 can perform a detection process on objects such as a person, a vehicle, an obstacle, a marker, or a symbol on the road surface, or a detection process of the distance to these objects.

[0182] The imaging unit 12031 is an optical sensor that receives light and outputs an electric signal corresponding to the amount of light of the received light. The imaging unit 12031 can output the electric signal as an image, or can output the electric signal as information about the measured distance. In addition, the light received by the imaging unit 12031 can be visible light, or can be invisible light such as infrared light.

[0183] The vehicle interior information detection unit 12040 detects information about the interior of the vehicle. For example, the vehicle interior information detection unit 12040 is connected to the driver state detection unit 12041 that detects the state of the driver. The driver state detection unit 12041 includes, for example, a camera that images the driver. Based on the detection information input from the driver state detection unit 12041, the vehicle interior information detection unit 12040 can calculate the degree of fatigue or the degree of concentration of the driver, or can determine whether the driver is dozing off.

[0184] The microcomputer 12051 can calculate control target values for the driving force generation device, steering mechanism, or braking device based on information about the inside or outside of the vehicle obtained by the out-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an advanced driver assistance system (ADAS: advanced driver assistance system), which includes: collision avoidance or impact mitigation of the vehicle, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, or vehicle lane departure warning, etc.

[0185] In addition, the microcomputer 12051 can perform cooperative control for autonomous driving, which makes the vehicle drive autonomously without relying on the driver's operation, etc. by controlling the driving force generation device, steering mechanism, braking device, etc. based on information about the inside or outside of the vehicle obtained by the out-vehicle information detection unit 12030 or the in-vehicle information detection unit 12040.

[0186] In addition, the microcomputer 12051 can output a control command to the body system control unit 12020 based on information about the outside of the vehicle obtained by the out-vehicle information detection unit 12030. For example, the microcomputer 12051 can perform cooperative control aimed at preventing glare by controlling the headlight to change from high beam to low beam according to the positions of the vehicle ahead or the oncoming vehicle detected by the out-vehicle information detection unit 12030.

[0187] The sound and image output unit 12052 transmits an output signal of at least one of sound and image to an output device that can notify information visually or auditorily to the passengers of the vehicle or the outside of the vehicle. In Figure 19 the example, the audio speaker 12061, the display unit 12062, and the instrument panel 12063 are shown as output devices. For example, the display unit 12062 can include at least one of an in-vehicle display and a head-up display.

[0188] Figure 20 is a diagram showing an example of the installation position of the imaging unit 12031.

[0189] In Figure 20 it, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0190] The imaging units 12101, 12102, 12103, 12104, and 12105 are, for example, arranged at positions on the front nose, side mirrors, rear bumper, and rear door of the vehicle 12100 and at a position on the upper part of the windshield inside the vehicle. The imaging unit 12101 arranged on the front nose and the imaging unit 12105 arranged on the upper part of the windshield inside the vehicle mainly obtain images of the front of the vehicle 12100. The imaging units 12102 and 12103 arranged on the side mirrors mainly obtain images of the sides of the vehicle 12100. The imaging unit 12104 arranged on the rear bumper or rear door mainly obtains images of the rear of the vehicle 12100. The imaging unit 12105 arranged on the upper part of the windshield inside the vehicle is mainly used to detect a vehicle ahead, a pedestrian, an obstacle, a signal, a traffic sign, a lane, or the like.

[0191] Incidentally, Figure 20 An example of the imaging ranges of the imaging units 12101 to 12104 is shown. The imaging range 12111 represents the imaging range of the imaging unit 12101 arranged on the front nose. The imaging ranges 12112 and 12113 represent the imaging ranges of the imaging units 12102 and 12103 arranged on the side mirrors, respectively. The imaging range 12114 represents the imaging range of the imaging unit 12104 arranged on the rear bumper or rear door. For example, by superimposing the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above is obtained.

[0192] At least one of the imaging units 12101 to 12104 may have a function of obtaining distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera composed of a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

[0193] For example, the microcomputer 12051 can determine the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the temporal change in the distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, and thus extract the nearest three-dimensional object as the vehicle ahead, particularly, the three-dimensional object that exists on the driving path of the vehicle 12100 and travels in substantially the same direction as the vehicle 12100 at a predetermined speed (e.g., equal to or greater than 0 km / h). In addition, the microcomputer 12051 can preset the inter-vehicle distance to be maintained in front of the vehicle ahead and perform automatic braking control (including following-stop control) or automatic acceleration control (including following-start control), etc. Therefore, cooperative control for autonomous driving can be executed, which enables the vehicle to autonomously travel without relying on the driver's operation or the like.

[0194] For example, the microcomputer 12051 can classify the three-dimensional object data of a three-dimensional object into the three-dimensional object data of a two-wheeled vehicle, a standard-sized vehicle, a large vehicle, a pedestrian, a utility pole, and other three-dimensional objects based on the distance information obtained from the imaging units 12101 to 12104, extract the classified three-dimensional object data, and use the extracted three-dimensional object data to automatically avoid obstacles. For example, the microcomputer 12051 identifies the obstacles around the vehicle 12100 as the obstacles that the driver of the vehicle 12100 can visually identify and the obstacles that the driver of the vehicle 12100 is difficult to visually identify. Then, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle. When the collision risk is equal to or higher than the set value and thus there is a possibility of collision, the microcomputer 12051 outputs a warning to the driver via the audio speaker 12061 or the display unit 12062, and performs forced deceleration or avoidance steering through the drive system control unit 12010. The microcomputer 12051 can thus assist driving to avoid collision.

[0195] At least one of the imaging units 12101 to 12104 can be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can identify a pedestrian by determining whether there is a pedestrian in the captured images of the imaging units 12101 to 12104. For example, this identification of a pedestrian is performed by a program for extracting feature points in the captured images of the imaging units 12101 to 12104 that are infrared cameras and a program for performing pattern matching processing on a series of feature points representing the object contour to determine whether it is a pedestrian. When the microcomputer 12051 determines that there is a pedestrian in the captured images of the imaging units 12101 to 12104 and thus identifies the pedestrian, the audio-visual output unit 12052 controls the display unit 12062 such that a square contour line for emphasis is displayed in a superimposed manner on the identified pedestrian. The audio-visual output unit 12052 can also control the display unit 12062 such that an icon representing the pedestrian or the like is displayed at a desired position.

[0196] Examples of the vehicle control system to which the technology according to the present disclosure can be applied have been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 in the above configuration. This enables the vehicle control system 12000 to detect a failure of the imaging unit 12031 with high accuracy. Therefore, in the vehicle control system 12000, even when a failure occurs in a vehicle collision avoidance function, a vehicle collision mitigation function, a following driving function based on the inter-vehicle distance, a function of maintaining the vehicle speed, a vehicle collision warning function, a lane departure warning function, etc., the failure can be detected. This makes it possible to improve the robustness of the system.

[0197] The present technology has been described above with reference to embodiments, variations, and specific application examples thereof. However, the present technology is not limited to the embodiments and the like and can be variously modified.

[0198] For example, in each of the above embodiments, four pixels PR, PGr, PGb, and PB are used to set the pixel array 11. However, this is non-limiting. Alternatively, for example, three pixels (a red pixel PR, a green pixel PG, and a blue pixel PB) can be used to set the pixel array.

[0199] Note that the effects described here are merely illustrative and non-limiting, and other effects may be provided.

[0200] Note that the present technology may have the following configuration. According to the present technology having the following configuration, determination can be made during normal operation.

[0201] (1) A signal processing device, comprising:

[0202] A plurality of first processors, which are arranged to be associated with a plurality of first signals, and each of the plurality of first processors is configured to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal;

[0203] A controller, which is configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result;

[0204] A first selection unit, which is configured to select, based on the selection control signal, the first signal among the plurality of first signals that will be provided to the selected first processor;

[0205] A second processor, which is configured to perform a predetermined process based on the first signal selected by the first selection unit among the plurality of first signals, thereby generating a third signal;

[0206] A second selection unit, which is configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and

[0207] A first comparison unit, which is configured to mutually compare the third signal and the second signal selected by the second selection unit among the plurality of second signals.

[0208] (2) The signal processing device according to (1) above, wherein the controller is configured to make a determination on the plurality of first processors and the second processor based on the comparison result obtained by the first comparison unit.

[0209] (3) The signal processing device according to (2) above, wherein the controller is configured to determine that the specific processor has failed in a case where the comparison result obtained by the first comparison unit shows inconsistency when the specific processor among the plurality of first processors is selected and the comparison result obtained by the first comparison unit shows consistency when one or more processors other than the specific processor among the plurality of first processors are sequentially selected.

[0210] (4) The signal processing device according to (1) or (2) above, wherein the controller is configured to determine that the second processor has failed in a case where the comparison result obtained by the first comparison unit shows inconsistency when two or more processors among the plurality of first processors are sequentially selected.

[0211] (5) The signal processing device according to (2) above, further comprising:

[0212] a replacement unit configured to replace one of the plurality of second signals with the third signal based on a control signal, thereby generating a plurality of fourth signals, wherein

[0213] the controller is configured to generate the control signal based on the determination result.

[0214] (6) The signal processing device according to (5) above, wherein the replacement unit includes a plurality of third selection units, the plurality of third selection units are provided in association with the plurality of first processors, and each is configured to select one of the third signal and the second signal output from the associated first processor among the plurality of first processors based on the control signal.

[0215] (7) The signal processing device according to (5) or (6) above, wherein

[0216] the controller is configured to

[0217] select the specific processor among the plurality of first processors in a case where the comparison result obtained by the first comparison unit shows inconsistency when the specific processor among the plurality of first processors is selected and the comparison result obtained by the first comparison unit shows consistency when one or more processors other than the specific processor among the plurality of first processors are sequentially selected,

[0218] generate the selection control signal based on the selection result, and

[0219] generate the control signal that gives an instruction to the replacement unit to replace the second signal generated by the specific processor with the third signal.

[0220] (8) The signal processing device according to any one of (5) to (7) above, wherein the controller is configured to generate a control signal in a case where a comparison result obtained by the first comparison unit shows inconsistency when two or more of the plurality of first processors are sequentially selected, and the control signal gives an instruction to the replacement unit to output the plurality of second signals as the plurality of fourth signals.

[0221] (9) The signal processing device according to any one of (5) to (8) above, further comprising:

[0222] a signal generator configured to generate a fifth signal; and

[0223] a second comparison unit configured to perform a comparison operation based on the plurality of fourth signals, wherein

[0224] each of the plurality of first processors is configured to perform the predetermined process in a first cycle,

[0225] the replacement unit is configured to output the fifth signal as two or more signals among the plurality of fourth signals in a second cycle outside the first cycle, and

[0226] the second comparison unit is configured to compare the two or more signals with each other.

[0227] (10) The signal processing device according to any one of (1) to (9) above, further comprising an output unit configured to output a signal based on a comparison result obtained by the first comparison unit.

[0228] (11) The signal processing device according to any one of (1) to (10) above, wherein the predetermined process includes at least one of a gain adjustment process, a white balance adjustment process, a black level adjustment process, an HDR synthesis process, a noise removal process, and a pixel defect correction process.

[0229] (12) A signal processing method, comprising:

[0230] causing each of a plurality of first processors set to be associated with a plurality of first signals to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal;

[0231] selecting any one of the plurality of first processors and generating a selection control signal based on the selection result;

[0232] selecting, based on the selection control signal, the first signal among the plurality of first signals that will be provided to the selected first processor;

[0233] Cause the second processor to perform the predetermined processing based on the selected first signal among the plurality of first signals, thereby generating a third signal;

[0234] Select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and

[0235] Mutually compare the third signal and the selected second signal among the plurality of second signals.

[0236] (13) An imaging device, comprising:

[0237] An imager configured to perform an imaging operation to generate an image signal including a plurality of first signals;

[0238] A plurality of first processors provided to be associated with the plurality of first signals, each of the plurality of first processors being configured to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal;

[0239] A controller configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result;

[0240] A first selection unit configured to select, based on the selection control signal, the first signal among the plurality of first signals to be provided to the selected first processor;

[0241] A second processor configured to perform a predetermined process based on the first signal selected by the first selection unit among the plurality of first signals, thereby generating a third signal;

[0242] A second selection unit configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and

[0243] A first comparison unit configured to mutually compare the third signal and the second signal selected by the second selection unit among the plurality of second signals.

[0244] This application claims priority based on Japanese Patent Application No. 2019-146248 filed with the Japan Patent Office on August 8, 2019, the entire content of which is incorporated herein by reference.

[0245] Those skilled in the art should understand that various deformations, combinations, sub - combinations and changes can occur within the scope of the appended claims or their equivalents according to design requirements and other factors.

Claims

1. A signal processing device, comprising: a plurality of first processors, which are arranged to be associated with a plurality of first signals, and each of the plurality of first processors is configured to perform a predetermined process based on the associated first signal among the plurality of first signals, so as to generate a second signal; a controller, which is configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result; a first selection unit, which is configured to select, based on the selection control signal, the first signal among the plurality of first signals that will be provided to the selected first processor; a second processor, which is configured to perform a predetermined process based on the first signal selected by the first selection unit among the plurality of first signals, so as to generate a third signal; a second selection unit, which is configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and a first comparison unit, which is configured to mutually compare the third signal and the second signal selected by the second selection unit among the plurality of second signals.

2. The signal processing device according to claim 1, wherein the controller is configured to determine the plurality of first processors and the second processor based on the comparison result obtained by the first comparison unit.

3. The signal processing device according to claim 2, wherein the controller is configured to determine that the specific processor has failed in a case where the comparison result obtained by the first comparison unit shows inconsistency when the specific processor among the plurality of first processors is selected and the comparison result obtained by the first comparison unit shows consistency when one or more processors other than the specific processor among the plurality of first processors are sequentially selected.

4. The signal processing device according to claim 2, wherein the controller is configured to determine that the second processor has failed in a case where the comparison result obtained by the first comparison unit shows inconsistency when two or more processors among the plurality of first processors are sequentially selected.

5. The signal processing device according to claim 2, further comprising: a replacement unit, which is configured to replace one of the plurality of second signals with the third signal based on a control signal, so as to generate a plurality of fourth signals, wherein the controller is configured to generate the control signal based on the determination result.

6. The signal processing device according to claim 5, wherein the replacement unit includes a plurality of third selection units, the plurality of third selection units are arranged to be associated with the plurality of first processors, and each of the plurality of third selection units is configured to select, based on the control signal, one of the third signal and the second signal output from the associated first processor among the plurality of first processors.

7. The signal processing device according to claim 5, wherein the controller is configured to In a case where a comparison result obtained by the first comparison unit shows inconsistency when a specific processor among the plurality of first processors is selected and the comparison results obtained by the first comparison unit show consistency when one or more processors other than the specific processor among the plurality of first processors are sequentially selected, the specific processor among the plurality of first processors is selected, a selection control signal is generated based on the selection result, and a control signal is generated, the control signal giving an instruction that the replacement unit will replace the second signal generated by the specific processor with the third signal.

8. The signal processing apparatus according to claim 5, wherein the controller is configured to generate a control signal in a case where a comparison result obtained by the first comparison unit shows inconsistency when two or more processors among the plurality of first processors are sequentially selected, the control signal giving an instruction that the replacement unit will output the plurality of second signals as the plurality of fourth signals.

9. The signal processing apparatus according to claim 5, further comprises: a signal generator configured to generate a fifth signal; and a second comparison unit configured to perform a comparison operation based on the plurality of fourth signals, wherein each of the plurality of first processors is configured to perform the predetermined process in a first cycle, the replacement unit is configured to output the fifth signal as two or more signals among the plurality of fourth signals in a second cycle outside the first cycle, and the second comparison unit is configured to mutually compare the two or more signals.

10. The signal processing apparatus according to any one of claims 1-9, further comprising an output unit configured to output a signal based on a comparison result obtained by the first comparison unit.

11. The signal processing apparatus according to any one of claims 1-9, wherein the predetermined process includes at least one of a gain adjustment process, a white balance adjustment process, a black level adjustment process, an HDR synthesis process, a noise removal process, and a pixel defect correction process.

12. A signal processing method, comprises: causing each of a plurality of first processors set to be associated with a plurality of first signals to perform a predetermined process based on the associated first signal among the plurality of first signals, thereby generating a second signal; selecting any one of the plurality of first processors and generating a selection control signal based on the selection result; selecting, based on the selection control signal, the first signal among the plurality of first signals that will be provided to the selected first processor; causing a second processor to perform the predetermined process based on the selected first signal among the plurality of first signals, thereby generating a third signal; selecting, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; and mutually comparing the third signal and the selected second signal among the plurality of second signals.

13. An imaging apparatus, comprises: an imager configured to perform an imaging operation to generate an image signal including a plurality of first signals; A plurality of first processors set to be associated with the plurality of first signals, each of the plurality of first processors being configured to perform a predetermined process based on the associated first signal among the plurality of first signals to generate a second signal; A controller configured to select any one of the plurality of first processors and generate a selection control signal based on the selection result; A first selection unit configured to select, based on the selection control signal, the first signal among the plurality of first signals to be provided to the selected first processor; A second processor configured to perform a predetermined process based on the first signal selected by the first selection unit among the plurality of first signals to generate a third signal; A second selection unit configured to select, based on the selection control signal, the second signal generated by the selected first processor among the plurality of second signals; And A first comparison unit configured to mutually compare the third signal and the second signal selected by the second selection unit among the plurality of second signals.

Images