Signal processing system

Abstract

A signal processing system according to the present disclosure comprises: an imaging device that has a plurality of pixels and outputs image data corresponding to subject light which is incident on the plurality of pixels; and a host that performs processing corresponding to the image data. One of the imaging device and the host detects for defective pixels among the plurality of pixels. The host makes a failure prediction expressing a sign that a failure will occur in the imaging device on the basis of the defective pixel detection result.

Description

Signal processing system 【0001】 The present disclosure relates to a signal processing system including an imaging device. 【0002】 For example, imaging devices and distance measuring devices are used in systems for automatic driving or driving support of vehicles. In such a system, if an abnormality (failure) occurs in the imaging device or the distance measuring device, there is a possibility that a problem may occur in the entire system. Therefore, it is desirable to detect an abnormality in the imaging device or the distance measuring device. In Patent Document 1, a technique for estimating an abnormality in a distance measuring device has been proposed. In Patent Documents 2 and 3, techniques for detecting an abnormality in a captured image of an imaging device have been proposed. 【0003】 Japanese Unexamined Patent Application Publication No. 2019-144186, Japanese Unexamined Patent Application Publication No. 9-113221, Japanese Unexamined Patent Application Publication No. 2006-229341 【0004】 It is considered that there is some sign before a failure occurs in the imaging device. In a system equipped with an imaging device, it is desirable to detect a sign before a failure occurs in the imaging device and predict the occurrence of a failure in the imaging device in order to prevent the occurrence of a failure caused by the imaging device. 【0005】 Therefore, it is desirable to provide a signal processing system capable of predicting a failure of an imaging device. 【0006】 A signal processing system according to an embodiment of the present disclosure includes an imaging device having a plurality of pixels and outputting image data corresponding to subject light incident on the plurality of pixels, and a host that performs processing corresponding to the image data. In either the imaging device or the host, defective pixels generated in the plurality of pixels are detected, and the host performs a failure prediction indicating a sign of the occurrence of a failure of the imaging device based on the detection result of the defective pixels. 【0007】 In the signal processing system according to an embodiment of the present disclosure, defective pixels generated in the plurality of pixels are detected in either the imaging device or the host. The host performs a failure prediction indicating a sign of the occurrence of a failure of the imaging device based on the detection result of the defective pixels. 【0008】Figure 1 is a schematic block diagram showing an example configuration of a signal processing system according to one embodiment of the present disclosure. Figure 2 is a schematic block diagram showing an example configuration of an imaging device in a signal processing system according to one embodiment. Figure 3 is a schematic block diagram showing an example configuration of a data processing unit in the imaging device shown in Figure 2. Figure 4 is an explanatory diagram showing an example of a late-onset defective pixel generation model used for fault prediction determination. Figure 5 is a flowchart showing an example of processing operations related to fault prediction by a signal processing system according to one embodiment. Figure 6 is a schematic block diagram showing an example configuration of a signal processing system according to Modification 1. Figure 7 is a flowchart showing an example of processing operations related to fault prediction by a signal processing system according to Modification 1. Figure 8 is a block diagram showing an example of a schematic configuration of a vehicle control system. Figure 9 is an explanatory diagram showing an example of the installation positions of an external information detection unit and an imaging unit. 【0009】 The embodiments of this disclosure will be described in detail below with reference to the drawings. The description will be in the following order: 1. One Embodiment 1.0 Overview 1.1 Configuration 1.2 Operation 1.3 Modifications 1.4 Effects 2. Application Examples to Mobile Devices 3. Other Embodiments 【0010】 <1. One Embodiment> [1.0 Overview] The signal processing system according to one embodiment of the present disclosure can be applied, for example, to a system that performs autonomous driving (AD) or driver assistance (ADAS (Advanced Driver Assistance System)) of a vehicle using an imaging device. ISO (International Organization for Standardization) 26262 is a functional safety standard formulated for the electrical and electronic systems of vehicles. ISO 26262 3rd Edition adds “Predictive Maintenance,” or “fault prediction,” and requires a higher level of safety. For example, it requires the prevention of failure of function through predictive maintenance and maintenance at the appropriate timing by detecting deterioration behavior that does not depend on human intervention. 【0011】Predicting random failures in imaging devices (image sensors) before manufacturing and shipment is difficult, so the development of technology to predict failures during startup after shipment is desirable. There should be some kind of warning sign before a failure occurs. On the other hand, it is known that after shipment, image sensors can be damaged by radiation, resulting in white spot defects. The number of defects depends on the environment and time, but white spot occurrence prediction data exists and can be used. Furthermore, by implementing a defect detection circuit inside the image sensor, it is possible to predict failures due to the increase in later defective pixels. If an unexpected number of defects occur after the image sensor has been shipped, it will lead to a deterioration in the performance of AD and ADAS using that image sensor. If the number of defects increases further, it may lead to misrecognition or failure of the system. This situation can be avoided by regularly checking whether the number of defects in the image sensor remains within the expected range. 【0012】 In one embodiment of the signal processing system, a predetermined threshold is set in advance based on a defect increase model, and it is checked whether the number of defective pixels detected by, for example, a defect detection circuit in the image sensor is below the predetermined threshold. If the number of defective pixels exceeds the predetermined threshold, it indicates that the image sensor is deteriorating faster than expected. In this case, it can be assumed that there is a high probability of causing a system malfunction early on, so notifying the user in advance can help avoid a major accident. 【0013】 [1.1 Configuration] Figure 1 is a schematic block diagram showing an example of the configuration of a signal processing system according to one embodiment of the present disclosure. 【0014】 One embodiment of the signal processing system comprises an imaging device 1 and a host 2 that performs processing according to the image data obtained by the imaging device 1. 【0015】Figure 2 is a schematic block diagram showing one example configuration of the imaging device 1. The imaging device 1 has multiple pixels and outputs image data corresponding to the subject light incident on the multiple pixels. The imaging device 1 may be, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor (CIS). 【0016】 The imaging device 1 includes a pixel array 10, an address generation circuit 11H, an address generation circuit 11V, an ADC (Analog to Digital Converter) 12, a line buffer 13, a data processing unit 14, a data output unit 15, and a logic controller 16. 【0017】 The pixel array 10 has a plurality of pixels arranged in a matrix. The pixel array 10 generates and outputs analog pixel signals corresponding to the subject light incident on the plurality of pixels. The address generation circuit 11H and the address generation circuit 11V generate the addresses of the pixels that output pixel signals from among the plurality of pixels. 【0018】 The ADC 12 converts the analog pixel signals output from the pixel array 10 into digital pixel signals. The digital pixel signals are output to the data processing unit 14 via the line buffer 13. 【0019】 The data processing unit 14 is composed of circuits that perform various signal processing operations on digital pixel signals. 【0020】 The data output unit 15 converts the pixel signals, after various signal processing by the data processing unit 14, into image data in a predetermined format and outputs it. The data output unit 15 outputs the image data to the host 2, for example, via the MIPI (Mobile Industry Processor Interface) interface. 【0021】 The logic controller 16 is configured to control each part of the imaging device 1. ２ It may be possible to communicate with host 2 via C (Inter Integrated Circuit) communication. 【0022】Figure 3 is a schematic block diagram showing one example configuration of the data processing unit 14 in the imaging device 1. 【0023】 The data processing unit 14 includes a clamping circuit 30, a defective pixel detection circuit 31, a noise reduction circuit 32, and a shading correction circuit 33. 【0024】 The clamp circuit 30 is a circuit that clamps the pixel signal. The noise reduction circuit 32 is a circuit that removes noise from the pixel signal. The shading correction circuit 33 is a circuit that performs shading correction on the pixel signal. 【0025】 The defective pixel detection circuit 31 is a circuit that detects defective pixels that have occurred in multiple pixels based on the pixel signal. The method for detecting defective pixels based on the pixel signal is not particularly limited, and general techniques can be used (see, for example, Japanese Patent Application Publication No. 9-227185). 【0026】 The imaging device 1 includes information about defective pixels detected by the defective pixel detection circuit 31 (for example, the number of defective pixels detected) in the image data, for example, as embedded data, and outputs it to the host 2 via the MIPI interface. The imaging device 1 also outputs the information about the defective pixels to the host 2 via the MIPI interface. ２ The output may also be sent to host 2 via C communication. 【0027】 Returning to Figure 1, let's explain the configuration of Host 2. Host 2 may consist of, for example, an ECU (Engine Control Unit) used in a system for autonomous driving or driver assistance of a vehicle. 【0028】 Host 2 performs fault prediction, indicating signs of impending failure of the imaging device 1, based on the detection results of defective pixels. Host 2 includes an image processing unit 20, a detection count holding unit 21, and a prediction determination unit 22. 【0029】 The image processing unit 20 performs processing according to the image data from the imaging device 1. 【0030】 The detection count retention unit 21 retains the number of defective pixels detected (Cnt) along with the time information of detection, as information about the defective pixels detected by the defective pixel detection circuit 31. 【0031】 The prediction and determination unit 22 performs fault prediction based on information about defective pixels. The prediction and determination unit 22 performs fault prediction of the imaging device 1 when the number of detected defective pixels exceeds a predetermined threshold. The prediction and determination unit 22 also notifies an external party of the fault prediction. External notification may be, for example, by display on a display unit (not shown) or by sound from an audio output unit (not shown). Thus, the prediction and determination unit 22 may notify the user of the fault prediction via a display unit or audio output unit. 【0032】 Figure 4 is an explanatory diagram showing an example of a late-onset defective pixel generation model used for fault prediction. 【0033】 The predetermined threshold used in the predictive determination unit 22 for fault prediction may be a threshold (Th(t)) based on a later-generation fault pixel generation model that shows the relationship between the elapsed time since the manufacture of the imaging device 1 and the number of faulty pixels, as shown in Figure 4. In Figure 4, the horizontal axis represents the elapsed time since the manufacture of the imaging device 1, and the vertical axis represents the number of faulty pixels (number of detected faulty pixels (Cnt)). In Figure 4, the solid line represents the actual number of faulty pixels detected, and the dashed line represents the threshold (Th(t)) based on the later-generation fault pixel generation model. If the number of detected faulty pixels (Cnt) is greater than the threshold (Th(t)) based on the later-generation fault pixel generation model, or if the total number of faulty pixels obtained by accumulating the number of detected faulty pixels (Cnt) over time is greater than the threshold (Th(t)) based on the later-generation fault pixel generation model, the predictive determination unit 22 predicts a fault in the imaging device 1 and notifies the user to take safety measures. 【0034】 [1.2 Operation] Figure 5 is a flowchart showing an example of the processing operation related to fault prediction by a signal processing system according to one embodiment. 【0035】 In one embodiment of the signal processing system, after the imaging device 1 is started up (step S101), the defective pixel detection circuit 31 detects defective pixels (step S102). Next, the imaging device 1 outputs information about the detected defective pixels (number of detected defective pixels (Cnt)) to the host 2 (step S103). 【0036】Host 2 receives information about defective pixels from imaging device 1 and stores the number of detected defective pixels (Cnt) as information about the received defective pixels, along with information about the time the defective pixels were detected, in the detection count holding unit 21 (step S201). Next, Host 2 uses the prediction determination unit 22 to determine whether the number of detected defective pixels (Cnt) is greater than or equal to a threshold (Th(t)) based on the subsequent defective pixel generation model (step S202). If it is determined that the number of detected defective pixels (Cnt) is within the threshold (Th(t)) based on the subsequent defective pixel generation model (step S202; N), the signal processing system transitions to the System Ready state. If the signal processing system is a system that performs automatic driving or driving assistance for a vehicle, it transitions to the normal driving mode as it enters the System Ready state. 【0037】 On the other hand, if the number of detected defective pixels (Cnt) is determined to be greater than the threshold (Th(t)) based on the model for the occurrence of later defective pixels (step S202; Y), the prediction determination unit 22 of the host 2 determines that a failure of the imaging device 1 is predicted (step S203), and notifies the user of the failure prediction via a display unit or audio output unit (not shown) (step S204). Subsequently, the signal processing system switches to a safe mode, for example, which restricts the operation of some parts of the system. This allows the signal processing system to prompt the user to take safety measures before the imaging device 1 fails. 【0038】 [1.3 Modifications] (Modification 1) Figure 6 is a block diagram schematically showing one example configuration of the signal processing system according to Modification 1. 【0039】In the above description, the defective pixel detection circuit 31 of the imaging device 1 detects defective pixels. However, the host 2 may also detect defective pixels based on the image data. Each pixel of the imaging device 1 changes its pixel value that can be taken for each frame if the imaging device 1 is normal. Therefore, the host 2 may detect defective pixels based on the difference between the pixel value of the current frame image data obtained by the imaging device 1 and the pixel value of the image data of the past frame. For example, by comparing the pixel value of the current frame image data with the pixel value of the image data of the past frame, for pixels whose difference value has been fixed for a certain period of time, they are determined as defective pixels. In this case, by performing pixel value comparison while imaging various scenes, it becomes easier to detect pixels whose values are fixed. 【0040】 In order to detect defective pixels based on the image data on the host 2 side, the host 2 may be provided with a defective pixel detection unit 40. The defective pixel detection unit 40 may include a frame memory 41, a pixel value comparison unit 42, and a time elapse determination unit 43. 【0041】 The frame memory 41 stores at least one frame of image data from the imaging device 1. The pixel value comparison unit 42 compares the pixel value of the current frame image data obtained by the imaging device 1 with the pixel value of the image data of the past frame stored in the frame memory 41. 【0042】 The time elapse determination unit 43 determines whether the difference value between the pixel value of the current frame image data and the pixel value of the image data of the past frame has been fixed for a certain period of time, and for pixels that have been fixed for a certain period of time, they are determined as defective pixels. The time elapse determination unit 43 outputs the number of defective pixels determined as defective pixels as the detection number (Cnt) of defective pixels to the detection number holding unit 21. 【0043】 FIG. 7 is a flowchart showing an example of a processing operation related to failure prediction by the signal processing system according to the first modification. 【0044】In the signal processing system according to the first modification, after the imaging device 1 is activated (step S111), the image data obtained by the imaging device 1 is output to the host 2 (step S112). 【0045】 In the host 2, the image data from the imaging device 1 is stored in the frame memory 41 for at least one frame (step S211). Next, in the host 2, the pixel value comparison unit 42 compares the pixel values of the image data of the current frame obtained by the imaging device 1 with the pixel values of the image data of the past frame stored in the frame memory 41 (step S212). Next, in the host 2, it is determined whether the difference value between the pixel values of the image data of the current frame and the pixel values of the image data of the past frame is fixed for a certain time Fix, and for pixels that are fixed for a certain time Fix, they are determined as defective pixels (step S213). The time elapse determination unit 43 outputs the number of defective pixels determined as defective pixels to the detection number holding unit 21 as the detection number (Cnt) of defective pixels. 【0046】 Next, in the host 2, the detection number holding unit 21 receives the information of the defective pixels, and holds the detection number (Cnt) of the defective pixels as the received information of the defective pixels together with the information of the time when the defective pixels were detected (step S201). Thereafter, the processing of steps S202 to S204 is the same as the processing of steps S202 to S204 shown in FIG. 5. 【0047】 Other configurations and operations may be substantially the same as the configurations and operations of the signal processing system according to the embodiment described using FIGS. 1 to 5 above. 【0048】 [1.4 Effect] As described above, according to the signal processing system according to the embodiment, in either the imaging device 1 or the host 2, defective pixels generated in a plurality of pixels are detected, and the host 2 performs a failure prediction indicating a sign of a failure occurrence of the imaging device 1 based on the detection result of the defective pixels. Thereby, it becomes possible to perform a failure prediction of the imaging device 1. 【0049】According to one embodiment of the signal processing system, when applied to a system for autonomous driving or driver assistance of a vehicle, for example, it becomes possible to create a system that complies with ISO 26262 3rd Edition. Furthermore, when applied to a system for autonomous driving or driver assistance of a vehicle, it becomes possible to prevent vehicle accidents through fault prediction. 【0050】 The effects described herein are merely illustrative and not limiting, and other effects may also exist. The same applies to the effects of other embodiments described later. 【0051】 <2. Examples of Application to Mobile Devices> The technology disclosed herein (the technology) can be applied to various products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots. 【0052】 Figure 8 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied. 【0053】 The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 8, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053. 【0054】The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle. 【0055】 The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc. 【0056】 The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing. 【0057】 The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light. 【0058】The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041. 【0059】 The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning. 【0060】 Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040. 【0061】 Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams. 【0062】The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying the vehicle occupants or those outside the vehicle of information. In the example in Figure 8, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display. 【0063】 Figure 9 shows an example of the installation position of the imaging unit 12031. 【0064】 In Figure 9, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105. 【0065】 The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes. 【0066】Figure 9 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained. 【0067】 At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection. 【0068】 For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, etc., that drives autonomously without driver operation, can be performed. 【0069】For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010. 【0070】 At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position. 【0071】The above describes an example of a vehicle control system to which the technology of this disclosure may be applied. The technology of this disclosure can be applied to the imaging unit 12031 and the microcomputer 12051 in the configuration described above. For example, in a signal processing system according to one embodiment of this disclosure, the imaging device 1 can be applied to the imaging unit 12031 and the host 2 to the microcomputer 12051. This makes it possible for the microcomputer 12051 to, for example, predict failures of the imaging unit 12031. When a failure of the imaging unit 12031 is predicted, the microcomputer 12051 can, for example, notify the user of the failure prediction via the audio speaker 12061 or the display unit 12062. 【0072】 Furthermore, for example, if the vehicle control system 12000 has two redundant imaging units as the imaging unit 12031, the other imaging unit may be activated if a failure prediction is made for one of the imaging units. Also, for example, if the vehicle control system 12000 has a distance measuring unit (for example, a LIDAR device (Light Detection and Ranging) or a TOF (Time Of Flight) image sensor) in addition to the imaging unit 12031 to detect the distance to an object, the distance measuring unit may be activated if a failure prediction is made for the imaging unit 12031. In this case, since at least the distance to the object can be detected, accidents caused by false detections due to failures of the imaging unit 12031 can be prevented. 【0073】 <3. Other Embodiments> The technology described herein is not limited to the above-described embodiment and can be implemented in various modified forms. 【0074】 For example, this technology can also take the following configuration. According to this technology with the following configuration, either the imaging device or the host detects defective pixels that have occurred in multiple pixels, and the host performs fault prediction, indicating signs of failure in the imaging device based on the detection results of the defective pixels. This makes it possible to provide a signal processing system that can predict failures in the imaging device. 【0075】(1) A signal processing system comprising: an imaging device having a plurality of pixels and outputting image data corresponding to subject light incident on the plurality of pixels; and a host that performs processing according to the image data, wherein either the imaging device or the host detects defective pixels that have occurred in the plurality of pixels, and the host performs fault prediction indicating signs of failure of the imaging device based on the detection results of the defective pixels. (2) The signal processing system according to (1) above, wherein the host performs fault prediction of the imaging device when the number of detected defective pixels exceeds a predetermined threshold. (3) The signal processing system according to (2) above, wherein the predetermined threshold is a threshold based on a defective pixel occurrence model that shows the relationship between the elapsed time since the manufacture of the imaging device and the number of defective pixels that have occurred. (4) The signal processing system according to any one of (1) to (3) above, wherein the host notifies the outside of the fault prediction. (5) The signal processing system according to any one of (1) to (4) above, wherein the imaging device has a defective pixel detection circuit. (6) The signal processing system according to any one of (1) to (4) above, wherein the host detects the defective pixels based on the image data. (7) The signal processing system according to (6) above, wherein the host detects the defective pixel based on the difference between the pixel value of the image data of the current frame obtained by the imaging device and the pixel value of the image data of a past frame. 【0076】 This application claims priority based on Japanese Patent Application No. 2024-218964, filed with the Japan Patent Office on 13 December 2024, and all contents of that application are incorporated herein by reference. 【0077】 Those skilled in the art will understand that various modifications, combinations, subcombinations, and changes can be conceived depending on design requirements and other factors, and that these fall within the scope of the attached claims and their equivalents.

Claims

1. A signal processing system comprising: an imaging device having a plurality of pixels and outputting image data corresponding to subject light incident on the plurality of pixels; and a host that performs processing according to the image data, wherein either the imaging device or the host detects a defective pixel that has occurred in the plurality of pixels, and the host performs fault prediction indicating signs of failure in the imaging device based on the detection result of the defective pixel.

2. The signal processing system according to claim 1, wherein the host predicts a failure of the imaging device when the number of detected defective pixels exceeds a predetermined threshold.

3. The signal processing system according to claim 2, wherein the predetermined threshold is a threshold based on a defective pixel generation model that shows the relationship between the elapsed time since the manufacture of the imaging device and the number of defective pixels generated.

4. The signal processing system according to claim 1, wherein the host notifies an external party of the fault prediction.

5. The signal processing system according to claim 1, wherein the imaging device has a defective pixel detection circuit.

6. The signal processing system according to claim 1, wherein the host detects the defective pixel based on the image data.

7. The signal processing system according to claim 6, wherein the host detects the defective pixel based on the difference between the pixel value of the image data of the current frame obtained by the imaging device and the pixel value of the image data of a past frame.

Images