Fault detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a fault detection method and device, a storage medium and electronic equipment. The method comprises the steps that a to-be-detected image device is acquired; at least one statistical node is deployed in the image signal processing module, the at least one statistical node is used for capturing first operation data of a first image signal input by the image signal processing module and the processing sensor module, and the first operation data is used for indicating whether the image device has a fault or not; under the condition that the first operation data indicates that the image device has a fault, acquiring an image signal processing module captured by at least one statistical node, and processing second operation data of a second image signal; under the condition that the second operation data indicates that the fault exists, determining that the image signal processing module has the fault; and determining that the sensor module has a fault under the condition that the second operation data indicates that the fault does not exist. According to the invention, the technical problem of low fault detection efficiency of the image device in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and more specifically, to a fault detection method, device, storage medium and electronic device. Background Art

[0002] In intelligent driving systems, it is crucial to ensure the reliability of the camera's sensor module and image signal processing (ISP) module. Current fault detection methods are often limited to the detection of a single module, lacking a comprehensive fail-safe mechanism for the entire image processing system from sensor to ISP path, and usually rely on complex algorithms and separate checks on each ISP functional module, which is not only time-consuming and labor-intensive, but may not respond quickly in real-time processing scenarios, resulting in problems that cannot be solved in a timely manner.

[0003] That is, there is a problem in the prior art that the efficiency of fault detection of the imaging device is low. Summary of the invention

[0004] The embodiments of the present application provide a fault detection method, device, storage medium and electronic device to at least solve the technical problem of low efficiency of fault detection of image devices in the related art.

[0005] According to one aspect of an embodiment of the present application, a fault detection method is provided, comprising: acquiring an image device to be detected, wherein the image device comprises a sensor module and an image signal processing module, and the image signal processing module is used to process an image signal captured by the sensor module; deploying at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture first operating data of a first image signal input by the sensor module, and the first operating data is used to indicate whether the image device has a fault; when the first operating data indicates that the image device has a fault, acquiring second operating data of the image signal processing module captured by at least one statistical node, and processing a second image signal, wherein the second image signal is an image signal not captured by the sensor module; when the second operating data indicates that a fault exists, determining that the image signal processing module has a fault; when the second operating data indicates that no fault exists, determining that the sensor module has a fault.

[0006] According to another aspect of an embodiment of the present application, a fault detection device is also provided, including: a first acquisition unit, used to acquire an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process the image signal captured by the sensor module; a deployment unit, used to deploy at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture the image signal processing module, and process first operating data of the first image signal input by the sensor module, and the first operating data is used to indicate whether the image device has a fault; a second acquisition unit, used to acquire second operating data of the image signal processing module captured by at least one statistical node and processed by the second image signal when the first operating data indicates that the image device has a fault, wherein the second image signal is an image signal not captured by the sensor module; a first determination unit, used to determine that the image signal processing module has a fault when the second operating data indicates that the fault exists; and a second determination unit, used to determine that the sensor module has a fault when the second operating data indicates that the fault does not exist.

[0007] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above fault detection method.

[0008] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the fault detection method through the computer program.

[0009] In an embodiment of the present application, at least one statistical node is deployed in the image signal processing module, wherein at least one statistical node is used to capture the image signal processing module, and process the first operation data of the first image signal input by the sensor module, and the first operation data is used to indicate whether the image device has a fault; when the first operation data indicates that the image device has a fault, the image signal processing module captured by at least one statistical node is obtained, and the second operation data of the second image signal is processed, wherein the second image signal is an image signal captured by the non-sensor module; when the second operation data indicates that there is a fault, it is determined that the image signal processing module has a fault; when the second operation data indicates that there is no fault, it is determined that the sensor module has a fault. By analyzing the statistical value (first operation data) of the image input by the sensor module and the statistical value (second operation data) of the image input by the non-sensor module, it is possible to accurately determine whether the fault occurs in the sensor module or the image signal processing module. Deploying statistical nodes at key nodes reduces the complexity of fault detection. The output of the statistical node (i.e., the first operation data and the second operation data) can be quickly analyzed by the fault determination module to determine the fault type, and then take appropriate measures, thereby achieving the technical effect of improving the fault detection efficiency of the image device. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] Figure 1 is a schematic diagram of a hardware environment of a fault detection method according to an embodiment of the present application;

[0013] Figure 2 is a schematic diagram of a process of an optional fault detection method according to an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of an optional image signal processing link according to an embodiment of the present application;

[0015] Figure 4 is a schematic diagram of another optional image signal processing link according to an embodiment of the present application;

[0016] Figure 5 is a schematic diagram of a process of an optional fail-safe judgment method according to an embodiment of the present application;

[0017] Figure 6 is a schematic diagram of another optional image signal processing link according to an embodiment of the present application;

[0018] Figure 7 is a schematic diagram of a process of another optional fail-safe judgment method according to an embodiment of the present application;

[0019] Figure 8 is a schematic diagram of a process of an optional fault handling method according to an embodiment of the present application;

[0020] Fig. 9 is a schematic diagram of an optional fault handling framework diagram according to an embodiment of the present application;

[0021] Fig.10 is a schematic diagram of an optional fault detection method according to an embodiment of the present application;

[0022] Fig.11 It is a schematic diagram of an optional electronic device based on a fault detection method according to an embodiment of the present application.

[0023] The drawings are not necessarily to scale and may present somewhat simplified representations of various preferred features of the present application as disclosed herein, including, for example, specific dimensions, orientations, locations, and shapes. The details associated with such features will be determined in part by the specific intended application and use environment. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] The drawings are in simplified form and are not precisely to scale. For convenience and clarity purposes, directional terms such as longitudinal, transverse, top, bottom, left, right, top, above, above, below, rear, and front may be used with respect to the drawings. These and similar directional terms should not be construed as limiting the scope of the present application. In addition, as shown and described herein, the present disclosure may be practiced in the absence of elements not specifically disclosed herein.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As used herein, the term "system" refers to the mechanical and electrical hardware, software, firmware, electronic control components, processing logic, and / or processor devices that, alone or in combination, provide the described functionality. This may include, but is not limited to, application specific integrated circuits (ASICs), electronic circuits, processors (shared, dedicated, or groups) that execute one or more software or firmware programs, memory containing software or firmware instructions, combinational logic circuits, and / or other components.

[0028] Optionally, as an optional implementation, the fault detection method can be, but is not limited to, applied to intelligent driving scenarios of vehicles. In intelligent driving scenarios, ensuring the safety of vehicles is the primary task, and image devices, such as cameras, play a very important role as the eyes of vehicles. The above fault detection method can be, but is not limited to, applied to fault detection of image devices on vehicles to further determine the specific source of the fault when the image device fails. Specifically, Figure 1 A side view of a vehicle 101 is shown, the vehicle 101 being disposed on and capable of traversing a travel surface 113. The vehicle 101 includes a vehicle onboard navigation system 103, a memory 102 of a digitized road map 104, a spatial monitoring system 117, a vehicle controller 109, a GPS (Global Positioning System) sensor 110, an HMI (Human / Machine Interface) device 111, and also includes an autonomous controller 112 and a telematics controller 114.

[0029] In one embodiment, the spatial monitoring system 117 includes: one or more spatial sensors and systems arranged to monitor the visible area 105 in front of the vehicle 101, and a spatial monitoring controller 118; the spatial sensors of the visible area 105 include, for example, a lidar sensor 106, a radar sensor 107, a camera 108, etc. The spatial sensors can monitor all or part of the visible area 105 to detect approaching remote objects. The spatial monitoring controller 118 generates a digital representation of the visible area 105 based on data input from the spatial sensors. The spatial monitoring controller 118 can evaluate the input from the spatial sensors to determine the linear range, relative speed and trajectory of the vehicle 101 in view of each approaching remote object. The spatial sensors can be located at various locations on the vehicle 101. The placement of the spatial sensors allows the spatial monitoring controller 118 to monitor traffic flow. The spatial sensors of the vehicle spatial monitoring system 117 may include object positioning sensing devices, which include range sensors that rely on effects such as Doppler effect measurements to locate objects in front.

[0030] The camera 108 is advantageously mounted and positioned on the vehicle 101 in a position that allows for capturing images of the visible area 105, wherein at least a portion of the visible area 105 includes an area in front of the vehicle 101 and a portion of the travel surface 113 of the trajectory of the vehicle 101. The visible area 105 may also include the surrounding environment. Other cameras may also be employed, for example, including a second camera disposed on a rear portion or side portion of the vehicle 101 to monitor the rear of the vehicle 101 or the left and right side areas of the vehicle 101.

[0031] The autonomous controller 112 is configured to implement autonomous driving or advanced driver assistance system (ADAS) vehicle functionality. Such functionality may include a vehicle onboard control system capable of providing a certain level of driving automation. Driving automation may include a range of dynamic driving and vehicle operations. Driving automation may include some level of automatic control or intervention involving a single vehicle function (e.g., steering, acceleration, and / or braking).

[0032] Operator controls may be included in a cockpit of the vehicle 101 and may include, by way of non-limiting example, a steering wheel, an accelerator pedal, a brake pedal, and an operator input device, which is an element of the HMI device 111. The operator controls enable a vehicle operator to interact with the operating vehicle 101 and direct the operation of the vehicle 101 to provide passenger transportation.

[0033] The HMI device 111 provides human-machine interaction for the purpose of directing the operation of the infotainment system, GPS (global positioning system) sensor 110, onboard navigation system 103, and the like, and includes a controller. The HMI device 111 monitors operator requests and provides the operator with status, service, and maintenance information of the vehicle system. The HMI device 111 communicates with a plurality of operator interface devices and / or controls the operation of a plurality of operator interface devices. The HMI device 111 may also communicate with one or more devices that monitor biometric data associated with the vehicle operator. For simplicity of description, the HMI device 111 is depicted as a single device, but in embodiments of the system described herein, may be configured as a plurality of controllers and associated sensing devices.

[0034] The onboard navigation system 103 uses the digitized road map 104 for the purpose of providing navigation support and information to the vehicle operator. The autonomous controller 112 uses the digitized road map 104 for the purpose of controlling autonomous vehicle operation or ADAS vehicle functions.

[0035] The vehicle 101 may include a telematics controller 114 that includes a wireless telematics communication system capable of off-vehicle communications, including communications with a communication network 115 having wireless and wired communication capabilities. The wireless telematics communication system is capable of short-range wireless communications with a handheld device. In one embodiment, the handheld device includes a software application that includes a wireless protocol to communicate with the telematics controller 114, and the handheld device performs off-vehicle communications, including communicating with an off-board server 116 via the communication network 115.

[0036] The term "controller" and related terms (e.g., microcontroller, control unit, processor, and similar terms) refer to one or various combinations of: application specific integrated circuit(s) (ASICs), field programmable gate arrays (FPGAs), electronic circuit(s), central processing unit(s), such as microprocessor(s), and associated non-transitory memory components (indicated by memory 102) in the form of memory and storage devices (read-only, programmable read-only, random access, hard drive, etc.). The non-transitory memory components are capable of storing machine-readable instructions in the form of one or more software or firmware programs or routines, combinational logic circuit(s), input / output circuit(s) and devices, signal conditioning and buffering circuitry, and other components accessible by one or more processors to provide the described functionality. The input / output circuit(s) and devices include analog / digital converters and related devices that monitor inputs from sensors, where such inputs are monitored at a preset sampling frequency or in response to a triggering event. Software, firmware, programs, instructions, control routines, code, algorithms, and similar terms mean controller executable instruction sets, including calibration and lookup tables. Each controller executes (multiple) control routines to provide the desired functionality. Routines may be executed at regular intervals, for example, once every 100 microseconds during ongoing operation. Alternatively, routines may be executed in response to the occurrence of a triggering event. Communication between controllers, actuators, and / or sensors may be implemented using a direct wired point-to-point link, a networked communication bus link, a wireless link, or another suitable communication link. Communication includes exchanging data signals in a suitable form, for example, including electrical signals via a conductive medium, electromagnetic signals via air, optical signals via optical waveguides, and the like. Data signals may include discrete, analog, or digitized analog signals representing inputs from sensors, actuator commands, and communications between controllers. The term "signal" refers to a physically identifiable indicator of information being transmitted, and may be a suitable waveform (e.g., electrical, optical, magnetic, mechanical, or electromagnetic), for example, DC, AC, sine waves, triangle waves, square waves, vibrations, and the like, which are capable of traveling through a medium. A parameter is defined as a measurable quantity representing a physical property of a device or other element that can be identified using one or more sensors and / or physical models. A parameter may have discrete values, such as "1" or "0", or may be infinitely variable in value.

[0037] As used herein, the terms 'dynamic' and 'dynamically' describe a step or process that is performed in real time and is characterized by monitoring or otherwise determining the state of a parameter and regularly or periodically updating the state of the parameter during execution of a routine or between iterations of execution of a routine.

[0038] Optionally, as an optional implementation, as Figure 2As shown, the fault detection method includes:

[0039] S202, acquiring an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process an image signal captured by the sensor module;

[0040] S204, deploying at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture the first operation data of the first image signal input by the sensor module and processed by the image signal processing module, and the first operation data is used to indicate whether the image device has a fault;

[0041] S206, when the first operation data indicates that the image device has a fault, obtaining an image signal processing module captured by at least one statistical node, and processing second operation data of a second image signal, wherein the second image signal is an image signal captured by a non-sensor module;

[0042] S208, determining that the image signal processing module has a fault if the second operation data indicates that there is a fault;

[0043] S210 , when the second operating data indicates that there is no fault, determining that there is a fault in the sensor module.

[0044] Optionally, in this embodiment, the image device refers to a device including a sensor module and an image signal processing module, which is used to capture and process image signals, such as a camera of an intelligent driving system.

[0045] Optionally, in this embodiment, the sensor module is a component in the imaging device, responsible for converting external light signals into electrical signals, and is the source of image data.

[0046] Optionally, in this embodiment, the image signal processing module (ISP) is used to receive and process raw image data from the sensor module, and optimize image quality through various algorithms, such as denoising, color correction, etc.

[0047] Optionally, in this embodiment, the statistical node may be, but is not limited to, a detection unit / detection module carried inside the ISP, for collecting and analyzing processed image data, such as automatic exposure data, automatic white balance data, brightness data, etc., to evaluate the system status. It should be noted that the statistical node captures the operating data by, but is not limited to, setting a corresponding statistical value module to obtain a corresponding statistical value, wherein the data representation of the statistical value is used to indicate whether a fault occurs.

[0048] Optionally, in this embodiment, the first image signal is an image signal directly captured by the sensor module, which is used as the original input for ISP processing. The second image signal is an image signal generated by a non-sensor module, such as a preset test pattern, which is used as a comparison benchmark for fault detection.

[0049] Optionally, in this embodiment, the image device is taken as a recognition camera as an example for description. As an image data capture device in an intelligent driving system, the device is equipped with a sensor module and an ISP module.

[0050] Optionally, in this embodiment, statistical nodes are deployed on key processing nodes of the ISP module to monitor the processed image data in real time and generate relevant operation data. These statistical nodes can generate first operation data based on information such as data distribution, brightness statistics, and color channel statistics, which are used to indicate whether the state of the image signal input by the sensor module is normal during ISP processing.

[0051] Optionally, in this embodiment, when the first operating data indicates that there is a fault in the image device, an image signal processing module captured by at least one statistical node is obtained to process second operating data of a second image signal: when the first operating data shows that there is a fault in the image device, an image signal (second image signal) generated by a non-sensor module is used to perform ISP processing again to obtain second operating data to distinguish whether the fault comes from the sensor module or the ISP module.

[0052] It is understandable that if the second operation data generated after the second image signal is processed by the ISP also shows a fault, it can be determined that the fault occurs in the ISP module. In this case, targeted measures such as restarting the ISP, resetting ISP parameters, and checking ISP hardware can be taken.

[0053] Furthermore, if the second operation data is normal, but the first operation data is still abnormal, it can be concluded that the fault occurs in the sensor module. At this time, measures such as increasing exposure parameters, checking lens occlusion or dirt, and restarting the sensor can be taken to recover from the fault.

[0054] It should be noted that the internal statistical value unit of the ISP module is used to monitor the image data in real time to quickly diagnose the health status of the image device. According to different image processing requirements, the statistical node can be, but is not limited to, located at different nodes of the ISP processing chain, such as the AE statistical node, the AWB statistical node, etc. When the processing result of the first image signal is detected to be abnormal, a test mode will be entered, and the preset second image signal will be used instead of the sensor input to verify the specific location of the fault.

[0055] It should also be noted that by comparing the output of the statistical node in normal mode and test mode (the first operating data and the second operating data), the source of the fault can be distinguished, so that corresponding recovery measures can be taken. For example, if the image is too dark or the color is distorted in normal mode, and the image returns to normal in test mode, it can be preliminarily determined that there is a problem with the sensor module, and then try to adjust the exposure parameters or perform occlusion detection. Conversely, if the image processing result is still abnormal in test mode, the ISP module may be faulty, and you should consider restarting the ISP or checking the ISP hardware status.

[0056] Through the embodiments provided by the present application, by analyzing the statistical values ​​of the sensor module input image (first operation data) and the non-sensor module input image (second operation data), it is possible to accurately determine whether the fault occurs in the sensor module or the image signal processing module. Deploying statistical nodes at key nodes reduces the complexity of fault detection. The output of the statistical node (i.e., the first operation data and the second operation data) can be quickly analyzed by the fault judgment module to determine the type of fault, and then take appropriate measures, thereby achieving the technical effect of improving the fault detection efficiency of the image device.

[0057] As an optional solution, at least one statistical node is deployed in the image signal processing module, including:

[0058] Deploy a statistics node at the output of the image signal processing module; or,

[0059] Dividing the image signal processing module to obtain N regions, wherein each of the N regions includes at least one functional module of the image signal processing module, and N is a positive integer greater than 1;

[0060] A statistics node is deployed at the output of each region.

[0061] Optionally, in this embodiment, the functional module is a sub-module inside the ISP module, and each sub-module is responsible for specific processing of image data, such as bad pixel correction (DPC), black level correction (BLC), automatic exposure (AE), automatic white balance (AWB), etc.

[0062] Optionally, in this embodiment, the ISP processing chain is divided into multiple logical areas, each area including one or more functional modules, to achieve more sophisticated fault detection and location.

[0063] Optionally, in this embodiment, a statistical node is installed at a key position of the ISP module, which can collect data features after image processing in real time, and determine whether a fault occurs by analyzing the data. The deployment location of the statistical node determines the precision of fault detection, and can be at the output end of the ISP or at the output end of multiple processing stages within it.

[0064] Optionally, in this embodiment, a statistical node is deployed at the final output point of the ISP processing chain to collect the final data after the overall image processing and perform overall fault judgment. This method is suitable for situations where overall image information is required for fault judgment.

[0065] Optionally, in this embodiment, the ISP module is divided into N areas according to function or processing stage, and each area contains at least one functional module. This division method can achieve more accurate fault location, and the value of N can be determined according to the complexity of the ISP module and the required fault detection accuracy. A statistical node is deployed at the output end of each functional area of ​​the ISP module to collect and analyze the local image data after processing in the area. By comparing the statistical node data of different areas, it is possible to quickly determine at which stage of the ISP processing chain the fault occurred, providing more specific information for fault recovery measures.

[0066] Through the embodiments provided by the present application, a statistical node is deployed at the final output end of the ISP module. This method can collect the final data after the entire image processing process, which is suitable for scenarios where it is necessary to detect whether the image device is operating normally as a whole. Alternatively, the ISP module is subdivided into N areas, each area contains one or more functional modules, and a statistical node is deployed at the output end of each area. This method can achieve more precise fault location and is suitable for scenarios that require high-precision fault detection and rapid recovery. Through the data collected by the statistical node, the system can quickly determine whether the source of the fault is the sensor module or the ISP module, and at which stage of the ISP processing chain the fault occurs, so as to take targeted recovery measures, such as adjusting exposure parameters, restarting the ISP module, checking the sensor status, etc.

[0067] As an optional solution, deploy a statistics node at the output end of each area, including:

[0068] In the case where the N regions include a first region and a second region, a first statistical node is deployed at an output end of the first region, and a second statistical node is deployed at an output end of the second region, wherein the second statistical node is located after the first statistical node;

[0069] After deploying a statistics node at the output end of each region, the method further includes:

[0070] Capturing, at the first statistical node, first running sub-data of the first image signal processed by the image signal processing module, wherein the first running data includes the first running sub-data;

[0071] In the event that the first operation sub-data indicates that a fault exists, it is determined that the first operation data indicates that a fault exists in the imaging device.

[0072] Optionally, in this embodiment, the first statistical node and the second statistical node refer to statistical nodes deployed at the output ends of the first area and the second area of ​​the ISP module, respectively, for collecting processed image data features. The first operation sub-data is data captured and generated by the first statistical node, and is used to indicate whether the state of the image signal processed by the first area is normal, and is a component of the first operation data.

[0073] Optionally, in this embodiment, the ISP module is divided into at least two areas, such as bad pixel correction and black level correction, automatic exposure processing and automatic white balance processing, etc., and a statistical node is deployed in each area after the processing is completed. This method can monitor the image quality according to the processing stage and provide more detailed information for fault location.

[0074] It should be noted that the second area is located after the first area in the processing order, and the data collected by the second statistical node is the result of further processing based on the first statistical node. This sequential deployment helps to track down the fault points in the image processing process.

[0075] Optionally, in this embodiment, data features after ISP processing are collected in real time, and the first operation sub-data reflects statistical information of the processing results of the first area, is a part of the overall first operation data, and is used to evaluate whether the image quality at this stage is normal.

[0076] It is understandable that if the first running sub-data captured by the first statistical node shows an abnormality (such as abnormal brightness, color distortion, etc.), it can be preliminarily determined that the entire image device (including the sensor module and the ISP module) has a fault. This step serves as a preliminary fault judgment process and provides a basis for subsequent more accurate fault location and recovery measures.

[0077] Through the embodiment provided by the present application, the method of sequentially deploying statistical nodes can not only quickly locate the specific stage of the fault, but also can capture the first running sub-data of the first image signal processed by the image signal processing module at the first statistical node by analyzing the first running sub-data at the first statistical node. The method also includes:

[0078] capturing, at the second statistical node, second running sub-data of the first image signal processed by the image signal processing module, wherein the first running data includes the second running sub-data;

[0079] If the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is a fault, determining that the first operation data indicates that the image signal processing module has a fault in the second area;

[0080] In the case where the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is no fault, it is determined that the first operation data indicates that there is no fault in the image device.

[0081] The first operation sub-data (captured by the first statistical node) determines whether there is a fault in the first area. If the first operation sub-data indicates that there is a fault, the entire image processing chain (i.e., the first operation data) will be considered abnormal, which provides clear guidance for subsequent targeted fault recovery measures. For example, if the first statistical node (located after AE processing) shows that the exposure is abnormal, and the second statistical node (located after AWB processing) shows that the color is normal, then the system can quickly determine that the source of the fault is the AE module, and then adjust the exposure parameters or restart the AE functional unit to try to restore the image quality online.

[0082] As an optional solution, in the process where the first statistical node captures the first running sub-data of the first image signal processed by the image signal processing module, the method further includes:

[0083] capturing, at the second statistical node, second running sub-data of the first image signal processed by the image signal processing module, wherein the first running data includes the second running sub-data;

[0084] If the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is a fault, determining that the first operation data indicates that the image signal processing module has a fault in the second area;

[0085] In the case where the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is no fault, it is determined that the first operation data indicates that there is no fault in the image device.

[0086] Optionally, in this embodiment, the first operation sub-data is data collected by the first statistical node, reflecting the state of the image signal after processing in the first region, and is used to preliminarily determine whether there is a fault in the region. The second operation sub-data is data collected by the second statistical node, reflecting the state of the image signal after processing in the second region, and is used to determine whether there is a fault in the image signal processing after the first region.

[0087] It is understandable that after the second area of ​​the ISP module is processed, the second statistical node will generate the second operating sub-data, which is used to evaluate whether the image quality after processing in the area is normal. The first operating data is a collection of output data of all statistical nodes in the entire ISP processing chain, including the first operating sub-data and the second operating sub-data, which are used to comprehensively evaluate the status of the image device. If the processing result of the first area is normal (that is, the first operating sub-data is normal), and the processing result of the second area shows a fault (that is, the second operating sub-data is abnormal), it can be determined that the fault occurred in the second area of ​​the ISP. If all the operating sub-data collected from the first statistical node to the second statistical node in the entire ISP processing chain are normal, it can be determined that the entire image device (including the sensor module and the ISP module) is currently not faulty. This comprehensive operating data monitoring provides a solid foundation for the safe operation of the image device.

[0088] Through the embodiments provided by the present application, a detailed process of image device fault detection is performed by deploying multiple statistical nodes. While the first statistical node captures the first operating sub-data, the second statistical node also collects the second operating sub-data, so that the processing process of the ISP module can be more comprehensively monitored. The core of this mechanism is to compare the operating sub-data of different regions. By comparing the first operating sub-data with the second operating sub-data, it can be accurately determined whether the fault occurs in the second region.

[0089] As an optional solution, after the image signal processing module deploys at least one statistical node, the method further includes at least one of the following:

[0090] At least one statistical node captures first brightness distribution data based on pixels during the process of processing the first image signal by the image signal processing module, wherein the first operation data includes the first brightness distribution data, and the first brightness distribution data is used to indicate the number of pixels at each brightness;

[0091] At least one statistical node captures second brightness distribution data based on blocks during the process of processing the first image signal by the image signal processing module, wherein the first operation data includes the second brightness distribution data, and the second brightness distribution data is used to indicate the proportion of blocks at each brightness;

[0092] In at least one statistical node, third brightness distribution data based on the original data domain is captured during the process of the image signal processing module processing the first image signal, wherein the first operating data includes the third brightness distribution data, and the third brightness distribution data is used to indicate statistical information of the RGB value of the block.

[0093] Optionally, in this embodiment, the first brightness distribution data is the brightness distribution data based on each pixel collected by the statistical node during the ISP module processing, reflecting the number distribution of pixels at different brightness levels. The second brightness distribution data is the brightness distribution data based on blocks collected by the statistical node during the ISP module processing, reflecting the proportion distribution of blocks at different brightness levels. The third brightness distribution data is the RGB value distribution data based on the original data domain (such as RGGB) collected by the statistical node during the ISP module processing, reflecting the statistical information of the RGB color value in the block.

[0094] It should be noted that blocks are multiple small areas into which images are divided during ISP module processing, which are used for statistics and analysis of local image quality. Pixels are the basic units that make up digital images, and each pixel has a specific brightness and color value. RGB values ​​refer to the intensity values ​​of the red (R), green (G), and blue (B) color channels of each pixel in the image, and are used to describe color information.

[0095] Optionally, in this embodiment, the statistical node in the ISP module generates first brightness distribution data based on pixel brightness, reflecting the distribution of pixels under different brightness values. These data are used as part of the first operating data to determine whether the image signal processing is normal.

[0096] Optionally, in this embodiment, the statistical node generates second brightness distribution data based on block brightness, and further evaluates the uniformity and stability of image processing by analyzing the proportion of blocks with different brightness levels, which serves as a basis for fault detection.

[0097] Optionally, in this embodiment, the statistical node also collects RGB value distribution data based on the original data domain (such as RGGB format), and determines whether the color processing is correct by counting the color channel information in the statistical block for fault detection and analysis.

[0098] It should be noted that by generating brightness distribution data, the image quality in the image signal processing process is accurately evaluated, so as to determine whether the ISP module has a fault. The first brightness distribution data, the second brightness distribution data, and the third brightness distribution data each provide a comprehensive evaluation of the image processing status from the perspective of the pixel, block, and raw data domain, and these data together constitute the first operation data for fault detection and analysis.

[0099] To further illustrate, if the first brightness distribution data shows that the image is too dark, that is, the number of pixels with brightness values ​​below the threshold is too high, it may mean that the exposure parameters are improperly set or the sensor module is faulty; if the second brightness distribution data shows that the block brightness distribution is uneven, that is, the brightness of some blocks deviates significantly from the average, this may indicate that some processing units inside the ISP module (such as local shadow correction) have failed; and if the third brightness distribution data shows that the RGB color values ​​within the block are abnormal, such as the color channel values ​​are too consistent, it may indicate that the color correction function is impaired.

[0100] Through the embodiments provided by the present application, by collecting and analyzing the above distribution data, the statistical node can generate comprehensive first operation data to provide key information for fault detection. According to the first brightness distribution data, the second brightness distribution data or the third brightness distribution data contained in the first operation data, the source of the fault can be quickly determined and corresponding recovery measures can be taken, such as adjusting exposure parameters, restarting the ISP module, checking the color correction function, etc., so as to effectively ensure the normal operation of the image device and ensure the safety and reliability of the intelligent driving system.

[0101] As an optional solution, after the image signal processing module deploys at least one statistical node, the method further includes:

[0102] When the first brightness distribution data indicates that there are at least a first proportion of pixels whose brightness is lower than the first threshold, or when the change amplitude of the first brightness distribution data in M ​​consecutive frames is less than the first change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold, determining that the first brightness distribution data indicates a fault;

[0103] When the second brightness distribution data indicates that there are at least a second proportion of blocks whose brightness is lower than the second threshold, or when the change amplitude of the second brightness distribution data in the continuous M frames is less than the second change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold, it is determined that the second brightness distribution data indicates that there is a fault;

[0104] When the third brightness distribution data indicates that there is at least a third proportion of blocks with RGB values ​​lower than the third threshold, or when the change amplitude of the third brightness distribution data in M ​​consecutive frames is less than the third change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold, it is determined that the third brightness distribution data indicates a fault.

[0105] Optionally, in this embodiment, the first threshold is a preset brightness value, which is used to determine whether the image is too dark. The first proportion is the ratio of the number of pixels with brightness lower than the first threshold to the total number of pixels, which is used to determine the severity of the fault. M frames are the number of consecutive image frames, which are used to evaluate the changes in image data over time. The first change threshold is the change amplitude standard of the first brightness distribution data within M consecutive frames, which is used to determine whether the image data is stable and unchanged. The motion speed is the driving speed of the platform (such as a vehicle) where the image device is located. The preset speed threshold is a preset driving speed standard, which is used to comprehensively judge the fault in combination with the changes in the image data.

[0106] Optionally, in this embodiment, the second threshold is a preset brightness value for determining whether the brightness of a block is too low. The second percentage is the percentage of the number of blocks whose brightness is lower than the second threshold, which is used to evaluate the impact range of the fault. The third threshold is a preset RGB value standard, which is used to determine whether a color channel is abnormal. The third percentage is the percentage of the number of blocks whose RGB values ​​in the block are lower than the third threshold, which is used to evaluate the color processing effect.

[0107] Optionally, in this embodiment, when the first brightness distribution data captured by the statistical node shows that the number of pixels with brightness lower than the first threshold reaches or exceeds the first proportion, this may indicate that there is a fault in the sensor module or the exposure unit in the ISP, resulting in an image that is too dark. When the image device (such as a vehicle-mounted camera) is in a high-speed moving state, if the first brightness distribution data of M consecutive frames changes very little (less than the first change threshold), it may indicate that some units in the ISP module (such as the statistical value module) are invalid or the parameters are improperly set.

[0108] Optionally, in this embodiment, when the second brightness distribution data shows that the proportion of blocks with brightness lower than the second threshold reaches or exceeds the second proportion, it may indicate that there is a fault in the local brightness correction unit in the ISP module, causing some blocks in the image to be too dark. When the image device moves at high speed, if the second brightness distribution data of M consecutive frames changes very little (less than the second change threshold), it may indicate that the statistical value module or processing unit in the ISP module fails to correctly respond to brightness changes in motion.

[0109] Optionally, in this embodiment, when the third brightness distribution data shows that the proportion of the number of RGB values ​​in the block below the third threshold reaches or exceeds the third proportion, it may indicate that there is a problem with the color correction unit in the ISP module, resulting in image color distortion or missing color information. When the image device moves at high speed, if the third brightness distribution data of M consecutive frames changes very little (below the third change threshold), it may mean that the color processing unit in the ISP module fails to adapt to the rapidly changing scene normally, resulting in damage to the color processing function.

[0110] Through the embodiments provided by the present application, it is possible to more accurately determine whether each processing unit of the ISP module is operating normally, thereby quickly identifying the source of the fault and guiding the adoption of corresponding recovery measures, such as adjusting exposure parameters, restarting the color correction unit, checking the statistical value module, etc., to maintain the normal operation of the image device and the safety and reliability of the intelligent driving system. These fault detection mechanisms based on statistical values ​​can not only detect faults online, but also evaluate the severity of the faults, and make a comprehensive judgment in combination with the motion state of the image device, greatly improving the effectiveness and accuracy of fault detection.

[0111] As an optional solution, after determining that the image signal processing module has a fault, the method further includes:

[0112] Performing an online recovery operation on the image signal processing module, wherein the online recovery operation is used to enable at least one functional module of the image signal processing module;

[0113] When the online recovery operation is invalid, resetting the image signal processing module;

[0114] After determining that the sensor module is faulty, the method further includes:

[0115] Performing a parameter adjustment operation on the sensor module, wherein the parameter adjustment is used to indicate adjusting an exposure parameter of the sensor module;

[0116] If the parameter adjustment operation is invalid, reset the sensor module.

[0117] Optionally, in this embodiment, the online recovery operation is used to indicate a series of recovery measures that are performed without interrupting the image signal processing process for a fault detected in the ISP module, in an attempt to restore the ISP module to a normal working state. Enable is used to indicate that a functional unit of the ISP module is reactivated or set to an enabled state so that it can process image data normally. The reset operation is used to indicate a more thorough recovery measure that is performed when the online recovery operation fails to resolve the ISP module fault, usually involving reinitializing the ISP module to restore its default or initialized state. The parameter adjustment operation is used to indicate that for a fault detected by the sensor module, an attempt is made to restore normal image data acquisition by adjusting the parameters of the sensor module (such as exposure parameters).

[0118] Optionally, in this embodiment, the exposure parameter is a parameter that affects the sensitivity of the sensor module to light and is used to control the brightness of the image. Adjusting the exposure parameter can change the brightness of the image. The reset operation is used to indicate a forced recovery measure when the parameter adjustment operation fails to solve the sensor module failure, which usually involves restarting the sensor module to restore its initialization state.

[0119] It is understandable that after detecting the ISP module failure, an attempt is made to restore its normal function online by enabling the functional modules inside the ISP, such as reactivating the automatic exposure (AE) or automatic white balance (AWB) units, without interrupting the image processing flow. When the online recovery measures fail to resolve the ISP module failure, a reset operation will be performed, usually restarting the ISP module or restoring its initialization state, in order to completely resolve the failure.

[0120] Optionally, in this embodiment, after the sensor module detects a fault, an attempt is made to restore the normal state of image acquisition online by adjusting sensor parameters such as exposure parameters. When adjusting the sensor module parameters cannot solve the fault, a reset operation is performed, usually restarting the sensor module to restore its initialization state to ensure the reliability of image data acquisition.

[0121] Through the embodiments provided by this application, the fault recovery capability of the image device can be improved, and the impact of the fault on the intelligent driving system can be reduced. In particular, when it is impossible to stop the car immediately or intervene manually, these recovery measures can be carried out online, greatly improving the system's autonomous repair capability and safety. At the same time, through the reset and resetting operations, it is ensured that when the online recovery measures are invalid, more thorough measures can be taken to ensure the long-term stable operation of the image device.

[0122] As an optional solution, the above fault detection method is applied to the fail-safe detection scenario of the image device. In this scenario, in the field of intelligent driving, ensuring vehicle safety is a direction that must be guaranteed. As a complex system, the fail-safe detection of each module is indispensable. Among them, the camera, as the eyes of the intelligent driving vehicle, has a very important position, and the image signal processing module (ISP for short), which directly determines the quality of the camera output and affects the subsequent perception module to recognize scenes and objects, also needs to have fail-safe detection capabilities. Especially when encountering an intrusion, such as importing some other pictures, the perception results vary greatly, and the consequences will be disastrous. Since the ISP module includes many processing units such as bad pixel correction, black level correction, shadow correction, white balance correction, raw domain denoising, demosaicing, color correction, RGB to YUV, sharpening, gamma correction, 2D denoising, etc., the complexity of the processing algorithm is relatively high, and it is somewhat complicated to perform fail-safe checks on each unit separately, so try to find one or two breakthrough points (key nodes) and check the data of a certain node to ensure the fail-safety of the entire ISP module, which can greatly improve the detection efficiency.

[0123] Optionally, in this embodiment, based on the above fault detection method, a camera fault detection method is provided to perform fault detection on a system composed of a sensor and an image signal processing (ISP). Optionally, the statistical value unit data in the ISP module is analyzed to determine a fault error type, and the next step of intervention or further detection is performed on this type.

[0124] It should be noted that the statistical value unit in the ISP module is mainly a result of statistics based on the image data, which can fully reflect the current image situation. For example, if the camera is completely blocked, the data of the statistical value unit is almost close to 0, and the system may need to enter a detection state or pre-processing state to confirm whether the data error is caused by lens blockage, sensor failure, ISP module failure, etc., and then hand it over to the post-processing module for unblocking operations or alarms and continuous close attention or downgrade of some driving functions.

[0125] Optionally, different processing measures are selected according to different characteristics of statistical value anomalies. First, normal processing is performed based on the online recoverable state. If the picture cannot be restored to normal after trying to solve the problem, a higher level report and processing is performed.

[0126] Optionally, if the statistical data is basically black, it may be caused by abnormal exposure parameters. Try to increase the exposure parameters. If it does not work, check the camera occlusion module to see if there is any occlusion and remove it; at the same time, check if there is any dirt. If there is, you can also enable dirt removal. If the above operations do not work, apply to check and restart the ISP and try to recover.

[0127] Optionally, if most of the statistical data are close to saturation, the exposure parameter may be set too high, try to reduce the exposure parameter. If it does not work, apply to check whether the camera is blocked, such as white objects.

[0128] Alternatively, if the statistical data is almost unchanged, it may be that the ISP function is frozen. Check the driving status (such as vehicle speed). If it is not stationary and the statistical data is almost unchanged, combined with the feedback, it may be that the ISP is invalid. Restart the ISP according to the situation.

[0129] Optionally, according to the type of data description, it can be classified into an online recoverable state or a state requiring external intervention.

[0130] To further illustrate, a schematic diagram of an image signal processing link is as follows: Figure 3As shown, the camera is first added to the ISP platform to provide the most original data, that is, sensorraw. The subsequent modules of image processing are DPC (Defectivate Pixel Concealment), BLC (Black Level Correction), LSC (Lens Shading Correction), AWB gain (Auto White Balance Gain), Demosaic (Demosaic module), Sharpen (Sharpening module), CM (Color Correction Matric Color Correction module), RGB2YUV (RGB to YUV module).

[0131] It should be noted that the camera is connected to the vehicle-mounted SOC chip. Due to the limitation of data transmission, the original raw data will generally be compressed and sent to the ISP, and then restored to the original data through the ISP decompression module. At this time, the original data can be sent to the statistical value module, and the hardware statistical module can calculate the value of the required data format.

[0132] For example, the statistical value of ae in the statistical value module may generally be based on dividing the picture into MxN grids, and counting the average brightness of each grid. Or, the pixel brightness of the entire picture is counted, and the number of pixels at each brightness is counted according to the brightness, and the distribution of pixels at each brightness is intuitively seen according to the brightness from low to high.

[0133] To give a further example, the awb statistical value in the statistical value module is generally expressed based on three different color channels R / G / B. Generally, the picture is divided into MxN grids, and then the R, G, and B values ​​of each grid are given; or there are platforms that directly give the R / G, B / G values ​​of each pixel in the form of a ratio, or give a block-based ratio.

[0134] It should be noted that the nodes where the statistical value modules are located on different platforms can be set differently. However, without exception, the statistical value modules must be available. Figure 3As shown, the statistical value module nodes may appear in 4 nodes in the figure (not limited to this), among which, the statistical value module node 1 is located between the SensorRaw graph and DPC, the statistical value module node 2 is located between AWB gain and Demosaic, the statistical value module node 3 is located between Sharpen and CCM, and the statistical value module node 4 is located between CCM and RGB2YUV. Some platforms also have different statistical data types in different nodes, which are divided into AEstat data required by the ae algorithm and AWB stat data required by the awb algorithm. According to different algorithm domains, some statistical value nodes are distributed in the raw domain, some statistical value nodes are distributed in the rgb domain, and some statistical value nodes are distributed in the yuv domain.

[0135] It should be noted that for Figure 3 The multiple statistical value module nodes shown can, but are not limited to, output the statistical values ​​required for 2A algorithm calculation after calculation by the hardware statistical module. The specific type can be, but is not limited to, the above-mentioned aestat or the above-mentioned awb stat. The specific form of the statistical value may also be the same for two different nodes, in order to determine the benefit obtained by the image after being processed by some image processing algorithms of the ISP, so as to determine whether the brightness or white balance of the current picture meets expectations.

[0136] For example, Figure 3 For example, it is possible that statistical value module node 1 and statistical value module node 4 are both statistical values ​​of picture brightness. Statistical value module node 1 is used as a reference to calculate the image exposure parameters of the next frame, and statistical value module node 4 is more about looking at the image brightness after the image passes through other brightness-related modules, in order to consider whether to compensate for the image brightness, or to make a reference to the previous exposure strategy based on the brightness data here.

[0137] For example, a schematic diagram of an optional image signal processing link is as follows Figure 4 As shown, the output of the statistical value module node 1 is AE stat, temporarily recorded as D1 data; the output of the statistical value module node 2 is AWB stat, recorded as D2 data, and the output of the statistical value module node 3 is AE stat, recorded as D3 data. First, the AE-related statistical data D1 data and D3 data are subjected to fault safety detection and judgment.

[0138] Further based on Figure 4 The content shown is an example of a schematic diagram of the process of a fault safety judgment method. Figure 5As shown, the raw data from the sensor first passes through the statistical value module node 1, and the hardware module counts the required AE statistical values ​​and outputs them to the fault judgment module, which performs further calculations and statistics. The current data distribution performance is recorded as Q1. If the current distribution Q1 meets the conditions of the pre-calibrated problem scene data R1, and the vehicle speed is greater than the threshold (indicating that at least the car is moving, not a static picture, and the statistical data of the static picture is basically unchanged and will also be judged as a problem scene), then it can be determined that there is a problem with the module before the statistical value module node 1. It means that a fault safety problem has occurred, and the next action can be taken according to the abnormal performance behavior of the statistical value. For example, if the statistical value is observed to be almost unchanged, it can be speculated that there may be a problem with the statistical value module node, restart the module, or re-enable other algorithm modules, or restart the sensor (check and process according to the possible line with the least impact, and the fewer units reset or reset, the better); it is observed that there is always an area or all areas of the statistical value distribution that are almost completely black. It is necessary to further check whether the exposure parameter is set too low, or whether the camera is dirty or blocked. The priority of the dirty detection processing can be increased, or the sensor is damaged. Subsequent measures that can be taken include but are not limited to:

[0139] (1) Report the error type and let the ISP-related modules perform recovery processing. For example, if the image is abnormally overexposed, lower the exposure parameters to see if the normal brightness of the image can be restored; if the image is abnormally dark, increase the exposure parameters to see if the brightness of the image can be restored; if the image turns black and white, increase the saturation to see if the image can be restored to color.

[0140] (2) Report the corresponding type of ISP failure to the superior, who will make a comprehensive judgment based on other sensors such as radar.

[0141] (3) Report the ISP fault safety corresponding type problem to the superior, restart the ISP or restart the sensor.

[0142] (4) Apply for dirt detection action in advance and perform dirt removal action.

[0143] (5) Degrade some driving functions.

[0144] (6) To shut down the system, you need to completely shut down the autonomous driving function (considering the entire system, if the necessary conditions for autonomous driving are not met, you can consider driving to the safe island area and ending the autonomous driving function).

[0145] Further based on Figure 4 The content shown is illustrated by examples, such as Figure 5As shown, when the data flows to the statistical value module node 2, it is awb statistical data with its own data characteristics. According to the current distribution situation Q2, it is confirmed whether it meets the conditions of the previously calibrated problem scene data R2. If it meets the calibration conditions and the vehicle speed is greater than the threshold (indicating that at least the vehicle is moving, not a static picture, and the statistical data of the static picture is basically unchanged and will also be judged as a problem scene), considering that the conditions of Q1 also meet the conditions of R1, then it is determined that: at least a safety problem has occurred before the statistical data node 1; if the conditions of Q1 do not meet the conditions of R1, and the conditions of Q2 meet the conditions of R2, then it can be determined that a safety problem has occurred between the statistical value module node 1 and the statistical value module node 2.

[0146] When the data flow reaches the statistical value module node 3, if the current distribution situation Q3 meets the conditions of the pre-calibrated problem scenario data R3, and the vehicle speed is greater than the threshold (indicating that at least the vehicle is moving, not a static picture, and the statistical data of the static picture is basically unchanged and will also be judged as a problem scene), considering the conditions of Q2 comprehensively, if it also meets the calibration conditions of R2, then at least a fault safety problem has occurred before the statistical value module node 2 node; if Q2 does not meet condition R2, then it can be determined that a fault safety problem has occurred between the statistical value module node 2 and the statistical value module node 3.

[0147] Depending on where the failsafe problem occurs, you can first try to perform actions that can be recovered online to see if normal image processing functions can be restored. You can try the following, but you are not limited to these. Here is an example:

[0148] (1) Restart the algorithm module between the statistical value module node 1 and the statistical value module node 2.

[0149] (2) Restart the algorithm module between the statistical value module node 2 and the statistical value module node 3.

[0150] (3) The image is too dark; increase the exposure parameter setting for the next frame.

[0151] (4) The image is too bright; reduce the exposure parameter setting for the next frame.

[0152] (5) Restart the sensor and retain the parameters of each algorithm module before the restart as the initial parameters of the algorithm module after the restart.

[0153] (6) Restart the sensor using the initialization parameters for cold restart of each algorithm module.

[0154] (7) Restart the ISP.

[0155] Optionally, in this embodiment, it is also possible but not limited to using only one statistical value module node to determine whether there is a fault and whether the fault is a sensor problem or an ISP module problem.

[0156] A schematic diagram of an optional image signal processing link is shown in Figure 6 As shown, there is only one data of the statistical value node 1. You can but are not limited to using a re-injection mode of the ISP. On the ISP module, in addition to providing an input map through the sensor, you can also re-inject another fixed map. Figure 6 At the testpattern node. Other image processing algorithm modules and Figure 4 It is the same, but for the following modules, sensor raw is not the original data, and the data input by testpattern is the original data. Figure 6 As can be seen from the diagram in the figure, this process can determine whether there is a problem with the sensor image or whether it is caused by the subsequent process (the use of this mode may affect the normal processing of the sensor image, so it is necessary to look at the idle time between two normal frames for processing. When the frame rate is not very high, inserting a test frame will not affect the overall data processing process. Or when it is not necessary to run the ISP at full frame rate, or in the application scenario inside the cabin, the time and method of enabling this process can be determined according to specific needs.) The flowchart of an optional fail-safe judgment method is as follows Figure 7 shown.

[0157] Specifically, Figure 7 As shown, in normal mode, the sensor enters the diagram and enters the ISP module, and the data reaches the statistical value node 1. Similarly, the statistical data output by the hardware is further statistically calculated to obtain the data analysis situation Q1 we need, and then the distribution situation Q1 is judged. If it meets the calibration scenario R1, it is further judged whether the vehicle speed exceeds the threshold K. If it does not meet the calibration scenario R1, it means that there is no fault safety problem, and the safety mode is marked. When the vehicle speed exceeds the threshold, further judgment is required to notify the ISP and the next frame enters the testpattern mode. If the vehicle speed does not exceed the threshold, the ISP module enters the alert state and requires close detection.

[0158] When entering the testpattern test mode, a fixed raw image will be injected into the ISP, so that when it reaches the statistical value node 1, the data is basically determined. When checking the statistical value distribution Q2 at this time, if it meets the calibration scene R1, it is determined that an error occurred before the image injection node, which may be due to unreasonable exposure parameter settings. You can try to adjust the exposure parameters; if it cannot be restored after re-adjusting the exposure parameters, it may be that the sensor is blocked, and you can prompt to perform occlusion detection or de-occlusion functions; finally, you can also choose to reset the sensor; if the statistical value distribution Q2 does not meet the calibration scene R1, it means that there is an error in the ISP hardware module, and you can enable the online recovery module and enable each algorithm module; if it is invalid, you can finally reset the ISP module.

[0159] It should be noted that in the testpattern mode, the ISP platform generally has several fixed images. Of course, you can also put a slightly more complex image according to your needs, which can better show the types of statistical data anomalies. This can be placed according to your needs.

[0160] To further illustrate, when the sensor is entered into the graph, most of the data in the statistical value node 1 is very small, while in the testpattern mode, the data in the statistical value module node 1 is normal, indicating that there is a problem with the data entered into the sensor. Subsequent processing measures may include but are not limited to:

[0161] (1) Check whether the exposure parameters are set too small and try adjusting the exposure parameters;

[0162] (2) Check high-precision maps and other information to see if the vehicle is in a remote area without street lights. When the exposure is adjusted to maximum, the ISP is prompted that it is no longer able to process the vehicle on its own.

[0163] (3) Checking whether the lens is blocked or dirty, performing dirt detection or blockage detection, and subsequently performing a dirt removal or blockage removal operation;

[0164] (4) If none of the above situations are found, it is necessary to report to the superior to request restart of the sensor and ISP operations.

[0165] It should be noted that if there are problems with the statistical data in both sensor input and testpattern modes, it is presumed that there is a problem with the ISP algorithm module. Subsequent processing measures may include but are not limited to:

[0166] (1) Reset the enable of the ISP algorithm module and try to see if it can be restored to normal;

[0167] (2) Restart the ISP.

[0168] Optionally, a statistical value determination method is described below.

[0169] Method 1: Based on the distribution of brightness of each pixel

[0170] Example 1: Count the number of pixels at each brightness in order from small to large to form statistical distribution data; determine the number of pixels in the dark area in the statistical data, and if the pixels with a brightness value less than 10 (can also be set to other brightness values) account for 80% (can also be set to other percentage thresholds), there may be camera obstruction or extremely dark areas. You can check the map to determine whether it is a scene without street lights or a remote scene. If not, the exposure parameter setting may be inappropriate, and you can increase the exposure parameter; or the lens may be blocked and needs to be deblocked.

[0171] Example 2: Based on the comparison of the brightness distribution statistics of the previous and next frames, if the brightness distribution statistics change very little and the vehicle speed exceeds a certain speed, it is very likely that a fault has occurred.

[0172] Method 2: Block-based brightness value matrix

[0173] Example 1: Observe the brightness value of each block. If the number of blocks with brightness less than the threshold Y1 exceeds a certain percentage, it is very likely that a fault has occurred.

[0174] Example 2: Count the changes in the brightness of each block. If the brightness of more than a certain percentage of blocks remains almost unchanged, it is very likely that a fault has occurred.

[0175] Method 3: Statistics based on raw fields

[0176] Example 1: Taking the RGGB format as an example, if more than a certain proportion of the R, G, and B data of the blocks are smaller than the threshold m, it is very likely that a fault has occurred.

[0177] Example 2: Count the changes in the brightness of each block. If the brightness of the R, G, and B values ​​of more than a certain proportion of blocks remains almost unchanged, it is very likely that a fault has occurred.

[0178] To further illustrate, the calibration scenarios Q1 and Q2 may be at least one of the following:

[0179] (1) Among the data obtained by the current node, the brightness Y value in 90% of the block data is less than 10.

[0180] (2) Among the data obtained by the current node, the highlight data in the brightness distribution statistics table accounts for 90%.

[0181] (3) Among the data obtained by the current node, the pixels with a brightness lower than 10 in the brightness distribution statistics table account for 90%.

[0182] (4) In the data obtained by the current node, the R, G, and B values ​​are almost equal. (The image is judged to be black and white, and the color is invalid).

[0183] (5) In the data obtained by the current node, the R, G, and B values ​​are close to data overflow. (It is judged that the exposure parameters are abnormal or the camera is exposed to strong light).

[0184] It should be noted that the above parameter values ​​can be adjusted according to actual needs, and this embodiment does not limit this.

[0185] It is understandable that the above only uses one statistical value module node to determine whether there is a fault and whether the fault is a sensor problem or an ISP module problem. A fault handling method can be but is not limited to the following: Figure 8 As shown, the camera sends the compressed Sensorraw graph to the chip system SOC. In the initial raw RAW mode, the chip system SOC sends the original Raw graph to the isp module, and the statistical value module node in the isp module obtains the corresponding statistical data and sends it to the isp functional safety analysis module for analysis, and then sends the analysis result to the decision module to obtain the decision result, and the action module sends (executes) the actual action according to the decision result. And, in the testpattern mode, the chip system SOC sends the testpattern graph to the isp module, and the statistical value module node in the isp module obtains the corresponding statistical data and sends it to the isp functional safety analysis module for analysis, and then sends the analysis result to the decision module to obtain the decision result, and the action module sends (executes) the actual action according to the decision result.

[0186] To further illustrate, a camera fault handling framework diagram is as follows: Fig. 9 As shown, the camera sends the image data to the ISP functional safety system. The ISP functional safety system includes an image signal processing module and a statistical value evaluation module. The statistical value evaluation module obtains relevant data in the process of image data processing by the image signal processing module, and performs a distribution evaluation of the statistical value to obtain the fault information of the camera and send it to the decision-making system to execute relevant actual actions, such as executing online recovery operations under online recovery conditions.

[0187] It should be noted that for situations where online recovery is possible, if the statistical value is almost completely black, you can try to increase the exposure parameter first to see if the picture can be restored. If the statistical value data is close to overflow, you can try to reduce the exposure parameter first to see if the picture can be restored. The drag statistical value data hardly changes. Combined with the vehicle condition and high-precision map, etc., it is judged that the picture should be changing at this time, but the statistical value hardly changes. You can try to re-enable the statistical value module, or re-enable the algorithm module, or call the unfreeze isp program operation or add a frame test mode to determine whether there is any abnormality.

[0188] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0189] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0190] According to another aspect of the embodiments of the present application, a fault detection device for implementing the above fault detection method is also provided. Fig.10 As shown, the device comprises:

[0191] A first acquisition unit 1002 is used to acquire an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process an image signal captured by the sensor module;

[0192] A deployment unit 1004 is configured to deploy at least one statistical node in the image signal processing module, wherein the at least one statistical node is configured to capture first operation data of a first image signal input by the image signal processing module and processed by the sensor module, wherein the first operation data is configured to indicate whether the image device has a fault;

[0193] A second acquisition unit 1006 is configured to acquire, when the first operation data indicates that the image device has a fault, second operation data of an image signal processing module captured by at least one statistical node and process a second image signal, wherein the second image signal is an image signal captured by a non-sensor module;

[0194] A first determining unit 1008, configured to determine that the image signal processing module has a fault if the second operation data indicates that a fault exists;

[0195] The second determining unit 1010 is configured to determine whether a fault exists in the sensor module when the second operating data indicates that no fault exists.

[0196] As an optional solution, the deployment unit 1004 includes:

[0197] A first deployment module is used to deploy a statistical node at the output end of the image signal processing module; or,

[0198] A division module, used for dividing the image signal processing module into N regions, wherein each of the N regions includes at least one functional module of the image signal processing module, and N is a positive integer greater than 1;

[0199] The second deployment module is used to deploy a statistical node at the output end of each area.

[0200] As an optional solution, the second deployment module includes:

[0201] A deployment submodule, configured to deploy a first statistical node at an output end of the first area and a second statistical node at an output end of the second area when the N areas include a first area and a second area, wherein the second statistical node is located after the first statistical node;

[0202] The device also includes:

[0203] A first capturing module is used for capturing, at a first statistical node, first operation sub-data of a first image signal processed by an image signal processing module after a statistical node is deployed at an output end of each region, wherein the first operation data includes first operation sub-data;

[0204] The first determination module is configured to, after deploying a statistical node at the output end of each region, determine that the first operation data indicates that the image device has a fault when the first operation sub-data indicates that a fault exists.

[0205] As an optional solution, the device also includes:

[0206] a second capturing module, configured to capture, at the second statistical node, second operating sub-data of the first image signal processed by the image signal processing module during the process in which the first operating sub-data of the first image signal is captured by the first statistical node, wherein the first operating data includes the second operating sub-data;

[0207] a second determining module, configured to determine, during a process in which the first statistical node captures first operating sub-data of the image signal processing module processing the first image signal, that the first operating sub-data indicates that there is no fault and the second operating sub-data indicates that there is a fault, that the first operating data indicates that there is a fault in the second area of ​​the image signal processing module;

[0208] The third determination module is used to determine that the first operation data indicates that there is no fault in the image device when the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is no fault during the process in which the first statistical node captures the first operation sub-data of the image signal processing module processing the first image signal, and when the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is no fault.

[0209] As an optional solution, the device further includes at least one of the following:

[0210] A third capturing module is configured to capture, at the at least one statistical node, first brightness distribution data based on pixels during the process of the image signal processing module processing the first image signal after the image signal processing module deploys at least one statistical node, wherein the first operation data includes the first brightness distribution data, and the first brightness distribution data is used to indicate the number of pixels at each brightness;

[0211] a fourth capturing module, configured to capture, at the at least one statistical node, second brightness distribution data based on blocks during processing of the first image signal by the image signal processing module after the image signal processing module deploys at least one statistical node, wherein the first operation data includes the second brightness distribution data, and the second brightness distribution data is used to indicate a block proportion at each brightness;

[0212] A fifth capture module is used to capture, after at least one statistical node is deployed in the image signal processing module, third brightness distribution data based on the original data domain during the process of the image signal processing module processing the first image signal, at at least one statistical node, wherein the first operating data includes the third brightness distribution data, and the third brightness distribution data is used to indicate statistical information of the RGB value of the block.

[0213] As an optional solution, the device also includes:

[0214] a fourth determination module, configured to, after the image signal processing module deploys at least one statistical node, determine that the first brightness distribution data indicates that there is a fault when the first brightness distribution data indicates that there are at least a first proportion of pixels whose brightness is lower than the first threshold, or when the change amplitude of the first brightness distribution data in M ​​consecutive frames is less than the first change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold;

[0215] a fifth determination module, configured to, after the image signal processing module deploys at least one statistical node, determine that the second brightness distribution data indicates that there is a fault when the second brightness distribution data indicates that there is at least a second proportion of blocks whose brightness is lower than the second threshold, or when the change amplitude of the second brightness distribution data in M ​​consecutive frames is less than the second change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold;

[0216] The sixth determination module is used to determine, after the image signal processing module deploys at least one statistical node, that the third brightness distribution data indicates that there is a fault when the third brightness distribution data indicates that there is at least a third proportion of blocks whose RGB values ​​are lower than a third threshold, or when the change amplitude of the third brightness distribution data in M ​​consecutive frames is less than the third change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold.

[0217] As an optional solution, the device also includes:

[0218] A recovery module, configured to perform an online recovery operation on the image signal processing module after determining that the image signal processing module has a fault, wherein the online recovery operation is used to enable at least one functional module of the image signal processing module;

[0219] A reset module, used for resetting the image signal processing module after determining that the image signal processing module has a fault and when an online recovery operation is invalid;

[0220] The device also includes:

[0221] An adjustment module, used for performing a parameter adjustment operation on the sensor module after determining that the sensor module has a fault, wherein the parameter adjustment is used to indicate adjustment of an exposure parameter of the sensor module;

[0222] The reset module is used to reset the sensor module after determining that the sensor module has a fault and when the parameter adjustment operation is invalid.

[0223] For specific embodiments, reference may be made to the examples shown in the above fault detection method, which will not be described in detail in this example.

[0224] According to another aspect of the embodiments of the present application, an electronic device for implementing the above fault detection method is also provided. Fig.11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the above method embodiments through the computer program.

[0225] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0226] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0227] S1, obtaining an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process an image signal captured by the sensor module;

[0228] S2, deploying at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture the first operation data of the first image signal input by the sensor module and processed by the image signal processing module, and the first operation data is used to indicate whether the image device has a fault;

[0229] S3, when the first operation data indicates that the image device has a fault, obtaining an image signal processing module captured by at least one statistical node, and processing second operation data of a second image signal, wherein the second image signal is an image signal captured by a non-sensor module;

[0230] S4, determining that the image signal processing module has a fault when the second operation data indicates that there is a fault;

[0231] S5 : When the second operation data indicates that there is no fault, determine that there is a fault in the sensor module.

[0232] Alternatively, a person skilled in the art may understand that: Fig.11 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.11 The structure of the electronic device is not limited. Fig.11 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.11 Different configurations are shown.

[0233] Among them, the memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the fault detection method and device in the embodiments of the present application. The processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, that is, to implement the above-mentioned fault detection method. The memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include a memory remotely located relative to the processor 1104, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1102 can be specifically, but not limited to, used to store first operating data, second operating data and other information. As an example, such as Fig.11 As shown, the memory 1102 may include, but is not limited to, the first acquisition unit 1002, the deployment unit 1004, the second acquisition unit 1006, the first determination unit 1008, and the second determination unit 1010 in the fault detection device. In addition, it may also include, but is not limited to, other module units in the fault detection device, which will not be repeated in this example.

[0234] Optionally, the transmission device 1106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 806 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0235] In addition, the electronic device further includes: a display 1108 for displaying information such as the first output result and the second output result; and a connection bus 1110 for connecting various module components in the electronic device.

[0236] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, the nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0237] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0238] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0239] It should be noted that the computer system of the electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0240] The computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) or the program loaded from the storage part to the random access memory (RAM). In the random access memory, various programs and data required for system operation are also stored. The central processing unit, the read-only memory and the random access memory are connected to each other through a bus. The input / output interface (I / O interface) is also connected to the bus.

[0241] The following components are connected to the input / output interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a local area network card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0242] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions defined in the system of the present application are executed.

[0243] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0244] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0245] S1, obtaining an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process an image signal captured by the sensor module;

[0246] S2, deploying at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture the first operation data of the first image signal input by the sensor module and processed by the image signal processing module, and the first operation data is used to indicate whether the image device has a fault;

[0247] S3, when the first operation data indicates that the image device has a fault, obtaining an image signal processing module captured by at least one statistical node, and processing second operation data of a second image signal, wherein the second image signal is an image signal captured by a non-sensor module;

[0248] S4, determining that the image signal processing module has a fault when the second operation data indicates that there is a fault;

[0249] S5 : When the second operation data indicates that there is no fault, determine that there is a fault in the sensor module.

[0250] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0251] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0252] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0253] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0254] In the several embodiments provided in the present application, it should be understood that the device embodiments described above are only schematic, such as the division of units, which is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0255] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0256] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0257] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A fault detection method, characterized in that: include: Acquire an image device to be detected, wherein the image device includes a sensor module and an image signal processing module, and the image signal processing module is used to process the image signal captured by the sensor module; At least one statistical node is deployed in the image signal processing module, wherein the at least one statistical node is used to capture first operation data of the image signal processing module processing the first image signal input by the sensor module, wherein the first operation data is used to indicate whether the image device has a fault; when the first operation data indicates that the image device has a fault, second operation data of the image signal processing module processing the second image signal captured by the at least one statistical node is obtained, wherein the second image signal is an image signal not captured by the sensor module; If the second operation data indicates that a fault exists, determining that the image signal processing module has a fault; In the event that the second operating data indicates that no fault exists, it is determined that the sensor module is faulty.

2. The method according to claim 1, characterized in that Deploying at least one statistical node in the image signal processing module includes: Deploy a statistical node at the output end of the image signal processing module; or, Dividing the image signal processing module to obtain N areas, wherein each of the N areas includes at least one functional module of the image signal processing module, and N is a positive integer greater than 1; A statistics node is deployed at the output end of each area.

3. The method according to claim 2, characterized in that Deploying a statistical node at the output end of each area includes: In the case where the N areas include a first area and a second area, a first statistical node is deployed at an output end of the first area, and a second statistical node is deployed at an output end of the second area, wherein the second statistical node is located after the first statistical node; After deploying a statistical node at the output end of each area, the method further includes: capturing, at the first statistical node, first operating sub-data of the first image signal processed by the image signal processing module, wherein the first operating data includes the first operating sub-data; In the event that the first operation sub-data indicates that a fault exists, it is determined that the first operation data indicates that a fault exists in the imaging device.

4. The method according to claim 3, characterized in that In the process where the first statistical node captures the first running sub-data of the first image signal processed by the image signal processing module, the method further includes: capturing, at the second statistical node, second running sub-data of the first image signal processed by the image signal processing module, wherein the first running data includes the second running sub-data; If the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is a fault, determining that the first operation data indicates that there is a fault in the second area of ​​the image signal processing module; In a case where the first operation sub-data indicates that there is no fault and the second operation sub-data indicates that there is no fault, it is determined that the first operation data indicates that there is no fault in the imaging device.

5. The method according to claim 1, characterized in that After the image signal processing module deploys at least one statistical node, the method further includes at least one of the following: In the at least one statistical node, first brightness distribution data based on pixels is captured during the process of the image signal processing module processing the first image signal, wherein the first operating data includes the first brightness distribution data, and the first brightness distribution data is used to indicate the number of pixels at each brightness; in the at least one statistical node, second brightness distribution data based on blocks is captured during the process of the image signal processing module processing the first image signal, wherein the first operating data includes the second brightness distribution data, and the second brightness distribution data is used to indicate the block proportion at each brightness; in the at least one statistical node, third brightness distribution data based on the original data domain is captured during the process of the image signal processing module processing the first image signal, wherein the first operating data includes the third brightness distribution data, and the third brightness distribution data is used to indicate statistical information of the RGB value of the block.

6. The method according to claim 5, characterized in that After the image signal processing module deploys at least one statistical node, the method further includes: When the first brightness distribution data indicates that there is at least a first proportion of pixels whose brightness is lower than a first threshold, or when the change amplitude of the first brightness distribution data in M ​​consecutive frames is less than a first change threshold and the image device is in a motion state with a motion speed greater than a preset speed threshold, it is determined that the first brightness distribution data indicates that there is a fault; When the second brightness distribution data indicates that there is at least a second proportion of blocks whose brightness is lower than a second threshold, or when the change amplitude of the second brightness distribution data in the continuous M frames is less than a second change threshold and the image device is in a motion state with a motion speed greater than the preset speed threshold, it is determined that the second brightness distribution data indicates a fault; When the third brightness distribution data indicates that there are at least a third proportion of blocks with RGB values ​​lower than the third threshold, or when the change amplitude of the third brightness distribution data in consecutive M frames is less than the third change threshold and the image device is in a motion state with a motion speed greater than the preset speed threshold, it is determined that the third brightness distribution data indicates a fault.

7. The method according to any one of claims 1 to 6, characterized in that: After determining that the image signal processing module has a fault, the method further includes: Performing an online recovery operation on the image signal processing module, wherein the online recovery operation is used to enable at least one functional module of the image signal processing module; When the online recovery operation is invalid, resetting the image signal processing module; After determining that the sensor module is faulty, the method further includes: Performing a parameter adjustment operation on the sensor module, wherein the parameter adjustment is used to indicate adjusting an exposure parameter of the sensor module; When the parameter adjustment operation is invalid, a reset operation is performed on the sensor module.

8. A fault detection device, characterized in that: include: A first acquisition unit, configured to acquire an image device to be detected, wherein the image device comprises a sensor module and an image signal processing module, and the image signal processing module is configured to process an image signal captured by the sensor module; a deployment unit, configured to deploy at least one statistical node in the image signal processing module, wherein the at least one statistical node is used to capture first operation data of the image signal processing module for processing a first image signal input by the sensor module, wherein the first operation data is used to indicate whether the image device has a fault; a second acquisition unit, configured to acquire, when the first operation data indicates that the image device has a fault, second operation data of the image signal processing module captured by the at least one statistical node and processing a second image signal, wherein the second image signal is an image signal not captured by the sensor module; a first determining unit, configured to determine that the image signal processing module has a fault if the second operation data indicates that the fault exists; The second determining unit is configured to determine that a fault exists in the sensor module when the second operating data indicates that a fault does not exist.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images