Camera fault detection method and device, vehicle and medium

Abstract

This application relates to the field of vehicle technology and discloses a method, device, vehicle, and medium for camera fault detection. The method includes: acquiring an occlusion detection control command; responding to the occlusion detection control command to perform occlusion detection on the camera and obtain an occlusion detection result; determining whether related equipment associated with the occlusion detection has malfunctioned and obtaining a fault diagnosis result; and displaying the occlusion detection result and the fault diagnosis result on an in-vehicle display screen to locate the cause of the camera fault. This application enables debugging and diagnostic personnel to display the occlusion detection result and the fault diagnosis result on an in-vehicle display screen, improving the speed of fault location and shortening the problem troubleshooting cycle.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, specifically to camera fault detection methods, devices, vehicles, and media. Background Technology

[0002] With the rapid development of intelligent driving technology, the Driver Monitoring System (DMS) has become a core in-vehicle component for ensuring driving safety. The DMS uses cameras to capture real-time images of the driver and determine if the camera is obstructed. Therefore, debugging and diagnosing the cameras and related equipment in the DMS to determine if these devices are faulty and to pinpoint the cause of camera malfunctions is a crucial step in ensuring the reliable operation of the DMS.

[0003] In the process of realizing this application, the inventors discovered that traditional solutions for debugging and diagnosing devices such as cameras require external dedicated equipment to read the operating logs of the cameras and other devices after debugging and diagnosis. This means that the fault location can only be determined by backtracking the logs, resulting in low fault diagnosis efficiency and long problem location cycle. Summary of the Invention

[0004] In view of this, embodiments of this application provide a camera fault detection method, device, vehicle, and medium to solve the problems of low fault diagnosis efficiency and long problem location cycle in traditional solutions.

[0005] In a first aspect, embodiments of this application provide a camera fault detection method, the method comprising: Obtain occlusion detection control instructions; In response to the occlusion detection control command, the camera is subjected to occlusion detection, and the occlusion detection result is obtained; Determine whether any associated equipment related to occlusion detection has malfunctioned, and obtain fault diagnosis results; The occlusion detection results and fault diagnosis results are displayed on the vehicle-mounted display screen to pinpoint the cause of the camera malfunction.

[0006] The method provided in this application acquires occlusion detection control commands, performs occlusion detection on the camera according to the occlusion detection control commands, and obtains occlusion detection results. It then performs debugging and diagnosis on associated devices related to occlusion detection to determine whether these devices have malfunctioned, obtaining fault diagnosis results. The occlusion detection results and fault diagnosis results are intuitively displayed on an in-vehicle display screen. This allows for direct fault location on the in-vehicle terminal when associated devices related to occlusion detection malfunction during fault diagnosis, without relying on external dedicated equipment or background log backtesting analysis. This effectively solves the shortcomings of traditional solutions, such as low fault diagnosis efficiency and long problem location cycles, significantly improving fault diagnosis efficiency.

[0007] In one optional implementation, the associated device includes a fill light, and the step of performing occlusion detection on the camera and obtaining the occlusion detection result includes: The brightness of the supplementary light, the working mode of the camera, and the occlusion detection threshold are adjusted; the supplementary light is used to provide supplementary lighting when the camera acquires image data inside the vehicle; wherein, the occlusion detection threshold is used to determine the occlusion status of the camera; Acquire image data captured by the camera; Based on the adjusted occlusion detection threshold, the occlusion region is identified in the image data to obtain the occlusion detection result.

[0008] The method provided in this application embodiment can adjust the brightness of the fill light, switch the working mode of the camera, adjust the occlusion detection threshold, acquire image data collected by the camera, and complete the occlusion area identification based on the adjusted occlusion detection threshold to complete the occlusion detection of the camera. It can avoid false detection and missed detection problems caused by parameter mismatch and realize flexible adjustment of working mode, judgment threshold, etc. in the occlusion detection process.

[0009] In one optional implementation, adjusting the brightness of the fill light, the operating mode of the camera, and the occlusion detection threshold includes: The system acquires a brightness adjustment command, a working mode switching command, and a threshold adjustment command; wherein the brightness adjustment command carries a brightness adjustment amount; the working mode switching command carries the working mode after the camera switches to the working mode; and the threshold adjustment command carries a threshold adjustment amount for the occlusion detection threshold. The brightness of the fill light, the working mode of the camera, and the occlusion detection threshold are adjusted according to the brightness adjustment command, the working mode switching command, and the threshold adjustment command, respectively.

[0010] The method provided in this application obtains the brightness adjustment command of the fill light, the working mode switching command of the camera, and the threshold adjustment command of the occlusion detection threshold. By using the above commands as occlusion detection control commands, the parameters of the fill light, camera, and occlusion detection threshold can be directly adjusted on the vehicle display screen. The occlusion detection parameters can be flexibly configured without the need for external debugging tools, adapting to the occlusion detection needs in different scenarios, simplifying the parameter adjustment process, and further improving the flexibility and adaptability of occlusion detection function debugging.

[0011] In one optional implementation, the associated device includes a supplemental light and at least one vehicle-mounted control device related to occlusion detection. The step of determining whether the associated device related to occlusion detection has malfunctioned and obtaining a fault diagnosis result includes: Based on the occlusion detection result, the image data collected by the camera is rendered to obtain the rendered image data. Determine the working status data of the fill light; the working status data includes: the expected on / off state, the actual on / off state, and the working mode of the fill light; Acquire link communication data between the vehicle control devices; the link communication data includes the connectivity status and transmission delay between the vehicle control devices. The image data after image rendering, the working status data, and the link communication data are used as the fault diagnosis results.

[0012] The method provided in this application embodiment performs image rendering processing on the image data collected by the camera based on the occlusion detection results, obtains the working status data of the supplementary light and the link communication data between the vehicle control equipment, and integrates the above data into the fault diagnosis results. It can collect the operation and communication information of the camera, supplementary light and vehicle control equipment in all aspects, fully present the status of the entire occlusion detection process, and intuitively reflect problems such as supplementary light failure and link communication abnormality. It can complete multi-dimensional diagnosis without disassembling the device or connecting external diagnostic equipment, realize rapid fault location, and improve the comprehensiveness and intuitiveness of debugging and diagnosis.

[0013] In one optional implementation, displaying the occlusion detection results and the fault diagnosis results on the vehicle-mounted display screen to locate the cause of the camera malfunction includes: The vehicle-mounted display screen shows a debugging and diagnostic interface; wherein the debugging and diagnostic interface includes: a first area and a second area; The occlusion detection result is displayed in the first area, and the fault diagnosis result is displayed in the second area.

[0014] The method provided in this application embodiment displays a segmented debugging and diagnostic interface on an in-vehicle display screen. By displaying the occlusion detection results in the first area and the fault diagnosis results in the second area, the result display and diagnosis presentation can be integrated into the same visual interface, which is more intuitive and orderly. The entire process information can be obtained without switching between multiple interfaces or tools, and the visualization and control of debugging and diagnosis can be realized.

[0015] In one optional implementation, the debugging and diagnostic interface further includes a third area, which is used to acquire the brightness adjustment amount of the fill light, the working mode of the camera, and the threshold adjustment amount of the occlusion detection threshold.

[0016] The method provided in this application embodiment allows for the adjustment of the fill light brightness, camera working mode, and occlusion detection threshold by displaying a regional debugging and diagnostic interface on the vehicle display screen, which is more convenient.

[0017] In an optional implementation, the debugging and diagnostic interface further includes a fourth area, and the method further includes: Acquire at least one target event during the occlusion detection process and the fault diagnosis process; the target event includes a first event of a change in the state of the camera and the associated device, and a second event of a change in the occlusion detection result and the fault diagnosis result; The fourth area displays an event timeline and the occurrence times of at least one target event corresponding to the event timeline. When a selection operation is detected for any of the target events, the detailed information of the corresponding target event is displayed.

[0018] The method provided in this application embodiment acquires target events during the occlusion detection process and fault diagnosis process. The event timeline and the corresponding event occurrence time are displayed in the fourth area of ​​the debugging and diagnosis interface. Selecting the target event allows viewing detailed information. It can completely record and present the changes in the equipment operating status, detection and diagnosis results, sort out key events in the time dimension, support the retrospective viewing of event details, facilitate the analysis of the triggering timing and correlation of abnormal events, quickly locate the cause of the fault, make the fault diagnosis process traceable and analyzable, improve the accuracy and efficiency of problem investigation, and improve the recording and analysis capabilities of debugging and diagnosis.

[0019] In an optional implementation, before determining whether the associated device related to the occlusion detection has malfunctioned and obtaining a fault diagnosis result, the method further includes: Generate a device testing process for the camera and the associated device; A device test button associated with the device test process is provided on the vehicle-mounted display screen. When the device test button is triggered, the step of determining whether the associated device related to the occlusion detection has malfunctioned and obtaining the fault diagnosis result is triggered.

[0020] The method provided in this application generates a device testing process for cameras and associated devices. A corresponding device testing button is set on the vehicle display screen. After the device testing button is triggered, link connectivity testing and functional testing are automatically carried out and a comprehensive test report is generated. It can complete the full-dimensional testing of occlusion detection related devices in an automated manner, avoiding the omissions and inefficiencies of manual item-by-item testing. It can complete the full-link health check and functional verification with one click, greatly simplifying the testing process, reducing the threshold for debugging and diagnosis operations, and improving the efficiency and standardization of device testing.

[0021] Secondly, embodiments of this application provide a camera fault detection device, the device comprising: The first processing module is used to acquire occlusion detection control instructions; The second processing module is used to respond to the occlusion detection control command, perform occlusion detection on the camera, and obtain the occlusion detection result. The third processing module is used to determine whether the associated equipment related to occlusion detection has malfunctioned and to obtain the fault diagnosis result. The fourth processing module is used to display the occlusion detection results and the fault diagnosis results on the vehicle-mounted display screen to locate the cause of the camera malfunction.

[0022] Thirdly, embodiments of this application provide a vehicle, including: an in-vehicle display screen and a controller; the controller includes: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the camera fault detection method of the first aspect or any of its corresponding optional embodiments described above.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions for causing a computer to execute the camera fault detection method of the first aspect or any of its corresponding optional embodiments.

[0024] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the camera fault detection method described in the first aspect or any of its corresponding optional embodiments. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the camera fault detection method according to an embodiment of this application; Figure 2 This is a hardware link diagram corresponding to the camera fault detection method in this application embodiment; Figure 3 This is a schematic diagram of the debugging and diagnostic interface according to an embodiment of this application; Figure 4 This is a schematic diagram of the LED status monitoring area according to an embodiment of this application; Figure 5 This is a schematic diagram of the DMS real-time screen area according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware link state area in an embodiment of this application; Figure 7 This is a structural block diagram of a camera fault detection device according to an embodiment of this application; Figure 8 This is a schematic diagram of the hardware structure of the vehicle according to an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It is understood that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, and usage scenarios of the personal information involved in this application should be informed to the debugging and diagnostic personnel in an appropriate manner in accordance with relevant laws and regulations, and authorization from the debugging and diagnostic personnel should be obtained.

[0029] Therefore, this application provides an embodiment of a camera fault detection method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides a camera fault detection method. Figure 1 This is a flowchart illustrating the camera fault detection method according to an embodiment of this application, as shown below. Figure 1 As shown, the process includes the following steps: S101, Obstruction detection control command.

[0031] In this embodiment, occlusion detection control commands can be input via an in-vehicle display screen, which is a touch display device driven by a vehicle cockpit domain controller.

[0032] This embodiment allows for direct input of occlusion detection control commands via the vehicle-mounted display screen, eliminating reliance on external debugging equipment and enabling real-time control of occlusion detection parameters on the vehicle side, thus improving the convenience and real-time nature of command issuance.

[0033] S102 responds to the occlusion detection control command, performs occlusion detection on the camera, and obtains the occlusion detection result.

[0034] In this embodiment, the occlusion detection result includes the occlusion state and occlusion rate of the image data. The occlusion state includes no occlusion, slight occlusion, partial occlusion, severe occlusion, and complete occlusion. The occlusion rate is the ratio of the occluded area to the effective detection area.

[0035] The camera can be a DMS infrared camera, used to capture infrared image data of the driver's facial area.

[0036] S103, determine whether the associated equipment related to the occlusion detection has malfunctioned, and obtain the fault diagnosis result.

[0037] In this embodiment, the fault diagnosis result includes image data after image rendering processing of the image data collected by the camera based on the occlusion detection result, the working status data of the supplementary light, and the link communication data between the vehicle control devices. In this embodiment, the image data after image rendering processing is synchronously overlaid in the form of a real-time video stream. The debugging and diagnostic interface of the vehicle display also supports screen zooming.

[0038] In this embodiment, the associated devices for occlusion detection include a supplementary light and at least one vehicle-mounted control device related to occlusion detection. The supplementary light provides illumination when the camera captures image data from inside the vehicle. The vehicle-mounted control device includes a driver-specific SoC (SoC controller, SoC detector), an MCU controller, and a cockpit-specific controller. The driver-specific SoC controller manages the camera's output format, exposure time, frame rate, and resolution. The driver-specific SoC detector receives image data, runs the occlusion detection algorithm, and outputs the occlusion detection result. The MCU controller controls the power supply and PWM dimming of the supplementary light. The cockpit-specific controller drives the vehicle-mounted display screen, providing rendering, interaction, and data display for the debugging and diagnostic interface. The driver-specific SoC detector's function of receiving image data, running the occlusion detection algorithm, and outputting the occlusion detection result is a conventional technique in the art and will not be described in detail here.

[0039] In this embodiment of the application, as an example, the camera may be a DMS infrared camera, used to collect infrared image data of the driver's facial area.

[0040] S104 displays the obstruction detection results and fault diagnosis results on the vehicle-mounted display screen to pinpoint the cause of the camera malfunction.

[0041] In this embodiment, the cause of a camera malfunction can be located based on the occlusion detection results and fault diagnosis results displayed on the vehicle-mounted display screen. For example, if the vehicle-mounted display screen shows a black screen with a communication interruption, it indicates that the communication link between the camera and the vehicle-mounted control device is abnormal. If the vehicle-mounted display screen shows an occlusion detection result, and the supplementary light is expected to be on but is actually off, it indicates that the supplementary light is not working properly, leading to a false occlusion detection.

[0042] In this embodiment, the occlusion detection results and fault diagnosis results are displayed on an in-vehicle display screen to pinpoint the cause of camera malfunctions. Specifically, the occlusion detection results and fault diagnosis results are displayed in separate areas on the debugging and diagnostic interface of the in-vehicle display screen. Debugging and diagnostic personnel or after-sales service personnel can intuitively view all information through the debugging and diagnostic interface, quickly determining the location and degree of occlusion, whether the supplementary lighting is faulty, and whether the link is interrupted, etc.

[0043] This embodiment integrates and visualizes the occlusion detection results and fault diagnosis results on the vehicle-mounted display screen. It can directly complete the anomaly location and cause judgment on the vehicle terminal without relying on the backend log back-tracking analysis. It effectively solves the shortcomings of traditional solutions in terms of low fault diagnosis efficiency and long problem location cycle, and significantly improves the efficiency and convenience of debugging and diagnosis.

[0044] This application embodiment obtains occlusion detection control commands, performs occlusion detection on the camera according to the occlusion detection control commands, and obtains occlusion detection results. It then performs debugging and diagnosis on associated devices related to occlusion detection to determine whether any of these devices have malfunctioned, obtaining fault diagnosis results. The occlusion detection results and fault diagnosis results are intuitively displayed on an in-vehicle display screen. This allows for direct location of the cause of the anomaly on the in-vehicle terminal when associated devices related to occlusion detection malfunction or fail during the fault diagnosis process, without relying on external dedicated equipment or background log backtracking analysis. This effectively solves the shortcomings of traditional solutions, such as low fault diagnosis efficiency and long problem location cycles, significantly improving fault diagnosis efficiency.

[0045] The hardware link diagram corresponding to the camera fault detection method in this application is as follows: Figure 2 As shown, it includes a module terminal, an autonomous driving control domain, and a cockpit domain controller. The modules interact and coordinate with each other through dedicated interfaces and networks.

[0046] The module includes a camera and a fill light. The camera transmits the acquired image data to the serializer via a MIPI interface, providing raw image information for occlusion detection and fault diagnosis.

[0047] The LED Driver IC is used to receive control commands from the serializer / MCU, drive the fill light to work, and at the same time, feed back the actual on / off status of the fill light to the serializer.

[0048] The serializer encodes and transmits the camera's image data to the autonomous driving control domain. Simultaneously, it receives status feedback signals from the LED Driver IC and forwards them to the backend deserializer. The MCU provides power control for the supplementary lights via the I / O interface, enabling the lights to start / stop and manage their power status.

[0049] The deserializer can receive image data and status signals transmitted by the serializer, decode them, and forward them to the SoC. Simultaneously, it sends initialization configuration commands to the module to complete the module's startup and parameter configuration.

[0050] The MCU is responsible for power supply control, basic status acquisition, and command forwarding at the module level, providing I / O power control signals for the fill light. The System-on-Chips (SoC) receives image data from the deserializer via the MCI interface and simultaneously obtains the actual on / off status of the fill light through the LED status detection interface. It then uses internal algorithms to perform occlusion detection, fault diagnosis, and threshold adjustment. It can also receive parameter adjustment commands from the cockpit domain controller to update the occlusion detection threshold.

[0051] The cockpit domain controller and the autonomous driving control domain achieve bidirectional communication via the in-vehicle network. It sends occlusion detection threshold adjustment commands and parameter configuration commands to the SoC to modify parameters during the debugging process. It receives camera image data, expected / actual on / off status of the auxiliary lights, occlusion detection results, etc., from the SoC, and finally visualizes them on the in-vehicle display, enabling in-vehicle debugging and diagnostics without external devices.

[0052] In one optional implementation, the associated device includes a fill light, and S102 performs occlusion detection on the camera to obtain an occlusion detection result, including: The brightness of the supplementary light, the operating mode of the camera, and the occlusion detection threshold are adjusted. The supplementary light is used to provide illumination when the camera acquires image data inside the vehicle. The occlusion detection threshold is used to determine the occlusion status of the camera.

[0053] Acquire image data captured by the camera.

[0054] Based on the adjusted occlusion detection threshold, the occlusion region is identified in the image data to obtain the occlusion detection result.

[0055] In this embodiment, the brightness adjustment command, working mode switching command, and threshold adjustment command belong to occlusion detection control commands. As an example, occlusion detection control commands include at least one of the following: a brightness adjustment command for the supplementary light, a camera IR (InFrared) / RGB (Red Green Blue) working mode switching command, and an occlusion detection threshold adjustment command. Diagnostic personnel can directly input and issue occlusion detection control commands through interactive controls such as virtual buttons, sliders, and touch switches on the vehicle display screen. These commands are transmitted sequentially along the link between the cockpit domain controller, the intelligent driving domain SoC, and the MCU controller.

[0056] In one optional implementation, the brightness of the supplementary light, the operating mode of the camera, and the occlusion detection threshold are adjusted. The supplementary light is used to provide supplementary illumination when the camera acquires image data inside the vehicle, specifically including: The system acquires a brightness adjustment command, a working mode switching command, and a threshold adjustment command; wherein the brightness adjustment command carries a brightness adjustment amount; the working mode switching command carries the working mode after the camera switches to the working mode; and the threshold adjustment command carries the threshold adjustment amount of the occlusion detection threshold.

[0057] The brightness of the fill light, the working mode of the camera, and the occlusion detection threshold are adjusted according to the brightness adjustment command, the working mode switching command, and the threshold adjustment command, respectively.

[0058] In this embodiment, the brightness adjustment amount, the working mode of the camera after switching working modes, and the threshold adjustment amount of the occlusion detection threshold can be determined by manual touch input on the vehicle display screen, or can be automatically calculated and generated by the vehicle system.

[0059] As an example, the brightness of the fill light can be adjusted by touching the slider, numerical input box, or gear switching control to generate corresponding brightness adjustment commands. The brightness adjustment commands are used to adjust the brightness of the fill light to adapt to the camera image acquisition needs under different ambient light intensities, and avoid misjudgment or missed judgment of occlusion detection due to excessive or insufficient fill light.

[0060] As an example, the camera's operating modes can include IR infrared mode, RGB visible light mode, etc. The operating mode can be selected via touch switch, mode selection button or drop-down menu to generate corresponding operating mode switching commands.

[0061] As an example, the occlusion detection threshold can be adjusted using a numerical slider, percentage adjustment control, or threshold level selector to obtain the threshold adjustment amount. Based on the threshold adjustment amount, a corresponding threshold adjustment command is generated. The threshold adjustment command is used to adjust the judgment criteria for occlusion detection to avoid frequent changes in occlusion status or detection lag.

[0062] In this embodiment of the application, as an example, when the debugging and diagnostic personnel observe that the image captured by the camera is too dark / overexposed, it is determined that the brightness of the fill light needs to be adjusted. When the debugging and diagnostic personnel observe that the image captured by the camera has uniform brightness and clear details, it is determined that the brightness of the fill light does not need to be adjusted.

[0063] When it is determined that the brightness of the fill light needs to be adjusted, a brightness adjustment command can be generated based on the target brightness value corresponding to the touch sliding position on the touch slider by the debugging and diagnostic personnel.

[0064] As another example, the brightness of the fill light can be adjusted based on factors such as the average brightness value and image signal-to-noise ratio of the image data captured by the camera.

[0065] For example, if the average brightness value is lower than a preset lower limit or higher than a preset upper limit, it is determined that the brightness of the supplementary light needs to be adjusted. If the average brightness value is within a preset reasonable range, it is determined that the brightness of the supplementary light does not need to be adjusted. The minimum value of the preset reasonable range is the preset lower limit, and the maximum value is the preset upper limit. The preset lower limit and preset upper limit can be set and modified according to actual needs.

[0066] When it is determined that the brightness of the fill light needs to be adjusted, the brightness adjustment amount can be determined based on the difference between the median and the average brightness value in a preset reasonable range, and a brightness adjustment command can be generated based on the brightness adjustment amount.

[0067] In this embodiment of the application, as an example, when the debugging and diagnostic personnel observe a decrease in the image clarity of the image displayed on the vehicle's display screen in real time, it is determined that the working mode of the camera needs to be switched.

[0068] For example, if it is determined that the camera's operating mode needs to be switched, a corresponding operating mode switching command can be generated based on the operating mode selected by the debugging and diagnostic personnel via the touch switch.

[0069] As another example, the operating mode of the camera can be switched based on the image clarity data monitored by the vehicle.

[0070] If the image clarity in the current working mode is less than a preset threshold, it is determined that the working mode of the camera needs to be switched.

[0071] For example, when it is determined that the camera's operating mode needs to be switched, the image sharpness level corresponding to each operating mode under different environmental scenarios can be preset. The current environmental data (such as light intensity) is acquired and matched with various environmental scenarios to determine the environmental scenario to which the current environmental data belongs. The image sharpness level corresponding to each operating mode under the current environmental scenario is then obtained. The operating mode with an image sharpness level one level higher than the current operating mode is selected as the target operating mode, and a operating mode switching command is generated, with the target operating mode being the switched-off operating mode.

[0072] If the image clarity in the current operating mode is greater than or equal to a preset threshold, it is determined that there is no need to switch the camera's operating mode. Simultaneously, the in-vehicle display can also update and display the camera's current operating mode indicator in real time; for example, the current operating mode indicator can be displayed in the upper center of the image, such as in a DMS Camera IR live feed.

[0073] In this embodiment, the occlusion detection threshold includes an occlusion determination threshold and an occlusion hysteresis threshold. The occlusion determination threshold is used to distinguish whether an image meets the occlusion determination conditions, thereby determining whether the camera is occluded. The occlusion hysteresis threshold is used to avoid frequent changes in occlusion state and improve detection stability. As an example, the initial value of the occlusion detection threshold is a preset default value. Whether to adjust the occlusion determination threshold can be determined based on manual adjustment instructions from debugging and diagnostic personnel, the number of false occlusion detection triggers within a preset time period, the frequency of occlusion state changes, etc.

[0074] The system can generate corresponding threshold adjustment commands based on the threshold adjustment amounts corresponding to the occlusion detection threshold and the occlusion hysteresis threshold entered by the debugging and diagnostic personnel through the numerical input boxes. Alternatively, it can directly generate corresponding threshold adjustment commands based on the occlusion detection threshold and the occlusion hysteresis threshold selected by the debugging and diagnostic personnel through the numerical slider.

[0075] When the number of false occlusion detection triggers detected by the vehicle exceeds a preset threshold, it can be determined that the occlusion judgment threshold needs adjustment. If adjustment is required, the corresponding adjustment amount can be determined based on the number of false occlusion detection triggers. A target occlusion judgment threshold can be determined based on the current threshold and the adjustment amount, and a threshold adjustment command can be generated based on the target threshold. Different adjustment amounts corresponding to different numbers of false occlusion detection triggers can be preset. Conversely, if the number of false occlusion detection triggers detected by the vehicle is less than or equal to a preset threshold, it can be determined that no adjustment is needed.

[0076] When the vehicle detects an occlusion state transition frequency greater than a preset frequency threshold, it can be determined that the occlusion hysteresis threshold needs adjustment. If adjustment is required, the corresponding hysteresis adjustment amount can be determined based on the occlusion state transition frequency. A target occlusion hysteresis threshold can be determined based on the current occlusion hysteresis threshold and the hysteresis adjustment amount, and a threshold adjustment command can be generated based on the target occlusion hysteresis threshold. Different hysteresis adjustment amounts corresponding to different occlusion state transition frequencies can be preset. Conversely, if the vehicle detects an occlusion state transition frequency less than or equal to a preset frequency threshold, it can be determined that no adjustment to the occlusion detection threshold is necessary.

[0077] This application embodiment inputs brightness adjustment commands for the supplementary light, working mode switching commands for the camera, and threshold adjustment commands for the occlusion detection threshold from the associated device via the vehicle display screen. These commands are used as occlusion detection control commands, enabling direct parameter adjustment of the supplementary light, camera, and occlusion detection threshold on the vehicle display screen. This allows for flexible configuration of occlusion detection parameters without the need for external debugging tools, adapting to occlusion detection needs in different scenarios, simplifying the parameter adjustment process, and further improving the flexibility and adaptability of occlusion detection function debugging.

[0078] In this embodiment, the occlusion detection result obtained by identifying occlusion areas in image data based on the adjusted occlusion detection threshold specifically includes: preprocessing the image data acquired by the camera, including grayscale conversion, Gaussian noise reduction, and cropping of invalid edge regions to obtain a preprocessed image. The preprocessed image is divided into several uniform sub-regions, and the effective texture features, average brightness, and effective pixel ratio of each sub-region are calculated. Sub-regions lacking effective texture features, with average brightness below a preset lower limit, or with insufficient effective pixel ratio are marked as suspected occlusion sub-regions. Connectivity analysis and merging are performed on the suspected occlusion sub-regions to determine continuous occlusion regions. The ratio of the occlusion region area to the effective detection region area is calculated to obtain the occlusion rate. The occlusion rate is compared with the adjusted occlusion judgment threshold, and image stabilization is performed in conjunction with the occlusion hysteresis threshold to determine the current occlusion state and obtain the occlusion detection result. In this application, the occlusion detection result can be obtained based on a single adjusted occlusion detection value.

[0079] As an example, taking the occlusion states as including unoccluded, slightly occluded, partially occluded, severely occluded, and completely occluded states, when the occlusion rate is 0 and there are no occluded connected components, the current occlusion state is determined to be unoccluded.

[0080] When the occlusion rate is greater than 0 and less than or equal to the first preset threshold, the current occlusion state is determined to be a slight occlusion state.

[0081] When the occlusion rate is greater than the first preset threshold and less than or equal to the second preset threshold, the current occlusion state is determined to be a partial occlusion state.

[0082] When the occlusion rate is greater than the second preset threshold and less than or equal to the third preset threshold, the current occlusion state is determined to be a severe occlusion state.

[0083] When the occlusion rate exceeds a third preset threshold, the current occlusion state is determined to be a complete occlusion state. The first, second, and third preset thresholds are occlusion determination thresholds.

[0084] In this embodiment of the application, considering that the ambient light intensity, image brightness, and state of the occlusion may still change in real time during the occlusion detection process, the brightness of the supplementary light, the working mode of the camera, and the occlusion detection threshold can also be adjusted during the occlusion detection process.

[0085] This application embodiment adjusts the brightness of the fill light according to the brightness adjustment command, switches the working mode of the camera according to the working mode switching command, and adjusts the occlusion detection threshold according to the threshold adjustment command. It acquires the image data collected by the camera and completes the occlusion area identification based on the adjusted occlusion detection threshold to obtain the occlusion detection result. It can avoid the problem of false detection and missed detection caused by parameter mismatch and realize the flexible adjustment of working mode, judgment threshold, etc. in the occlusion detection process.

[0086] In one alternative implementation, the associated device includes a supplementary light and at least one vehicle control device related to occlusion detection. S103 determines whether the associated device related to occlusion detection has malfunctioned and obtains fault diagnosis results, including Sa1 to Sa4.

[0087] Sa1: Based on the occlusion detection results, the image data collected by the camera is processed to obtain the image data after image rendering.

[0088] In this embodiment, image rendering processing is performed on the image data collected by the camera, including overlaying annotation information such as occlusion area, effective detection area, occlusion rate, current frame rate and resolution information of the image data on the image data.

[0089] As an example, the specific rendering method can be as follows: the occluded area is highlighted with a transparent red rectangle, the effectively detected area is marked with a blue dashed border, the real-time occlusion rate is overlaid in the upper right corner of the image, and the current frame rate and resolution information are overlaid in the upper left corner of the image, forming the image data after visualization and rendering. The rendered image data is transmitted to the cockpit domain controller via the vehicle Ethernet in the form of a real-time video stream for real-time display on the debugging and diagnostic interface of the vehicle display screen.

[0090] This embodiment uses image rendering to fuse the occlusion detection results with the original image for visualization, allowing debugging and diagnostic personnel to intuitively determine the occlusion location, degree of occlusion, image quality, and detection accuracy without analyzing background data, greatly improving the intuitiveness of debugging.

[0091] Sa2: Determines the working status data of the fill light.

[0092] In this embodiment of the application, the working status data includes: the expected on / off state, the actual on / off state, and the working mode of the fill light.

[0093] In this embodiment, the expected switching state is determined by the supplementary light control command during the occlusion detection process. If the supplementary light control command instructs the supplementary light to be turned on, then the expected switching state is the on state. The actual switching state is obtained based on the hardware feedback signal, drive level signal, or status detection data acquired by the supplementary light. Whether the supplementary light is faulty can be determined based on the expected and actual switching states. For example, if the expected switching state is on and the actual switching state is off, then a faulty supplementary light can be determined. Alternatively, this application can also directly display the working status of the supplementary light, including normal and fault states. The working modes include ultra-short pulse mode and frequency hopping mode.

[0094] In this embodiment, as an example, the expected on / off state, actual on / off state, and operating mode of the supplementary light can be obtained by parsing the supplementary light control commands, collecting hardware driver feedback signals, and LED driver chip status information. In this embodiment, the supplementary light's operating status data is uploaded to the intelligent driving domain SoC control terminal in real time, and then synchronized to the cockpit domain controller via the vehicle network. This data is then visualized as virtual indicator lights on the vehicle display's debugging and diagnostic interface. A green circular light indicates that the supplementary light is on, and a gray circle indicates that the supplementary light is off. The text "IRLED: ON / OFF" is displayed next to / below the virtual indicator light.

[0095] This embodiment can realize real-time acquisition and visualization feedback of the full-dimensional working status of the auxiliary light. It can quickly determine whether the auxiliary light is malfunctioning or whether the brightness is abnormal without disassembling the device or connecting an external on-board diagnostic (OBD) device, which significantly improves the speed of auxiliary light fault location.

[0096] Sa3: Acquires link communication data between vehicle control devices.

[0097] In this embodiment of the application, the link communication data includes the connectivity status and transmission delay between vehicle control devices.

[0098] In this embodiment of the application, each vehicle control device can send heartbeat frames and status handshake frames to each other at a preset period (such as 100ms). The cockpit domain controller summarizes and parses the heartbeat frames to obtain the connectivity status and transmission delay values ​​of each link segment.

[0099] In the embodiments of this application, connectivity states include normal connectivity, high communication latency, and communication interruption. Transmission latency is used to characterize the real-time performance of data interaction; the lower the latency, the higher the link communication efficiency.

[0100] This embodiment can realize real-time monitoring of the entire DMS communication status, and can quickly locate link anomalies such as communication interruption and excessive latency, solving the problems of difficult fault location and long troubleshooting cycle in traditional multi-device collaborative systems.

[0101] Sa4: Uses the image data after image rendering, working status data, and link communication data as the fault diagnosis results.

[0102] In this embodiment, the fault diagnosis results support real-time refreshing, screenshot saving, video recording, and log export, which facilitates debugging process backtracking and problem analysis.

[0103] This application embodiment performs image rendering processing on the image data collected by the camera based on the occlusion detection results, obtains the working status data of the supplementary light and the link communication data between the vehicle control equipment, and integrates the above data into the fault diagnosis results. It can collect the operation and communication information of the camera, supplementary light and vehicle control equipment in all aspects, fully present the status of the entire occlusion detection process, and intuitively reflect problems such as supplementary light failure and link communication abnormality. It can complete multi-dimensional diagnosis without disassembling the device or connecting external diagnostic equipment, realize rapid fault location, and improve the comprehensiveness and intuitiveness of debugging and diagnosis.

[0104] In one optional implementation, the occlusion detection results and fault diagnosis results are displayed on an in-vehicle display screen to pinpoint the cause of the camera malfunction, including: The in-vehicle display shows the debugging and diagnostic interface. This interface includes two sections: a first area and a second area.

[0105] The first area displays the occlusion detection results, and the second area displays the fault diagnosis results.

[0106] In one optional implementation, the debugging and diagnostic interface further includes a third area, which is used to acquire the brightness adjustment amount of the fill light, the working mode of the camera, and the threshold adjustment amount of the occlusion detection threshold.

[0107] In this embodiment, the third area of ​​the vehicle-mounted display screen may also be equipped with an LED on / off control, a screenshot control, a recording control, and a log export control. The LED on / off control is used to control the start / stop status of the LED, and can be manually controlled. The screenshot control is used to capture all or part of the debugging and diagnostic interface in real time, such as the image frames currently captured by the camera, facilitating direct observation of the lighting effect and occlusion status. The recording control is used to record all or part of the debugging and diagnostic interface, such as the video stream captured by the camera, recording image changes and occlusion detection response processes during debugging. The log export control is used to export the occlusion detection operation log, debugging and diagnostic log, brightness, working mode, threshold modification log, etc., facilitating offline analysis of parameter adjustment effects and abnormal operating conditions.

[0108] In the embodiments of this application, such as Figure 3 The debugging and diagnostic interface shown has the following sections: the first area is the occlusion detection result area; the second area is the LED status monitoring area; the third area is the debugging operation area, the DMS real-time screen area, and the hardware link status area.

[0109] As an example, the LED status monitoring area is as follows: Figure 4As shown, the display includes a fill light switch status indicator, a fill light fault status indicator, and a fill light operating mode display. The fill light switch status is indicated by a green light indicating it's on and a gray light indicating it's off. The fill light fault status is indicated by a green light indicating normal operation and a solid red light indicating a fault. The fill light operating mode display shows whether it's in ultra-short pulse mode or frequency-hopping mode. Clicking the fault status indicator expands to show the fault type, occurrence time, and fault code.

[0110] As an example, the DMS real-time screen area is as follows: Figure 5 As shown, the image includes real-time IR grayscale images captured by the camera, highlighted bounding boxes for occluded areas, bounding boxes for effective detection areas, real-time occlusion rate values, and image frame rate and resolution information. Occluded areas are marked with transparent red rectangles, and effective detection areas are marked with blue dashed boxes. A 5% margin is reserved on both sides for invalid areas. The image supports two-finger zoom to view details.

[0111] As an example, the hardware link state area is as follows: Figure 6 As shown, the node identifiers and communication link connections of the MCU controller, intelligent driving domain SoC control terminal, intelligent driving domain SoC detection terminal, and cockpit domain controller are displayed. The communication links are marked with green to indicate normal connection, yellow to indicate high latency, and red to indicate communication interruption. The transmission latency and interruption duration of each link are displayed in real time.

[0112] In this embodiment, the debugging and diagnostic interface can be accessed via menu path operation on the vehicle display screen or triggered by OBD diagnostic interface commands. Method 1: Vehicle display screen -> Settings -> Engineering Mode -> Enter developer password -> DMS debugging and diagnostics. Method 2: Activate the vehicle display screen's debugging and diagnostic interface by sending a specific diagnostic command through the OBD diagnostic interface.

[0113] As an example, the process of entering the debugging and diagnostic interface includes: turning on the vehicle display screen, entering the settings menu, selecting the engineering mode option in the settings menu, and the engineering mode verification interface prompting for the developer password. If the developer password is entered correctly, the user will directly enter the DMS debugging and diagnostic interface and load data for each functional area. If the developer password is entered incorrectly, an error message will be displayed and the user will be returned to the previous menu. After entering the debugging and diagnostic interface, debugging and diagnostics will be performed, loading display data for each area. After the debugging and diagnostics are completed, relevant data (occlusion detection results, fault diagnosis results) can be saved, and the user can exit the debugging and diagnostic mode.

[0114] In this embodiment, the triggering of occlusion detection and the triggering of debugging and diagnosis can be started simultaneously and executed in parallel. After entering the debugging and diagnosis interface, the cockpit domain controller synchronously sends a start command to the intelligent driving domain SoC, and the vehicle system automatically starts occlusion detection of the camera and simultaneously starts debugging and diagnosis of the camera and related devices, without requiring separate triggering by debugging and diagnosis personnel. When debugging and diagnosis personnel issue occlusion detection control commands such as brightness adjustment commands, working mode switching commands, and threshold adjustment commands in the third area, the occlusion detection is immediately re-triggered after the occlusion detection control command is issued, and the fault diagnosis results are updated synchronously to ensure that the occlusion detection results and fault diagnosis results are linked in real time after parameter adjustment. When the debugging and diagnosis personnel save the relevant data and exit the debugging and diagnosis mode, the vehicle system synchronously stops the occlusion detection and debugging and diagnosis process.

[0115] This implementation method uses a regionally visualized layout to display debugging and control, occlusion detection results, device status, and link health. Combined with screenshot, video recording, log export, and event backtracking capabilities, it can complete the entire debugging and diagnosis process in the vehicle without the need for external dedicated debugging equipment, disassembly testing, or background log backtracking analysis. This significantly improves the speed of fault location, shortens the problem investigation cycle, and lowers the threshold for development and after-sales maintenance.

[0116] This application embodiment displays a segmented debugging and diagnostic interface on an in-vehicle display screen. The first area displays the occlusion detection results, the second area displays the fault diagnosis results, and the third area obtains the brightness adjustment amount of the supplementary light, the working mode of the camera, and the threshold adjustment amount of the occlusion detection threshold. This integrates command input, result display, and diagnostic presentation into a single visual interface, which is more intuitive and orderly. It allows for the acquisition of full-process information without switching between multiple interfaces or tools, thus achieving visualized control of debugging and diagnosis.

[0117] In an optional implementation, the debugging and diagnostic interface further includes a fourth area, and the camera fault detection method further includes: Acquire at least one target event during the occlusion detection and fault diagnosis processes. The target event includes a first event indicating a change in the state of the camera and associated devices, and a second event indicating a change in the occlusion detection result and the fault diagnosis result. As an example, the target event could be a fill light state change event, an occlusion state transition event, a link state change event, an occlusion detection related parameter manipulation event, or a fault diagnosis result transition event. Specifically, the first event could include a fill light state change event, a link state change event, and an occlusion detection related parameter manipulation event, while the second event could include an occlusion state transition event and a fault diagnosis result transition event.

[0118] The fourth area displays the event timeline and the occurrence time of each of the at least one target event corresponding to the event timeline.

[0119] When a selection action is detected for any target event, the detailed information of the corresponding target event is displayed.

[0120] In this embodiment of the application, an event marker can be set at the time when the event corresponding to the target event occurs.

[0121] like Figure 3 The debugging and diagnostic interface shown has a fourth area: the diagnostic event timeline. Located at the bottom of the interface, this timeline uses a sliding horizontal timeline to record and backtrack key events. The timeline records all target events throughout the occlusion detection and debugging process, visualizing them over time and supporting event backtracking and multi-event correlation analysis.

[0122] As an example, the recorded events related to fill light status changes can include fill light on / off, fault triggering, fault recovery, and brightness adjustment completion. Events related to occlusion status transitions include camera state switching between different occlusion states, while also recording the occlusion rate and its change curve at the corresponding time. Events related to link status changes include interruptions, recovery, and latency anomalies in the communication link between the MCU controller, the intelligent driving domain SoC control terminal, the intelligent driving domain SoC detection terminal, and the cockpit domain controller. Events related to occlusion detection parameter manipulation include fill light on / off operations, camera working mode switching operations, occlusion detection threshold adjustment operations, parameter reset operations, and screenshot / video / log export operations performed by debugging and diagnostic personnel through the debugging and diagnostic interface. Camera anomaly events include decreased camera capture frame rate, black screen images, overly dark / overexposed images, abnormal resolution, and capture interruptions.

[0123] In this embodiment, the diagnostic event timeline uses different colors and icons to distinguish and mark various target events, facilitating rapid identification of event types. When a diagnostic operator clicks or touches an event marker, the diagnostic interface displays a details card for the corresponding event, showing details such as event type, precise timestamp, triggering conditions, duration, and values ​​before and after the state change. Simultaneously, the diagnostic event timeline supports multi-event correlation analysis. Diagnostic operators can select two or more target event markers simultaneously, and the vehicle system automatically calculates the time interval between events, analyzes the sequence and causal relationships of the events, and quickly locates the cause of the anomaly. For example, it can correlate a supplementary lighting malfunction event with a surge in obstruction rate, or a link interruption event with a detection failure event, improving fault location accuracy and diagnostic efficiency.

[0124] This application embodiment acquires target events during the occlusion detection and fault diagnosis processes. The event timeline and corresponding event markers are displayed in the fourth area of ​​the debugging and diagnosis interface. Selecting an event marker allows viewing detailed information. It can completely record and present the changes in equipment operating status, detection and diagnosis results, and sort out key events in the time dimension. It supports backtracking of event details, which facilitates the analysis of the triggering timing and correlation of abnormal events, quickly locates the cause of the fault, makes the fault diagnosis process traceable and analyzable, improves the accuracy and efficiency of problem investigation, and improves the recording and analysis capabilities of debugging and diagnosis.

[0125] In an optional implementation, before determining whether the associated device related to occlusion detection has malfunctioned and obtaining a fault diagnosis result, the camera fault detection method further includes: Generate device testing procedures for cameras and associated devices.

[0126] Set up a device test button on the vehicle display screen that is associated with the device testing process.

[0127] When the device test button is triggered, the process is executed to determine whether the associated device related to the occlusion detection has malfunctioned and to obtain the fault diagnosis result.

[0128] As another example, the device test button can function as a standalone test button, independently testing the camera and associated devices. This is independent of the steps involved in determining whether the associated devices related to occlusion detection are malfunctioning and obtaining fault diagnosis results. Upon triggering the device test button, link connectivity and functional tests are performed on the camera and associated devices, yielding test results. A comprehensive test report is generated based on these results. Specifically, link connectivity tests are performed on the vehicle control equipment, while functional tests are performed separately on the camera and associated devices.

[0129] In this embodiment of the application, the device testing process is shown in Table 1.

[0130] Table 1 Equipment Testing Procedure

[0131] In this embodiment, the device testing process is a one-click automated self-test sequence. It can automatically complete the health check and functional verification of the entire DMS system in sequence without the need for manual operation by debugging and diagnostic personnel. It covers the entire process of communication link, camera, fill light, and occlusion detection algorithm.

[0132] In this embodiment, the device test button can be set in the third area (debugging control area) of the vehicle display screen debugging and diagnostic interface, so that debugging and diagnostic personnel can trigger it with one click.

[0133] In this embodiment, the test results are visualized on the vehicle-mounted display screen in the form of a checklist report card: a green √ indicates the test passed, a red × indicates the test failed, and a yellow √ indicates the test failed. This indicates that the test has timed out or is in a boundary value state. After all test steps are completed, the vehicle system automatically calculates the overall health score (0-100 points) based on the test results of each step, and generates corresponding fault handling suggestions and debugging measures. It also supports one-click saving of the comprehensive test report (including test steps, test results, health score, and suggested measures) to the vehicle's local storage medium for easy fault tracing and after-sales record keeping.

[0134] In this embodiment, after the debugging and diagnostic personnel trigger the equipment test button, they execute the tests step by step. If any test step fails, the corresponding fault is recorded. After all tests are completed, a comprehensive test report is generated and displayed on the debugging and diagnostic interface, realizing fully automated, visualized, and standardized testing.

[0135] This embodiment achieves rapid self-testing of the entire link and all functions of the camera and related devices by pre-setting an automated equipment testing process, combined with one-click triggering and visual report display on the vehicle display screen. This avoids omissions, errors and inefficiencies of manual testing, and can complete health assessment and fault location without external equipment, greatly reducing the threshold for debugging and diagnosis and improving the standardization and efficiency of testing.

[0136] This application embodiment generates a device testing process for cameras and associated devices. A corresponding device testing button is set on the vehicle display screen. After the device testing button is triggered, link connectivity testing and functional testing are automatically carried out and a comprehensive test report is generated. It can complete the full-dimensional testing of occlusion detection related devices in an automated manner, avoiding the omissions and inefficiencies of manual item-by-item testing. It can complete the full-link health check and functional verification with one click and generate a comprehensive test report, which greatly simplifies the testing process, lowers the threshold for debugging and diagnosis operations, and improves the efficiency and standardization of device testing.

[0137] In this application, the debugging and diagnostic interface of the in-vehicle display screen can be replaced by remote viewing and control via a mobile app, tablet, or other portable smart terminal. The vehicle can simultaneously transmit real-time debugging video streams, occlusion detection results, supplementary lighting status data, hardware link communication data, and diagnostic event timeline data from the DMS camera to the portable smart terminal via in-vehicle WiFi, Bluetooth, or in-vehicle Ethernet, enabling remote debugging, remote diagnostics, and remote parameter adjustment outside the vehicle. This further expands the application scenarios for debugging and diagnostics, eliminating the limitations of in-vehicle space.

[0138] In this embodiment of the application, the video transmission bandwidth optimization scheme reduces the transmission mode of the real-time image of the DMS camera from full frame rate video stream to low frame rate key frame snapshot, thumbnail data stream or interval frame transmission (e.g., 2-3 frames per second). While ensuring the occlusion detection effect and the visibility of the device status, it reduces the bandwidth occupation of the vehicle network and reduces the data processing pressure of the cockpit domain controller and intelligent driving domain SoC, enabling this application to better adapt to low computing power and low bandwidth vehicle hardware platforms.

[0139] In this embodiment of the application, the cloud-based diagnostic and problem reporting solution adds a one-click problem reporting function to the debugging and diagnostic interface. When an abnormality in occlusion detection, a faulty auxiliary light, an abnormal link communication, or an abnormal fault diagnosis result is detected, the debugging and diagnostic personnel can trigger the one-click reporting control. The system will automatically capture the current debugging and diagnostic interface, real-time camera footage, occlusion detection results, auxiliary light status, hardware link status, and debugging and diagnostic logs, and automatically generate a standardized diagnostic report containing the abnormality type, occurrence time, equipment status, and key parameters. This report will then be uploaded to the cloud service system via the vehicle-to-everything (V2X) T-Box, facilitating rapid fault analysis, problem location, and remote technical support in the background.

[0140] In this application embodiment, the DMS camera occlusion detection visualization debugging and diagnostic interface, debugging and diagnostic method, parameter adjustment logic, one-click self-test process, event timeline and link monitoring scheme can be reused as a whole to other vehicle camera systems such as panoramic surround view system, automatic parking system, blind spot monitoring system for occlusion detection, function debugging and fault diagnosis scenarios. Only adaptive modifications are needed for different camera types, installation locations, supplementary lighting methods and detection logic to quickly achieve one-stop visualization debugging and diagnosis for multiple scenarios and multiple systems, improving the versatility and reusability of the solution.

[0141] This embodiment also provides a camera fault detection device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0142] Figure 7 This is a structural block diagram of a camera fault detection device according to an embodiment of this application.

[0143] This embodiment provides a camera fault detection device, such as... Figure 7 As shown, the camera fault detection device includes: The first processing module 11 is used to acquire occlusion detection control instructions.

[0144] The second processing module 12 is used to respond to the occlusion detection control command, perform occlusion detection on the camera, and obtain the occlusion detection result.

[0145] The third processing module 13 is used to determine whether the associated equipment related to the occlusion detection has malfunctioned and to obtain the fault diagnosis result.

[0146] The fourth processing module 14 is used to display the occlusion detection results and fault diagnosis results on the vehicle display screen to locate the cause of the camera malfunction.

[0147] In one optional implementation, the associated device includes a supplementary light and a second processing module 12, specifically used to adjust the brightness of the supplementary light, the operating mode of the camera, and the occlusion detection threshold. The supplementary light is used to provide supplementary illumination when the camera acquires image data inside the vehicle.

[0148] Acquire image data captured by the camera.

[0149] Based on the adjusted occlusion detection threshold, the occlusion region is identified in the image data to obtain the occlusion detection result.

[0150] In one optional implementation, the second processing module 12 includes a first processing unit for... The system acquires a brightness adjustment command, a working mode switching command, and a threshold adjustment command; wherein the brightness adjustment command carries a brightness adjustment amount; the working mode switching command carries the working mode after the camera switches to the working mode; and the threshold adjustment command carries the threshold adjustment amount of the occlusion detection threshold.

[0151] The brightness of the fill light, the working mode of the camera, and the occlusion detection threshold are adjusted according to the brightness adjustment command, the working mode switching command, and the threshold adjustment command, respectively.

[0152] In one alternative implementation, the associated device includes a fill light and at least one vehicle control device related to occlusion detection. The third processing module 13 is used to perform image rendering processing on the image data collected by the camera based on the occlusion detection result to obtain image data after image rendering processing.

[0153] Determine the working status data of the fill light. This data includes: the expected on / off status, actual on / off status, and operating mode of the fill light.

[0154] Acquire link communication data between vehicle control devices. This link communication data includes the connectivity status and transmission latency between the vehicle control devices.

[0155] Image data after image rendering, working status data, and link communication data are used as fault diagnosis results.

[0156] In one optional implementation, the fourth processing module 14 is specifically used to display a debugging and diagnostic interface on the vehicle-mounted display screen. The debugging and diagnostic interface includes a first area and a second area.

[0157] The first area displays the occlusion detection results, and the second area displays the fault diagnosis results.

[0158] In one optional implementation, the debugging and diagnostic interface further includes a third area, which is used to acquire the brightness adjustment amount of the fill light, the working mode of the camera, and the threshold adjustment amount of the occlusion detection threshold.

[0159] In an optional implementation, when the debugging and diagnostic interface further includes a fourth area, the fourth processing module 14 is also configured to acquire at least one target event during the occlusion detection process and the fault diagnosis process. The target event includes a first event of a change in the state of the camera and associated device, and a second event of changes in the occlusion detection result and the fault diagnosis result.

[0160] The fourth area displays the event timeline and the occurrence time of each of the at least one target event corresponding to the event timeline.

[0161] When a selection action is detected for any target event, the detailed information of the corresponding target event is displayed.

[0162] In one optional implementation, the camera fault detection device further includes a fifth processing module, used to generate a device test process for the camera and associated devices before determining whether the associated devices related to the occlusion detection have malfunctioned and obtaining fault diagnosis results.

[0163] Set up a device test button on the vehicle display screen that is associated with the device testing process.

[0164] When the device test button is triggered, the process is executed to determine whether the associated device related to the occlusion detection has malfunctioned and to obtain the fault diagnosis result.

[0165] In this embodiment, the camera fault detection device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0166] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0167] This application also provides a vehicle having the above-described features. Figure 7 The image shows a camera fault detection device and an in-vehicle display screen.

[0168] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a vehicle provided in an optional embodiment of this application, such as... Figure 8 As shown, the vehicle may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 201, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 202 or a program loaded from memory 208 into random access memory (RAM) 203. The RAM 203 also stores various programs and data required for vehicle operation. The processor 201, ROM 202, and RAM 203 are interconnected via bus 204. An input / output (I / O) interface 205 is also connected to bus 204.

[0169] Typically, the following devices can be connected to I / O interface 205: input devices 206 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 207 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory 208 including, for example, magnetic tape, hard disk, etc.; and communication devices 209. Communication device 209 allows the vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 8 Vehicles with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0170] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 209, or installed from memory 208, or installed from ROM 202. When the computer program is executed by processor 201, it performs the functions defined in the camera fault detection method of embodiments of this application.

[0171] Figure 8 The vehicle shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0172] This application also provides a vehicle that includes the resource allocation device or vehicle described above, so that the vehicle can implement the camera fault detection method of the above example.

[0173] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc. Further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessors, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the camera fault detection method shown in the above embodiments is implemented.

[0174] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0175] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for detecting camera faults, characterized in that, The method includes: Obtain occlusion detection control instructions; In response to the occlusion detection control command, the camera is subjected to occlusion detection, and the occlusion detection result is obtained; Determine whether any associated equipment related to occlusion detection has malfunctioned, and obtain fault diagnosis results; The occlusion detection results and fault diagnosis results are displayed on the vehicle-mounted display screen to pinpoint the cause of the camera malfunction.

2. The method according to claim 1, characterized in that, The associated device includes a fill light, and the occlusion detection of the camera to obtain the occlusion detection result includes: The brightness of the supplementary light, the working mode of the camera, and the occlusion detection threshold are adjusted; the supplementary light is used to provide supplementary lighting when the camera acquires image data inside the vehicle; wherein, the occlusion detection threshold is used to determine the occlusion status of the camera; Acquire image data captured by the camera; Based on the adjusted occlusion detection threshold, the occlusion region is identified in the image data to obtain the occlusion detection result.

3. The method according to claim 2, characterized in that, The adjustment of the brightness of the fill light, the working mode of the camera, and the occlusion detection threshold includes: The system acquires a brightness adjustment command, a working mode switching command, and a threshold adjustment command; wherein the brightness adjustment command carries a brightness adjustment amount; the working mode switching command carries the working mode after the camera switches to the working mode; and the threshold adjustment command carries a threshold adjustment amount for the occlusion detection threshold. The brightness of the fill light, the working mode of the camera, and the occlusion detection threshold are adjusted according to the brightness adjustment command, the working mode switching command, and the threshold adjustment command, respectively.

4. The method according to claim 1, characterized in that, The associated devices include supplemental lighting and at least one vehicle-mounted control device related to occlusion detection. Determining whether the associated devices related to occlusion detection have malfunctioned and obtaining fault diagnosis results includes: Based on the occlusion detection result, the image data collected by the camera is rendered to obtain the rendered image data. Determine the working status data of the fill light; the working status data includes: the expected on / off state, the actual on / off state, and the working mode of the fill light; Acquire link communication data between the vehicle control devices; the link communication data includes the connectivity status and transmission delay between the vehicle control devices. The image data after image rendering, the working status data, and the link communication data are used as the fault diagnosis results.

5. The method according to claim 1, characterized in that, The step of displaying the occlusion detection results and the fault diagnosis results on the vehicle-mounted display screen to locate the cause of the camera malfunction includes: The vehicle-mounted display screen shows a debugging and diagnostic interface; wherein the debugging and diagnostic interface includes: a first area and a second area; The occlusion detection result is displayed in the first area, and the fault diagnosis result is displayed in the second area.

6. The method according to claim 5, characterized in that, The debugging and diagnostic interface also includes a third area, which is used to obtain the brightness adjustment amount of the fill light, the working mode of the camera, and the threshold adjustment amount of the occlusion detection threshold.

7. The method according to claim 5, characterized in that, The debugging and diagnostic interface also includes a fourth area, and the method further includes: Acquire at least one target event during the occlusion detection process and the fault diagnosis process; the target event includes a first event of a change in the state of the camera and the associated device, and a second event of a change in the occlusion detection result and the fault diagnosis result; The fourth area displays an event timeline and the occurrence times of at least one target event corresponding to the event timeline. When a selection operation is detected for any of the target events, the detailed information of the corresponding target event is displayed.

8. The method according to claim 1, characterized in that, Before determining whether the associated equipment related to the occlusion detection has malfunctioned and obtaining the fault diagnosis result, the method further includes: Generate a device testing process for the camera and the associated device; A device test button associated with the device test process is provided on the vehicle-mounted display screen. When the device test button is triggered, the step of determining whether the associated device related to the occlusion detection has malfunctioned and obtaining the fault diagnosis result is triggered.

9. A camera fault detection device, characterized in that, The device includes: The first processing module is used to acquire occlusion detection control instructions; The second processing module is used to respond to the occlusion detection control command, perform occlusion detection on the camera, and obtain the occlusion detection result. The third processing module is used to determine whether the associated equipment related to occlusion detection has malfunctioned and to obtain the fault diagnosis result. The fourth processing module is used to display the occlusion detection results and the fault diagnosis results on the vehicle-mounted display screen to locate the cause of the camera malfunction.

10. A vehicle, characterized in that, include: In-vehicle displays and controllers; The controller includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method of any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 8.

Images