Electronic rearview mirror system, vehicle vision aid system, and method of failure handling thereof

Abstract

The application relates to an electronic rearview mirror system and a fault processing method thereof, a computer readable storage medium and a computer program product for implementing the fault processing method, a vehicle vision auxiliary system and a fault processing method thereof, a computer readable storage medium and a computer program product for implementing the fault processing method. An electronic rearview mirror system in one aspect of the application comprises: a processing unit configured to compare images of two adjacent frames of a rearview environment outside a vehicle to generate a freeze detection result; a control unit coupled with the processing unit and configured to: for a first frame image, determine a sending time of sending an exposure command of the first frame image and an acquisition time of the processing unit acquiring the first frame image; determine a time difference between the acquisition time and the sending time; and perform a fault processing operation based on one or more of the freeze detection result and the time difference, the fault processing operation comprising generating a control signal for superimposedly displaying fault alarm information on a display screen.

Description

Technical Field

[0001] This application generally relates to the field of vehicles, and more specifically to an electronic rearview mirror system, a fault handling method for an electronic rearview mirror system, a computer-readable storage medium and computer program product for implementing the fault handling method for an electronic rearview mirror system, a vehicle vision assistance system, a fault handling method for a vehicle vision assistance system, and a computer-readable storage medium and computer program product for implementing the fault handling method for a vehicle vision assistance system. Background Technology

[0002] Vehicle vision assistance is an important component of modern intelligent vehicles. Through cameras, sensors, and computer vision algorithms, it provides drivers with comprehensive visual support, enhancing driving safety and convenience.

[0003] For example, an increasing number of smart cars are adopting electronic rearview mirrors (Camera Monitor Systems, CMS) to replace traditional physical rearview mirrors. Compared to traditional physical rearview mirrors, electronic rearview mirrors provide a wider field of view through cameras, reducing blind spots. After camera calibration, they can clearly display the rear view of the vehicle at night and in inclement weather conditions, thus solving the problems of limited field of view and unclear display at night and in inclement weather that are inherent to physical rearview mirrors. Furthermore, the streamlined design of electronic rearview mirrors helps reduce air resistance, thereby reducing wind noise and increasing the vehicle's range. During driving, the driver can use the rear view displayed on the electronic rearview mirror's screen to determine whether there are other vehicles behind the vehicle, thereby controlling the vehicle's driving, such as changing lanes or turning. However, if the rear view displayed on the electronic rearview mirror's screen is delayed or sluggish, and the driver relies on this image to control the vehicle, it may negatively impact driving safety.

[0004] For example, an increasing number of smart cars are equipped with reversing camera systems. These systems use multiple cameras installed around the vehicle to capture images of the surrounding environment in real time. Image processing technology then stitches these images together to create a complete bird's-eye view, helping drivers to observe the surroundings more safely and conveniently when reversing. However, if the image displayed on the reversing camera system's screen is delayed or lags, and the driver relies on this image to control the vehicle's reversing, it may negatively impact driving safety.

[0005] Therefore, improving the reliability and safety of vehicle vision assistance systems is an urgent problem to be solved. Summary of the Invention

[0006] To address or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0007] According to a first aspect of this application, an electronic rearview mirror system is provided, the electronic rearview mirror system comprising: a processing unit configured to compare images of two adjacent frames of the external rear view environment of a vehicle to generate a freeze detection result; and a control unit coupled to the processing unit and configured to: determine, for a first frame image, a sending time of an exposure command for sending the first frame image and an acquisition time of the processing unit acquiring the first frame image; determine a time difference between the acquisition time and the sending time; and perform a fault handling operation based on one or more of the freeze detection result and the time difference, the fault handling operation including generating a control signal for overlaying fault alarm information on a display screen.

[0008] An electronic rearview mirror system according to one or more embodiments of this application, wherein the processing unit is a bridging chip of the electronic rearview mirror system, the bridging chip being configured to display the fault alarm information superimposed on the display screen via the OSD mapping function of the bridging chip in response to the control signal.

[0009] The processing unit is further configured to: generate a freeze detection result indicating that the current frame's image has a freeze fault in response to determining that the image of the current frame is consistent with the image of the previous frame in two adjacent frames; and generate a freeze detection result indicating that the image of the current frame has not a freeze fault in response to determining that the image of the current frame is inconsistent with the image of the previous frame in two adjacent frames.

[0010] The processing unit is further configured to determine whether the image of the current frame is consistent with the image of the previous frame in two adjacent frames by: comparing each pixel value of the region of interest in the current frame with the corresponding pixel value of the region of interest in the previous frame; determining that the image of the current frame is consistent with the image of the previous frame in response to the fact that each pixel value of the region of interest in the current frame is the same as the corresponding pixel value of the region of interest in the previous frame; and determining that the image of the current frame is inconsistent with the image of the previous frame in response to the fact that one or more pixel values ​​of the region of interest in the current frame are different from the corresponding pixel values ​​of the region of interest in the previous frame.

[0011] The control unit is further configured to perform fault handling operations by: determining an initial count value in response to the first frame image; incrementing or decrementing the initial count value based on the freeze detection result to obtain a fault count value; and selectively performing the fault handling operations based on a comparison of the fault count value with a fault threshold.

[0012] The control unit is further configured to increment or decrement the initial count value by: incrementing the initial count value by two in response to the freeze detection result indicating that the current frame image has a freeze fault; and decrementing the initial count value by one in response to the freeze detection result indicating that the current frame image has not a freeze fault.

[0013] The control unit is further configured to perform fault handling operations in the following manner: performing the fault handling operation in response to the time difference being greater than or equal to the fault tolerance time interval of the electronic rearview mirror system; determining the number of times a freeze detection result indicating a freeze fault has occurred in the current frame in response to the time difference being less than the fault tolerance time interval of the electronic rearview mirror system; determining the fault duration based on the number of times and the time difference; and selectively performing the fault handling operation based on a comparison of the fault duration with the fault tolerance time interval of the electronic rearview mirror system.

[0014] The control unit is further configured to determine the duration of the fault based on the number of occurrences and the time difference by: determining the product of the number of occurrences and the image update cycle of the electronic rearview mirror system; and determining the duration of the fault by summing the product with the time difference.

[0015] The fault handling operation also includes generating a control signal for controlling the vehicle to turn on its hazard warning lights.

[0016] According to a second aspect of this application, a fault handling method for an electronic rearview mirror system is provided. The fault handling method includes: comparing images of two adjacent frames of the vehicle's external rearview environment to generate a freeze detection result; determining a sending time of an exposure command for sending a first frame image and an acquisition time of acquiring the first frame image; determining a time difference between the acquisition time and the sending time; and performing a fault handling operation based on one or more of the freeze detection result and the time difference, the fault handling operation including generating a control signal for overlaying fault alarm information on a display screen.

[0017] According to one or more embodiments of the fault handling method described in this application, the fault alarm information is overlaid on the display screen through the OSD mapping function of the bridging chip of the electronic rearview mirror system.

[0018] The method of comparing two adjacent frames of the vehicle's external rear-view environment to generate a freeze detection result includes: generating a freeze detection result indicating that the current frame's image has a freeze fault in response to determining that the current frame's image is consistent with the previous frame's image; and generating a freeze detection result indicating that the current frame's image has not a freeze fault in response to determining that the current frame's image is inconsistent with the previous frame's image.

[0019] The determination of whether the image of the current frame is consistent with the image of the previous frame in two adjacent frames includes: comparing each pixel value of the region of interest in the current frame with the corresponding pixel value of the region of interest in the previous frame; determining that the image of the current frame is consistent with the image of the previous frame in response to the fact that each pixel value of the region of interest in the current frame is the same as the corresponding pixel value of the region of interest in the previous frame; and determining that the image of the current frame is inconsistent with the image of the previous frame in response to the fact that one or more pixel values ​​of the region of interest in the current frame are different from the corresponding pixel values ​​of the region of interest in the previous frame.

[0020] The fault handling operation based on one or more of the freeze detection result and the time difference includes: determining an initial count value in response to the first frame image; incrementing or decrementing the initial count value based on the freeze detection result to obtain a fault count value; and selectively performing the fault handling operation based on a comparison of the fault count value with a fault threshold.

[0021] Increasing or decreasing the initial count value based on the freeze detection result includes: increasing the initial count value by two in response to the freeze detection result indicating that the current frame image has a freeze fault; and decreasing the initial count value by one in response to the freeze detection result indicating that the current frame image has not a freeze fault.

[0022] The fault handling operation based on one or more of the freeze detection result and the time difference includes: performing the fault handling operation in response to the time difference being greater than or equal to the fault tolerance time interval of the electronic rearview mirror system; determining the number of times a freeze detection result indicating a freeze fault has occurred in the current frame image in response to the time difference being less than the fault tolerance time interval of the electronic rearview mirror system; determining the fault duration based on the number of times and the time difference; and selectively performing the fault handling operation based on a comparison between the fault duration and the fault tolerance time interval of the electronic rearview mirror system.

[0023] Determining the fault duration based on the number of occurrences and the time difference includes: determining the product of the number of occurrences and the image update cycle of the electronic rearview mirror system; and determining the fault duration by summing the product and the time difference.

[0024] The fault handling operation also includes generating a control signal for controlling the vehicle to turn on its hazard warning lights.

[0025] According to a third aspect of this application, a vehicle vision assistance system is provided, the vehicle vision assistance system comprising: a processing unit configured to compare two adjacent frames of images of an external environment of a vehicle to generate a freeze detection result; and a control unit coupled to the processing unit and configured to: determine, for a first frame image, a sending time of an exposure command for sending the first frame image and an acquisition time of the processing unit acquiring the first frame image; determine a time difference between the acquisition time and the sending time; and perform a fault handling operation based on one or more of the freeze detection result and the time difference, the fault handling operation including generating a control signal for overlaying fault alarm information on a display screen of the vehicle vision assistance system.

[0026] According to one or more embodiments of the present application, a vehicle vision assistance system is provided, wherein the processing unit is a bridging chip of the vehicle vision assistance system, and the bridging chip is configured to display the fault alarm information superimposed on the display screen through the OSD mapping function of the bridging chip in response to the control signal.

[0027] The processing unit is further configured to: in response to determining that the image of the current frame in two adjacent frames is consistent with the image of the previous frame, generate a freeze detection result indicating that the image of the current frame has a freeze fault; and send the freeze detection result indicating that the image of the current frame has a freeze fault to the control unit.

[0028] The processing unit is further configured to determine whether the image of the current frame is consistent with the image of the previous frame in two adjacent frames by: comparing each pixel value of the region of interest in the current frame with the corresponding pixel value of the region of interest in the previous frame; determining that the image of the current frame is consistent with the image of the previous frame in response to the fact that each pixel value of the region of interest in the current frame is the same as the corresponding pixel value of the region of interest in the previous frame; and determining that the image of the current frame is inconsistent with the image of the previous frame in response to the fact that one or more pixel values ​​of the region of interest in the current frame are different from the corresponding pixel values ​​of the region of interest in the previous frame.

[0029] The control unit is further configured to perform fault handling operations by: determining the number of times a freeze detection result indicating a freeze fault has occurred in the current frame; and selectively performing the fault handling operations based on a comparison of the number of times with a threshold number.

[0030] The vehicle vision assistance system includes an electronic rearview mirror system and a reversing camera system.

[0031] The control unit is further configured to perform fault handling operations in the following manner: performing the fault handling operation in response to the time difference being greater than or equal to the fault tolerance time interval of the vehicle vision assistance system; determining the number of times a freeze detection result indicating a freeze fault has occurred in the current frame is continuously received in response to the time difference being less than the fault tolerance time interval of the vehicle vision assistance system, determining the fault duration based on the number of consecutive receptions and the time difference; and selectively performing the fault handling operation based on a comparison between the fault duration and the fault tolerance time interval of the vehicle vision assistance system.

[0032] The control unit is further configured to determine the fault duration based on the number of occurrences and the time difference by: determining the product of the number of occurrences and the image update cycle of the vehicle vision assistance system; and determining the fault duration by summing the product with the time difference.

[0033] The control unit is further configured to: in response to the vehicle vision assistance system including the reversing camera system, determine the timing of sending an exposure command for sending the first frame image based on the timing of receiving the reversing gear signal.

[0034] According to a fourth aspect of this application, a fault handling method for a vehicle vision assistance system is provided. The fault handling method includes: comparing two adjacent frames of images of the vehicle's external environment to generate a freeze detection result; determining a sending time of an exposure command for sending the first frame image and an acquisition time of acquiring the first frame image; determining a time difference between the acquisition time and the sending time; and performing a fault handling operation based on one or more of the freeze detection result and the time difference, wherein the fault handling operation includes generating a control signal for superimposing fault alarm information on a display screen of the vehicle vision assistance system.

[0035] According to one or more embodiments of the fault handling method described in this application, the fault alarm information is overlaid on the display screen through the OSD mapping function of the bridging chip of the vehicle vision assistance system.

[0036] The process of comparing two adjacent frames of the vehicle's external environment to generate a freeze detection result includes: in response to determining that the image of the current frame in the two adjacent frames is consistent with the image of the previous frame, generating a freeze detection result indicating that a freeze fault has occurred in the image of the current frame.

[0037] The determination of whether the image of the current frame is consistent with the image of the previous frame in two adjacent frames includes: comparing each pixel value of the region of interest in the current frame with the corresponding pixel value of the region of interest in the previous frame; determining that the image of the current frame is consistent with the image of the previous frame in response to the fact that each pixel value of the region of interest in the current frame is the same as the corresponding pixel value of the region of interest in the previous frame; and determining that the image of the current frame is inconsistent with the image of the previous frame in response to the fact that one or more pixel values ​​of the region of interest in the current frame are different from the corresponding pixel values ​​of the region of interest in the previous frame.

[0038] The fault handling operation based on one or more of the freeze detection results and the time difference includes: determining the number of times to continuously generate freeze detection results indicating that the image of the current frame has a freeze fault; and selectively performing the fault handling operation based on a comparison of the number of times with a threshold number.

[0039] The vehicle vision assistance system includes an electronic rearview mirror system and a reversing camera system.

[0040] The fault handling operation based on one or more of the freeze detection results and the time difference includes: performing the fault handling operation in response to the time difference being greater than or equal to the fault tolerance time interval of the vehicle vision assistance system; determining the number of times a freeze detection result indicating a freeze fault has occurred in the current frame image is continuously generated in response to the time difference being less than the fault tolerance time interval of the vehicle vision assistance system, determining the fault duration based on the number of times and the time difference; and selectively performing the fault handling operation based on a comparison between the fault duration and the fault tolerance time interval of the vehicle vision assistance system.

[0041] Determining the fault duration based on the number of occurrences and the time difference includes: determining the product of the number of occurrences and the image update cycle of the vehicle vision assistance system; and determining the fault duration by summing the product and the time difference.

[0042] The fault handling method further includes: in response to the vehicle vision assistance system including the reversing image system, determining the sending time of the exposure command for sending the first frame image based on the time when the reversing gear signal is received.

[0043] According to a fifth aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, characterized in that, when the instructions are executed by a processor, the processor causes the processor to perform a fault handling method for an electronic rearview mirror system according to a second aspect of this application or a fault handling method for a vehicle vision assistance system according to a fourth aspect.

[0044] According to a sixth aspect of this application, a computer program product is provided, the computer program product including instructions, which, when executed by a processor, implement the fault handling method of the electronic rearview mirror system according to the second aspect of this application or the fault handling method of the vehicle vision assistance system according to the fourth aspect.

[0045] The electronic rearview mirror system according to one or more embodiments of this application detects the time difference between the sending time of the exposure command for the first frame image and the acquisition time of the first frame image by detecting two adjacent frames of the vehicle's external rear view environment, thereby enabling timely detection of display screen malfunctions and controlling the display screen to overlay fault alarm information, thus improving the reliability and safety of the electronic rearview mirror system and ensuring vehicle driving safety. The fault handling scheme of the electronic rearview mirror system according to one or more embodiments of this application has simple judgment logic, does not rely on additional detection and control circuits, has high reliability, and has advantages such as cost savings, simple control, and ease of implementation. The vehicle vision assistance system according to one or more embodiments of this application detects the time difference between the sending time of the exposure command for the first frame image and the acquisition time of the first frame image by detecting two adjacent frames of the vehicle's external environment, thereby enabling timely detection of display screen malfunctions and controlling the display screen to overlay fault alarm information, thus improving the reliability and safety of the vehicle vision assistance system and ensuring vehicle driving safety. The fault handling scheme of the vehicle vision assistance system according to one or more embodiments of this application has simple judgment logic, does not rely on additional detection and control circuits, has high reliability, and has the advantages of cost saving, simple control and easy implementation. Attached Figure Description

[0046] The above and / or other aspects and advantages of this application will become clearer and more readily understood from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are denoted by the same reference numerals. In the drawings:

[0047] Figure 1 A structural block diagram of an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0048] Figure 2A structural block diagram of an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0049] Figure 3 A flowchart illustrating a fault handling method for an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0050] Figure 4 A flowchart illustrating a fault handling method for an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0051] Figure 5 A structural block diagram of a vehicle vision assistance system according to one or more embodiments of this application is shown.

[0052] Figure 6 A structural block diagram of a reversing camera system according to one or more embodiments of this application is shown.

[0053] Figure 7 A flowchart is shown of a fault handling method for a vehicle vision assistance system according to one or more embodiments of this application. Detailed Implementation

[0054] The present application will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments thereof. However, the present application may be implemented in various forms and should not be construed as being limited to the embodiments given herein. The foregoing embodiments are intended to make the disclosure herein complete and thorough, so as to more fully convey the scope of protection of the present application to those skilled in the art.

[0055] In the context of this application, terms such as “comprising” and “including” indicate that, in addition to having units and steps that are directly and explicitly stated in the specification and claims, the technical solution of this application does not exclude the presence of other units and steps that are not directly or explicitly stated.

[0056] In the context of this application, unless otherwise specified, terms such as “first” and “second” do not indicate the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0057] In the context of this application, "coupling" should be understood to include situations in which electrical energy or electrical signals are directly transmitted between two units, or in situations in which electrical energy or electrical signals are indirectly transmitted through one or more intermediate units.

[0058] In the following, various exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings.

[0059] Figure 1 A structural block diagram of an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0060] like Figure 1 As shown, the electronic rearview mirror system 100 includes a processing unit 110 and a control unit 120 coupled to the processing unit 110.

[0061] The processing unit 110 is configured to compare images of two adjacent frames of the vehicle's external rear-view environment to generate a freeze detection result.

[0062] In one embodiment, the processing unit 110 can be configured to compare the image of the current frame with the image of the previous frame in two adjacent frames of the vehicle's external rear-view environment to determine whether the image of the current frame is consistent with the image of the previous frame. When it is determined that the image of the current frame is consistent with the image of the previous frame, the processing unit 110 can generate a freeze detection result indicating that the image of the current frame has a freeze fault; when it is determined that the image of the current frame is inconsistent with the image of the previous frame, the processing unit 110 can generate a freeze detection result indicating that the image of the current frame has not a freeze fault. For example, assuming the images of the vehicle's external rear-view environment include a first frame image I1, a second frame image I2, a third frame image I3, a fourth frame image I4, and a fifth frame image I5, the processing unit 110 can determine whether the second frame image I2 is consistent with the first frame image I1 to generate a freeze detection result for the second frame image I2, determine whether the third frame image I3 is consistent with the second frame image I2 to generate a freeze detection result for the third frame image I3, determine whether the fourth frame image I4 is consistent with the third frame image I3 to generate a freeze detection result for the fourth frame image I4, and determine whether the fifth frame image I5 is consistent with the fourth frame image I4 to generate a freeze detection result for the fifth frame image I5.

[0063] In one embodiment, the processing unit 110 can be configured to compare each pixel value of the region of interest (ROI) in the current frame image with the corresponding pixel value of the ROI in the previous frame image. When each pixel value of the ROI in the current frame image is the same as the corresponding pixel value of the ROI in the previous frame image, it can be determined that the current frame image is consistent with the previous frame image; when one or more pixel values ​​of the ROI in the current frame image are different from the corresponding pixel values ​​of the ROI in the previous frame image, it can be determined that the current frame image is inconsistent with the previous frame image. For example, the central region of the image can be selected as the ROI. For example, for a rectangular image with height H and width W, the center coordinates of the rectangular image can be determined as (W / 2, H / 2). Assuming the size of the central region is roi_width × roi_height, the coordinates of the top left corner of the central region are (W / 2 - roi_width / 2, H / 2 - roi_height / 2), and the coordinates of the bottom right corner are (W / 2 + roi_width / 2, H / 2 + roi_height / 2). The central region can be extracted from the rectangular image using the coordinates of the top left and bottom right corners of the central region. For example, suppose the region of interest in the current frame image has pixels (x1,y1), (x2,y2), and (x3,y3), where (x1,y1), (x2,y2), and (x3,y3) represent the coordinates of each pixel. The pixel values ​​of pixels (x1,y1), (x2,y2), and (x3,y3) in the current frame image can be compared with the pixel values ​​of pixels (x1,y1), (x2,y2), and (x3,y3) in the previous frame image.

[0064] Control unit 120 is configured to, for a first frame image, determine the transmission time of the exposure command for sending the first frame image and the acquisition time of the first frame image acquired by processing unit 110, determine the time difference between the acquisition time and the transmission time, and perform a fault handling operation based on one or more of the freeze detection result and the time difference. The fault handling operation includes generating a control signal for overlaying fault alarm information onto a display screen. Optionally, the fault handling operation may further include generating a control signal for controlling the vehicle to activate hazard warning lights. This control signal may, for example, be transmitted via a CAN bus to the body controller to control the vehicle to activate the hazard warning lights, thereby alerting other vehicles behind the vehicle.

[0065] In one embodiment, the processing unit 110 can be implemented as a bridging chip for an electronic rearview mirror system. This bridging chip can be configured to overlay fault alarm information onto a display screen in response to a control signal from the control unit 120 for overlaying fault alarm information onto the display screen, using the chip's OSD (On-Screen Display) mapping function. For example, the fault alarm information could be the text message "Rearview mirror display fault, please drive carefully," thus reminding the driver to drive cautiously. By utilizing the bridging chip to generate freeze detection results and overlay fault alarm information, no other hardware and software processing units need to be set up in the electronic rearview mirror system, resulting in high reliability and advantages such as cost savings, simple control, and ease of implementation.

[0066] In one embodiment, the control unit 120 may be configured to determine an initial count value upon detecting the first frame image, increment or decrement the initial count value based on the freeze detection result to obtain a fault count value, and selectively perform fault handling operations based on a comparison of the fault count value with a fault threshold. In one embodiment, when the fault count value is greater than or equal to the fault threshold, the control unit 120 may perform a fault handling operation; when the fault count value is less than the fault threshold, the control unit 120 may continue to increment or decrement the fault count value based on the freeze detection result.

[0067] In one embodiment, when the freeze detection result indicates that the current frame image has a freeze fault, the control unit 120 may increment the initial count value by two; when the freeze detection result indicates that the current frame image has not a freeze fault, the control unit 120 may decrement the initial count value by one. Exemplarily, the control unit 120 may include a counter, which may set the initial count value to zero and increment or decrement the initial count value based on the freeze detection result of each frame image to obtain a fault count value. Exemplarily, the fault threshold A may be set according to the image update cycle T1 (i.e., the update cycle of each frame image) and the fault tolerance time interval T2 of the electronic rearview mirror system. For example, the fault threshold A may be set according to the following formula (1):

[0068] T1×A≤T2×2 formula (1)

[0069] For example, assuming the image update cycle T1 of the electronic rearview mirror system is 16.66ms and the fault tolerance time interval T2 of the electronic rearview mirror system is 200ms, then according to the above formula (1), the fault threshold A can be set to A≤24, for example, the fault threshold A can be set to 12.

[0070] In one embodiment, the control unit 120 may be configured to selectively perform fault handling operations based on a comparison of the time difference with the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the control unit 120 may perform fault handling operations; when the time difference is less than the fault tolerance time interval of the electronic rearview mirror system, the control unit 120 may determine the number of times a freeze detection result indicating a freeze fault has occurred in the current frame, determine the duration of the fault based on the number of times and the time difference, and selectively perform fault handling operations based on a comparison of the duration of the fault with the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the duration of the fault is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the control unit 120 may perform fault handling operations; when the duration of the fault is less than the fault tolerance time interval of the electronic rearview mirror system, the control unit 120 may continue to determine the duration of the fault. In one embodiment, the control unit 120 may determine the product of the number of times a freeze detection result indicating a freeze fault has occurred in the current frame and the image update cycle of the electronic rearview mirror system, and determine the sum of the product and the time difference as the fault duration.

[0071] For example, the duration T of the fault can be determined by the following formula (2):

[0072] T = K × T1 + △T (Formula 2)

[0073] Where K represents the number of times a freeze detection result is generated to indicate that the image of the current frame has frozen, T1 represents the image update cycle of the electronic rearview mirror system, and ΔT represents the time difference.

[0074] By converting the delay in detecting each frame of an image into the delay in detecting the first frame of an image (i.e., the time difference between the acquisition time of the first frame of an image and the transmission time of the exposure command for the first frame of an image, as described above) and the freezing of each frame of an image (i.e., the freezing detection result generated by comparing two adjacent frames of the vehicle's external rear-view environment, as described above), the design difficulty of detecting the delay in detecting each frame of an image is reduced (i.e., the detection of the transmission time of the exposure command for each frame of an image and the acquisition time of each frame of an image is difficult and inaccurate), and the accuracy of fault detection is improved.

[0075] It should be noted that the processing unit 110 can compare the images of two adjacent frames in the left and right display screens of the electronic rearview mirror system to generate the freeze detection result of the left display screen and the freeze detection result of the right display screen. The control unit 120 can determine the time difference of the first frame image of the left display screen and the time difference of the first frame image of the right display screen for the first frame image in the left and right display screens respectively. Based on one or more of the freeze detection result of the left display screen and the time difference of the first frame image of the left display screen, the control unit 120 performs the fault handling operation of the left display screen. Based on one or more of the freeze detection result of the right display screen and the time difference of the first frame image of the right display screen, the control unit 120 performs the fault handling operation of the right display screen.

[0076] The electronic rearview mirror system according to one or more embodiments of this application detects two adjacent frames of images of the vehicle's external rear view environment and the time difference between the sending time of the exposure command for the first frame image and the acquisition time of the first frame image. This enables timely detection of display screen malfunctions and allows the system to control the display screen to overlay fault alarm information, thereby improving the reliability and safety of the electronic rearview mirror system and ensuring vehicle driving safety. The electronic rearview mirror system according to one or more embodiments of this application meets the ISO26262 functional safety requirements of ASIL B level.

[0077] The following will combine Figure 2 This application provides a detailed description of specific implementations of an electronic rearview mirror system according to one or more embodiments.

[0078] Figure 2 A structural block diagram of an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0079] like Figure 2 As shown, the electronic rearview mirror system 200 includes a camera module 210, a controller 220, and a screen module 230. The camera module 210 includes a CMOS chip 2101, the controller 220 includes an image signal processing unit 2201 and a first control unit 2202, and the screen module 230 includes a bridge chip 2301, a second control unit 2302, a driving unit 2303, and a display screen 2304. In one embodiment, Figure 1 The processing unit 110 shown can be used by means of Figure 2 The bridging chip 2301 shown is used to implement this. Figure 1 The control unit 120 shown can be used by means of Figure 2The second control unit 2302 shown is used to implement this. Optionally, the camera module 210 may include a left camera module and a right camera module respectively mounted on the left and right sides of the vehicle to acquire images of the rear side of the vehicle, and the screen module 230 may include a left screen module for processing and displaying images captured by the left camera module and a right screen module for processing and displaying images captured by the right camera module.

[0080] The CMOS chip 2101 is used to acquire raw image data of the vehicle's external rear-view environment and, for example, transmit the raw image data to the image signal processing unit 2201 via a Gigabit Multimedia Serial Link (GMSL).

[0081] The image signal processing unit 2201 is used to process the raw image data, such as denoising, color correction, dynamic range optimization, etc., to obtain a processed image, and sends the processed image to the bridge chip 2301, for example, via GMSL.

[0082] The bridging chip 2301 is used to convert the processed image into a format, such as converting the processed image from RAW format to RGB format or YUV format, to obtain a format-converted image, and then sending the format-converted image to the driver unit 2303.

[0083] The driving unit 2303 can drive the display screen 2304 to display based on the format-converted image.

[0084] The first control unit 2202 is used to control the exposure of the CMOS chip 2101, for example, it can control the exposure of the CMOS chip 2101 in response to receiving an exposure command from the second control unit 2302. For example, the second control unit 2302 can send the exposure command to the first control unit 2202 via a general purpose input / output port.

[0085] In one embodiment, the bridging chip 2301 can compare two adjacent frames of images received from the image signal processing unit 2201 to generate a freeze detection result, and send the freeze detection result to the second control unit 2302. In one embodiment, when it is determined that the image of the current frame in two adjacent frames is consistent with the image of the previous frame, the bridging chip 2301 can generate a freeze detection result indicating that the image of the current frame has a freeze fault; when it is determined that the image of the current frame in two adjacent frames is inconsistent with the image of the previous frame, the bridging chip 2301 can generate a freeze detection result indicating that the image of the current frame has not a freeze fault. Exemplarily, the bridging chip 2301 and the second control unit 2302 can be communicatively connected via a Serial Peripheral Interface (SPI).

[0086] In one embodiment, the second control unit 2302 can set an initial count value, increment or decrement the initial count value based on the freeze detection result from the bridging chip 2301 to obtain a fault count value, and selectively perform fault handling operations based on a comparison of the fault count value with a fault threshold. In one embodiment, when the fault count value is greater than or equal to the fault threshold, the second control unit 2302 can perform a fault handling operation; when the fault count value is less than the fault threshold, the second control unit 2302 can continue to receive freeze detection results from the bridging chip 2301 and increment or decrement the fault count value. In one embodiment, when the freeze detection result indicates that the current frame image has a freeze fault, the second control unit 2302 can increment the initial count value by two; when the freeze detection result indicates that the current frame image has not a freeze fault, the second control unit 2302 can decrement the initial count value by one.

[0087] In one embodiment, the second control unit 2302 can determine the sending time of the exposure command for sending the first frame image and the acquisition time of the first frame image by the bridging chip 2301, determine the time difference between the acquisition time and the sending time, and perform a fault handling operation based on one or more of the freeze detection result from the bridging chip 2301 and the time difference. For example, the bridging chip 2301 can record the acquisition time of the first frame image from the image signal processing unit 2201 and send that acquisition time to the second control unit 2302. In one embodiment, the second control unit 2302 can selectively perform a fault handling operation based on a comparison of the time difference with the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the second control unit 2302 can perform a fault handling operation; when the time difference is less than the fault tolerance time interval of the electronic rearview mirror system, the second control unit 2302 can determine the number of times it receives a freeze detection result from the bridging chip 2301 indicating that the image of the current frame has frozen, determine the duration of the fault based on the number of times and the time difference, and selectively perform a fault handling operation based on a comparison of the duration of the fault with the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the duration of the fault is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the second control unit 2302 can perform a fault handling operation; when the duration of the fault is less than the fault tolerance time interval of the electronic rearview mirror system, the second control unit 2302 can continue to determine the duration of the fault.

[0088] In one embodiment, the fault handling operation may include the second control unit 2302 sending a control command to the bridging chip 2301 to control the bridging chip 2301 to overlay fault alarm information on the display screen 2304 using the OSD mapping function. In another embodiment, the fault handling operation may further include the second control unit 2302 sending a control command to the body controller via the CAN bus to control the vehicle to activate the hazard warning lights.

[0089] In one embodiment, in the electronic rearview mirror system 200, the bridging chip 2301 and the second control unit 2302 can meet the ISO26262 functional safety requirements of ASIL B level, while the other modules have no functional safety requirements, thereby improving the reliability and safety of the electronic rearview mirror system at low cost.

[0090] Figure 3 A flowchart illustrating a fault handling method for an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0091] like Figure 3 As shown, in step S301, images of two adjacent frames of the vehicle's external rear-view environment are compared to generate a freeze detection result, and the sending time of the exposure command for sending the first frame image and the acquisition time of acquiring the first frame image are determined.

[0092] Optionally, in step S301, it can be determined whether the image of the current frame in two adjacent frames is consistent with the image of the previous frame. When it is determined that the image of the current frame in two adjacent frames is consistent with the image of the previous frame, a freeze detection result can be generated to indicate that the image of the current frame has a freeze fault; and when it is determined that the image of the current frame in two adjacent frames is inconsistent with the image of the previous frame, a freeze detection result can be generated to indicate that the image of the current frame has not a freeze fault.

[0093] In step S303, the time difference between the acquisition time of the first frame image and the transmission time of the exposure command for the first frame image is determined.

[0094] In step S305, a fault handling operation is performed based on one or more of the freeze detection results and time differences. The fault handling operation includes displaying fault alarm information superimposed on the display screen.

[0095] Optionally, in step S305, an initial count value can be determined in response to the first frame image, the initial count value can be incremented or decremented based on the freeze detection result to obtain a fault count value, and a fault handling operation can be selectively performed based on a comparison between the fault count value and a fault threshold. In one embodiment, when the fault count value is greater than or equal to the fault threshold, a fault handling operation can be performed; when the fault count value is less than the fault threshold, the fault count value can continue to be incremented or decremented based on the freeze detection result. In one embodiment, when the freeze detection result indicates that the current frame image has a freeze fault, the initial count value can be incremented by two; when the freeze detection result indicates that the current frame image has not a freeze fault, the initial count value can be decremented by one.

[0096] Optionally, in step S305, a fault handling operation can be selectively performed based on a comparison between the time difference and the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, a fault handling operation can be performed; when the time difference is less than the fault tolerance time interval of the electronic rearview mirror system, the number of times a freeze detection result indicating a freeze fault in the current frame can be generated can be determined, the duration of the fault can be determined based on the number of times and the time difference, and a fault handling operation can be selectively performed based on a comparison between the duration of the fault and the fault tolerance time interval of the electronic rearview mirror system. In one embodiment, when the duration of the fault is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, a fault handling operation can be performed; when the duration of the fault is less than the fault tolerance time interval of the electronic rearview mirror system, the duration of the fault can be further determined.

[0097] According to one aspect of this application, the fault handling method for an electronic rearview mirror system detects two adjacent frames of images of the vehicle's external rear view environment and the time difference between the sending time of the exposure command for the first frame image and the acquisition time of the first frame image. This allows for timely detection of a fault in the electronic rearview mirror system's display screen, thereby controlling the display screen to overlay fault alarm information. This improves the reliability and safety of the electronic rearview mirror system and ensures vehicle driving safety. The fault handling method for the electronic rearview mirror system according to one or more embodiments of this application has simple judgment logic, does not rely on additional detection and control circuits, has high reliability, and offers advantages such as cost savings, simple control, and ease of implementation.

[0098] Figure 4 A flowchart illustrating a fault handling method for an electronic rearview mirror system according to one or more embodiments of this application is shown.

[0099] like Figure 4As shown, in step S401, images of two adjacent frames of the vehicle's external rear-view environment are compared to generate a freeze detection result, and the time difference between the acquisition time of the first frame image and the transmission time of the exposure command for the first frame image is determined.

[0100] In step S403, the fault count value is determined based on the freeze detection results.

[0101] Optionally, in step S403, an initial count value can be determined in response to the first frame image, and the initial count value can be incremented or decremented based on the freeze detection result to obtain a fault count value. In one embodiment, when the freeze detection result indicates that the current frame image has a freeze fault, the initial count value can be incremented by two; when the freeze detection result indicates that the current frame image has not a freeze fault, the initial count value can be decremented by one.

[0102] In step S405, it is determined whether the fault count value is greater than or equal to the fault threshold. If the fault count value is greater than or equal to the fault threshold, proceed to step S407; otherwise, proceed to step S403 to determine the fault count value based on the freeze detection result of the next frame image.

[0103] In step S407, a fault handling operation is performed.

[0104] In step S409, it is determined whether the time difference is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system. If the time difference is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, proceed to step S411; otherwise, proceed to step S413.

[0105] In step S411, a fault handling operation is performed.

[0106] In step S413, the duration of the fault is determined based on the number of times and the time difference of generating freeze detection results that indicate that the image in the current frame has frozen.

[0107] In step S415, it is determined whether the duration of the fault is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system. If the duration of the fault is greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, proceed to step S417; otherwise, proceed to step S413 to determine the duration of the fault based on the freeze detection result of the next frame image.

[0108] In step S417, a fault handling operation is performed.

[0109] In one embodiment, steps S403, S405, and S407 can be used to detect and handle a freezing fault in the display screen of the electronic rearview mirror system, and steps S409, S411, S413, S415, and S417 can be used to detect and handle a delay fault in the display screen of the electronic rearview mirror system. In one embodiment, steps S403, S405, and S407 can be executed in parallel with steps S409, S411, S413, S415, and S417. It should be noted that... Figure 4 Using an electronic rearview mirror system as an example, this paper describes a fault handling method for an electronic rearview mirror system according to one or more embodiments of the present application, without departing from the spirit and scope of the present application. Figure 4 The troubleshooting methods shown can be applied to other vehicle vision assistance systems, such as, but not limited to, reversing camera systems.

[0110] Figure 5 A structural block diagram of a vehicle vision assistance system according to one or more embodiments of this application is shown.

[0111] like Figure 5 As shown, the vehicle vision assistance system 500 includes a processing unit 510 and a control unit 520 coupled to the processing unit 510.

[0112] The processing unit 510 is configured to compare images of two adjacent frames of the vehicle's external environment to generate a freeze detection result.

[0113] In one embodiment, the processing unit 510 can be configured to compare the image of the current frame with the image of the previous frame in two adjacent frames of the vehicle's external environment to determine whether the image of the current frame is consistent with the image of the previous frame. When it is determined that the image of the current frame is consistent with the image of the previous frame, the processing unit 510 can generate a freeze detection result indicating that the image of the current frame has a freeze fault, and send the freeze detection result indicating that the image of the current frame has a freeze fault to the control unit 520; when it is determined that the image of the current frame is inconsistent with the image of the previous frame, the processing unit 510 can continue to compare whether the image of the next frame is consistent with the image of the current frame, and so on.

[0114] In one embodiment, the processing unit 510 can be configured to compare each pixel value of the region of interest in the current frame with the corresponding pixel value of the region of interest in the previous frame. When each pixel value of the region of interest in the current frame is the same as the corresponding pixel value of the region of interest in the previous frame, it can be determined that the current frame is consistent with the previous frame. When one or more pixel values ​​of the region of interest in the current frame are different from the corresponding pixel values ​​of the region of interest in the previous frame, it can be determined that the current frame is inconsistent with the previous frame. For example, the central region of the image can be selected as the region of interest.

[0115] The control unit 520 is configured to, for a first frame image, determine the transmission time of the exposure command for transmitting the first frame image and the acquisition time of the first frame image acquired by the processing unit 510, determine the time difference between the acquisition time and the transmission time, and perform a fault handling operation based on one or more of the freeze detection result and the time difference. The fault handling operation includes generating a control signal for overlaying fault alarm information onto the display screen of the vehicle vision assistance system. Optionally, the fault handling operation may also include generating a voice prompt to remind the driver to drive cautiously.

[0116] In one embodiment, the processing unit 510 can be implemented as a bridging chip for a vehicle vision assistance system. This bridging chip can be configured to overlay fault alarm information onto a display screen in response to a control signal from the control unit 520 for overlaying fault alarm information onto the display screen, using the OSD mapping function of the bridging chip. By utilizing the bridging chip to generate freeze detection results and overlay fault alarm information, no other hardware and software processing units are required in the vehicle vision assistance system, resulting in high reliability and advantages such as cost savings, simple control, and ease of implementation. In one embodiment, the vehicle vision assistance system 500 may include an electronic rearview mirror system and a reversing camera system.

[0117] In one embodiment, the control unit 520 can be configured to determine the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received, and to selectively perform a fault handling operation based on a comparison of that number with a threshold number. In one embodiment, when the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received is greater than or equal to the threshold number, the control unit 520 can perform a fault handling operation; when the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received is less than the threshold number, the control unit 520 can continue to determine the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received based on the freeze detection result. For example, the threshold number can be set according to the image update period T1 (i.e., the update period per frame) and the fault tolerance time interval T2 of the vehicle vision assistance system. For example, the fault threshold A can be set according to the following formula (3):

[0118] T1×A≤T2×2 formula (3)

[0119] In one embodiment, the control unit 520 can be configured to selectively perform fault handling operations based on a comparison of the time difference with the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the control unit 520 can perform fault handling operations; when the time difference is less than the fault tolerance time interval of the vehicle vision assistance system, the control unit 520 can determine the number of consecutive freeze detection results indicating a freeze fault in the current frame, determine the fault duration based on the number of consecutive freeze detection results and the time difference, and selectively perform fault handling operations based on a comparison of the fault duration with the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the fault duration is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the control unit 520 can perform fault handling operations; when the fault duration is less than the fault tolerance time interval of the vehicle vision assistance system, the control unit 520 can continue to determine the fault duration. In one embodiment, the control unit 520 can determine the product of the number of consecutive freeze detection results indicating a freeze fault in the current frame and the image update period of the vehicle vision assistance system, and determine the fault duration by summing the product with the time difference.

[0120] For example, the duration T of the fault can be determined by the following formula (4):

[0121] T = K × T1 + △T Formula (4)

[0122] Where K represents the number of times a freeze detection result indicating a freeze fault has occurred in the current frame is received consecutively, T1 represents the image update cycle of the vehicle vision assistance system, and ΔT represents the time difference.

[0123] By converting the delay in detecting each frame of an image into the delay in detecting the first frame of an image (i.e., the time difference between the acquisition time of the first frame of an image and the transmission time of the exposure command for the first frame of an image, as described above) and the freezing of each frame of an image (i.e., the frozen detection result generated by comparing two adjacent frames of the vehicle's external environment, as described above), the design difficulty of detecting the delay in detecting each frame of an image is reduced (i.e., the detection of the transmission time of the exposure command for each frame of an image and the acquisition time of each frame of an image is difficult and inaccurate), thereby improving the accuracy of fault detection in the vehicle vision assistance system.

[0124] In one embodiment, when the vehicle vision assistance system 500 is implemented as an electronic rearview mirror system, the processing unit 510 can compare the images of two adjacent frames in the display screens on the left and right sides of the electronic rearview mirror system to generate a freeze detection result for the left display screen and a freeze detection result for the right display screen. The control unit 520 can determine the time difference of the first frame image of the left display screen and the time difference of the first frame image of the right display screen for the first frame image in the display screens on the left and right sides respectively. Based on one or more of the freeze detection result of the left display screen and the time difference of the first frame image of the left display screen, the control unit 520 performs a fault handling operation for the left display screen and performs a fault handling operation for the right display screen based on one or more of the freeze detection result of the right display screen and the time difference of the first frame image of the right display screen.

[0125] In one embodiment, when the vehicle vision assistance system 500 is implemented as a reversing camera system, the processing unit 510 can compare the images of two adjacent frames acquired by each camera module of the reversing camera system (e.g., front, rear, left, and right camera modules) to generate a freeze detection result for each camera module. The control unit 520 can determine the time difference between the acquisition time of the first frame image and the transmission time of the exposure command for the first frame image for each camera module, and perform fault handling operations based on one or more of the freeze detection result and the time difference of the first frame image for each camera module. In one embodiment, when the vehicle vision assistance system 500 is implemented as a reversing camera system, the control unit 520 can be configured to determine the transmission time of the exposure command for the first frame image based on the time when the reverse gear signal is received. For example, the time when the reverse gear signal is received can be determined as the transmission time of the exposure command for the first frame image to reduce design difficulty and design cost.

[0126] The vehicle vision assistance system according to one or more embodiments of this application detects the time difference between the sending time of the exposure command of the first frame image and the acquisition time of the first frame image by detecting two adjacent frames of the vehicle's external environment and the time difference between the acquisition time of the first frame image. This enables timely detection of a fault in the display screen of the vehicle vision assistance system and controls the display screen to overlay fault alarm information, thereby improving the reliability and safety of the vehicle vision assistance system and ensuring vehicle driving safety.

[0127] Figure 6 A structural block diagram of a reversing camera system according to one or more embodiments of this application is shown.

[0128] like Figure 6 As shown, the reversing camera system 600 includes a camera module 610, a controller 620, and a display screen 630. The camera module 610 includes a CMOS chip 6101, the controller 620 includes an image signal processing unit 6201 and a first control unit 6202, and the screen module 630 includes a bridge chip 6301, a second control unit 6302, a drive unit 6303, and a display panel 6304. In one embodiment, Figure 5 The processing unit 510 shown can be used by means of Figure 6 The bridge chip 6301 shown is used to implement this. Figure 5 The control unit 520 shown can be used by means of Figure 6 The second control unit 6302 shown is used to implement this. Optionally, the camera module 610 may include multiple cameras at the front, rear, left, and right of the vehicle, the controller 620 may be implemented as a cockpit controller, and the screen module 630 may be implemented as a central control display screen.

[0129] The CMOS chip 6101 is used to acquire raw image data of the vehicle's external environment and transmits the raw image data to the image signal processing unit 6201, for example, via a Gigabit Multimedia Serial Link (GMSL).

[0130] The image signal processing unit 6201 is used to process the raw image data, such as denoising, color correction, dynamic range optimization, etc., to obtain a processed image, and sends the processed image to the bridge chip 6301, for example, via GMSL.

[0131] The bridging chip 6301 is used to convert the processed image into a format, such as converting the processed image from RAW format to RGB format or YUV format, to obtain a format-converted image, and then sending the format-converted image to the driver unit 6303.

[0132] The driving unit 6303 can drive the display panel 6304 to display based on the format-converted image.

[0133] The first control unit 6202 is used to control the exposure of the CMOS chip 6101, for example, it can control the exposure of the CMOS chip 6101 in response to receiving an exposure command from the second control unit 6302. For example, the second control unit 6302 can send the exposure command to the first control unit 6202 via a general purpose input / output port.

[0134] In one embodiment, the bridging chip 6301 can compare two adjacent frames of images received from the image signal processing unit 6201 to generate a freeze detection result and send the freeze detection result to the second control unit 6302. In one embodiment, when it is determined that the image of the current frame in two adjacent frames is consistent with the image of the previous frame, the bridging chip 6301 can generate a freeze detection result indicating that the image of the current frame has a freeze fault, and send the freeze detection result indicating that the image of the current frame has a freeze fault to the second control unit 6302; when it is determined that the image of the current frame in two adjacent frames is inconsistent with the image of the previous frame, the bridging chip 6301 can continue to compare whether the image of the next frame is consistent with the image of the current frame, and so on. In one embodiment, the bridging chip 6301 can determine the acquisition time of the first frame image and send the acquisition time of the first frame image to the second control unit 6302. By using the bridging chip 6301 to detect frame freeze of the image and sending the acquisition time of the first frame image to the second control unit 6302, the design chain of fault detection can be reduced and the reliability of fault detection can be improved.

[0135] In one embodiment, the second control unit 6302 can determine the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received, and selectively perform a fault handling operation based on a comparison of the number of times with a threshold number. In one embodiment, when the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received is greater than or equal to the threshold number, the second control unit 6302 can perform a fault handling operation; when the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received is less than the threshold number, the second control unit 6302 can continue to determine the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received.

[0136] In one embodiment, the second control unit 6302 can determine the sending time of the exposure command for sending the first frame image and the acquisition time of the first frame image by the bridging chip 6301, determine the time difference between the acquisition time and the sending time, and perform a fault handling operation based on one or more of the freeze detection result from the bridging chip 6301 and the time difference. For example, the bridging chip 6301 can record the acquisition time of the first frame image from the image signal processing unit 6201 and send that acquisition time to the second control unit 6302. In one embodiment, the second control unit 6302 can determine the sending time of the exposure command for sending the first frame image based on the time when the reverse gear signal is received, for example, determining the time when the reverse gear signal is received as the sending time of the exposure command for sending the first frame image. By utilizing the aforementioned time difference to perform delay detection on the first frame image, detection accuracy is improved and design complexity is reduced.

[0137] In one embodiment, the second control unit 6302 can selectively perform fault handling operations based on a comparison of the time difference with the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the second control unit 6302 can perform fault handling operations; when the time difference is less than the fault tolerance time interval of the vehicle vision assistance system, the second control unit 6302 can determine the number of times it continuously receives freeze detection results from the bridging chip 6301 indicating a freeze fault in the current frame, determine the fault duration based on the number of times and the time difference, and selectively perform fault handling operations based on a comparison of the fault duration with the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the fault duration is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the second control unit 6302 can perform fault handling operations; when the fault duration is less than the fault tolerance time interval of the vehicle vision assistance system, the second control unit 6302 can continue to determine the fault duration. By selectively performing fault handling operations based on the comparison between the time difference and the fault tolerance time interval of the vehicle vision assistance system, the delay detection of each frame of image can be transformed into delay detection of the first frame of image and freeze detection of each frame of image. This design reduces the design difficulty of using a timestamp mechanism for each frame of image.

[0138] In one embodiment, the fault handling operation may include sending a control command from the second control unit 6302 to the bridging chip 6301 to control the bridging chip 6301 to overlay fault alarm information onto the display screen 6304 using the OSD mapping function. By using the OSD mapping function of the bridging chip, clear fault reminders can be displayed on the screen of the vehicle vision assistance system through OSD mapping, improving the alertness to vehicle users and eliminating the need to occupy other display devices in the vehicle. In one embodiment, the fault handling operation may further include generating voice prompts to indicate that the vehicle vision assistance system has malfunctioned.

[0139] In one embodiment, in the reversing camera system 600, the bridging chip 6301 and the second control unit 6302 can meet the ISO26262 functional safety requirements of ASIL A level, while the remaining modules have no functional safety requirements, thereby improving the reliability and safety of the reversing camera system at low cost. In one embodiment, the first control unit 6202 and the second control unit 6302 can be implemented as microcontrollers, respectively. One or more embodiments of the vehicle vision assistance system in this application do not add additional components and utilize an end-to-end diagnostic mechanism, reducing the safety requirements for intermediate modules (e.g., besides...). Figure 6 Apart from the bridging chip 6301 and the second control unit 6302 shown, no functional safety requirements are put forward for other modules, which reduces the design difficulty and design cost while meeting the functional safety requirements.

[0140] Figure 7 A flowchart is shown of a fault handling method for a vehicle vision assistance system according to one or more embodiments of this application.

[0141] like Figure 7 As shown, in step S701, images of two adjacent frames of the vehicle's external environment are compared to generate a freeze detection result, and the sending time of the exposure command for sending the first frame image and the acquisition time of acquiring the first frame image are determined.

[0142] In step S703, the time difference between the acquisition time of the first frame image and the transmission time of the exposure command for the first frame image is determined.

[0143] In step S705, a fault handling operation is performed based on one or more of the freeze detection results and time differences, which includes displaying fault alarm information overlaid on the display screen.

[0144] Optionally, in step S705, a fault handling operation can be selectively performed based on a comparison between the time difference and the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the time difference is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, a fault handling operation can be performed; when the time difference is less than the fault tolerance time interval of the vehicle vision assistance system, the number of times a freeze detection result indicating a freeze fault in the current frame is continuously received can be determined, the fault duration can be determined based on the number of times and the time difference, and a fault handling operation can be selectively performed based on a comparison between the fault duration and the fault tolerance time interval of the vehicle vision assistance system. In one embodiment, when the fault duration is greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, a fault handling operation can be performed; when the fault duration is less than the fault tolerance time interval of the vehicle vision assistance system, the fault duration can be further determined.

[0145] According to one aspect of this application, the fault handling method for a vehicle vision assistance system detects two adjacent frames of images of the vehicle's external environment and the time difference between the sending time of the exposure command for the first frame image and the acquisition time of the first frame image. This enables timely detection of a fault in the display screen of the vehicle vision assistance system and allows for the control of the display screen to overlay fault alarm information, thereby improving the reliability and safety of the vehicle vision assistance system and ensuring vehicle driving safety. The fault handling method for the vehicle vision assistance system according to one or more embodiments of this application has simple judgment logic, does not rely on additional detection and control circuits, has high reliability, and offers advantages such as cost savings, simple control, and ease of implementation.

[0146] Alternatively, this application can also be implemented as a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a fault handling method for an electronic rearview mirror system according to one aspect of this application. Alternatively, this application can also be implemented as a computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a fault handling method for a vehicle vision assistance system according to one aspect of this application.

[0147] The term "computer-readable medium" as used in this application includes various types of computer storage media, and can be any available medium accessible to a general-purpose or special-purpose computer. For example, a computer-readable medium may include RAM, ROM, EPROM, E2PROM, registers, hard disk, removable disk, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or any other temporary or non-temporary medium capable of carrying or storing desired program code units in the form of instructions or data structures and accessible by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. As used herein, disks typically magnetically copy data, while discs optically copy data using lasers. Combinations of the above should also be included within the scope of protection for computer-readable media. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0148] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Those skilled in the art can conceive of other feasible variations or substitutions based on the technical scope disclosed in this application, and such variations or substitutions are all covered within the scope of protection of this application. Where there is no conflict, the embodiments and features described in these embodiments can also be combined with each other.

Claims

1. An electronic rearview mirror system, characterized in that, The electronic rearview mirror system includes: The processing unit is configured to compare two adjacent frames of images of the vehicle's external rear-view environment to generate a freeze detection result; The control unit is coupled to the processing unit and configured to: For the first frame image, the sending time of the exposure command for sending the first frame image and the acquisition time of the processing unit for acquiring the first frame image are determined; Determine the time difference between the acquisition time and the transmission time; and Based on one or more of the freeze detection results and the time difference, a fault handling operation is performed, the fault handling operation including generating a control signal for overlaying fault alarm information on a display screen. The control unit is further configured to perform fault handling operations in the following manner: In response to the time difference being less than the fault tolerance time interval of the electronic rearview mirror system, the number of times a freeze detection result indicating a freeze fault has occurred in the current frame is generated is determined; the duration of the fault is determined based on the number of times and the time difference; and the fault handling operation is selectively performed based on a comparison between the duration of the fault and the fault tolerance time interval of the electronic rearview mirror system. In response to the time difference being greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the fault handling operation is performed. The control unit is further configured to determine the duration of the fault based on the number of occurrences and the time difference in the following manner: Determine the product of the number of times and the image update cycle of the electronic rearview mirror system; and The sum of the product and the time difference is determined as the duration of the fault.

2. The electronic rearview mirror system according to claim 1, wherein the processing unit is a bridging chip of the electronic rearview mirror system, and the bridging chip is configured to display the fault alarm information superimposed on the display screen through the OSD mapping function of the bridging chip in response to the control signal. The processing unit is further configured to: In response to determining that the image of the current frame in the two adjacent frames is consistent with the image of the previous frame, a freeze detection result is generated to indicate that the image of the current frame has a freeze fault; as well as In response to determining that the image of the current frame in two adjacent frames is inconsistent with the image of the previous frame, a freeze detection result is generated to indicate that the image of the current frame has not experienced a freeze fault. The processing unit is further configured to determine whether the image of the current frame in two adjacent frames is identical to the image of the previous frame by means of the following method: Compare the pixel value of each pixel in the region of interest in the current frame with the pixel value of the corresponding position in the region of interest in the previous frame; In response to the fact that each pixel value of the region of interest in the current frame is the same as the pixel value of the corresponding position of the region of interest in the previous frame, it is determined that the current frame is consistent with the previous frame. as well as If one or more pixel values ​​in the region of interest (ROI) of the current frame are different from the corresponding pixel values ​​in the ROI of the previous frame, it is determined that the current frame is inconsistent with the previous frame. The control unit is further configured to perform fault handling operations in the following manner: An initial count value is determined in response to the first frame image; The initial count value is incremented or decremented based on the freeze detection results to obtain the fault count value; and The fault handling operation is selectively performed based on a comparison between the fault count value and the fault threshold. The control unit is further configured to increment or decrement the initial count value in the following manner: In response to the freeze detection result indicating that the current frame image has a freeze fault, the initial count value is incremented by two; and In response to the freeze detection result indicating that the current frame image has not experienced a freeze fault, the initial count value is decremented by one. The fault handling operation also includes generating a control signal for controlling the vehicle to turn on its hazard warning lights.

3. A fault handling method for an electronic rearview mirror system, characterized in that, The fault handling method includes: Compare two adjacent frames of the vehicle's external rear-view environment to generate a freeze detection result, and determine the sending time of the exposure command for sending the first frame image and the acquisition time of the first frame image; Determine the time difference between the acquisition time and the transmission time; and Based on one or more of the freeze detection results and the time difference, a fault handling operation is performed, the fault handling operation including generating a control signal for overlaying fault alarm information on a display screen. The fault handling operations performed based on one or more of the freeze detection results and the time difference include: In response to the time difference being less than the fault tolerance time interval of the electronic rearview mirror system, the number of times a freeze detection result indicating a freeze fault has occurred in the current frame is generated is determined; the duration of the fault is determined based on the number of times and the time difference; and the fault handling operation is selectively performed based on a comparison between the duration of the fault and the fault tolerance time interval of the electronic rearview mirror system. In response to the time difference being greater than or equal to the fault tolerance time interval of the electronic rearview mirror system, the fault handling operation is performed. Determining the fault duration based on the number of occurrences and the time difference includes: Determine the product of the number of times and the image update cycle of the electronic rearview mirror system; and The sum of the product and the time difference is determined as the duration of the fault.

4. The fault handling method according to claim 3, wherein the fault alarm information is superimposed and displayed on the display screen through the OSD mapping function of the bridging chip of the electronic rearview mirror system. The comparison of two adjacent frames of the vehicle's external rear-view environment to generate freeze detection results includes: In response to determining that the image of the current frame in the two adjacent frames is consistent with the image of the previous frame, a freeze detection result is generated to indicate that the image of the current frame has a freeze fault; as well as In response to determining that the image of the current frame in two adjacent frames is inconsistent with the image of the previous frame, a freeze detection result is generated to indicate that the image of the current frame has not experienced a freeze fault. Determining whether the image of the current frame in two adjacent frames is identical to the image of the previous frame includes: Compare the pixel value of each pixel in the region of interest in the current frame with the pixel value of the corresponding position in the region of interest in the previous frame; In response to the fact that each pixel value of the region of interest in the current frame is the same as the pixel value of the corresponding position of the region of interest in the previous frame, it is determined that the current frame is consistent with the previous frame; and If one or more pixel values ​​in the region of interest (ROI) of the current frame are different from the corresponding pixel values ​​in the ROI of the previous frame, it is determined that the current frame is inconsistent with the previous frame. The fault handling operations performed based on one or more of the freeze detection results and the time difference include: An initial count value is determined in response to the first frame image; The initial count value is incremented or decremented based on the freeze detection results to obtain the fault count value; and The fault handling operation is selectively performed based on a comparison between the fault count value and the fault threshold. Increasing or decreasing the initial count value based on the freeze detection result includes: In response to the freeze detection result indicating that the current frame image has a freeze fault, the initial count value is incremented by two; and In response to the freeze detection result indicating that the current frame image has not experienced a freeze fault, the initial count value is decremented by one. The fault handling operation also includes generating a control signal for controlling the vehicle to turn on its hazard warning lights.

5. A vehicle vision assistance system, characterized in that, The vehicle vision assistance system includes: The processing unit is configured to compare two adjacent frames of images of the vehicle's external environment to generate a freeze detection result; The control unit is coupled to the processing unit and configured to: For the first frame image, the sending time of the exposure command for sending the first frame image and the acquisition time of the processing unit for acquiring the first frame image are determined; Determine the time difference between the acquisition time and the transmission time; and Based on one or more of the freeze detection results and the time difference, a fault handling operation is performed, the fault handling operation including generating a control signal for overlaying fault alarm information onto the display screen of the vehicle vision assistance system. The control unit is further configured to perform fault handling operations in the following manner: In response to the time difference being less than the fault tolerance time interval of the vehicle vision assistance system, the system determines the number of times a freeze detection result indicating a freeze fault has occurred in the current frame is continuously received; the system determines the fault duration based on the number of consecutive receptions and the time difference; and the system selectively performs the fault handling operation based on a comparison between the fault duration and the fault tolerance time interval of the vehicle vision assistance system. In response to the time difference being greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the fault handling operation is performed. The control unit is further configured to determine the duration of the fault based on the number of occurrences and the time difference in the following manner: Determine the product of the number of times and the image update period of the vehicle vision assistance system; and The sum of the product and the time difference is determined as the duration of the fault.

6. The vehicle vision assistance system according to claim 5, wherein the processing unit is a bridging chip of the vehicle vision assistance system, and the bridging chip is configured to display the fault alarm information superimposed on the display screen through the OSD mapping function of the bridging chip in response to the control signal. The processing unit is further configured to: In response to determining that the image of the current frame in the two adjacent frames is consistent with the image of the previous frame, a freeze detection result is generated to indicate that the image of the current frame has a freeze fault; as well as The freeze detection result, which indicates that the image in the current frame has frozen, is sent to the control unit. The processing unit is further configured to determine whether the image of the current frame in two adjacent frames is identical to the image of the previous frame by means of the following method: Compare the pixel value of each pixel in the region of interest in the current frame with the pixel value of the corresponding position in the region of interest in the previous frame; In response to the fact that each pixel value of the region of interest in the current frame is the same as the pixel value of the corresponding position of the region of interest in the previous frame, it is determined that the current frame is consistent with the previous frame. as well as If one or more pixel values ​​in the region of interest (ROI) of the current frame are different from the corresponding pixel values ​​in the ROI of the previous frame, it is determined that the current frame is inconsistent with the previous frame. The control unit is further configured to perform fault handling operations in the following manner: Determine the number of consecutive freeze detection results received, indicating a freeze fault in the current frame; and The fault handling operation is selectively performed based on a comparison between the stated number of occurrences and a threshold number of occurrences. The vehicle vision assistance system includes an electronic rearview mirror system and a reversing camera system. The control unit is further configured to: In response to the vehicle vision assistance system including the reversing camera system, the timing of sending the exposure command for sending the first frame image is determined based on the timing of receiving the reversing gear signal.

7. A fault handling method for a vehicle vision assistance system, characterized in that, The fault handling method includes: Compare two adjacent frames of images of the vehicle's external environment to generate a freeze detection result, and determine the sending time of the exposure command for sending the first frame image and the acquisition time of the first frame image; Determine the time difference between the acquisition time and the transmission time; and Based on one or more of the freeze detection results and the time difference, a fault handling operation is performed, the fault handling operation including generating a control signal for overlaying fault alarm information onto the display screen of the vehicle vision assistance system. The execution of fault handling operations based on one or more of the freeze detection results and the time difference includes: In response to the time difference being greater than or equal to the fault tolerance time interval of the vehicle vision assistance system, the fault handling operation is performed. In response to the time difference being less than the fault tolerance time interval of the vehicle vision assistance system, the number of times a freeze detection result indicating a freeze fault in the current frame is continuously generated is determined, and the fault duration is determined based on the number of times and the time difference; and The fault handling operation is selectively executed based on a comparison between the duration of the fault and the fault tolerance time interval of the vehicle vision assistance system. Determining the fault duration based on the number of occurrences and the time difference includes: Determine the product of the number of times and the image update period of the vehicle vision assistance system; and The sum of the product and the time difference is determined as the duration of the fault.

8. The fault handling method according to claim 7, wherein the fault alarm information is overlaid on the display screen through the OSD mapping function of the bridging chip of the vehicle vision assistance system. The comparison of two adjacent frames of the vehicle's external environment to generate freeze detection results includes: In response to determining that the image of the current frame in two adjacent frames is identical to the image of the previous frame, a freeze detection result is generated to indicate that a freeze fault has occurred in the image of the current frame. Determining whether the image of the current frame in two adjacent frames is identical to the image of the previous frame includes: Compare the pixel value of each pixel in the region of interest in the current frame with the pixel value of the corresponding position in the region of interest in the previous frame; In response to the fact that each pixel value of the region of interest in the current frame is the same as the pixel value of the corresponding position of the region of interest in the previous frame, it is determined that the current frame is consistent with the previous frame; and If one or more pixel values ​​in the region of interest (ROI) of the current frame are different from the corresponding pixel values ​​in the ROI of the previous frame, it is determined that the current frame is inconsistent with the previous frame. The fault handling operations performed based on one or more of the freeze detection results and the time difference include: Determine the number of times a freeze detection result indicating a freeze fault in the current frame is continuously generated; and The fault handling operation is selectively performed based on a comparison between the stated number of occurrences and a threshold number of occurrences. The vehicle vision assistance system includes an electronic rearview mirror system and a reversing camera system. The fault handling method further includes: In response to the vehicle vision assistance system including the reversing camera system, the timing of sending the exposure command for sending the first frame image is determined based on the timing of receiving the reversing gear signal.

9. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed by the processor, the processor performs the fault handling method for the electronic rearview mirror system according to any one of claims 3-4 or the fault handling method for the vehicle vision assistance system according to any one of claims 7-8.

10. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, implement the fault handling method for the electronic rearview mirror system according to any one of claims 3-4 or the fault handling method for the vehicle vision assistance system according to any one of claims 7-8.

Images