Electronic rearview mirror system and electronic rearview mirror display method

Abstract

The invention relates to the technical field of vehicle-mounted display, in particular to an electronic rearview mirror system and an electronic rearview mirror display method, and aims to solve the technical problems that an electronic rearview mirror is small in display visual field and poor in dazzle light eliminating effect. In order to achieve the purpose, the electronic rearview mirror system comprises an environment sensing module, a sensing result processing module and a display module. The environment sensing module obtains environment sensing results of multiple visual angles based on the environment sensing equipment of the multiple visual angles; the sensing result processing module obtains an environment sensing processing result according to the environment sensing results of the multiple visual angles; and the display module visually displays the environment perception processing result based on the display module group, so that a wider visual field can be realized, and dazzle light in the environment perception result can be removed. Meanwhile, when one environment sensing device is abnormal, other environment sensing devices can still be used normally, the function of the electronic rearview mirror system can be effectively ensured, and the safer function of the electronic rearview mirror system is achieved.

Description

Technical Field

[0001] The present application relates to the field of vehicle-mounted display technology, and in particular to an electronic rearview mirror system and an electronic rearview mirror display method. Background Art

[0002] CMS (Camera Monitor System, electronic rearview mirror) is an emerging in-vehicle display technology. It has the advantages of clear images, wide field of view, and low wind resistance, and is considered to be an alternative to traditional rearview mirrors in the future. With the advancement of in-vehicle technology, CMS technology has achieved rapid development in recent years. At present, related companies or R&D units have also launched some related products or demos (trial products). However, from a technical point of view, there is still a lack of mature CMS architecture solutions in the in-vehicle market. The main solution is "image sensor with conventional display screen", combined with some specific image processing algorithms to achieve image enhancement or "glare" elimination functions. Its technical solution only stays at the level of realizing the basic concept of CMS, and lacks systematic high-value solutions, which is mainly reflected in the following aspects:

[0003] 1) Small display field of view: The road condition information captured by the monocular camera is limited. Even if a wide-angle lens is added, the driver's field of view is still relatively narrow;

[0004] 2) Poor glare removal effect: Even if the road condition images taken by a monocular camera are combined with mainstream anti-glare algorithms, it is difficult to restore the real road condition scene after the glare is removed due to the occlusion of the glare area.

[0005] Accordingly, the art needs a new electronic rearview mirror solution to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem of how to solve the small display field of the electronic rearview mirror and the poor glare removal effect.

[0007] In a first aspect, an electronic rearview mirror system is provided, the system comprising an environment perception module, a perception result processing module and a display module; the environment perception module comprises an environment perception device of multiple viewing angles; the display module comprises a display module;

[0008] The environment perception module is configured as an environment perception device based on multiple perspectives to obtain environment perception results from multiple perspectives;

[0009] The perception result processing module is configured to obtain an environment perception processing result according to the environment perception results of the multiple perspectives;

[0010] The display module is configured to visually display the environmental perception processing result based on the display module.

[0011] In a technical solution of the above electronic rearview mirror system, the environment perception device is a camera; the environment perception result is an environment image collected by the camera; the perception result processing module includes an image fusion unit and a display driving unit;

[0012] The image fusion unit is configured to perform image fusion according to the environment images of the multiple perspectives to obtain an image fusion result;

[0013] The display driving unit is configured to obtain the environment perception processing result based on the display driving program and according to the image fusion result.

[0014] In a technical solution of the above electronic rearview mirror system, the image fusion unit includes a glare detection subunit and a fusion subunit;

[0015] The glare detection subunit is configured to perform glare detection on the environment image of each viewing angle respectively, and obtain the glare detection result of the environment image of each viewing angle;

[0016] The fusion subunit is configured to perform image fusion on the environmental images of multiple perspectives according to the glare detection results of the environmental images of multiple perspectives to obtain the image fusion result.

[0017] In a technical solution of the above electronic rearview mirror system, the cameras with multiple viewing angles are binocular cameras; and the fusion subunit is further configured as follows:

[0018] If both viewing angles of the binocular camera are normal, then according to the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environment images of the two viewing angles to obtain the image fusion result;

[0019] If one viewing angle of the binocular camera is normal, de-glare processing is performed on the environment image of the normal viewing angle according to the glare detection result of the normal viewing angle, and the image fusion result is obtained according to the processing result.

[0020] In a technical solution of the above electronic rearview mirror system, the display driver includes a display driver subroutine and a backlight driver subroutine; the display driver unit includes a display driver subunit and a backlight driver subunit; the environment perception processing result includes a display driver result and a backlight driver result;

[0021] The display driving subunit is configured to perform a grayscale compensation operation on the image fusion result based on the display driving subroutine, and obtain the display driving result based on the grayscale compensation operation result;

[0022] The backlight driving subunit is configured to perform a backlight extraction operation on the image fusion result based on the backlight driving subroutine, and obtain the backlight driving result based on the backlight extraction operation result.

[0023] In a technical solution of the above electronic rearview mirror system, the environment perception device is a camera; the camera is provided with a state detection sensor; the environment perception module includes a state detection unit and an environment perception unit;

[0024] The state detection unit is configured to obtain a state detection result of each of the cameras according to the state detection sensor;

[0025] The environment perception unit is configured to obtain the environment perception results of the multiple perspectives according to the state detection results and the cameras of the multiple perspectives.

[0026] In a technical solution of the electronic rearview mirror system, the state detection sensor includes a water mist sensor and / or a lens dirt sensor; the state detection result includes the presence of water mist abnormality and / or dirt abnormality on the camera;

[0027] The environmental perception unit is further configured to obtain the environmental perception result based on cameras of other perspectives where the water mist abnormality and the dirt abnormality do not exist when the state detection result of the camera is that there is water mist abnormality and / or dirt abnormality.

[0028] In a technical solution of the above electronic rearview mirror system, the state detection sensor includes an ambient light sensor; the state detection result includes an ambient light detection result; the camera includes a fill light;

[0029] The environmental perception unit is further configured to control the fill light of the camera to turn on when the ambient light detection result obtained by the environmental sensor is less than a preset ambient light threshold, so as to obtain the environmental perception results of the multiple perspectives according to the camera of the multiple perspectives.

[0030] In a technical solution of the above electronic rearview mirror system, the state detection unit is further configured to send the state detection result to the display module when the state detection result indicates that the camera is abnormal;

[0031] The display module is further configured to visually display the received status detection result based on the display module.

[0032] In a technical solution of the above electronic rearview mirror system, the system further includes a fault warning module;

[0033] The state detection unit is further configured to send the state detection result to the fault warning module when the state detection result indicates that the camera is abnormal;

[0034] The fault warning module is configured to provide a camera fault prompt based on a preset fault prompt device and according to the received status detection result.

[0035] In a technical solution of the above electronic rearview mirror system, the display module is a local dimming display module.

[0036] In a technical solution of the above-mentioned electronic rearview mirror system, the local dimming display module includes an LCD module and a Mini LED backlight module; the environmental perception processing result includes a display driving result and a backlight driving result; the display module includes a display unit and a backlight unit;

[0037] The display unit is configured to perform visual display of the display driving result based on the LCD module;

[0038] The backlight unit is configured to perform a visual display of the backlight driving result based on the Mini LED backlight module.

[0039] In a second aspect, an electronic rearview mirror display method is provided, the method comprising:

[0040] Based on the environment perception device with multiple perspectives, the environment perception results with multiple perspectives are obtained;

[0041] Obtaining environmental perception processing results according to the environmental perception results of the multiple perspectives;

[0042] Based on the display module, the environmental perception processing result is visualized and displayed.

[0043] In a technical solution of the above-mentioned electronic rearview mirror display method, the environment perception device is a camera; the environment perception result is an environment image captured by the camera;

[0044] The obtaining of the environmental perception processing results according to the environmental perception results of the multiple perspectives includes:

[0045] Performing image fusion according to the environment images from the multiple perspectives to obtain an image fusion result;

[0046] Based on the display driver, the environment perception processing result is obtained according to the image fusion result.

[0047] In a technical solution of the above-mentioned electronic rearview mirror display method, performing image fusion according to the environment images of the multiple perspectives to obtain the image fusion result includes:

[0048] Perform glare detection on the environment image of each viewing angle respectively, and obtain the glare detection result of the environment image of each viewing angle;

[0049] According to the glare detection results of the environmental images of multiple perspectives, image fusion is performed on the environmental images of the multiple perspectives to obtain the image fusion result.

[0050] In a technical solution of the above-mentioned electronic rearview mirror display method, the cameras with multiple viewing angles are binocular cameras;

[0051] The step of performing image fusion on the environment images of the multiple perspectives according to the glare detection results of the environment images of the multiple perspectives to obtain the image fusion result includes:

[0052] If both viewing angles of the binocular camera are normal, then according to the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environment images of the two viewing angles to obtain the image fusion result;

[0053] If one viewing angle of the binocular camera is normal, de-glare processing is performed on the environment image of the normal viewing angle according to the glare detection result of the normal viewing angle, and the image fusion result is obtained according to the processing result.

[0054] In a technical solution of the above-mentioned electronic rearview mirror display method, the display driver includes a display driver subroutine and a backlight driver subroutine; the environment perception processing result includes a display driver result and a backlight driver result;

[0055] The obtaining the environment perception processing result based on the display driver and according to the image fusion result includes:

[0056] Based on the display driving subroutine, grayscale compensation operation is performed on the image fusion result, and based on the grayscale compensation operation result, a display driving result is obtained;

[0057] Based on the backlight driving subroutine, a backlight extraction operation is performed on the image fusion result, and based on the backlight extraction operation result, a backlight driving result is obtained.

[0058] In a technical solution of the above electronic rearview mirror display method, the environment perception device is a camera; the camera is provided with a state detection sensor;

[0059] The environment perception device based on multiple perspectives obtains the environment perception results of multiple perspectives, including:

[0060] According to the state detection sensor, obtaining a state detection result of each of the cameras;

[0061] According to the state detection result and the cameras of the multiple viewing angles, the environmental perception results of the multiple viewing angles are obtained.

[0062] In a technical solution of the above-mentioned electronic rearview mirror display method, the state detection sensor includes a water mist sensor and / or a lens dirt sensor; the state detection result includes the presence of water mist abnormality and / or dirt abnormality on the camera;

[0063] The step of obtaining the environment perception results of the multiple perspectives according to the state detection result and the cameras of the multiple perspectives includes:

[0064] When the state detection result of the camera is that there is water mist abnormality and / or dirt abnormality, the environmental perception result is obtained based on cameras of other perspectives where the water mist abnormality and the dirt abnormality do not exist.

[0065] In a technical solution of the above electronic rearview mirror display method, the state detection sensor includes an ambient light sensor; the state detection result includes an ambient light detection result; the camera includes a fill light;

[0066] The step of obtaining the environment perception results of the multiple perspectives according to the state detection result and the cameras of the multiple perspectives includes:

[0067] When the ambient light detection result obtained by the environmental sensor is less than a preset ambient light threshold, the fill light of the camera is controlled to turn on, so as to obtain the environmental perception results of the multiple perspectives according to the camera of the multiple perspectives.

[0068] In a technical solution of the above-mentioned electronic rearview mirror display method, the method further includes:

[0069] When the status detection result indicates that the camera is abnormal, the abnormal status detection result is visually displayed based on the display module; and / or, based on a preset fault prompt device, a camera fault prompt is given according to the abnormal status detection result.

[0070] In a technical solution of the above-mentioned electronic rearview mirror display method, the display module is a local dimming display module.

[0071] In a technical solution of the above-mentioned electronic rearview mirror display method, the local dimming display module includes an LCD module and a Mini LED backlight module; the environmental perception processing result includes a display driving result and a backlight driving result;

[0072] The step of visually displaying the environmental perception processing result based on the display module includes:

[0073] Performing visual display of the display driving result based on the LCD module;

[0074] The backlight driving result is visualized based on the Mini LED backlight module.

[0075] And run to execute the method described in any one of the technical solutions of the above-mentioned electronic rearview mirror system.

[0076] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0077] In the implementation of the electronic rearview mirror system technical solution provided by the present application, the electronic rearview mirror system of the present application includes an environment perception module, a perception result processing module and a display module; the environment perception module obtains the environment perception results of multiple perspectives based on the environment perception devices of multiple perspectives; the perception result processing module obtains the environment perception processing results according to the environment perception results of multiple perspectives; the display module visualizes the environment perception processing results based on the display module. Through the above configuration, the present application can achieve a wider field of view because the environment perception devices of multiple perspectives are provided in the environment perception module, thereby ensuring that more road condition information can be captured. Furthermore, the environment perception results of the road conditions obtained by each perspective of the environment perception devices of multiple perspectives will have differences in the glare area, and the environment perception results based on multiple perspectives can more flexibly remove the glare in the environment perception results. At the same time, when an environment perception device is abnormal, other environment perception devices can still be used normally, and the environment perception devices are redundantly designed to each other, which can effectively ensure the function of the electronic rearview mirror system and realize the function of a safer electronic rearview mirror system. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:

[0079] Figure 1 is a schematic diagram of the main components of an electronic rearview mirror system according to an embodiment of the present application;

[0080] Figure 2is a schematic flow chart of the main steps of the electronic rearview mirror display method according to an embodiment of the present application;

[0081] Figure 3 is a schematic diagram of the principle of an electronic rearview mirror system according to an implementation of an embodiment of the present application;

[0082] Figure 4 is a schematic diagram of the main components of an electronic rearview mirror system according to an implementation of an embodiment of the present application;

[0083] Figure 5 is a schematic diagram of the main components of an electronic rearview mirror system according to another implementation of an embodiment of the present application;

[0084] Figure 6 It is a schematic flow chart of the main steps of performing image fusion according to the state detection result of the sensor according to an implementation of an embodiment of the present application;

[0085] FIG. 7 is a schematic diagram of a glare removal comparison according to an example of an embodiment of the present application. DETAILED DESCRIPTION

[0086] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0087] In the description of this application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memories, and may also include software parts, such as program codes, or a combination of software and hardware. The term "A and / or B" means all possible combinations of A and B, such as just A, just B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include just A, just B, or A and B. The singular terms "a" and "the" may also include plural forms.

[0088] See attached Figure 1 , Figure 1 FIG. 1 is a schematic diagram of the main components of an electronic rearview mirror system according to an embodiment of the present application. Figure 1As shown, the electronic rearview mirror system in the embodiment of the present application mainly includes an environment perception module, a perception result processing module and a display module. The environment perception module may include an environment perception device with multiple perspectives; the display module may include a display module. The environment perception module may be configured to obtain environment perception results from multiple perspectives based on the environment perception device with multiple perspectives. The perception result processing module may be configured to obtain environment perception processing results based on the environment perception results from multiple perspectives. The display module may be configured to visualize the environment perception processing results based on the display module.

[0089] In one implementation, the environment perception device may be a camera, a laser radar, a millimeter wave radar, or other device capable of environment perception.

[0090] In one implementation, the camera may be a wide-angle camera, a binocular camera, an infrared camera, or the like.

[0091] In one embodiment, the display module may be a local dimming (LD) display module. The local dimming display module may include an LCD (Liquid Crystal Display) module and a Mini LED (Mini Light Emitting Diode) backlight module. Since the local dimming display module has a backlight local dimming control function, it can improve the display contrast while also reducing the display power consumption.

[0092] In one embodiment, the display module may also be a BD Cell display module, wherein the BD Cell display module is a display module that achieves micron-level light control accuracy by adding a light control liquid crystal box between the liquid crystal display layer and the backlight source and using pixel-level light control technology.

[0093] The environment perception module, the perception result processing module and the display module are further explained below.

[0094] In one implementation of the embodiment of the present application, the environment perception device may be a camera, a state detection sensor may be provided on the camera, and the environment perception module may further include a state detection unit and an environment perception unit. The state detection unit may be configured to obtain a state detection result of each camera according to the state detection sensor. The environment perception unit may be configured to obtain environment perception results of multiple perspectives according to the state detection result and cameras of multiple perspectives.

[0095] In one embodiment, the state detection sensor may include a water mist sensor and a lens dirt sensor; the state detection result may include the presence of water mist abnormality or dirt abnormality on the camera. The environment perception unit may be further configured to, when the state detection result of the camera is the presence of water mist abnormality or dirt abnormality, obtain the environment perception result based on cameras of other viewing angles without water mist abnormality or dirt abnormality.

[0096] In one embodiment, the state detection sensor may include an ambient light sensor; the state detection result may include an ambient light detection result; and the camera may include a fill light. The environment perception unit may be further configured to control the fill light of the camera to turn on when the ambient light detection result obtained by the environment sensor is less than a preset ambient light threshold, so as to obtain environment perception results of multiple perspectives according to the camera of multiple perspectives. Among them, those skilled in the art may set the value of the ambient light threshold according to the needs of actual applications.

[0097] In one embodiment, the state detection unit may be further configured to send the state detection result to the display module when the state detection result indicates that the camera is abnormal. The display module may be further configured to visually display the received state detection result based on the display module.

[0098] In one embodiment, the electronic rearview system may further include a fault warning module. The state detection unit may be further configured to send the state detection result to the fault warning module when the state detection result indicates that the camera is abnormal. The fault warning module may be configured to provide a camera fault prompt based on a preset fault prompt device and the received state detection result.

[0099] In one implementation of the embodiment of the present application, the environment perception device may be a camera; the environment perception result may be an environment image captured by the camera; and the perception result processing module may include an image fusion unit and a display driver unit. The image fusion unit may be configured to perform image fusion based on environment images from multiple perspectives to obtain an image fusion result. The display driver unit may be configured to obtain an environment perception processing result based on the image fusion result based on a display driver.

[0100] In one implementation, the image fusion unit may be implemented based on a Field Programmable Gate Array (FPGA) chip.

[0101] In another implementation, the image fusion unit may be implemented based on an ASIC (Application Specific Integrated Circuit) chip.

[0102] In one embodiment, the image fusion unit may include a glare detection subunit and a fusion subunit. The glare detection subunit may be configured to perform glare detection on the environmental image of each perspective respectively, and obtain the glare detection result of the environmental image of each perspective. The fusion subunit may be configured to perform image fusion on the environmental images of multiple perspectives according to the glare detection results of the environmental images of multiple perspectives, and obtain the image fusion result. Among them, a glare detection algorithm commonly used in the field may be applied to perform glare detection to obtain the glare detection result. The glare detection result may include whether there is a "glare" area in the environmental image, and the location of the "glare" area. The size of the glare area, edge coordinate information, etc. may be obtained in real time as the glare detection result.

[0103] In one embodiment, the camera with multiple viewing angles can be a binocular camera; the fusion subunit can be further configured as follows: if both viewing angles of the binocular camera are normal, then based on the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environmental images of the two viewing angles to obtain an image fusion result; if one viewing angle of the binocular camera is normal, then based on the glare detection result of the normal viewing angle, de-glare processing is performed on the environmental image of the normal viewing angle, and the image fusion result is obtained based on the processing result.

[0104] Among them, a sensor can be set at each viewing angle of the binocular camera, and the detection result of the sensor can be used to judge whether the binocular camera is normal. The sensor can be a water mist sensor, a dirt sensor, etc.; a status indicator light can be set at each viewing angle of the binocular camera, and the binocular camera can be judged whether it is normal according to the state of the indicator light. For example, when the indicator light is on, it can be considered that the binocular camera is abnormal; monitoring software can be used to monitor the working status of each viewing angle of the binocular camera in real time, and judge whether the binocular camera is normal according to the monitoring results. Of course, those skilled in the art can also use the method commonly used in the field to judge whether the viewing angle of the binocular camera is normal to judge whether the binocular camera is normal, or combine multiple methods for comprehensive judgment.

[0105] In one embodiment, the display driver may include a display driver subroutine and a backlight driver subroutine; the display driver unit may include a display driver subunit and a backlight driver subunit. The environmental perception processing result may include a display driver result and a backlight driver result. The display driver subunit may be configured to perform a grayscale compensation operation on the image fusion result based on the display driver subroutine, and obtain a display driver result based on the grayscale compensation operation result. The backlight driver subunit may be configured to perform a backlight extraction operation on the image fusion result based on the backlight driver subroutine, and obtain a backlight driver result based on the backlight extraction operation result.

[0106] Among them, the grayscale compensation operation is an operation process for optimizing the display effect of the display device. The backlight extraction operation is the process of extracting background information from the display drive result. In one embodiment, the image fusion result can be subjected to grayscale analysis to identify areas with lower grayscale values. These areas usually appear as darker or less detailed parts. Compensation calculations are performed based on the grayscale information of surrounding pixels. Through weighted averaging or dynamic adjustment based on the grayscale differences of neighboring pixels, the pixels in the low grayscale area gradually transition to a more suitable grayscale value range. The compensated pixel values ​​are resynthesized into the image to complete the grayscale compensation operation to obtain the display drive result.

[0107] In one implementation, the backlight extraction operation may be implemented using an inter-frame difference method, a background difference method, a ViBe (Visual Background Extractor) algorithm, or the like.

[0108] In one implementation of the embodiment of the present application, the display module may include a display unit and a backlight unit. The display unit may be configured to perform a visual display of the display driving result based on the LCD module. The backlight unit may be configured to perform a visual display of the backlight driving result based on the Mini LED backlight module.

[0109] The following is combined with Figure 3 and attached Figure 4 , further describing the electronic rearview mirror system of the present application, Figure 3 is a schematic diagram of the principle of an electronic rearview mirror system according to an implementation of an embodiment of the present application; Figure 4 : is a schematic diagram of the main components of an electronic rearview mirror system according to an embodiment of the present application. Figure 3 and Figure 4 The number 1 is the environment perception module, the number 2 is the perception result processing module, and the number 3 is the display module.

[0110] like Figure 3 and Figure 4 As shown, the environment perception module uses a binocular camera to collect environment images, and can obtain environment images from two perspectives, which can ensure a wider field of view. At the same time, due to the different perspectives of the two cameras of the binocular camera, the glare positions in the collected environment images can be different, making it easier to remove the glare area in the environment image.

[0111] Furthermore, the environment perception module also includes a sensor, which is used to detect the dirtiness of the binocular camera and the ambient light detection result of the surrounding environment. The sensor may include a water mist sensor, a lens dirtiness sensor, and an ambient light sensor.

[0112] Water mist sensor: used to detect water, fog, other liquids and other contamination on the binocular camera lens; the water mist sensor determines whether there is water mist abnormality in the binocular camera by detecting the presence of water mist on the binocular camera. For example, if it is detected that the water mist area on the binocular camera lens exceeds the first preset threshold, it can be considered that there is water mist abnormality in the binocular camera.

[0113] Lens dirt sensor: used to detect dust and occlusion on the binocular camera lens; the lens dirt sensor determines whether the binocular camera is abnormally dirty by detecting the presence of dirt on the binocular camera. If the area of ​​dirt on the binocular camera lens is detected to exceed the second preset threshold, it can be considered that the binocular camera is abnormally dirty.

[0114] Ambient light sensor: used to detect the light conditions of the surrounding environment to obtain the ambient light detection results; when the light is dim, when the light is dim (that is, the ambient light detection result is lower than the preset ambient light threshold), the ambient light can be sent to the perception result processing module for feedback information, and the fill light of the binocular camera can be turned on through the relevant control module to ensure that the binocular camera can still capture high-definition images under dark light conditions.

[0115] See attached Figure 6 , Figure 6 The figure is a flow chart of the main steps of image fusion according to the state detection results of the sensor according to an implementation of an embodiment of the present application. The sensor can provide the dirtiness or working conditions of the cameras of two viewing angles to the perception result processing module through the bus implementation, and based on the Figure 6 The method shown is used to deal with abnormal dirt or abnormal operation of the camera:

[0116] 1. If both cameras are working properly, the binocular camera working mode can be adopted through the perception result processing module to achieve image fusion of the two environment images;

[0117] 2. If only one camera is working properly, the perception result processing module adopts the monocular camera working mode to implement glare removal processing for a single environmental image and handle the warning of abnormal status of the other camera.

[0118] Among them, early warning can be carried out in the following two ways:

[0119] 1. The fault warning module controls the preset fault prompt device in the vehicle to give a fault prompt to indicate that the camera is in an abnormal state;

[0120] 2. The OSD (on-screen display) submodule of the display model is controlled by the perception result processing module to display the fault prompt information corresponding to the status detection result in real time on the display screen of the electronic rearview mirror.

[0121] The image fusion unit in the environmental perception processing module is implemented by an FPGA chip. The image fusion processing part of the FPGA chip is used to process the environmental images collected by the binocular camera in real time, and finally remove the glare and fuse a fused image (i.e., image fusion result) after a series of related processing. The specific related processing may include but is not limited to anti-distortion (i.e., image correction), noise removal, glare detection and positioning, so as to finally obtain the image fusion result of the two environmental images collected by the camera after removing the glare, specifically:

[0122] Camera image correction: Correct the environmental image captured by the camera based on the anti-distortion algorithm to obtain an undistorted environmental image;

[0123] Image preprocessing: used to remove noise information from environmental images, including image cropping, Gaussian filtering, abnormal noise detection and processing units;

[0124] Glare detection and positioning: Based on the glare detection algorithm, the "glare" area in the environmental image is detected and positioned, the size and edge coordinate information of the "glare" area is obtained in real time, and the back-end module is provided;

[0125] Image fusion: The image fusion of two environmental images is realized based on the image fusion algorithm. In the image fusion link, the "glare" area in the two environmental images is removed to obtain a high-definition fused image after removing the "glare". Specifically, please refer to Figure 7, which is a schematic diagram of glare removal comparison according to an example of an embodiment of the present application. As shown in Figure 7, Figure 7 (a) is an environmental image captured by two perspectives of the binocular camera, where the area corresponding to label 4 is the "glare". It can be seen from Figure 7 (a) that the glare areas of the environmental images of the two perspectives are different, so that when the environmental images of the two perspectives are fused, the image fusion algorithm can be combined to combine the environmental images taken from two different perspectives into one, and the glare area can be effectively removed. Among them, after the image fusion, the image fusion result with the glare area removed is shown in Figure 7 (b).

[0126] It should be noted that if there is more than one glare area in the two environmental images taken by the binocular camera, multiple glare areas can be identified and located in the glare detection and positioning link, and the glare areas can be removed based on the image fusion algorithm to obtain the image fusion result; if there is a glare area in only one of the two environmental images taken by the binocular camera, or there is no glare area in both environmental images, it will not affect the image fusion process, and the image fusion algorithm does not need to remove the glare area, and can directly perform image fusion.

[0127] The display driver unit of the perception result processing module is an LD display driver unit. The LD display driver unit is used to realize the driving of the display module in the display module, mainly including LCD grayscale compensation calculation, Mini LED backlight calculation, LCD (Driver) driving, Mini LED (Driver) driving and other modules.

[0128] LCD grayscale compensation operation: used to realize grayscale compensation operation of pixel grayscale after backlight extraction of image fusion results, as well as pixel-level compensation or correction operations such as Gamma mapping;

[0129] Mini LED backlight calculation: used to realize the backlight value of each element in the Mini LED backlight matrix, mainly including average value extraction, maximum (minimum) brightness extraction, Gaussian filtering and other calculation links;

[0130] LCD driver: used to realize the functions of the following two parts:

[0131] (1) Driver IC of LCD Panel, including IC register configuration, data mapping, and driver implementation of underlying P2P protocol;

[0132] (2) The GOA (Gate On Array) driver of the LCD Panel generates corresponding waveforms such as STV (a video signal) and CK (a clock signal) suitable for GOA signals.

[0133] Mini LED driver: used to implement the driver of Mini LED Driver IC, including IC register configuration, SPI (Serial Peripheral Interface) driver (or other bus protocols suitable for LED Driver IC), etc.

[0134] OSD control: When the sensor detects an abnormality in the binocular camera, the FPGA chip gives an abnormal "subtitle prompt" through the OSD control unit, and drives the display module to provide real-time feedback to the driver to handle the corresponding abnormality in time.

[0135] Camera status feedback: used to process the camera information detected by the sensor in real time, mainly including the following:

[0136] Water mist and dirt detection: When the relevant sensors detect abnormal conditions such as rain, dirt, etc. blocking the camera, the FPGA chip gives a fault prompt, including driving the warning prompt light, driving the display screen to display warning subtitles through OSD control, etc.; in addition, if a single camera has an abnormality (for example, a camera is blocked or cannot be used normally due to other abnormal reasons), the FPGA chip gives a warning prompt information and also starts switching to the monocular camera working mode. At this time, the electronic rearview mirror system can still work normally with a single camera, but the image fusion algorithm can switch to the glare detection and removal mode.

[0137] Ambient light detection: When the relevant sensor detects that the ambient light drops below the ambient light threshold, the FPGA chip will control the camera to turn on the fill light to ensure that the camera can still capture clear ambient images in dim light.

[0138] The display module in the display module is a high-contrast LD display module. The LD display module is composed of an LCD module and a MiniLED backlight module. Since the LD display module has a local dimming control function, it can improve the display contrast while reducing the power consumption of the display module.

[0139] LCD module: The display screen is controlled by the "LCD driver" module of FPGA, which consists of two parts: the driver IC and the GOA.

[0140] Mini LED backlight module: The display image is controlled by the FPGA's "backlight driver" module.

[0141] In one embodiment, see the attached Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the main components of an electronic rearview mirror system according to another embodiment of the present application. Figure 5 As shown, the display module of the display module can be a BD cell display module. The corresponding display driver unit can be a BD Cell display driver unit, which can obtain the Main Cell calculation result through the Main Cell calculation to realize the main (Main drive); obtain the Sub Cell calculation result through the Sub Cell calculation to realize the slave (Sub) drive. Based on the Main drive and the Sub drive, a high-contrast BD Cell display is realized through the Main Cell panel (Panel). At the same time, the BD display driver unit can also realize OSD control.

[0142] In one embodiment, the display driving result and the backlight driving result can be transmitted to the display module through a video interface. The video interface may include but is not limited to eDP (Embedded Display Port), HDMI (High Definition Multimedia Interface), LVDS (Low Voltage Differential Signaling), and other general video interfaces.

[0143] Furthermore, the present application also provides an electronic rearview mirror display method.

[0144] See attached Figure 2 , Figure 2 FIG. 1 is a flow chart of the main steps of the electronic rearview mirror display method according to an embodiment of the present application. Figure 2 As shown, the electronic rearview mirror display method of the present application mainly includes the following steps S101 to S103.

[0145] Step S101: Based on the environment perception device of multiple perspectives, obtain the environment perception results of multiple perspectives.

[0146] In this embodiment, the environment perception device may be a camera; the camera may be provided with a state detection sensor. Step S101 may further include the following steps S1011 and S1012:

[0147] Step S1011: Obtain the status detection result of each camera according to the status detection sensor.

[0148] Step S1012: Obtain environmental perception results from multiple perspectives based on the state detection results and cameras from multiple perspectives.

[0149] In one embodiment, the state detection sensor may include a water mist sensor and a lens dirt sensor; the state detection result may include the presence of water mist anomaly and dirt anomaly in the camera. Step S1012 may further include the following steps S10121:

[0150] Step S10121: When the state detection result of the camera is that there is water mist abnormality and dirt abnormality, the environmental perception result is obtained based on cameras of other perspectives where there is no water mist abnormality and dirt abnormality.

[0151] In one implementation, the state detection sensor may include an ambient light sensor; the state detection result may include an ambient light detection result; the camera may include a fill light; step S1012 may further include the following steps S10122:

[0152] Step S10122: When the ambient light detection result obtained by the environmental sensor is less than a preset ambient light threshold, the fill light of the camera is controlled to turn on, so as to obtain environmental perception results of multiple perspectives based on the camera of multiple perspectives.

[0153] In one implementation, when the status detection result indicates that there is an abnormality in the camera, the abnormal status detection result can be visualized based on a display module.

[0154] In another embodiment, when the status detection result indicates that there is an abnormality in the camera, a camera fault prompt may be issued based on a preset fault prompt device according to the abnormal status detection result.

[0155] Step S102: Obtaining environmental perception processing results based on environmental perception results from multiple perspectives.

[0156] In this embodiment, the environment perception device may be a camera; the environment perception result may be an environment image captured by the camera. Step S102 may further include the following steps S1021 and S1022:

[0157] Step S1021: performing image fusion based on the environment images from multiple perspectives to obtain an image fusion result.

[0158] In this implementation, step S1021 may further include the following steps S10211 and S10212:

[0159] Step S10211: Perform glare detection on the environmental image of each viewing angle respectively, and obtain the glare detection result of the environmental image of each viewing angle.

[0160] Step S10212: performing image fusion on the environment images from multiple perspectives according to the glare detection results of the environment images from multiple perspectives to obtain an image fusion result.

[0161] In this embodiment, the camera with multiple viewing angles may be a binocular camera. Step S10212 may further include the following steps S102121 and S102122:

[0162] Step S102121: If both viewing angles of the binocular camera are normal, then according to the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environmental images of the two viewing angles to obtain an image fusion result.

[0163] Step S102122: If one viewing angle of the binocular camera is normal, deglare processing is performed on the environment image of the normal viewing angle according to the glare detection result of the normal viewing angle, and an image fusion result is obtained according to the processing result.

[0164] Step S1022: Based on the display driver and the image fusion result, an environment perception processing result is obtained.

[0165] In this embodiment, the display driver may include a display driver subroutine and a backlight driver subroutine; the environment perception processing result may include a display driver result and a backlight driver result; step S1022 may further include the following steps S10221 and S10222:

[0166] Step S10221: Based on the display driving subroutine, grayscale compensation operation is performed on the image fusion result, and based on the grayscale compensation operation result, a display driving result is obtained.

[0167] Step S10222: Based on the backlight driving subroutine, perform a backlight extraction operation on the image fusion result, and obtain a backlight driving result based on the backlight extraction operation result.

[0168] Step S103: Based on the display module, the environmental perception processing result is visualized and displayed.

[0169] In this embodiment, the display module may be a local dimming display module; the local dimming display module may include an LCD module and a Mini LED backlight module; the environmental perception processing result may include a display driving result and a backlight driving result. Step S103 may further include the following steps S1031 and S1032:

[0170] Step S1031: Visually display the display driving result based on the LCD module.

[0171] Step S1032: Visually display the backlight driving results based on the Mini LED backlight module.

[0172] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.

[0173] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0174] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. An electronic rearview mirror system, characterized in that: The system includes an environment perception module, a perception result processing module and a display module; the environment perception module includes environment perception devices of multiple viewing angles; the display module includes a display module; The environment perception module is configured as an environment perception device based on multiple perspectives to obtain environment perception results from multiple perspectives; The perception result processing module is configured to obtain an environment perception processing result according to the environment perception results of the multiple perspectives; The display module is configured to visually display the environmental perception processing result based on the display module.

2. The electronic rearview mirror system according to claim 1, characterized in that: The environment perception device is a camera; the environment perception result is an environment image collected by the camera; the perception result processing module includes an image fusion unit and a display driving unit; The image fusion unit is configured to perform image fusion according to the environment images of the multiple perspectives to obtain an image fusion result; The display driving unit is configured to obtain the environment perception processing result based on the display driving program and according to the image fusion result.

3. The electronic rearview mirror system according to claim 2, characterized in that: The image fusion unit includes a glare detection subunit and a fusion subunit; The glare detection subunit is configured to perform glare detection on the environment image of each viewing angle respectively, and obtain the glare detection result of the environment image of each viewing angle; The fusion subunit is configured to perform image fusion on the environmental images of multiple perspectives according to the glare detection results of the environmental images of multiple perspectives to obtain the image fusion result.

4. The electronic rearview mirror system according to claim 3, characterized in that: The cameras with multiple viewing angles are binocular cameras; the fusion subunit is further configured as follows: If both viewing angles of the binocular camera are normal, then according to the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environment images of the two viewing angles to obtain the image fusion result; If one viewing angle of the binocular camera is normal, de-glare processing is performed on the environment image of the normal viewing angle according to the glare detection result of the normal viewing angle, and the image fusion result is obtained according to the processing result.

5. The electronic rearview mirror system according to claim 2, characterized in that: The display driver includes a display driver subroutine and a backlight driver subroutine; the display driver unit includes a display driver subunit and a backlight driver subunit; the environment perception processing result includes a display driver result and a backlight driver result; The display driving subunit is configured to perform a grayscale compensation operation on the image fusion result based on the display driving subroutine, and obtain the display driving result based on the grayscale compensation operation result; The backlight driving subunit is configured to perform a backlight extraction operation on the image fusion result based on the backlight driving subroutine, and obtain the backlight driving result based on the backlight extraction operation result.

6. The electronic rearview mirror system according to claim 1, characterized in that: The environment perception device is a camera; the camera is provided with a state detection sensor; the environment perception module includes a state detection unit and an environment perception unit; The state detection unit is configured to obtain a state detection result of each of the cameras according to the state detection sensor; The environment perception unit is configured to obtain the environment perception results of the multiple perspectives according to the state detection results and the cameras of the multiple perspectives.

7. The electronic rearview mirror system according to claim 6, characterized in that: The state detection sensor includes a water mist sensor and / or a lens dirt sensor; the state detection result includes the presence of water mist abnormality and / or dirt abnormality on the camera; The environmental perception unit is further configured to obtain the environmental perception result based on cameras of other perspectives where the water mist abnormality and the dirt abnormality do not exist when the state detection result of the camera is that there is water mist abnormality and / or dirt abnormality.

8. The electronic rearview mirror display system according to claim 6, characterized in that: The state detection sensor includes an ambient light sensor; the state detection result includes an ambient light detection result; the camera includes a fill light; The environmental perception unit is further configured to control the fill light of the camera to turn on when the ambient light detection result obtained by the environmental sensor is less than a preset ambient light threshold, so as to obtain the environmental perception results of the multiple perspectives according to the camera of the multiple perspectives.

9. The electronic rearview mirror system according to claim 6, characterized in that: The state detection unit is further configured to send the state detection result to the display module when the state detection result indicates that the camera is abnormal; The display module is further configured to visually display the received status detection result based on the display module.

10. The electronic rearview mirror system according to claim 6, characterized in that: The system also includes a fault warning module; The state detection unit is further configured to send the state detection result to the fault warning module when the state detection result indicates that the camera is abnormal; The fault warning module is configured to provide a camera fault prompt based on a preset fault prompt device and according to the received status detection result.

11. The electronic rearview mirror system according to claim 1, characterized in that: The display module is a local dimming display module.

12. The electronic rearview mirror system according to claim 11, characterized in that: The local dimming display module includes an LCD module and a MiniLED backlight module; the environmental perception processing result includes a display driving result and a backlight driving result; the display module includes a display unit and a backlight unit; The display unit is configured to perform visual display of the display driving result based on the LCD module; The backlight unit is configured to perform a visual display of the backlight driving result based on the Mini LED backlight module.

13. An electronic rearview mirror display method, characterized in that: The method comprises: Based on the environment perception device with multiple perspectives, the environment perception results with multiple perspectives are obtained; Obtaining environmental perception processing results according to the environmental perception results of the multiple perspectives; Based on the display module, the environmental perception processing result is visualized and displayed.

14. The electronic rearview mirror display method according to claim 13, characterized in that: The environment perception device is a camera; the environment perception result is an environment image captured by the camera; The obtaining of the environmental perception processing results according to the environmental perception results of the multiple perspectives includes: Performing image fusion according to the environment images from the multiple perspectives to obtain an image fusion result; Based on the display driver, the environment perception processing result is obtained according to the image fusion result.

15. The electronic rearview mirror display method according to claim 14, characterized in that: The step of performing image fusion according to the environment images from the multiple perspectives to obtain an image fusion result includes: Perform glare detection on the environment image of each viewing angle respectively, and obtain the glare detection result of the environment image of each viewing angle; According to the glare detection results of the environment images of multiple perspectives, image fusion is performed on the environment images of the multiple perspectives to obtain the image fusion result.

16. The electronic rearview mirror display method according to claim 15, characterized in that: The cameras with multiple viewing angles are binocular cameras; The step of performing image fusion on the environment images of the multiple perspectives according to the glare detection results of the environment images of the multiple perspectives to obtain the image fusion result includes: If both viewing angles of the binocular camera are normal, then according to the glare detection results of the two viewing angles of the binocular camera, image fusion is performed on the environment images of the two viewing angles to obtain the image fusion result; If one viewing angle of the binocular camera is normal, de-glare processing is performed on the environment image of the normal viewing angle according to the glare detection result of the normal viewing angle, and the image fusion result is obtained according to the processing result.

17. The electronic rearview mirror display method according to claim 14, characterized in that: The display driver includes a display driver subroutine and a backlight driver subroutine; the environment perception processing result includes a display driver result and a backlight driver result; The obtaining the environment perception processing result based on the display driver and according to the image fusion result includes: Based on the display driving subroutine, grayscale compensation operation is performed on the image fusion result, and based on the grayscale compensation operation result, a display driving result is obtained; Based on the backlight driving subroutine, a backlight extraction operation is performed on the image fusion result, and based on the backlight extraction operation result, a backlight driving result is obtained.

18. The electronic rearview mirror display method according to claim 13, characterized in that: The environment perception device is a camera; the camera is provided with a state detection sensor; The environment perception device based on multiple perspectives obtains the environment perception results of multiple perspectives, including: According to the state detection sensor, obtaining a state detection result of each of the cameras; According to the state detection result and the cameras of the multiple viewing angles, the environmental perception results of the multiple viewing angles are obtained.

19. The electronic rearview mirror display method according to claim 18, characterized in that: The state detection sensor includes a water mist sensor and / or a lens dirt sensor; the state detection result includes the presence of water mist abnormality and / or dirt abnormality on the camera; The step of obtaining the environment perception results of the multiple perspectives according to the state detection result and the cameras of the multiple perspectives includes: When the state detection result of the camera is that there is water mist abnormality and / or dirt abnormality, the environmental perception result is obtained based on cameras of other perspectives where the water mist abnormality and the dirt abnormality do not exist.

20. The electronic rearview mirror display method according to claim 18, characterized in that: The state detection sensor includes an ambient light sensor; the state detection result includes an ambient light detection result; the camera includes a fill light; The step of obtaining the environment perception results of the multiple perspectives according to the state detection result and the cameras of the multiple perspectives includes: When the ambient light detection result obtained by the environmental sensor is less than a preset ambient light threshold, the fill light of the camera is controlled to turn on, so as to obtain the environmental perception results of the multiple perspectives according to the camera of the multiple perspectives.

21. The electronic rearview mirror display method according to claim 18, characterized in that: The method further comprises: When the status detection result indicates that the camera is abnormal, the abnormal status detection result is visually displayed based on the display module; and / or, based on a preset fault prompt device, a camera fault prompt is given according to the abnormal status detection result.

22. The electronic rearview mirror display method according to claim 13, characterized in that: The display module is a local dimming display module.

23. The electronic rearview mirror display method according to claim 22, characterized in that: The local dimming display module includes an LCD module and a MiniLED backlight module; The environmental perception processing result includes a display driving result and a backlight driving result; The step of visually displaying the environmental perception processing result based on the display module includes: Performing visual display of the display driving result based on the LCD module; The backlight driving result is visualized based on the MiniLED backlight module.

Images