Image compensation method and device based on electronic outside rear-view mirror and storage medium
By using color and infrared cameras combined with light sensors in the electronic exterior rearview mirror system to match and process image features, the problem of insufficient sensor perception ability in harsh environments is solved, and stable image output under various conditions is achieved. Improved driving safety.
Patent Information
- Application Number
- CN202411065328.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-12
AI Technical Summary
In severe weather, poor night light, dazzling light and rainy, snowy and foggy weather, the existing electronic exterior mirror system has insufficient sensor perception ability and cannot provide stable image output, which affects the driver's perception of the rear environment of the car and increases Driving risk.
Use color cameras and infrared cameras to collect color and infrared images on the outside of the vehicle, and detect the environmental brightness through the light sensor. Based on the user's infrared compensation function, the characteristics of color images and infrared images are matched, the characteristic matching level is determined, and the appropriate image output is selected. mode, process and output the image.
It improves the image perception capability and output stability of the electronic exterior mirror system under various lighting and weather conditions, improves the driver's perception of the surrounding environment of the vehicle, and enhances driving safety.
Smart Images

Figure CN118945488A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image compensation method, device and storage medium based on an electronic exterior rearview mirror. Background Art
[0002] As an important component of the trend of automobile intelligence and electrification, electronic exterior rearview mirrors have developed rapidly in recent years. With the increasing demand of consumers for driving safety and the improvement of automobile intelligence, the market size of electronic exterior rearview mirrors has grown year by year.
[0003] At present, the conventional method of the electronic exterior rearview mirror system is to collect the side and rear images of the vehicle through the RGB module, pre-process the output image through image enhancement technology, and then transmit it to the screen inside the car to prompt the user. However, the existing technology still has the following shortcomings:
[0004] Insufficient sensor perception: In bad weather, poor lighting at night, glaring light, and rain, snow, and fog, traditional CMOS sensors have weak perception capabilities and are unable to effectively identify surrounding target objects, resulting in the driver being unable to obtain accurate road conditions information, seriously affecting driving safety.
[0005] Image continuity and stability issues: Since electronic exterior rearview mirrors eliminate the need for traditional physical rearview mirrors, drivers rely entirely on the image information provided by the in-car screen. Once the system fails to output continuous and stable images in dim light or bad weather, the driver will lose perception of the environment behind the car, increasing driving risks.
[0006] According to the CMS regulations GB15084, the electronic exterior rearview mirror system must have the function of "detecting optical defocus and prompting the user", which makes image redundancy design a regular requirement of the system. How to provide stable image output in all scenarios and at all times has become a major problem in the current CMS system image processing technology. Since existing technologies are difficult to meet these requirements, how to improve the image perception ability and output stability of the electronic exterior rearview mirror system under various lighting and weather conditions has become an issue that the industry needs to solve urgently. Summary of the invention
[0007] In view of this, the embodiments of the present application provide an image compensation method, device and storage medium based on an electronic exterior rearview mirror to solve the problems of poor image perception ability and imaging effect in the prior art, which reduces vehicle driving safety.
[0008] In a first aspect of an embodiment of the present application, there is provided an image compensation method based on an electronic exterior rearview mirror, comprising: using a color camera and an infrared camera installed on the electronic exterior rearview mirror of the vehicle to respectively collect a color image and an infrared image of the outside of the vehicle; using a light sensor installed on the electronic exterior rearview mirror of the vehicle to detect brightness information of the vehicle's external environment; in response to a user's operation of turning on an infrared compensation function, performing feature matching on the color image and the infrared image based on brightness information to determine the similarity between target objects in the color image and the infrared image; determining a feature matching level based on the feature matching result, processing the color image and / or infrared image according to an image output mode corresponding to a predetermined feature matching level, and outputting the processed image to a display screen inside the vehicle for display.
[0009] According to a second aspect of an embodiment of the present application, there is provided an image compensation device based on an electronic exterior rearview mirror, comprising: an acquisition module, configured to use a color camera and an infrared camera installed on the electronic exterior rearview mirror of the vehicle to respectively acquire a color image and an infrared image of the outside of the vehicle; a detection module, configured to use a light sensor installed on the electronic exterior rearview mirror of the vehicle to detect brightness information of the vehicle's external environment; a matching module, configured to respond to a user's operation of turning on an infrared compensation function, and to perform feature matching between the color image and the infrared image based on brightness information to determine the similarity between target objects in the color image and the infrared image; a processing module, configured to determine a feature matching level based on a feature matching result, process the color image and / or infrared image according to an image output mode corresponding to a predetermined feature matching level, and output the processed image to a display screen inside the vehicle for display.
[0010] According to a third aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.
[0011] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0012] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:
[0013] By using the color camera and infrared camera installed on the electronic exterior rearview mirror of the vehicle, the color image and infrared image of the outside of the vehicle are collected respectively; the brightness information of the vehicle's external environment is detected by the light sensor installed on the electronic exterior rearview mirror of the vehicle; in response to the user's operation of turning on the infrared compensation function, the color image and the infrared image are feature matched based on the brightness information to determine the similarity between the target objects in the color image and the infrared image; the feature matching level is determined according to the feature matching result, and the color image and / or infrared image are processed according to the image output mode corresponding to the predetermined feature matching level, and the processed image is output to the display screen inside the vehicle for display. The present application can improve the image perception ability and imaging effect of the electronic exterior rearview mirror, thereby improving the driving safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0015] Figure 1 It is a schematic diagram of the arrangement structure of system components involved in the actual scenario provided by the embodiment of the present application;
[0016] Figure 2 is a structural block diagram of an electronic exterior rearview mirror system provided in an embodiment of the present application;
[0017] Figure 3 is a flow chart of an image compensation method based on an electronic exterior rearview mirror provided in an embodiment of the present application;
[0018] Figure 4 is a schematic diagram of the working process of the electronic exterior rearview mirror system provided in an embodiment of the present application;
[0019] Figure 5 It is a schematic diagram of the process of matching and fusing a color image and an infrared image provided in an embodiment of the present application;
[0020] Figure 6 is a structural schematic diagram of a super-resolution image processing device based on a digital camera provided in an embodiment of the present application;
[0021] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0023] As an important component of the trend of automobile intelligence and electrification, electronic exterior rearview mirrors have developed rapidly in recent years. With the increasing demand of consumers for driving safety and the improvement of automobile intelligence, the market size of electronic exterior rearview mirrors has grown year by year.
[0024] At present, the conventional method of the electronic exterior rearview mirror system is to collect the side and rear images of the vehicle through the RGB module, pre-process the output image through image enhancement technology, and transmit it to the screen inside the car to prompt the user. Due to the limitations of existing sensor perception and image processing, the perception ability of traditional CMOS sensors is weak in bad weather, poor lighting at night, glare, rain, snow and fog weather conditions, and it is unable to effectively identify surrounding targets, causing the driver to directly lose road condition information, seriously affecting driving safety. At the same time, due to the cancellation of the traditional physical rearview mirror, the electronic exterior rearview mirror screen is the only input that the driver relies on to perceive the environment behind the car. How to ensure the continuity of the system's output image in dark light and rain and fog scenes and the ability to perceive surrounding objects, and avoid the loss of the electronic rearview mirror screen image has become a problem that the industry urgently needs to solve.
[0025] CMS regulations GB15084 clearly stipulate that "detecting optical defocus and prompting users" is a necessary requirement for the functional safety of electronic exterior rearview mirror systems, making image redundancy design a regular requirement of the system. How to provide stable image output in all scenarios and at all times also puts forward new requirements for the current CMS system image processing technology.
[0026] In view of the problems existing in the prior art, the present application provides an electronic exterior rearview mirror system redundancy and dark light compensation technology, which mainly adds an IR infrared module on the basis of the traditional structured light module, and monitors the surrounding light in real time through the light sensor. When the weather around the vehicle is bad, the lighting is poor at night, there is glare, and the CMOS imaging is poor in rain, snow and fog, and does not meet the preset image output threshold, RGB / IR dual-light image fusion is performed, and the conventional RGB image features are compensated and displayed by utilizing the stable imaging effect of IR infrared perception in low light, rain, snow and fog scenes, thereby improving the safety of vehicle driving.
[0027] First, the vehicle structure and installation positions of system components involved in the embodiments of the present application in actual scenarios are described in detail below in conjunction with the accompanying drawings and specific embodiments. Figure 1is a schematic diagram of the arrangement structure of system components involved in the actual scenario provided by the embodiment of the present application, Figure 2 is a structural block diagram of the electronic exterior rearview mirror system provided in the embodiment of the present application. Figure 1 and Figure 2 As shown, the vehicle structure involved in the electronic exterior rearview mirror system may specifically include:
[0028] The whole system mainly includes an environmental perception module, a decision control module and a display module. The environmental perception module includes the electronic exterior rearview mirror body ③, an RGB camera ④, an IR infrared camera ⑤ and a light sensor. The decision control module includes an electronic exterior rearview mirror controller ⑥. The display module includes a combination instrument ① and an LCD display ②.
[0029] The functions of the entire infrared compensation system and the system data flow link are as follows:
[0030] 1) The RGB camera and infrared IR camera are responsible for collecting visible light images and infrared images of the vehicle's current side and rear and storing them in the CMS controller storage module;
[0031] 2) The light sensor module on the RGB camera detects real-time brightness information and transmits it to the CMS controller decision module;
[0032] 3) The compensation function switch is set on the IVI central control interface to control the on / off of the infrared compensation function of the entire system. When the function is turned off, the conventional image taken by the RGB module is output by default. When the function is turned on, the control end determines whether the image contrast deviation in the current period meets the preset threshold, and outputs different levels of fusion images (such as S1 / S2 / S3) accordingly;
[0033] 4) The LCD display screen displays the above-listed fused or non-fused images (such as P0 / P1 / P2 / P3) in the vehicle;
[0034] 5) When the fusion level continues to be high, the instrument pops up a prompt.
[0035] The contents of the technical solution of the present application are described in detail below with reference to the accompanying drawings and specific embodiments.
[0036] Figure 3 It is a flow chart of an image compensation method based on an electronic exterior rearview mirror provided in an embodiment of the present application. Figure 3 The image compensation method based on the electronic exterior rearview mirror can be performed by the electronic exterior rearview mirror system.
[0037] like Figure 3 As shown, the image compensation method based on the electronic exterior rearview mirror may specifically include:
[0038] S301, using a color camera and an infrared camera installed on the electronic exterior rearview mirror of the vehicle to respectively collect a color image and an infrared image of the exterior of the vehicle;
[0039] S302, detecting brightness information of the vehicle's external environment using a light sensor installed on the vehicle's electronic exterior rearview mirror;
[0040] S303, in response to the user's operation of turning on the infrared compensation function, feature matching is performed on the color image and the infrared image based on the brightness information to determine the similarity between the target object in the color image and the infrared image;
[0041] S304, determining a feature matching level according to the feature matching result, processing the color image and / or infrared image according to an image output mode corresponding to a predetermined feature matching level, and outputting the processed image to a display screen inside the vehicle for display.
[0042] In some embodiments, a color camera and an infrared camera installed on an electronic exterior rearview mirror of a vehicle are used to collect a color image and an infrared image of the exterior of the vehicle, respectively, including:
[0043] A color camera is used to collect images from the side and rear of the vehicle in real time to obtain a color image, and an infrared camera is used to collect images from the side and rear of the vehicle in real time to obtain an infrared image. The collected color image and infrared image are stored in a storage module of the electronic exterior rearview mirror controller.
[0044] Specifically, the RGB camera installed on the vehicle's electronic rearview mirror collects the visible light image of the side and rear of the vehicle in real time to generate a color image. The collected color image is transmitted to the storage module of the electronic rearview mirror controller (CMS controller) via a data line.
[0045] Furthermore, the storage module can perform preliminary processing on the color image, including image signal processing (ISP) operations such as noise reduction, de-distortion and image enhancement, etc. The processed color image is stored in the storage unit of the CMS controller for subsequent processing and display.
[0046] In addition, the IR camera installed on the electronic exterior rearview mirror of the vehicle is used to collect infrared images of the side and rear of the vehicle in real time to generate infrared images. The collected infrared images are transmitted to the storage module of the CMS controller through the data line. The storage module performs preliminary processing on the infrared images to ensure the clarity and accuracy of the images. The processed infrared images are stored in the storage unit of the CMS controller for subsequent processing and display.
[0047] Furthermore, the storage module built into the CMS controller is responsible for receiving and storing image data from the RGB camera and the IR camera. The color image and the infrared image are stored in separate storage units for subsequent image processing and fusion. The storage module also has basic image processing functions and can perform preliminary processing and optimization on the received image data.
[0048] It should be noted that under normal daylight conditions, the system mainly relies on the color images collected by the RGB camera to provide clear and realistic information about the vehicle's side and rear environment. Infrared cameras can be used as auxiliary equipment to detect heat sources or other special environmental features.
[0049] The system collects infrared images through infrared cameras to ensure clear images in low-light environments. Combined with low-light color images collected by RGB cameras, the system performs image fusion to improve image visibility and details.
[0050] In adverse weather conditions such as rain, snow and fog, the advantages of infrared cameras are more obvious. They can penetrate environmental interference and provide stable image information. The system combines the images of RGB cameras and IR cameras, and uses image fusion technology to ensure that the driver can clearly perceive the surrounding environment and improve driving safety.
[0051] Through the method of the present embodiment described above, the electronic exterior rearview mirror system can provide clear and reliable image display under various lighting and weather conditions, thereby enhancing vehicle safety and driving experience.
[0052] In some embodiments, before responding to the user's operation of turning on the infrared compensation function, the method further includes:
[0053] After the vehicle is powered on, the electronic exterior rearview mirror system is started and a fault self-check is performed. When a fault is detected in the electronic exterior rearview mirror system, the infrared compensation function is turned off, and the color image is processed using image enhancement technology. The processed color image is output to the display screen inside the vehicle for display. When no fault is detected in the electronic exterior rearview mirror system, whether to turn on the infrared compensation function is determined based on user operation.
[0054] Specifically, the working process of the electronic exterior rearview mirror system provided in the embodiment of the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. Figure 4 Schematic diagram of the working process of the electronic exterior rearview mirror system provided in the embodiment of the present application. Figure 4 As shown, the workflow of the electronic exterior rearview mirror system may specifically include the following contents:
[0055] When the vehicle is powered on, the electronic exterior mirror system automatically starts. After the electronic exterior mirror system starts, it will automatically perform a fault self-check to ensure that all hardware and software modules are operating normally. The self-check includes the status check of the RGB camera, IR camera, light sensor, display screen and controller.
[0056] If any fault is detected during the system self-check, the mode selection button on the central control interface will be grayed out and the infrared compensation function cannot be selected. At this time, the system defaults to outputting the conventional unfused electronic exterior mirror color image P0 after image signal processing (ISP). The P0 image includes noise reduction, de-distortion and image enhancement processing to ensure the clearest possible image without infrared compensation function.
[0057] If the system self-test does not detect any faults, the system will allow the user to choose whether to turn on the infrared compensation function through the central control button.
[0058] The system will record the mode selection made by the user in the previous cycle, and the previous cycle selection will be executed by default each time the power is turned on. The user can choose whether to turn on the infrared compensation function by pressing the central control button:
[0059] If the infrared compensation function is not enabled, the eMMC storage unit of the controller directly outputs the conventional color image P0 processed by the ISP module.
[0060] Enable infrared compensation: The system will determine whether to enter different fusion image output modes S1, S2 or S3 based on the current ambient brightness information and RGB / IR image feature matching results.
[0061] S1 mode: When the RGB image (color image) and the IR image (infrared image) are highly matched, the system outputs the enhanced RGB image P1 after ISP processing. This mode is suitable for environments with good lighting conditions.
[0062] S2 mode: When the RGB image and the IR image partially match, the system uses the deep learning model to fuse the images and outputs the RGB image P2 that incorporates the IR features. This mode is suitable for environments with insufficient lighting but still with some visible light.
[0063] S3 mode: When the RGB image does not match the IR image, the system outputs a pure IR infrared image P3. This mode is suitable for extremely low-light or no-light environments.
[0064] Through the above method of this embodiment, the electronic exterior rearview mirror system can automatically perform fault self-check after the vehicle is powered on to ensure the reliability and safety of the system. Under normal working conditions, the system provides a variety of image output modes, and users can choose whether to turn on the infrared compensation function according to actual needs, thereby providing clear and reliable image display under various lighting and weather conditions, improving driving safety and user experience.
[0065] In some embodiments, based on brightness information, feature matching is performed on the color image and the infrared image, including:
[0066] Perform edge detection on color images to detect the edges of target objects, determine the coordinate information of target objects through edge points, and extract the contour information of target objects using gradient-based edge detection algorithms;
[0067] The edge detection algorithm based on temperature gradient is used to detect the edge of infrared images, so as to extract the contour information and coordinate information of the target object in the infrared image;
[0068] Extract the contour features of the target object from the contour information, and use the similarity measurement method to calculate the similarity between the contour features of the target object in the color image and the contour features of the target object in the infrared image;
[0069] According to the similarity between the contour features, the feature matching level between the target object in the color image and the target object in the infrared image is determined.
[0070] Specifically, the process of matching and fusing color images and infrared images provided in the embodiments of the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. Figure 5 Schematic diagram of the matching and fusion process of color images and infrared images provided in the embodiment of the present application. Figure 5 As shown, the process of matching and fusing the color image and the infrared image may specifically include the following contents:
[0071] First, a color image is collected by an RGB camera installed on the vehicle's electronic exterior mirror. Canny edge detection is performed on the color image to detect the edge of the target object. The gradient of the image is calculated to identify the area with significant intensity changes in the image to determine the edge of the object.
[0072] The coordinate information of the target object is determined by the edge points. The contour information of the target object is extracted by using a gradient-based edge detection algorithm (such as Sobel, Prewitt, etc.). The contour of the object is extracted by calculating the gradient of the pixels in the image.
[0073] In addition, the infrared image collected by the IR camera installed on the vehicle's electronic exterior rearview mirror is used to perform edge detection based on temperature gradient to extract the contour and coordinate information of the target object.
[0074] The temperature change characteristics in infrared images are used to detect the edges of objects, ensuring accurate identification of target objects even in low-light environments.
[0075] Furthermore, contour features of the target object, including shape, size, direction, etc., are extracted from the contour information of the color image and the infrared image.
[0076] Furthermore, a similarity measurement method (such as Hausdorff distance, Chamfer distance, etc.) is used to calculate the similarity between the contour features of the target object in the color image and the contour features of the target object in the infrared image.
[0077] Among them, Hausdorff distance is used to measure the maximum distance between two point sets and is suitable for calculating the similarity of image contours. Chamfer distance is used to calculate the average distance between two point sets and measure the similarity of contour features.
[0078] Furthermore, based on the calculated contour feature similarity, the feature matching level between the target object in the color image and the target object in the infrared image is determined. In practical applications, the feature matching levels include but are not limited to the following:
[0079] High matching level: If the distance D between the corresponding coordinate points of the target object in the two images is less than 5 pixels, and the contour similarity score S is higher than 0.8 (the full score is 1), and the ratio P of the size difference between the target object in the RGB image and the IR image is less than 5%, then the two contours are considered to match and are judged as the first feature matching level.
[0080] Medium matching level: If the distance D between the corresponding coordinate points of the target object in the two images is greater than 5 pixels and less than 500 pixels, or the contour similarity score S is greater than 0.1 and less than 0.8, or the ratio P of the size difference between the target object in the RGB image and the IR image is greater than 5% and less than 90%, then it is determined that the RGB image and the IR image do not completely match, and it is judged as the second feature matching level.
[0081] Low matching level: If the distance D between the corresponding coordinate points of the target object in the two images is greater than 500 pixels, or the contour similarity score S is less than 0.1, or the ratio P of the size difference between the target object in the RGB image and the IR image is less than 5%, then the two contours are judged to be mismatched and the third feature matching level is determined.
[0082] Through the method of the present embodiment described above, the system can perform feature matching between color images and infrared images based on brightness information, thereby achieving accurate perception and display of the side and rear environment of the vehicle and improving driving safety.
[0083] In some embodiments, judging the feature matching level between the target object in the color image and the target object in the infrared image according to the similarity between the contour features includes:
[0084] Calculate the distance, contour similarity score and size difference ratio between the corresponding coordinate points of the target object in the color image and the target object in the infrared image respectively;
[0085] The distance between the coordinate points, the contour similarity score, and the size difference ratio are compared with the thresholds corresponding to the preset feature matching levels, and the feature matching level is determined based on the comparison results.
[0086] Specifically, this embodiment uses the Canny edge detection algorithm to detect the edge of the target object in the color image, and determines the coordinate information of the target object through the edge points. A gradient-based edge detection algorithm (such as Sobel, Prewitt, etc.) is used to extract the contour information of the target object. A temperature gradient-based edge detection algorithm is used to extract the contour and coordinate information of the target object in the infrared image.
[0087] Furthermore, the contour features (such as shape, size, direction, etc.) in the color image and the infrared image are extracted, and the similarity between the contours is calculated using a similarity measurement method (such as Hausdorff distance, Chamfer distance, etc.), and the target objects in the RGB image and the IR image are judged to match based on the similarity.
[0088] Furthermore, the calculated distance D, similarity score S and size difference ratio P are compared with the preset thresholds corresponding to the feature matching level. According to the comparison results, the feature matching level is determined, and the image output state is adjusted in real time.
[0089] For example, in one example, first, the distance D between the corresponding coordinate points of the target object in the color image and the target object in the infrared image is calculated. Then, a similarity measurement method (such as Hausdorff distance or Chamfer distance) is used to calculate the similarity score S between the contour features of the target object in the color image and the contour features of the target object in the infrared image. Then, the difference ratio P between the size of the target object in the color image and the size of the target object in the infrared image is calculated.
[0090] The following table describes in detail the three feature matching level determination methods and principles provided in the embodiment of the present application.
[0091] Table 1 Definition of image output levels
[0092]
[0093] Table 2 Output image type definition
[0094]
[0095]
[0096] In practical applications, the present application embodiment provides the following three feature matching level judgment processes, which may specifically include the following contents:
[0097] 1. First feature matching level:
[0098] When the distance D between the coordinate points is less than 5 pixels, the contour similarity score S is higher than 0.8 (full score is 1), and the size difference ratio P is less than 5%, the contours of the target objects in the two images match, and the output target position coordinates and contours on the current RGB image meet the display requirements, it is determined to be the first feature matching level.
[0099] 2. Second feature matching level:
[0100] When the distance D between the coordinate points is greater than 5 pixels and less than 500 pixels, or the contour similarity score S is greater than 0.1 and less than 0.8 (full score is 1), or the size difference ratio P is greater than 5% and less than 90%, the target object in the RGB image and the IR image does not completely match, and the target position coordinates or contour size on the RGB image are missing, and it is determined to be the second feature matching level.
[0101] 3. Third feature matching level:
[0102] When the distance D between the coordinate points is greater than 500 pixels, or the contour similarity score S is less than 0.1 (full score is 1), or the size difference ratio P is less than 5%, the contours of the target object in the two images do not match, and it is determined to be the third feature matching level.
[0103] It should be noted that when comparing the coordinates and contours of RGB images and IR images, the setting of the threshold (TBD) needs to be determined according to the specific application requirements and data characteristics. As a calibrable quantification of image comparison, this application focuses on clarifying the methods and strategies of image comparison, and only lists the conventional RGB image and IR image coordinate and contour comparison methods. The other threshold settings and image processing techniques listed above are also within the scope of protection of this application.
[0104] Through the method of the present embodiment described above, the system can perform feature matching on color images and infrared images based on brightness information, thereby ensuring accurate perception of the features of target objects in various environments, providing clear and reliable image display, and improving driving safety.
[0105] In some embodiments, the color image and / or infrared image is processed according to the image output mode corresponding to the predetermined feature matching level, and the processed image is output to a display screen inside the vehicle for display, including:
[0106] When it is determined that the first feature matching level is met, the color image is processed using the image enhancement technology according to the first image output mode, and the processed color image is output to a display screen inside the vehicle for display;
[0107] When it is determined that it belongs to the second feature matching level, the color image and the infrared image are fused using the image fusion module according to the second image output mode, and the fused image is output to the display screen inside the vehicle for display;
[0108] When it is determined that it belongs to the third feature matching level, the infrared image is processed using image enhancement technology according to the third image output mode, and the processed infrared image is output to a display screen inside the vehicle for display.
[0109] Specifically, when it is judged to belong to the first feature matching level, that is, the distance D between the coordinate points is less than 5 pixels, the contour similarity score S is higher than 0.8 (the full score is 1), and the size difference ratio P is less than 5%. According to the first image output mode, the color image is processed using image enhancement technology. For example, it includes image denoising, dedistortion and image enhancement to improve the clarity and detail of the image. The image fusion module only outputs the color image P1 processed by the ISP. The processed color image P1 is output to the display screen inside the vehicle for display.
[0110] Further, when it is judged to belong to the second feature matching level, that is, the distance D between the coordinate points is greater than 5 pixels and less than 500 pixels, or the contour similarity score S is greater than 0.1 and less than 0.8 (the full score is 1), or the size difference ratio P is greater than 5% and less than 90%. According to the second image output mode, the color image and the infrared image are fused using the image fusion module. The image fusion process may include the following:
[0111] The mapping relationship between infrared images and color images is learned based on deep learning models (such as convolutional neural networks). During the model training process, a large number of labeled data sets are used for training so that the model can accurately map the features of infrared images to color images.
[0112] The key feature points in the infrared image are replaced with the corresponding coordinate positions in the color image to ensure that the features in the two images are aligned.
[0113] The fused image is color balanced to adjust the color ratio to make the image present a more natural visual effect. Finally, contrast enhancement is performed to improve the clarity and detail of the image.
[0114] The image fusion module outputs a fused image P2 that fuses the IR target features. The fused image P2 is output to a display screen inside the vehicle for display.
[0115] Further, when it is judged to belong to the third feature matching level, that is, the distance D between the coordinate points is greater than 500 pixels, or the contour similarity score S is less than 0.1 (the full score is 1), or the size difference ratio P is less than 5%.
[0116] According to the third image output mode, the infrared image is processed using image enhancement technology, for example, including contrast enhancement and detail enhancement of the infrared image to ensure a clear image in a low-light environment.
[0117] The image fusion module outputs the IR infrared image P3. The processed infrared image P3 is output to a display screen inside the vehicle for display.
[0118] In one example, edge detection and feature extraction are performed on color images and infrared images respectively. The distance between coordinate points, contour similarity score and size difference ratio are calculated. These feature parameters are compared with preset thresholds to determine the feature matching level. The images are processed and output according to the following image output modes:
[0119] S1 mode: When the image output mode is S1 (ie, the first image output mode), the image fusion module only outputs the color image P1 processed by the ISP.
[0120] S2 mode: When the image output mode is S2 (ie, the second image output mode), the image fusion module outputs a fused image P2 that fuses the IR target features.
[0121] S3 mode: When the image output mode is S3 (i.e., the third image output mode), it means that the current RGB environment and target contour do not meet the fusion output requirements, and the image fusion module outputs the IR infrared image P3.
[0122] Through the method of the present embodiment described above, the system can select an appropriate image output mode according to the feature matching level, and perform corresponding processing on the color image and infrared image, thereby ensuring clear and reliable image display under various lighting and environmental conditions, thereby improving driving safety and user experience.
[0123] In some embodiments, the color image and the infrared image are fused using an image fusion module, including:
[0124] The deep learning image fusion model is trained using a pre-labeled data set so that the deep learning image fusion model can learn the mapping relationship between infrared images and color images;
[0125] Use the trained deep learning image fusion model to identify key feature points in infrared images;
[0126] The key feature points in the infrared image are replaced with the corresponding coordinate positions of the color image so as to align the features in the infrared image and the color image to obtain a fused image; the fused image is subjected to color balancing and contrast enhancement processing.
[0127] Specifically, the deep learning image fusion model is trained using pre-labeled color image and infrared image datasets. Through training, the deep learning image fusion model (such as a convolutional neural network) can learn the mapping relationship from infrared images to color images. Using large-scale datasets for training can improve the generalization ability and accuracy of the model.
[0128] Furthermore, the trained deep learning image fusion model is used to identify key feature points in the infrared image. Key contour feature points in the infrared image are identified to ensure high accuracy during the fusion process. Key feature points in the infrared image are replaced with the corresponding coordinates of the color image. Features in the infrared image and the color image are aligned to obtain the fused image.
[0129] Furthermore, the fused image is color balanced and the color ratio is adjusted to make the image present a more natural visual effect. Contrast enhancement is performed to improve the clarity and detail of the image and ensure that the image is more in line with human visual perception.
[0130] In a specific example, the P2 image fusion method: learn the mapping relationship between IR image and RGB image based on a deep learning model (such as a convolutional neural network). Replace the feature points on the IR image with the corresponding coordinate positions of the RGB image. Perform post-processing such as color balancing and contrast enhancement on the fused image to improve the image quality and make it more consistent with human visual perception. Obtain the fused image P2 and output it to the display screen inside the vehicle for display.
[0131] Through the method of the present embodiment, the electronic exterior rearview mirror system can use the deep learning model to efficiently fuse color images and infrared images, and provide clear and reliable image display under various lighting and environmental conditions. In addition, through the system alarm and prompt function, the driver can timely understand the working status of the electronic exterior rearview mirror, improving driving safety and user experience.
[0132] In some embodiments, after processing the infrared image using the image enhancement technology according to the third image output mode and outputting the processed infrared image to a display screen inside the vehicle for display, the method further includes:
[0133] When the system output mode is the third image output mode, the infrared image processed by the image enhancement technology is output to the display screen for display, and the alarm signal is output synchronously, and the alarm signal is transmitted to the display screen interface for visual warning;
[0134] When the system detects that the continuous time in the third image output mode exceeds the preset first time threshold, a warning window is popped up by the central control system of the vehicle to prompt the user that the electronic exterior rearview mirror function in the current environment is abnormal;
[0135] When the system detects that the state of exiting the third image output mode has continued for more than a preset second time threshold, the central control system of the vehicle is used to automatically cancel the warning window so that the display screen can return to normal display.
[0136] Specifically, when the system is determined to be in the third image output mode (S3 mode), the infrared image is processed using image enhancement technology. The image enhancement processing includes contrast enhancement, noise reduction and detail enhancement to ensure a clear image in low light or complex environments.
[0137] Furthermore, the processed infrared image P3 is output to a display screen inside the vehicle for display, ensuring that the driver can obtain clear visual information.
[0138] Furthermore, when the system enters S3 mode, the controller not only outputs the fused video, but also outputs an alarm signal synchronously. The alarm signal is transmitted to the display interface to provide a visual warning to remind the driver that the electronic exterior rearview mirror is malfunctioning in the current environment.
[0139] Furthermore, the system continuously monitors the state time in the S3 mode. If the system detects that the state time in the S3 mode exceeds a preset first time threshold (e.g., 2000 milliseconds), a warning window is popped up using the vehicle's central control system. The warning window prompts the user that the electronic exterior rearview mirror function is abnormal in the current environment, so as to remind the driver to pay attention to driving safety.
[0140] Furthermore, the system monitors the exit state of the S3 mode. If the system detects that the exit state of the S3 mode lasts for more than a preset second time threshold (e.g., 500 milliseconds), the warning window is automatically cancelled using the vehicle's central control system. After the warning is cancelled, the display screen returns to normal display, ensuring that the driver can use the electronic exterior rearview mirror normally.
[0141] For example, in one example, when the system output mode is S3 mode (third image output mode), in addition to the fused video, the controller synchronously outputs an alarm signal and transmits it to the LCD display interface to provide a warning. When the fused image mode jumps to the S3 state for a continuous time exceeding the first time threshold (e.g., 2000 milliseconds), the central control pop-up window prompts the user that the electronic exterior rearview mirror function in the current environment is abnormal. When exiting the S3 mode exceeds the second time threshold (e.g., 500 milliseconds), the central control alarm pop-up window is canceled.
[0142] According to the technical solution provided in the embodiments of the present application, the technical solution of the present application has at least the following advantages:
[0143] This technical solution effectively improves the driver's interactive experience and the intelligence of the system. By fusing RGB images and IR images, the system can provide clear and stable video images under various environmental conditions, allowing the driver to accurately perceive the environmental conditions around the vehicle body, reducing misjudgments and operational errors, thereby improving driving safety.
[0144] This technical solution has a high degree of integration, and only one IR infrared camera is needed to achieve full range scene coverage. This design not only simplifies the system structure, but also reduces the hardware cost, while enhancing the system's functionality and reliability. Since the infrared camera has a stable imaging effect under low light and bad weather conditions, it can effectively compensate for the shortcomings of the RGB camera in these situations, thereby ensuring the normal operation of the system in various scenarios.
[0145] Through the image compensation technology of the fusion of multiple visual sensors with different output types, the problem of blurred images and unclear target positions of electronic exterior rearview mirror cameras in bad weather, poor lighting at night, glare, rain, snow and fog is effectively solved. The driver can obtain clearer and more stable video images, which greatly improves driving safety. This technical solution not only has high product advancement, but also has significant practical value and broad promotion prospects.
[0146] This technical solution forms a multi-sensor redundant design by adding a set of sensor equipment, which makes up for the inherent deficiencies of a single sensor, thereby improving the reliability and safety of the system. The multi-sensor redundant design can still ensure the normal operation of the system when a single sensor fails, meeting the functional safety requirements of the system level.
[0147] To sum up, the technical solution of the present application improves the intelligence and safety of the system by optimizing the image processing capability of the electronic exterior rearview mirror system, while achieving efficient scene coverage and system-level functional safety, with significant technical effects and broad application prospects.
[0148] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0149] Figure 6 Schematic diagram of the structure of a super-resolution image processing device based on a digital camera provided in an embodiment of the present application. Figure 6 As shown, the super-resolution image processing device based on a digital camera includes:
[0150] The acquisition module 601 is configured to use a color camera and an infrared camera installed on the electronic exterior rearview mirror of the vehicle to respectively acquire a color image and an infrared image of the exterior of the vehicle;
[0151] The detection module 602 is configured to detect the brightness information of the vehicle's external environment using a light sensor installed on the vehicle's electronic exterior rearview mirror;
[0152] The matching module 603 is configured to perform feature matching between the color image and the infrared image based on the brightness information in response to the user's operation of turning on the infrared compensation function, so as to determine the similarity between the target object in the color image and the infrared image;
[0153] The processing module 604 is configured to determine the feature matching level based on the feature matching results, process the color image and / or infrared image according to the image output mode corresponding to the predetermined feature matching level, and output the processed image to a display screen inside the vehicle for display.
[0154] In some embodiments, Figure 6 After the vehicle is powered on, the self-check module 605 starts the electronic exterior rearview mirror system and performs a fault self-check. When a fault is detected in the electronic exterior rearview mirror system, the infrared compensation function is turned off, and the color image is processed using image enhancement technology, and the processed color image is output to the display screen inside the vehicle for display; when no fault is detected in the electronic exterior rearview mirror system, it is determined whether to turn on the infrared compensation function based on the user's operation.
[0155] In some embodiments, Figure 6The matching module 603 performs edge detection on the color image to detect the edge of the target object, determines the coordinate information of the target object through the edge points, and extracts the contour information of the target object using a gradient-based edge detection algorithm; performs edge detection on the infrared image using a temperature gradient-based edge detection algorithm to extract the contour information and coordinate information of the target object in the infrared image; extracts the contour features of the target object from the contour information, and calculates the similarity between the contour features of the target object in the color image and the contour features of the target object in the infrared image using a similarity measurement method; and determines the feature matching level between the target object in the color image and the target object in the infrared image based on the similarity between the contour features.
[0156] In some embodiments, Figure 6 The matching module 603 calculates the distance, contour similarity score and size difference ratio between the corresponding coordinate points of the target object in the color image and the target object in the infrared image respectively; compares the distance between the coordinate points, the contour similarity score and the size difference ratio with the threshold values corresponding to the preset feature matching level, and determines the feature matching level according to the comparison results.
[0157] In some embodiments, Figure 6 When the processing module 604 determines that it belongs to the first feature matching level, it uses the image enhancement technology to process the color image according to the first image output mode, and outputs the processed color image to the display screen inside the vehicle for display; when it is determined that it belongs to the second feature matching level, it uses the image fusion module to fuse the color image and the infrared image according to the second image output mode, and outputs the fused image to the display screen inside the vehicle for display; when it is determined that it belongs to the third feature matching level, it uses the image enhancement technology to process the infrared image according to the third image output mode, and outputs the processed infrared image to the display screen inside the vehicle for display.
[0158] In some embodiments, Figure 6 The processing module 604 uses a pre-labeled data set to train the deep learning image fusion model so that the deep learning image fusion model learns the mapping relationship between the infrared image and the color image; uses the trained deep learning image fusion model to identify the key feature points in the infrared image; replaces the key feature points in the infrared image with the corresponding coordinate positions of the color image so as to align the features in the infrared image and the color image to obtain a fused image; and performs color balancing and contrast enhancement processing on the fused image.
[0159] In some embodiments, Figure 6After the processing module 604 processes the infrared image using the image enhancement technology according to the third image output mode and outputs the processed infrared image to the display screen inside the vehicle for display, when the system output mode is the third image output mode, the infrared image processed by the image enhancement technology is output to the display screen for display, and an alarm signal is output synchronously, and the alarm signal is transmitted to the display screen interface for visual warning; when the system detects that the continuous time in the third image output mode exceeds the preset first time threshold, the vehicle's central control system pops up a warning window to prompt the user that the electronic exterior rearview mirror function in the current environment is abnormal; when the system detects that the continuous time in the exit from the third image output mode exceeds the preset second time threshold, the vehicle's central control system automatically cancels the warning window so that the display screen can resume normal display.
[0160] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0161] Figure 7 Schematic diagram of the structure of the electronic device 7 provided in the embodiment of the present application. Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. When the processor 701 executes the computer program 703, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0162] Exemplarily, the computer program 703 may be divided into one or more modules / units, which are stored in the memory 702 and executed by the processor 701 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 703 in the electronic device 7.
[0163] The electronic device 7 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 7 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 It is only an example of the electronic device 7 and does not constitute a limitation of the electronic device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.
[0164] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0165] The memory 702 may be an internal storage unit of the electronic device 7, for example, a hard disk or memory of the electronic device 7. The memory 702 may also be an external storage device of the electronic device 7, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 702 may also include both an internal storage unit of the electronic device 7 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device. The memory 702 may also be used to temporarily store data that has been output or is to be output.
[0166] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0167] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0168] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0169] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0170] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0172] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0173] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. An image compensation method based on an electronic exterior rearview mirror, characterized in that: include: The color camera and infrared camera installed on the electronic exterior rearview mirror of the vehicle are used to collect the color image and infrared image of the exterior of the vehicle respectively; The brightness information of the vehicle's external environment is detected by using a light sensor installed on the vehicle's electronic exterior rearview mirror; In response to a user's operation of turning on the infrared compensation function, feature matching is performed on the color image and the infrared image based on the brightness information to determine the similarity between the target object in the color image and the infrared image; The feature matching level is determined according to the feature matching result, the color image and / or the infrared image is processed according to the image output mode corresponding to the predetermined feature matching level, and the processed image is output to a display screen inside the vehicle for display.
2. The method according to claim 1, characterized in that Before responding to the user's operation of turning on the infrared compensation function, the method further includes: After the vehicle is powered on, the electronic exterior rearview mirror system is started and a fault self-check is performed. When a fault is detected in the electronic exterior rearview mirror system, the infrared compensation function is turned off, and the color image is processed using image enhancement technology. The processed color image is output to a display screen inside the vehicle for display. When no fault is detected in the electronic exterior rearview mirror system, whether to turn on the infrared compensation function is determined based on user operation.
3. The method according to claim 1, characterized in that The performing feature matching between the color image and the infrared image based on the brightness information includes: Performing edge detection on the color image to detect the edge of a target object, determining the coordinate information of the target object through edge points, and extracting the contour information of the target object using a gradient-based edge detection algorithm; Performing edge detection on the infrared image using a temperature gradient-based edge detection algorithm to extract contour information and coordinate information of the target object in the infrared image; Extracting contour features of the target object from the contour information, and calculating the similarity between the contour features of the target object in the color image and the contour features of the target object in the infrared image using a similarity measurement method; The feature matching level between the target object in the color image and the target object in the infrared image is determined according to the similarity between the contour features.
4. The method according to claim 3, characterized in that The step of determining the feature matching level between the target object in the color image and the target object in the infrared image according to the similarity between the contour features includes: Calculating the distance, contour similarity score and size difference ratio between corresponding coordinate points of the target object in the color image and the target object in the infrared image respectively; The distance between the coordinate points, the contour similarity score, and the size difference ratio are respectively compared with thresholds corresponding to preset feature matching levels, and the feature matching level is determined according to the comparison results.
5. The method according to claim 4, characterized in that The processing of the color image and / or the infrared image according to the predetermined image output mode corresponding to the feature matching level, and outputting the processed image to a display screen inside the vehicle for display, comprises: When it is determined that the color image belongs to the first feature matching level, the color image is processed by using the image enhancement technology according to the first image output mode, and the processed color image is output to the display screen inside the vehicle for display; When it is determined that it belongs to the second feature matching level, the color image and the infrared image are fused using an image fusion module according to a second image output mode, and the fused image is output to a display screen inside the vehicle for display; When it is determined that it belongs to the third feature matching level, the infrared image is processed using image enhancement technology according to the third image output mode, and the processed infrared image is output to a display screen inside the vehicle for display.
6. The method according to claim 5, characterized in that The step of fusing the color image and the infrared image using an image fusion module includes: The deep learning image fusion model is trained using a pre-labeled data set so that the deep learning image fusion model can learn the mapping relationship between infrared images and color images; Using the trained deep learning image fusion model, key feature points in the infrared image are identified; The key feature points in the infrared image are replaced with the corresponding coordinate positions of the color image so as to align the features in the infrared image and the color image to obtain a fused image; and the fused image is subjected to color balancing and contrast enhancement processing.
7. The method according to claim 5, characterized in that After processing the infrared image using the image enhancement technology according to the third image output mode and outputting the processed infrared image to a display screen inside the vehicle for display, the method further includes: When the system output mode is the third image output mode, the infrared image processed by the image enhancement technology is output to the display screen for display, and an alarm signal is output synchronously, and the alarm signal is transmitted to the display screen interface for visual warning; When the system detects that the continuous time in the state of the third image output mode exceeds a preset first time threshold, a warning window is popped up by the central control system of the vehicle to prompt the user that the electronic exterior rearview mirror function in the current environment is abnormal; When the system detects that the continuous time of exiting the third image output mode exceeds a preset second time threshold, the central control system of the vehicle is used to automatically cancel the warning window so that the display screen can resume normal display.
8. An image compensation device based on an electronic exterior rearview mirror, characterized in that: include: The acquisition module is configured to respectively acquire a color image and an infrared image of the outside of the vehicle using a color camera and an infrared camera installed on the electronic exterior rearview mirror of the vehicle; The detection module is configured to detect brightness information of the vehicle's external environment using a light sensor installed on the vehicle's electronic exterior rearview mirror; a matching module, configured to perform feature matching between the color image and the infrared image based on the brightness information in response to a user's operation of turning on the infrared compensation function, so as to determine the similarity between the target object in the color image and the infrared image; The processing module is configured to determine the feature matching level according to the feature matching result, process the color image and / or the infrared image according to the image output mode corresponding to the predetermined feature matching level, and output the processed image to a display screen inside the vehicle for display.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
