Electronic rearview mirror flicker mitigation method and related devices

Abstract

The application provides a kind of electronic rearview mirror flicker suppression method and related device, method includes: according to original image data, vehicle state data and environmental data, whether target vehicle is in flicker trigger working condition is judged;If it is judged that target vehicle is in flicker trigger working condition, then according to flicker trigger working condition type, target flicker suppression strategy is determined;According to original image data, target flicker suppression strategy is executed, and target image data is obtained.The frame rate of image acquisition equipment, the frame rate of electronic rearview mirror chip image processing and the refresh rate of image display device are set to the same 50HZ, which is conducive to solving the phenomenon of imaging lagging and incoherence, suppressing power frequency flicker, and by identifying different flicker trigger working conditions and executing different flicker suppression strategies, it is conducive to improving the accuracy of flicker suppression.

Description

Technical Field

[0001] This application belongs to the field of electronic rearview mirrors, specifically relating to a method and device for suppressing flicker in electronic rearview mirrors. Background Technology

[0002] In real-world driving environments, cameras are often affected by interference (such as streetlights) when capturing image data, resulting in screen flickering. Electronic rearview mirrors exhibiting screen flickering due to streetlight illumination pose a real driving hazard and increase the risk of accidents.

[0003] This phenomenon of screen flickering on electronic rearview mirror displays due to streetlight illumination is commonly known as LED flickering or light source pulse flicker. It's not a malfunction of the display itself, but rather a mismatch between the pulsed light generated by the external light source and the camera's imaging system. Specifically, LED streetlights often operate with high-frequency pulses, which are imperceptible to the human eye, but the camera sensor records these changes in brightness when capturing images, causing flickering or striped effects on the display screen. Summary of the Invention

[0004] This application provides an electronic rearview mirror flicker suppression method and related apparatus. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it is beneficial to solve the phenomenon of image stuttering and discontinuity, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker triggering conditions and executing different flicker suppression strategies accordingly.

[0005] In a first aspect, embodiments of this application provide a method for suppressing flicker in an electronic rearview mirror, the method comprising:

[0006] Acquire raw image data, vehicle status data, and environmental data;

[0007] Based on the original image data, the vehicle status data, and the environmental data, it is determined whether the target vehicle is in a flashing trigger condition. The flashing trigger conditions include continuous street light conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights.

[0008] If it is determined that the target vehicle is in a flashing-triggered condition, then a target flashing suppression strategy is determined based on the type of flashing-triggered condition.

[0009] The target flicker suppression strategy is executed based on the original image data to obtain the target image data;

[0010] The target image data is sent to the image display device.

[0011] In one possible example, determining whether the target vehicle is in a flashing-triggered condition includes:

[0012] Identify streetlights based on the original image data and environmental data;

[0013] If a street light is identified, the vehicle speed of the target vehicle is determined based on the vehicle status data.

[0014] If it is determined that the vehicle speed is greater than the preset speed, then it is determined that the target vehicle is in a high-speed driving condition at night.

[0015] If it is determined that the vehicle speed is less than or equal to a preset speed, the system determines whether the target vehicle is in a tunnel based on the tunnel recognition model, the original image data, and the environmental data.

[0016] If it is determined that the target vehicle is in a tunnel, then it is determined whether the original image data contains a region of alternating light and dark that meets a preset condition;

[0017] If it is determined that the original image data contains a region of alternating brightness and darkness that meets preset conditions, then it is determined that the target vehicle is in the tunnel continuous street light condition.

[0018] If it is determined that the target vehicle is not in the tunnel, then based on the original image data, the environmental data, and the rain recognition model, it is determined whether the target vehicle is in a rainy environment;

[0019] If it is determined that the target vehicle is in a rainy environment, then the presence of road surface water is determined based on the original image data.

[0020] If it is determined that there is standing water on the current road, then the target vehicle is in a rainy night with water surface reflection conditions.

[0021] In one possible example, determining the target flicker suppression strategy based on the flicker triggering condition type includes:

[0022] If the flicker triggering condition type is the tunnel continuous street light condition, then the target flicker suppression strategy is determined as follows:

[0023] Determine the first frequency of the flashing;

[0024] The first exposure period to be adjusted is determined based on the first frequency, wherein the first exposure period is an integer multiple of the first frequency;

[0025] Determine the luminance variance of multiple consecutive frames in the original image data;

[0026] If it is determined that the brightness variance is greater than the preset variance, then the weight matrix of the first frame is determined;

[0027] The original image data is fused according to the first frame weight matrix to obtain the first image data;

[0028] Stripe detection is performed on the first image data. If stripes are detected, the stripe region is determined.

[0029] Spatial frequency domain filtering is performed on the stripe region in the first image data to obtain the third image data, and the third image data is used as the target image data.

[0030] In one possible example, the stripe detection of the first image data includes:

[0031] Extract the high-frequency components of the first image data;

[0032] A binarized image is generated based on the high-frequency components;

[0033] If it is determined that the binarized image simultaneously satisfies the preset morphological features, preset frequency features, and preset directional features, then it is determined that the first image data contains stripes.

[0034] In one possible example, determining the target flicker suppression strategy based on the flicker triggering condition type includes:

[0035] If the flickering triggering condition is nighttime high-speed driving, then the target flicker suppression strategy is determined as follows:

[0036] Based on the original image data, predict the street light occurrence cycle;

[0037] The required adjustment of the second exposure duration is determined based on the vehicle speed and the street light occurrence cycle;

[0038] Identify the streetlight highlight areas in the original image data;

[0039] Determine the brightness jump range of the street light's highlight area;

[0040] Based on the brightness jump amplitude, determine the second frame weight matrix of the motion compensation temporal filtering;

[0041] The original image data is subjected to temporal filtering based on the weight matrix of the second frame to obtain the fourth image data;

[0042] Determine the enhancement adjustment parameters for the high-beam area of ​​the streetlight;

[0043] The parameters of the street light highlight area in the fourth image data are adjusted according to the enhancement adjustment parameters to obtain the fifth image data, and the fifth image data is used as the target image data.

[0044] In one possible example, determining the target flicker suppression strategy based on the flicker triggering condition type includes:

[0045] If the flicker triggering condition is a rainy night water surface reflection condition, then the target flicker suppression strategy is determined as follows:

[0046] The original image data is divided into regions to obtain multiple regions, including water surface reflection region, normal road surface, vehicle headlight region and street light region.

[0047] Determine the third exposure time for the water surface reflective area and the fourth exposure time for the non-water surface reflective area;

[0048] Extract non-overexposed detail data of the water surface reflection area from multiple frames of the original image data;

[0049] The water surface reflection area in multiple frames of the original image data is repaired using the non-overexposed detail data to obtain the sixth image data;

[0050] Determine the color shift calibration parameters for the water surface reflection area in the sixth image data, and perform color shift calibration on the water surface reflection area in the sixth image data according to the color shift calibration parameters;

[0051] Determine the transition region and corresponding boundary pixel interpolation of the water surface reflection area in the sixth image data;

[0052] The transition region is interpolated based on the boundary pixel interpolation to obtain the seventh image data, and the seventh image data is used as the target image data.

[0053] In one possible example, after acquiring the raw image data, vehicle status data, and environmental data, the method further includes:

[0054] The original image data is divided into multiple pixel regions according to preset rules;

[0055] Determine the average pixel brightness for each pixel region;

[0056] Determine the brightness difference between the brightness of each pixel in the pixel region and the average brightness of the pixels;

[0057] Perform the following operation on each pixel to correct bad pixels:

[0058] If it is determined that the pixel brightness difference corresponding to the currently processed pixel is greater than a preset brightness threshold, then the currently processed pixel is determined to be a bad pixel.

[0059] Replace the brightness of the currently processed pixel with the average brightness of the pixel.

[0060] Secondly, embodiments of this application provide an electronic rearview mirror flicker suppression device, applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment system for a target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment system. The frame rate at which the image acquisition device acquires images, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. The electronic rearview mirror flicker suppression device includes an acquisition unit, a judgment unit, a determination unit, an execution unit, and a transmission unit.

[0061] The acquisition unit is used to acquire raw image data, vehicle status data, and environmental data;

[0062] The judgment unit is used to determine whether the target vehicle is in a flashing trigger condition based on the original image data, the vehicle status data and the environmental data. The flashing trigger condition includes continuous street light condition in tunnel, high-speed driving condition at night and water surface reflection condition at rainy night.

[0063] The determining unit is used to determine a target flicker suppression strategy based on the flicker triggering condition type if it is determined that the target vehicle is in a flicker triggering condition.

[0064] The execution unit is used to execute the target flicker suppression strategy based on the original image data to obtain the target image data;

[0065] The transmission unit is used to send the target image data to the image display device.

[0066] A third aspect of this application provides an electronic device including: a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for some or all of the steps as described in the first aspect.

[0067] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform some or all of the steps described in the first aspect of this application.

[0068] A fifth aspect of this application provides a computer program product, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. This computer program product may be a software installation package.

[0069] As can be seen, in this embodiment, the original image data, vehicle status data, and environmental data are first acquired. Then, based on the original image data, vehicle status data, and environmental data, it is determined whether the target vehicle is in a flicker-triggered condition. Flicker-triggered conditions include continuous streetlight conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights. If it is determined that the target vehicle is in a flicker-triggered condition, a target flicker suppression strategy is determined according to the flicker-triggered condition type. The target flicker suppression strategy is executed based on the original image data to obtain the target image data. Finally, the target image data is sent to the image display device. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it is beneficial to solve the phenomenon of image stuttering and discontinuity, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker-triggered conditions and executing different flicker suppression strategies accordingly. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is a schematic diagram of the architecture of an electronic rearview mirror system provided in an embodiment of this application;

[0072] Figure 2 This is a flowchart illustrating an electronic rearview mirror flicker suppression method provided in an embodiment of this application;

[0073] Figure 3 This is a flowchart illustrating a method for determining a flashing trigger condition, provided in an embodiment of this application.

[0074] Figure 4 This is a flowchart illustrating a method for determining a flicker suppression strategy, as provided in an embodiment of this application.

[0075] Figure 5 This is a flowchart illustrating another method for determining a flicker suppression strategy provided in an embodiment of this application.

[0076] Figure 6 This is a flowchart illustrating another method for determining a flicker suppression strategy provided in an embodiment of this application;

[0077] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0078] Figure 8 This is a functional unit block diagram of an electronic rearview mirror flicker suppression device provided in an embodiment of this application;

[0079] Figure 9 A pin diagram of an electronic rearview mirror chip provided for an embodiment of this application;

[0080] Figure 10 This is a schematic diagram of the architecture of an electronic rearview mirror chip control board provided in an embodiment of this application. Detailed Implementation

[0081] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0082] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0083] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0084] In the embodiments of this application, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist simultaneously; B exists alone. Among them, A and B can be singular or plural.

[0085] In this embodiment, the symbol " / " can indicate that the preceding and following objects are in an "or" relationship. Alternatively, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.

[0086] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0087] In the embodiments of this application, "equal to" can be used with "greater than" and is applicable to technical solutions used when "greater than" is used; it can also be used with "less than" and is applicable to technical solutions used when "less than" is used. When "equal to" is used with "greater than", it is not used with "less than"; when "equal to" is used with "less than", it is not used with "greater than".

[0088] To better understand the solutions of the embodiments of this application, the electronic devices, related concepts and background that may be involved in the embodiments of this application will be introduced below.

[0089] The electronic device in this application embodiment is a device with wireless communication capabilities, and may be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal device, vehicle-mounted terminal device, industrial control terminal device, UE unit, UE station, mobile station, remote station, remote terminal device, mobile device, UE terminal device, wireless communication device, UE agent, or UE device, etc. The terminal device can be fixed or mobile. It should be noted that the terminal device can support at least one wireless communication technology, such as LTE, New Radio (NR), Wideband Code Division Multiple Access (WCDMA), etc. For example, terminal devices can be mobile phones, tablets, desktop computers, laptops, all-in-one computers, in-vehicle terminals, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, electronic devices or other processing devices connected to a wireless modem, wearable devices, terminal devices in future mobile communication networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc.

[0090] Please see Figure 1 , Figure 1This is a schematic diagram of the architecture of an electronic rearview mirror system provided in an embodiment of this application. The electronic rearview mirror system 1 includes an electronic rearview mirror chip 10, an image acquisition device 20, an image display device 30, and a vehicle control system 40. The electronic rearview mirror chip 10 is connected to the image acquisition device 20, the image display device 30, and the vehicle control system 40, respectively.

[0091] The electronic rearview mirror chip includes an ISP image signal processor and an NPU neural network processing unit.

[0092] The image acquisition device 20 can be a camera, which can acquire raw image data and transmit the raw image data to the electronic rearview mirror chip 10.

[0093] The image display device 30 can be a display screen, used to display target image data. The location of the image display device 30 within the target vehicle is not limited.

[0094] The vehicle control system 40 can transmit vehicle status and / or environmental data to the electronic rearview mirror chip 10.

[0095] In one possible example, the electronic rearview mirror chip 10 can first acquire raw image data, vehicle status data, and environmental data. Then, based on these data, the chip determines whether the target vehicle is in a flicker-triggered condition. Flicker-triggered conditions include continuous streetlight conditions in tunnels, high-speed driving at night, and water surface reflection conditions in rainy nights. If the chip determines that the target vehicle is in a flicker-triggered condition, it determines a target flicker suppression strategy based on the type of flicker-triggered condition. The chip executes the target flicker suppression strategy based on the raw image data to obtain the target image data. Finally, the chip sends the target image data to the image display device. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it helps to solve the problem of image stuttering and inconsistency, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker-triggered conditions and implementing different flicker suppression strategies accordingly.

[0096] Please see Figure 2 , Figure 2This is a flowchart illustrating an electronic rearview mirror flicker suppression method provided in an embodiment of this application. The method is applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment system for the target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment system. The frame rate at which the image acquisition device acquires images, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. The method includes:

[0097] Step S201: Obtain raw image data, vehicle status data, and environmental data.

[0098] The image acquisition device has the same frame rate for acquiring images, the electronic rearview mirror chip has the same image processing frame rate, and the image display device has the same refresh rate of 50Hz, in order to match the frequency of lighting equipment (such as streetlights) on the road and improve the flickering of the image.

[0099] The image acquisition device can be triggered by the synchronization signal output by the electronic rearview mirror chip to ensure that the acquisition frame is aligned with the chip processing timing. The image acquisition device is configured with a frame rate of 50Hz and a frame interval of 20ms. The image acquisition device and the electronic rearview mirror chip use a high-speed interface to transmit image data, ensuring that each frame of data is transmitted within 20ms.

[0100] When the electronic rearview mirror chip outputs a frame, it aligns with the synchronization signal of the image display device. The image display device generates a synchronization signal every 20ms, and the electronic rearview mirror chip outputs a frame within a certain validity period of the synchronization signal of the image display device.

[0101] Among them, the electronic rearview mirror chip can acquire raw image data from the image acquisition device, as well as vehicle status data and environmental data from the vehicle control system.

[0102] Vehicle status data includes, but is not limited to, vehicle speed, acceleration, driving direction, location, and navigation information; environmental data includes, but is not limited to, lighting environment data, weather, and road condition data.

[0103] Step S202: Based on the original image data, the vehicle status data, and the environmental data, determine whether the target vehicle is in a flashing trigger condition. The flashing trigger conditions include continuous streetlight conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights.

[0104] In tunnel continuous streetlight operation, which refers to the presence of continuous LED streetlights within a tunnel, the light clusters form long, bright bands in the image. The alternating contrast between bright and dark areas enhances the visual impact of the flickering. During high-speed driving at night, the streetlights intermittently and rapidly enter and exit the electronic rearview mirror's view due to the high speed. Each entry causes a sharp increase in local brightness, making the flickering frequency appear more concentrated. In rainy nights with water reflection, the streetlights and water reflection create a double pulse, resulting in superimposed flickering.

[0105] Step S203: If it is determined that the target vehicle is in a flashing trigger condition, then a target flashing suppression strategy is determined according to the flashing trigger condition type.

[0106] Different flicker suppression strategies can be adopted for the three different operating conditions due to the differences in the causes of flicker.

[0107] Step S204: Execute the target flicker suppression strategy based on the original image data to obtain the target image data.

[0108] Specifically, the target flicker suppression strategy is implemented to optimize the original image data, resulting in optimized target image data.

[0109] This includes color restoration and display adaptation of the target image data so that the image display device can display it normally.

[0110] Step S205: Send the target image data to the image display device.

[0111] Among them, the image display device is used to display the target image data.

[0112] As can be seen, in this embodiment, the original image data, vehicle status data, and environmental data are first acquired. Then, based on the original image data, vehicle status data, and environmental data, it is determined whether the target vehicle is in a flicker-triggered condition. Flicker-triggered conditions include continuous streetlight conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights. If it is determined that the target vehicle is in a flicker-triggered condition, a target flicker suppression strategy is determined according to the flicker-triggered condition type. The target flicker suppression strategy is executed based on the original image data to obtain the target image data. Finally, the target image data is sent to the image display device. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it is beneficial to solve the phenomenon of image stuttering and discontinuity, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker-triggered conditions and executing different flicker suppression strategies accordingly.

[0113] Please see Figure 3 , Figure 3This is a flowchart illustrating a method for determining a flashing trigger condition, provided in an embodiment of this application. The method includes:

[0114] Step S301: Identify streetlights based on the original image data and environmental data.

[0115] In this process, by combining time and environmental data to determine that it is currently nighttime with low illumination, image analysis is then performed on the original image to identify streetlights. Alternatively, streetlights can be identified using a streetlight recognition model.

[0116] Step S302: If a street light is identified, the vehicle speed of the target vehicle is determined based on the vehicle status data.

[0117] Step S303: If it is determined that the vehicle speed is greater than the preset speed, then it is determined that the target vehicle is in a high-speed driving condition at night.

[0118] The preset speed can be set manually or by the system default; no specific setting is made here.

[0119] Step S304: If it is determined that the vehicle speed is less than or equal to a preset speed, determine whether the target vehicle is in a tunnel based on the tunnel recognition model, the original image data, and the environmental data.

[0120] The system can input raw image data and environmental data into a pre-trained AI tunnel recognition model and output tunnel recognition results. If the tunnel recognition results indicate that a tunnel has been identified, then the target vehicle is determined to be in a tunnel.

[0121] Step S305: If it is determined that the target vehicle is in a tunnel, then determine whether there is a region of alternating light and dark that meets the preset conditions in the original image data.

[0122] Among them, it can detect whether there are light-dark alternation areas under preset conditions in the original image data. The light-dark alternation areas under preset conditions are long strip-shaped highlight areas with equal spacing and periodic light-dark alternation.

[0123] Step S306: If it is determined that the original image data contains a region of alternating light and dark that meets preset conditions, then it is determined that the target vehicle is in the tunnel continuous street light condition.

[0124] Step S307: If it is determined that the target vehicle is not in the tunnel, then based on the original image data, the environmental data, and the rain recognition model, determine whether the target vehicle is in a rainy environment.

[0125] The system can input raw image data and environmental data into the AI ​​rain recognition model to obtain rain recognition results. If the rain recognition results indicate that it is raining, then the target vehicle is determined to be in a rainy environment.

[0126] Step S308: If it is determined that the target vehicle is in a rainy environment, then determine whether there is water accumulation on the road surface based on the original image data.

[0127] Specifically, the system can locate the road surface area in the original image data and detect whether there are irregularly shaped highlight areas with gray values ​​greater than a certain value in the road surface area. It can also analyze the irregularly shaped highlight areas in three consecutive frames. If the shape and position of the area change with the frame, it indicates that the area is a dynamic highlight. Then, the texture clarity of the road surface area is calculated. If the texture clarity is less than or equal to the preset clarity, it indicates that the road surface texture is covered by water and appears as a mirror blur. In summary, it indicates that there is water accumulation on the road surface.

[0128] Step S309: If it is determined that there is water on the current road, then the target vehicle is determined to be in a rainy night water surface reflection condition.

[0129] As can be seen, in this example, the determination of high-speed driving conditions at night, continuous street lighting conditions in tunnels, and water surface reflection conditions at rainy nights can be performed sequentially, which helps to improve the accuracy of flashing trigger condition identification.

[0130] Please see Figure 4 Regarding determining the target flicker suppression strategy based on the flicker triggering condition type, the above method may include the following steps:

[0131] Step S401: If the flickering triggering condition type is tunnel continuous street light condition, then the target flickering suppression strategy is determined as follows: determine the first frequency of flickering;

[0132] Specifically, the NPU can call the AI ​​flickering model to perform brightness analysis on 10 consecutive frames of the original image data, and extract the dominant flickering frequency, i.e., the first frequency, through Fourier transform. The ISP is used to adjust the corresponding parameters based on the output of the AI ​​flickering model.

[0133] Step S402: Determine the first exposure period to be adjusted based on the first frequency, wherein the first exposure period is an integer multiple of the first frequency.

[0134] Among these measures, the ISP adjusts the exposure cycle and limits the exposure duration, setting an upper limit for the exposure duration, for example, 20ms, to prevent dynamic motion blur in the tunnel.

[0135] The first exposure period being an integer multiple of the first frequency is to eliminate inter-frame jumps caused by only shooting bright or dark areas.

[0136] Step S403: Determine the brightness variance of multiple consecutive frame images in the original image data.

[0137] This could be the determination of the brightness variance of 5 consecutive frames of images.

[0138] Step S404: If it is determined that the brightness variance is greater than the preset variance, then the weight matrix of the first frame is determined.

[0139] The preset variance can be set manually or by system default, and no restriction is made here.

[0140] The weight matrix of the first frame can be preset, for example: the specific weights are 0.6 for the current frame, 0.2 for the previous frame, and 0.2 for the next frame.

[0141] Step S405: Perform frame fusion on the original image data according to the first frame weight matrix to obtain the first image data.

[0142] Step S406: Perform stripe detection on the first image data. If stripes are detected, determine the stripe region.

[0143] Step S407: Spatial frequency domain filtering is performed on the stripe region in the first image data to obtain third image data, and the third image data is used as the target image data.

[0144] The AI ​​flickering model performs a Fourier transform on the stripe region to locate the spatial frequency peak corresponding to the stripe. Based on the spatial frequency peak, it determines the frequency component that needs to be attenuated. The filter kernel coefficients of the 3×3 filter kernel are calibrated according to the frequency component. Sliding window filtering is performed based on the filter kernel coefficients. Specifically, the nine pixels in a single window are weighted and summed according to the filter kernel coefficients to obtain the filtered pixel brightness value.

[0145] As can be seen, in this example, adjusting exposure parameters to eliminate inter-frame jumps and balance inter-frame brightness, along with dynamic frame fusion and targeted stripe elimination, helps improve the accuracy of flicker suppression.

[0146] In one possible example, the stripe detection of the first image data includes the following steps: extracting high-frequency components of the first image data; generating a binarized image based on the high-frequency components; and determining that the first image data contains stripes if the binarized image simultaneously satisfies preset morphological features, preset frequency features, and preset direction features.

[0147] In this process, low-frequency background noise is filtered out by high-pass filtering, the edge features of the stripes are preserved, the high-frequency components are obtained, a brightness threshold is set, and the stripe area is marked as white (1) and the background is marked as black (0) according to the brightness threshold, thus generating a binarized image.

[0148] Among them, the preset morphological features are equally spaced, continuous long strip-shaped bright areas with a length greater than or equal to 50 pixels and a width range of 2-5 pixels. The preset frequency features are that the stripe spacing corresponds to a fixed spatial frequency. The preset direction features are that the stripe direction is horizontal and consistent with the tunnel extension direction.

[0149] As can be seen, in this example, enhancing the stripe features by binarizing the image and then performing stripe feature matching determination helps to improve the accuracy of stripe detection.

[0150] Please see Figure 5 Regarding determining the target flicker suppression strategy based on the flicker triggering condition type, the above method may include the following steps:

[0151] Step S501: If the flickering triggering condition type is nighttime high-speed driving condition, then the target flickering suppression strategy is determined to be: predict the street light occurrence cycle based on the original image data.

[0152] Among them, the AI ​​flickering model predicts the frequency of streetlight appearance based on raw image data.

[0153] Step S502: Determine the required adjustment of the second exposure duration based on the vehicle speed and the street light appearance cycle.

[0154] The greater the vehicle speed, the shorter the street light appearance period, and the shorter the second exposure time; conversely, the lower the vehicle speed, the longer the street light appearance period, and the longer the second exposure time.

[0155] Step S503: Determine the street light highlight area in the original image data.

[0156] Among these features, AI models can be used to identify the high-brightness areas of streetlights and distinguish them from oncoming vehicle lights and billboard lights, ensuring that only the high-brightness areas of streetlights are enhanced.

[0157] Step S504: Determine the brightness jump amplitude of the street light's high-brightness area.

[0158] Specifically, the average brightness L1 and L2 of the same streetlight highlight area are calculated in the current frame and the previous frame, respectively, and the relative brightness jump amplitude is = .

[0159] If there are multiple street light highlight areas in a frame, the maximum value of the multiple relative brightness jump amplitudes is taken as the global brightness jump amplitude of the frame.

[0160] If the standard deviation of the jump amplitude of the high beam area of ​​all street lights is calculated, and the standard deviation is greater than or equal to the preset standard deviation, it indicates that the jump degree of different street lights is quite different, and it is necessary to suppress the high beam area of ​​each street light separately.

[0161] The system can preset three jump amplitude ranges: a first jump amplitude range, a second jump amplitude range, and a third jump amplitude range. Different jump amplitude ranges correspond to different frame weight matrices. When the global brightness jump amplitude is within the first jump amplitude range, it indicates that the jump amplitude is small and no enhanced suppression is needed. When the global brightness jump amplitude is within the second jump amplitude range, it indicates that the jump amplitude is moderate, and the weights are determined to be 0.7 for the current frame and 0.3 for the previous frame. When the global brightness jump amplitude is within the third jump amplitude range, it indicates that the jump amplitude is large, and the weights are determined to be 0.6 for the current frame, 0.2 for the previous frame, and 0.2 for the next frame.

[0162] Step S505: Perform temporal filtering on the original image data according to the second frame weight matrix to obtain the fourth image data.

[0163] Step S506: Determine the enhancement adjustment parameters for the high-brightness area of ​​the street light.

[0164] Among them, the bright area of ​​the street light can be further divided into the core bright area and the transition dark area according to the gray value.

[0165] Among them, the AI ​​flickering model determines the brightness and contrast adjustment coefficients, highlight compression thresholds, and dark area enhancement thresholds for the core highlight area and the transition dark area, respectively. out1 =min(L in1 ×K1,T high ), L out1 L represents the adjusted pixel brightness in the core highlight area. in1 K1 represents the original pixel brightness of the core highlight area, and T represents the brightness / contrast adjustment coefficient. high L represents the specular compression threshold. out2 =max(L in2 ×K1,T low ), L out2 L represents the pixel brightness after adjustment in the transition dark area. in2 T represents the original pixel brightness of the transition dark area. low This indicates the highlight compression threshold.

[0166] In determining the brightness contrast adjustment coefficient, the brightness difference between the core highlight area and the transition dark area can be determined first to obtain the local contrast index. The local contrast index is calculated as follows: (average brightness of the core highlight area - average brightness of the transition dark area) / (average brightness of the core highlight area + average brightness of the transition dark area) × 100%.

[0167] The higher the local contrast ratio, the higher the flicker risk level, and the larger the corresponding brightness contrast adjustment coefficient. For example, for a high flicker risk level, K1 is 0.5-0.7; for a medium flicker risk level, K1 is 0.8-0.9; and for a low flicker risk level, K1 is 1.1-1.2.

[0168] Step S507: Adjust the parameters of the street light highlight area in the fourth image data according to the enhancement adjustment parameters to obtain the fifth image data, and use the fifth image data as the target image data.

[0169] As can be seen in this example, adjusting the exposure time based on vehicle speed and street light occurrence cycle, analyzing the brightness jump amplitude of the street light highlight area, performing motion compensation temporal filtering, and dividing the street light highlight area to analyze the flicker risk level and adjust the brightness are all beneficial to improving the accuracy of flicker suppression under high-speed driving conditions at night.

[0170] Please see Figure 6 Regarding determining the target flicker suppression strategy based on the flicker triggering condition type, the above method may include the following steps:

[0171] Step S601: If the flicker triggering condition type is a rainy night water surface reflection condition, then the target flicker suppression strategy is determined to be: dividing the original image data into regions to obtain multiple regions, including water surface reflection region, normal road surface, vehicle headlight region and street light region.

[0172] Among them, the image can be divided into regions by calling the AI ​​flickering model through the NPU, and the regions of water reflection, normal road surface, vehicle headlights and streetlights can be obtained.

[0173] Step S602: Determine the third exposure time for the water surface reflective area and the fourth exposure time for the non-water surface reflective area.

[0174] Specifically, a zoned exposure mode is used for both reflective and non-reflective areas on the water surface. The third exposure time is shorter than the fourth exposure time. Short exposures, such as 1ms, are set for reflective areas on the water surface, while relatively long exposure times, such as 2-3ms, are set for non-reflective areas on the water surface.

[0175] Step S603: Extract the non-overexposed detail data of the water surface reflection area from multiple frames of the original image data.

[0176] This allows for the extraction of non-overexposed detail data from multiple consecutive frames of images. It also enables the capture of non-overexposed detail data from reflective areas on the water surface using a short-exposure mode.

[0177] Step S604: Perform pixel repair on the water surface reflection area of ​​multiple frames in the original image data based on the non-overexposed detail data to obtain the sixth image data.

[0178] The pixel restoration method involves fusing pixels from multiple consecutive frames of images to restore road surface texture. Specifically, the reflective area of ​​the water surface is further divided into sub-regions to obtain overexposed reflective areas and non-overexposed reflective areas. For overexposed reflective areas, a very large weight can be set for non-overexposed detail data and a very small weight can be set for pixel fusion, for example, 0.9-1.0 and 0-0.1 respectively. For non-overexposed reflective areas, a relatively large weight can be set for non-overexposed detail data and a relatively small weight can be set for original pixel data, for example, 0.55-0.65 and 0.35-0.45 respectively.

[0179] Step S605: Determine the color shift calibration parameters of the water surface reflection area in the sixth image data, and perform color shift calibration on the water surface reflection area in the sixth image data according to the color shift calibration parameters.

[0180] In particular, rainy night reflections cause the reflective areas on the water surface to appear whitish or yellowish. Specifically, the color cast type of the water surface reflective area can be determined based on the standard color difference: whitish or yellowish. For a whitish cast, a first color cast calibration parameter can be determined based on the degree of color cast; for a yellowish cast, a second color cast calibration parameter can be determined based on the degree of color cast. Then, the color cast of the water surface reflective area in the sixth image data can be calibrated using the first color cast calibration parameter / the second color cast calibration parameter.

[0181] The transition area of ​​the water surface reflective area refers to the area connecting the water surface reflective area and the non-water surface reflective area. It can be a transition zone with a total width of 4 pixels, extending 2 pixels inward and 2 pixels outward from the water surface reflective area.

[0182] The process involves calculating the average brightness difference between the reflective and non-reflective sides within the transition region, determining the interpolation intensity parameter K based on this average brightness difference (the larger the average brightness difference, the larger the interpolation intensity parameter), and calculating the position weight of each pixel in the transition region. The position weight w = number of pixels from the edge / total width of the transition region. The formula for calculating the interpolated pixel brightness is as follows:

[0183] L C (x, y) = K × [w × L1(x, y) + (1 w)×L2(x,y)]+(1 K)×L3(x,y).

[0184] Among them, L C(x, y) represents the interpolated pixel brightness, L1(x, y) represents the average brightness of the pixels in the reflective area of ​​the transition region, L2(x, y) represents the average brightness of the pixels in the non-reflective area of ​​the transition region, and L3(x, y) represents the original brightness value of the pixel.

[0185] Step S606: Interpolate the transition region according to the boundary pixel interpolation to obtain the seventh image data, and use the seventh image data as the target image data.

[0186] As can be seen, in this example, pixel repair and color shift calibration of the water surface reflective area, and interpolation calibration of the transition area, are beneficial to improving the accuracy of flicker suppression under the condition of water surface reflection in rainy night.

[0187] In one possible example, after acquiring the original image data, vehicle status data, and environmental data, the method further includes: dividing the original image data into multiple pixel regions according to a preset rule; determining the average pixel brightness of each pixel region; determining the brightness difference between the brightness of each pixel in the pixel region and the average pixel brightness; and performing the following operation on each pixel to correct bad pixels: if it is determined that the pixel brightness difference corresponding to the currently processed pixel is greater than a preset brightness threshold, then the currently processed pixel is determined to be a bad pixel; and replacing the brightness of the currently processed pixel with the average pixel brightness.

[0188] The preset rules can be set manually or by system default, and are not limited here. For example, the original image can be divided into multiple rectangular pixel regions of the same size.

[0189] The preset brightness threshold can be set manually or by system default, and no restriction is made here.

[0190] As can be seen, in this example, detecting and replacing bad pixels can improve the accuracy of subsequent flicker suppression.

[0191] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment control system for the target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment control system. The frame rate at which the image acquisition device acquires images, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. Figure 7As shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to be executed by the processor according to the following instructions:

[0192] Acquire raw image data, vehicle status data, and environmental data;

[0193] Based on the original image data, the vehicle status data, and the environmental data, it is determined whether the target vehicle is in a flashing trigger condition. The flashing trigger conditions include continuous street light conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights.

[0194] If it is determined that the target vehicle is in a flashing-triggered condition, then a target flashing suppression strategy is determined based on the type of flashing-triggered condition.

[0195] The target flicker suppression strategy is executed based on the original image data to obtain the target image data;

[0196] The target image data is sent to the image display device.

[0197] As can be seen, in this embodiment, the electronic device first acquires raw image data, vehicle status data, and environmental data. Then, based on the raw image data, vehicle status data, and environmental data, it determines whether the target vehicle is in a flicker-triggered condition. Flicker-triggered conditions include continuous streetlight conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights. If the target vehicle is determined to be in a flicker-triggered condition, a target flicker suppression strategy is determined based on the flicker-triggered condition type. The target flicker suppression strategy is then executed based on the raw image data to obtain the target image data. Finally, the target image data is sent to the image display device. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it is beneficial to solve the phenomenon of image stuttering and discontinuity, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker-triggered conditions and executing different flicker suppression strategies accordingly.

[0198] In one possible example, regarding the determination of whether the target vehicle is in a flashing-triggered condition, the above procedure includes instructions for performing the following steps:

[0199] Identify streetlights based on the original image data and environmental data;

[0200] If a street light is identified, the vehicle speed of the target vehicle is determined based on the vehicle status data.

[0201] If it is determined that the vehicle speed is greater than the preset speed, then it is determined that the target vehicle is in a high-speed driving condition at night.

[0202] If it is determined that the vehicle speed is less than or equal to a preset speed, the system determines whether the target vehicle is in a tunnel based on the tunnel recognition model, the original image data, and the environmental data.

[0203] If it is determined that the target vehicle is in a tunnel, then it is determined whether the original image data contains a region of alternating light and dark that meets a preset condition;

[0204] If it is determined that the original image data contains a region of alternating brightness and darkness that meets preset conditions, then it is determined that the target vehicle is in the tunnel continuous street light condition.

[0205] If it is determined that the target vehicle is not in the tunnel, then based on the original image data, the environmental data, and the rain recognition model, it is determined whether the target vehicle is in a rainy environment;

[0206] If it is determined that the target vehicle is in a rainy environment, then the presence of road surface water is determined based on the original image data.

[0207] If it is determined that there is standing water on the current road, then the target vehicle is in a rainy night with water surface reflection conditions.

[0208] In one possible example, regarding the determination of the target flicker suppression strategy based on the flicker triggering condition type, the above procedure includes instructions for performing the following steps:

[0209] If the flicker triggering condition type is the tunnel continuous street light condition, then the target flicker suppression strategy is determined as follows:

[0210] Determine the first frequency of the flashing;

[0211] The first exposure period to be adjusted is determined based on the first frequency, wherein the first exposure period is an integer multiple of the first frequency;

[0212] Determine the luminance variance of multiple consecutive frames in the original image data;

[0213] If it is determined that the brightness variance is greater than the preset variance, then the weight matrix of the first frame is determined;

[0214] The original image data is fused according to the first frame weight matrix to obtain the first image data;

[0215] Stripe detection is performed on the first image data. If stripes are detected, the stripe region is determined.

[0216] Spatial frequency domain filtering is performed on the stripe region in the first image data to obtain the third image data, and the third image data is used as the target image data.

[0217] In one possible example, regarding the stripe detection of the first image data, the above procedure includes instructions for performing the following steps:

[0218] Extract the high-frequency components of the first image data;

[0219] A binarized image is generated based on the high-frequency components;

[0220] If it is determined that the binarized image simultaneously satisfies the preset morphological features, preset frequency features, and preset directional features, then it is determined that the first image data contains stripes.

[0221] In one possible example, regarding the determination of the target flicker suppression strategy based on the flicker triggering condition type, the above procedure further includes instructions for performing the following steps:

[0222] If the flickering triggering condition is nighttime high-speed driving, then the target flicker suppression strategy is determined as follows:

[0223] Based on the original image data, predict the street light occurrence cycle;

[0224] The required adjustment of the second exposure duration is determined based on the vehicle speed and the street light occurrence cycle;

[0225] Identify the streetlight highlight areas in the original image data;

[0226] Determine the brightness jump range of the street light's highlight area;

[0227] Based on the brightness jump amplitude, determine the second frame weight matrix of the motion compensation temporal filtering;

[0228] The original image data is subjected to temporal filtering based on the weight matrix of the second frame to obtain the fourth image data;

[0229] Determine the enhancement adjustment parameters for the high-beam area of ​​the streetlight;

[0230] The parameters of the street light highlight area in the fourth image data are adjusted according to the enhancement adjustment parameters to obtain the fifth image data, and the fifth image data is used as the target image data.

[0231] In one possible example, regarding the determination of the target flicker suppression strategy based on the flicker triggering condition type, the above procedure further includes instructions for performing the following steps:

[0232] If the flicker triggering condition is a rainy night water surface reflection condition, then the target flicker suppression strategy is determined as follows:

[0233] The original image data is divided into regions to obtain multiple regions, including water surface reflection region, normal road surface, vehicle headlight region and street light region.

[0234] Determine the third exposure time for the water surface reflective area and the fourth exposure time for the non-water surface reflective area;

[0235] Extract non-overexposed detail data of the water surface reflection area from multiple frames of the original image data;

[0236] The water surface reflection area in multiple frames of the original image data is repaired using the non-overexposed detail data to obtain the sixth image data;

[0237] Determine the color shift calibration parameters for the water surface reflection area in the sixth image data, and perform color shift calibration on the water surface reflection area in the sixth image data according to the color shift calibration parameters;

[0238] Determine the transition region and corresponding boundary pixel interpolation of the water surface reflection area in the sixth image data;

[0239] The transition region is interpolated based on the boundary pixel interpolation to obtain the seventh image data, and the seventh image data is used as the target image data.

[0240] In one possible example, after acquiring the raw image data, vehicle status data, and environmental data, the above procedure further includes instructions for performing the following steps:

[0241] The original image data is divided into multiple pixel regions according to preset rules;

[0242] Determine the average pixel brightness for each pixel region;

[0243] Determine the brightness difference between the brightness of each pixel in the pixel region and the average brightness of the pixels;

[0244] Perform the following operation on each pixel to correct bad pixels:

[0245] If it is determined that the pixel brightness difference corresponding to the currently processed pixel is greater than a preset brightness threshold, then the currently processed pixel is determined to be a bad pixel.

[0246] Replace the brightness of the currently processed pixel with the average brightness of the pixel.

[0247] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0248] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0249] When dividing each function into modules according to its corresponding function. Figure 8 A functional block diagram of an electronic rearview mirror flicker suppression device is provided, applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment system for the target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment system. The frame rate of image acquisition by the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. Figure 8 As shown, the electronic rearview mirror flicker suppression device includes an acquisition unit 801, a judgment unit 802, a determination unit 803, an execution unit 804, and a transmission unit 805; wherein,

[0250] The acquisition unit 801 is used to acquire raw image data, vehicle status data, and environmental data.

[0251] The judgment unit 802 is used to determine whether the target vehicle is in a flashing trigger condition based on the original image data, the vehicle status data and the environmental data. The flashing trigger condition includes continuous street light condition in tunnel, high-speed driving condition at night and water surface reflection condition at rainy night.

[0252] The determining unit 803 is used to determine a target flicker suppression strategy based on the flicker triggering condition type if it is determined that the target vehicle is in a flicker triggering condition.

[0253] The execution unit 804 is used to execute the target flicker suppression strategy based on the original image data to obtain the target image data;

[0254] The transmission unit 805 is used to send the target image data to the image display device.

[0255] As can be seen from the embodiments of this application, the electronic rearview mirror flicker suppression device can first acquire original image data, vehicle status data, and environmental data. Then, based on the original image data, vehicle status data, and environmental data, it determines whether the target vehicle is in a flicker-triggered condition. Flicker-triggered conditions include continuous streetlight conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights. If it is determined that the target vehicle is in a flicker-triggered condition, a target flicker suppression strategy is determined according to the flicker-triggered condition type. The target flicker suppression strategy is executed based on the original image data to obtain the target image data. Finally, the target image data is sent to the image display device. By setting the frame rate of the image acquisition device, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device to the same 50Hz, it is beneficial to solve the phenomenon of image stuttering and discontinuity, suppress power frequency flicker, and improve the accuracy of flicker suppression by identifying different flicker-triggered conditions and executing different flicker suppression strategies accordingly.

[0256] In one possible example, regarding determining whether the target vehicle is in a flashing-triggered condition, the determining unit 803 is specifically used for:

[0257] Identify streetlights based on the original image data and environmental data;

[0258] If a street light is identified, the vehicle speed of the target vehicle is determined based on the vehicle status data.

[0259] If it is determined that the vehicle speed is greater than the preset speed, then it is determined that the target vehicle is in a high-speed driving condition at night.

[0260] If it is determined that the vehicle speed is less than or equal to a preset speed, the system determines whether the target vehicle is in a tunnel based on the tunnel recognition model, the original image data, and the environmental data.

[0261] If it is determined that the target vehicle is in a tunnel, then it is determined whether the original image data contains a region of alternating light and dark that meets a preset condition;

[0262] If it is determined that the original image data contains a region of alternating brightness and darkness that meets preset conditions, then it is determined that the target vehicle is in the tunnel continuous street light condition.

[0263] If it is determined that the target vehicle is not in the tunnel, then based on the original image data, the environmental data, and the rain recognition model, it is determined whether the target vehicle is in a rainy environment;

[0264] If it is determined that the target vehicle is in a rainy environment, then the presence of road surface water is determined based on the original image data.

[0265] If it is determined that there is standing water on the current road, then the target vehicle is in a rainy night with water surface reflection conditions.

[0266] In one possible example, regarding the determination of the target flicker suppression strategy based on the flicker triggering condition type, the determining unit 803 is specifically used for:

[0267] If the flicker triggering condition type is the tunnel continuous street light condition, then the target flicker suppression strategy is determined as follows:

[0268] Determine the first frequency of the flashing;

[0269] The first exposure period to be adjusted is determined based on the first frequency, wherein the first exposure period is an integer multiple of the first frequency;

[0270] Determine the luminance variance of multiple consecutive frames in the original image data;

[0271] If it is determined that the brightness variance is greater than the preset variance, then the weight matrix of the first frame is determined;

[0272] The original image data is fused according to the first frame weight matrix to obtain the first image data;

[0273] Stripe detection is performed on the first image data. If stripes are detected, the stripe region is determined.

[0274] Spatial frequency domain filtering is performed on the stripe region in the first image data to obtain the third image data, and the third image data is used as the target image data.

[0275] In one possible example, regarding the stripe detection of the first image data, the determining unit 803 is specifically used for:

[0276] Extract the high-frequency components of the first image data;

[0277] A binarized image is generated based on the high-frequency components;

[0278] If it is determined that the binarized image simultaneously satisfies the preset morphological features, preset frequency features, and preset directional features, then it is determined that the first image data contains stripes.

[0279] In one possible example, in determining the target flicker suppression strategy based on the flicker triggering condition type, the determining unit 803 is further specifically used for:

[0280] If the flickering triggering condition is nighttime high-speed driving, then the target flicker suppression strategy is determined as follows:

[0281] Based on the original image data, predict the street light occurrence cycle;

[0282] The required adjustment of the second exposure duration is determined based on the vehicle speed and the street light occurrence cycle;

[0283] Identify the streetlight highlight areas in the original image data;

[0284] Determine the brightness jump range of the street light's highlight area;

[0285] Based on the brightness jump amplitude, determine the second frame weight matrix of the motion compensation temporal filtering;

[0286] The original image data is subjected to temporal filtering based on the weight matrix of the second frame to obtain the fourth image data;

[0287] Determine the enhancement adjustment parameters for the high-beam area of ​​the streetlight;

[0288] The parameters of the street light highlight area in the fourth image data are adjusted according to the enhancement adjustment parameters to obtain the fifth image data, and the fifth image data is used as the target image data.

[0289] In one possible example, regarding the determination of the target flicker suppression strategy based on the flicker triggering condition type, the determining unit 803 is further specifically configured to:

[0290] If the flicker triggering condition is a rainy night water surface reflection condition, then the target flicker suppression strategy is determined as follows:

[0291] The original image data is divided into regions to obtain multiple regions, including water surface reflection region, normal road surface, vehicle headlight region and street light region.

[0292] Determine the third exposure time for the water surface reflective area and the fourth exposure time for the non-water surface reflective area;

[0293] Extract non-overexposed detail data of the water surface reflection area from multiple frames of the original image data;

[0294] The water surface reflection area in multiple frames of the original image data is repaired using the non-overexposed detail data to obtain the sixth image data;

[0295] Determine the color shift calibration parameters for the water surface reflection area in the sixth image data, and perform color shift calibration on the water surface reflection area in the sixth image data according to the color shift calibration parameters;

[0296] Determine the transition region and corresponding boundary pixel interpolation of the water surface reflection area in the sixth image data;

[0297] The transition region is interpolated based on the boundary pixel interpolation to obtain the seventh image data, and the seventh image data is used as the target image data.

[0298] In one possible example, after acquiring the raw image data, vehicle status data, and environmental data, the determining unit 803 is further specifically used for:

[0299] The original image data is divided into multiple pixel regions according to preset rules;

[0300] Determine the average pixel brightness for each pixel region;

[0301] Determine the brightness difference between the brightness of each pixel in the pixel region and the average brightness of the pixels;

[0302] Perform the following operation on each pixel to correct bad pixels:

[0303] If it is determined that the pixel brightness difference corresponding to the currently processed pixel is greater than a preset brightness threshold, then the currently processed pixel is determined to be a bad pixel.

[0304] Replace the brightness of the currently processed pixel with the average brightness of the pixel.

[0305] Optional, please refer to Figure 9 , Figure 9 A pin diagram of an electronic rearview mirror chip provided in an embodiment of this application is shown below. Figure 9As shown, the electronic rearview mirror chip (LQ560) includes at least 12 pins, which are as follows: 1. LVDS_TX0_CH0_D0N, function is to receive serial video data from the image acquisition device (low level); 2. LVDS_TX0_CH0_D0P, function is to receive serial video data from the image acquisition device (high level); 3. LVDS_TX0_CLK_N, function is the clock synchronization signal for data transmission (low level); 4. LVDS_TX0_CLK_P, function is the clock synchronization signal for data transmission (high level); 5. A_CAN1_TX, function is to send control commands to the vehicle's infotainment system and receive status data; 6. A_CAN1_... RX, function is to receive driving status data (vehicle speed, relative position, etc.) from the vehicle's infotainment system; 7, GND, circuit reference zero potential point; 8, VCC, function is to provide the chip with operating voltage (usually 3.3V); 9, LVDS_TX1_CH0_D0P, function is to send the processed image data to the display screen (high level); 10, LVDS_TX1_CH0_D0N, function is to send the processed image data to the display screen (low level); 11, AM_SENSOR0_VS, function is to synchronize with the image acquisition device and provide a vertical synchronization signal; 12, AM_SENSOR0_CLK, function is to synchronize with the image acquisition device and ensure image acquisition timing.

[0306] Understandable. Figure 9 This only lists part of the structure of the electronic rearview mirror chip. It should also include other pins used to implement the method of this application, as well as general pins of the chip, such as the reset pin (RESET), the reference voltage pin (VREF), etc.

[0307] For example, please refer to Figure 10 , Figure 10 This application provides a schematic diagram of the architecture of an electronic rearview mirror chip control board, as shown in the embodiments below. Figure 10 As shown, it includes a camera deserializer connected to the LQ560 (electronic rearview mirror chip), a CAN Transceiver (CAN transceiver), a video serializer / video bridge IC, LPDDR4 (memory), eMMC (storage), and a PMIC (power management IC).

[0308] The system includes a camera deserializer that converts serial camera data into parallel data; it connects to the electronic rearview mirror chip via an LVDS input interface. A CAN Transceiver enables CAN bus communication, exchanging data with the vehicle's infotainment system; it connects to the electronic rearview mirror chip via a CAN bus interface. A video serializer / video bridge IC converts the chip's LVDS output into a display-compatible signal; it connects to the electronic rearview mirror chip via its LVDS output. LPDDR4 memory is used for high-speed caching of image data, supporting frame-to-frame comparison; it connects to the electronic rearview mirror chip via a memory interface. An eMMC (emulated metal memory module) stores system software, configuration files, and processing results; it connects to the electronic rearview mirror chip via an eMMC interface. A PMIC manages the power supply for the entire system, ensuring stable voltage for all components; it connects to the electronic rearview mirror chip via VCC and GND interfaces.

[0309] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0310] The electronic device provided in this embodiment is used to execute the above-described electronic rearview mirror flicker suppression method, and thus can achieve the same effect as the above-described implementation method.

[0311] When using integrated units, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device; for example, it can support the electronic device in executing the steps performed by the aforementioned functional units. The storage module can support the electronic device in executing stored program code and data. The communication module can support communication between the electronic device and other devices.

[0312] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0313] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0314] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer includes a control platform.

[0315] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0316] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0317] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0318] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0319] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0320] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0321] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0322] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for suppressing flicker in an electronic rearview mirror, characterized in that, An electronic rearview mirror chip is used in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment system for a target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment system. The frame rate at which the image acquisition device acquires images, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. The method includes: Acquire raw image data, vehicle status data, and environmental data; Based on the original image data, the vehicle status data, and the environmental data, it is determined whether the target vehicle is in a flashing trigger condition. The flashing trigger conditions include continuous street light conditions in tunnels, high-speed driving conditions at night, and water surface reflection conditions in rainy nights. If it is determined that the target vehicle is in a flashing-triggered condition, then a target flashing suppression strategy is determined based on the type of flashing-triggered condition. The target flicker suppression strategy is executed based on the original image data to obtain the target image data; The target image data is sent to the image display device; The step of determining the target flicker suppression strategy based on the flicker triggering condition type includes: If the flicker triggering condition is a rainy night water surface reflection condition, then the target flicker suppression strategy is determined as follows: The original image data is divided into regions to obtain multiple regions, including water surface reflection region, normal road surface, vehicle headlight region and street light region. Determine the third exposure time for the water surface reflective area and the fourth exposure time for the non-water surface reflective area; Extract non-overexposed detail data of the water surface reflection area from multiple frames of the original image data; The water surface reflection area in multiple frames of the original image data is repaired using the non-overexposed detail data to obtain the sixth image data; The color shift calibration parameters of the water surface reflection area in the sixth image data are determined according to the color shift type of the water surface reflection area, and the color shift calibration of the water surface reflection area in the sixth image data is performed according to the color shift calibration parameters. Determine the transition region and corresponding boundary pixel interpolation of the water surface reflection area in the sixth image data; The transition region is interpolated based on the boundary pixel interpolation to obtain the seventh image data, and the seventh image data is used as the target image data.

2. The method according to claim 1, characterized in that, The determination of whether the target vehicle is in a flashing trigger condition includes: Identify streetlights based on the original image data and environmental data; If a street light is identified, the vehicle speed of the target vehicle is determined based on the vehicle status data. If it is determined that the vehicle speed is greater than the preset speed, then it is determined that the target vehicle is in a high-speed driving condition at night. If it is determined that the vehicle speed is less than or equal to a preset speed, the system determines whether the target vehicle is in a tunnel based on the tunnel recognition model, the original image data, and the environmental data. If it is determined that the target vehicle is in a tunnel, then it is determined whether the original image data contains a region of alternating light and dark that meets a preset condition; If it is determined that the original image data contains a region of alternating brightness and darkness that meets preset conditions, then it is determined that the target vehicle is in the tunnel continuous street light condition. If it is determined that the target vehicle is not in the tunnel, then based on the original image data, the environmental data, and the rain recognition model, it is determined whether the target vehicle is in a rainy environment; If it is determined that the target vehicle is in a rainy environment, then the presence of road surface water is determined based on the original image data. If it is determined that there is water on the current road, then the target vehicle is in a rainy night with water surface reflection conditions.

3. The method according to claim 2, characterized in that, The step of determining the target flicker suppression strategy based on the flicker triggering condition type includes: If the flicker triggering condition type is the tunnel continuous street light condition, then the target flicker suppression strategy is determined as follows: Determine the first frequency of the flashing; The first exposure period to be adjusted is determined based on the first frequency, wherein the first exposure period is an integer multiple of the first frequency; Determine the luminance variance of multiple consecutive frames in the original image data; If it is determined that the brightness variance is greater than the preset variance, then the weight matrix of the first frame is determined; The original image data is fused according to the first frame weight matrix to obtain the first image data; Stripe detection is performed on the first image data. If stripes are detected, the stripe region is determined. Spatial frequency domain filtering is performed on the stripe region in the first image data to obtain the third image data, and the third image data is used as the target image data.

4. The method according to claim 3, characterized in that, The stripe detection of the first image data includes: Extract the high-frequency components of the first image data; A binarized image is generated based on the high-frequency components; If it is determined that the binarized image simultaneously satisfies the preset morphological features, preset frequency features, and preset directional features, then it is determined that the first image data contains stripes.

5. The method according to claim 2, characterized in that, The step of determining the target flicker suppression strategy based on the flicker triggering condition type includes: If the flickering triggering condition is nighttime high-speed driving, then the target flicker suppression strategy is determined as follows: Based on the original image data, predict the street light occurrence cycle; The required adjustment of the second exposure duration is determined based on the vehicle speed and the street light occurrence cycle; Identify the streetlight highlight areas in the original image data; Determine the brightness jump range of the street light's highlight area; Based on the brightness jump amplitude, determine the second frame weight matrix of the motion compensation temporal filtering; The original image data is subjected to temporal filtering based on the weight matrix of the second frame to obtain the fourth image data; Determine the enhancement adjustment parameters for the high-beam area of ​​the streetlight; The parameters of the street light highlight area in the fourth image data are adjusted according to the enhancement adjustment parameters to obtain the fifth image data, and the fifth image data is used as the target image data.

6. The method according to claim 1, characterized in that, After acquiring the raw image data, vehicle status data, and environmental data, the method further includes: The original image data is divided into multiple pixel regions according to preset rules; Determine the average pixel brightness for each pixel region; Determine the brightness difference between the brightness of each pixel in the pixel region and the average brightness of the pixels; Perform the following operation on each pixel to correct bad pixels: If it is determined that the pixel brightness difference corresponding to the currently processed pixel is greater than a preset brightness threshold, then the currently processed pixel is determined to be a bad pixel. Replace the brightness of the currently processed pixel with the average brightness of the pixel.

7. An electronic rearview mirror flicker suppression device, characterized in that, An electronic rearview mirror chip is used in an electronic rearview mirror system. The electronic rearview mirror system includes the electronic rearview mirror chip, an image acquisition device, an image display device, and a vehicle infotainment system for the target vehicle. The electronic rearview mirror chip is connected to the image acquisition device, the image display device, and the vehicle infotainment system. The frame rate at which the image acquisition device acquires images, the image processing frame rate of the electronic rearview mirror chip, and the refresh rate of the image display device are the same. The electronic rearview mirror flicker suppression device includes an acquisition unit, a judgment unit, a determination unit, an execution unit, and a transmission unit. The acquisition unit is used to acquire raw image data, vehicle status data, and environmental data; The judgment unit is used to determine whether the target vehicle is in a flashing trigger condition based on the original image data, the vehicle status data and the environmental data. The flashing trigger condition includes continuous street light condition in tunnel, high-speed driving condition at night and water surface reflection condition at rainy night. The determining unit is configured to, if it is determined that the target vehicle is in a flashing trigger condition, determine a target flashing suppression strategy based on the flashing trigger condition type; the determination of the target flashing suppression strategy based on the flashing trigger condition type includes: if the flashing trigger condition type is a rainy night water surface reflection condition, then the target flashing suppression strategy is determined as follows: dividing the original image data into regions to obtain multiple regions, the multiple regions including a water surface reflection region, a normal road surface, a vehicle headlight region, and a street light region; determining the third exposure time for the water surface reflection region, and determining the fourth exposure time for the non-water surface reflection region; extracting the image from multiple frames in the original image data. Non-overexposed detail data of the water surface reflection area; pixel repair of the water surface reflection area in multiple frames of the original image data based on the non-overexposed detail data to obtain the sixth image data; color cast calibration parameters of the water surface reflection area in the sixth image data are determined according to the color cast type of the water surface reflection area, and color cast calibration is performed on the water surface reflection area in the sixth image data according to the color cast calibration parameters; the transition area and corresponding boundary pixel interpolation of the water surface reflection area in the sixth image data are determined; the transition area is interpolated according to the boundary pixel interpolation to obtain the seventh image data, and the seventh image data is used as the target image data; The execution unit is used to execute the target flicker suppression strategy based on the original image data to obtain the target image data; The transmission unit is used to send the target image data to the image display device.

8. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store one or more programs and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to perform the method as described in any one of claims 1-6.

Images