Image processing method and device, electronic equipment and storage medium

Abstract

The application provides an image processing method and device, electronic equipment and storage medium. The method comprises: acquiring first image data from an image acquisition device; in response to determining that the image acquisition device is blocked by liquid according to the first image data, acquiring first state data, the first state data comprising driving state data of a first vehicle adjacent to the vehicle; determining whether the liquid blockage belongs to a first situation according to the first state data, the first situation being that the first vehicle runs over accumulated water to cause the liquid blockage; and in response to the liquid blockage not belonging to the first situation, performing rain and fog removal processing on the first image data to obtain second image data. The influence of a rain and fog environment on a rear view image can be overcome.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] A Camera Monitor System (CMS), or electronic rearview mirror system, replaces traditional optical rearview mirrors (interior and exterior mirrors) with a combination of cameras and displays, and is an important component of automotive vision assistance systems. However, in actual driving, rainy or foggy conditions can cause water droplets or fog to form on the camera lens surface, blurring the image, reducing contrast, and causing loss of detail. This severely affects the driver's judgment of the rear environment and can even lead to safety accidents. Overcoming the impact of rainy and foggy conditions on rearview images is a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0003] This application provides an image processing method, apparatus, electronic device, and storage medium. By identifying whether the liquid obstruction of the camera is caused by adjacent vehicles running over water, and after ruling out such cases, corresponding de-raining and de-fogging processing is performed. This not only overcomes the influence of rain and fog on the rear view image, but also avoids performing invalid image enhancement in non-rain and fog environments, thereby reducing system power consumption and latency.

[0004] In a first aspect, this application provides an image processing method applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system also includes an image acquisition device for acquiring rear-view images of a vehicle. The method includes:

[0005] Acquire first image data from the image acquisition device;

[0006] In response to determining from the first image data that there is liquid obstruction in the image acquisition device, first state data is acquired, the first state data including the driving state data of the first vehicle adjacent to this vehicle;

[0007] Based on the first state data, determine whether the liquid obstruction belongs to the first situation. The first situation is that the liquid obstruction is caused by the first vehicle running over the water.

[0008] In response to the fact that liquid occlusion does not fall under the first case, the first image data is subjected to de-raining and defogging processing to obtain the second image data.

[0009] As can be seen, in this application, after detecting liquid obstruction, the electronic rearview mirror chip actively calls the first vehicle driving status data provided by the vehicle system to determine whether the obstruction is caused by adjacent vehicles running over water (i.e., the first scenario). If this scenario is ruled out, de-fogging processing is triggered; otherwise, it is skipped. This mechanism distinguishes the causes of liquid obstruction, avoiding redundant image enhancement calculations in non-rainy / foggy environments (vehicle splashing water), thereby significantly reducing the computational load and power consumption of the image processing module. Simultaneously, by activating the de-fogging algorithm only in real rainy / foggy environments, processing efficiency and system response speed are improved, ensuring that the output image still has high clarity and contrast under rainy / foggy conditions, effectively overcoming the impact of rain and fog on the rearview image and ensuring driving safety.

[0010] In a feasible example, the first state data includes vehicle speed and a first relative position, where the first relative position is the position of the first vehicle relative to itself. Determining whether liquid obstruction falls under the first scenario based on the first state data includes:

[0011] The lateral distance threshold is determined based on vehicle speed; the higher the vehicle speed, the larger the lateral distance threshold.

[0012] The lateral distance is determined based on the first relative position, and the lateral distance is the lateral interval between this vehicle and the first vehicle.

[0013] In response to a lateral distance threshold not being less than the lateral distance, determine whether the first vehicle has driven over accumulated water;

[0014] In response to the fact that the first vehicle ran over the accumulated water, it was determined that the liquid obstruction was the first scenario.

[0015] In this application, by adaptively setting a lateral distance threshold based on vehicle speed, the rationality of the threshold setting is improved. By comparing the actual lateral distance between the vehicle and the first vehicle with the threshold, it is initially determined whether the vehicle is within the splashing water influence range. Furthermore, by determining whether the first vehicle has run over the water accumulation, it is further confirmed whether the liquid obstruction is directly caused by running over the water accumulation, which can improve the accuracy of the first situation judgment.

[0016] In a feasible example, determining whether the first vehicle drove over the accumulated water includes:

[0017] In response to the identification of road surface water within a first time period before the first moment, the second relative position of the road surface water at the first moment is determined, the second relative position being the position of the road surface water relative to the vehicle, and the first moment being the moment when the image acquisition device acquires the first image data.

[0018] In response to the coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle has driven over the accumulated water.

[0019] Since there is no coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle did not run over the accumulated water.

[0020] In this application, by introducing correlation analysis between the road surface water and the second relative position and the first relative position of the first vehicle at the first moment, the accuracy of judging the vehicle's behavior of running over the water is improved. Furthermore, based on the dual judgment of time and space, the accuracy of subsequently determining that the liquid obstruction belongs to the first situation is improved.

[0021] In a feasible example, the method also includes:

[0022] The first image data is converted to a different color space to obtain the third image data;

[0023] Clustering is performed on the third image data based on hue, saturation, and brightness to obtain multiple regions in the third image data;

[0024] The fourth image data acquired by the image acquisition device in the frame before the acquisition of the first image data is obtained, and the fourth image data is converted in color space to obtain the fifth image data.

[0025] The brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether there is liquid obstruction in the image acquisition device.

[0026] In this application, by extracting independent features in the color space and dividing them into regions, and combining the brightness comparison analysis of the previous and next frames, the presence of liquid occlusion can be identified. This color space-based clustering process can not only effectively divide the regions that may have liquid occlusion, but also enhance the robustness to complex background interference. Furthermore, by comparing the brightness of the different regions after division to identify whether liquid occlusion exists, the accuracy of liquid occlusion detection is improved.

[0027] In a feasible example, the brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether the image acquisition device is obstructed by liquid, including:

[0028] In response to the fact that the brightness difference between the first region and the second region in multiple regions of the third image data is greater than a first preset threshold, the morphological feature parameters of the first region are obtained. The morphological feature parameters include roundness. The first region includes at least one region. The ratio between the area of ​​the first region and the area of ​​multiple regions is greater than a second preset threshold. The second region is the corresponding region of the first region in the fifth image data.

[0029] If the circularity of the first region is greater than a third preset threshold, it is determined that there is liquid obstruction in the image acquisition device.

[0030] In this application, high-precision identification of liquid occlusion is achieved by introducing dual criteria of brightness difference and morphological features. Simultaneously, the area proportion condition eliminates the influence of minor noise or localized reflections, ensuring that only salient areas are evaluated. This multi-dimensional, layered screening mechanism improves the accuracy of identifying whether an image acquisition device is obstructed by liquid.

[0031] In a feasible example, the method also includes:

[0032] In response to the liquid occlusion being the first case, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data.

[0033] In this application, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data. This allows for effective recovery of visual information without the need to initiate complex rain and fog removal algorithms. It leverages the characteristic that the background environment remains largely unchanged in the time domain, replacing computationally expensive rain and fog removal processing with image reconstruction technology, thus significantly reducing system latency and resource consumption while ensuring image clarity.

[0034] In a feasible example, the first image data is de-rained and de-fog processed to obtain the second image data, including:

[0035] The first image data is downsampled to obtain the seventh image data;

[0036] The brightness difference is calculated based on the first image data and the fourth image data to obtain the first image features. The fourth image data is the image data acquired by the image acquisition device in the frame before the acquisition of the first image data.

[0037] The seventh image data and the first image features are input into the first model for processing to obtain the first parameter set, which is used to characterize the state of rain and fog.

[0038] The first image data and the first parameter set are input into the second model for processing to obtain the second parameter set, which includes the image parameters of the first image data after dehazing.

[0039] The first image data is dehazed based on the second parameter set to obtain the second image data.

[0040] In this application, the computational complexity of subsequent model processing is reduced by downsampling the first image data. Brightness difference calculation is performed based on the first and fourth image data to capture dynamically changing rain and fog textures and motion artifacts, thereby enhancing the perception of rain and fog events. The seventh image data and the features of the first image are input into the first model for processing to obtain the first parameter set, thereby achieving global modeling of the rain and fog state, extracting corresponding rain and fog characterization parameters, and subsequently generating targeted image restoration parameters. Finally, rain and fog interference is effectively removed while maintaining the integrity of the image structure, improving visual clarity and achieving efficient, accurate, and robust rain and fog removal processing technology.

[0041] Secondly, this application provides an image processing device applied to an electronic rearview mirror chip in an electronic rearview mirror system. The electronic rearview mirror system also includes an image acquisition device for acquiring rear-view images of a vehicle. The device includes:

[0042] The communication unit is used to acquire first image data from the image acquisition device;

[0043] In response to the processing unit determining, based on the first image data, that the image acquisition device is obstructed by liquid, the communication unit is further configured to acquire first state data, which includes the driving state data of a first vehicle adjacent to this vehicle.

[0044] The processing unit is also used to determine whether the liquid obstruction belongs to the first situation based on the first state data. The first situation is that the liquid obstruction is caused by the first vehicle running over the water.

[0045] The processing unit is also configured to, in response to the fact that liquid occlusion does not fall under the first case, perform de-raining and defogging processing on the first image data to obtain the second image data.

[0046] Thirdly, this application provides an electronic device including a processor, a memory, and a communication interface. The processor, memory, and communication interface are interconnected and perform communication with each other. The memory stores executable program code, the communication interface is used for wireless communication, and the processor is used to retrieve the executable program code stored in the memory and execute some or all of the steps described in any of the methods in the first aspect.

[0047] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements some or all of the steps described in the first aspect of this application.

[0048] Fifthly, this application provides a computer program product, including a computer program that, when processed and executed, implements some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the structure of an electronic rearview mirror system provided in an embodiment of this application;

[0051] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0052] Figure 3 A schematic flowchart illustrating another image processing method provided in an embodiment of this application;

[0053] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application;

[0054] Figure 5 A schematic flowchart illustrating another image processing method provided in an embodiment of this application;

[0055] Figure 6 A functional unit block diagram of an image processing apparatus provided in an embodiment of this application;

[0056] Figure 7 A functional unit block diagram of another image processing apparatus provided in the embodiments of this application;

[0057] Figure 8 A structural block diagram of an electronic device provided in an embodiment of this application;

[0058] Figure 9 This is a schematic diagram of the structure of an electronic rearview mirror chip provided in an embodiment of this application;

[0059] Figure 10 A schematic diagram of the appearance of another electronic rearview mirror chip provided in an embodiment of this application;

[0060] Figure 11 This is a schematic diagram of the structure of an electronic rearview mirror chip control board provided in an embodiment of this application. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0062] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps is not limited to the steps listed, but may optionally include steps not listed, or may optionally include other steps inherent to these processes, methods, products, or apparatuses.

[0063] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0064] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of an electronic rearview mirror system provided in an embodiment of this application, as shown below. Figure 1 As shown, the electronic rearview mirror system 100 includes an image acquisition device 101, an electronic rearview mirror chip 102, and a display 103.

[0065] The image acquisition device 101 can be a camera device installed on the exterior of the vehicle for real-time acquisition of images of the rear environment, providing raw visual information of the vehicle's rear view area. Furthermore, the image acquisition device 101 can include one or more of the following, depending on its installation location: a left exterior rearview mirror camera, a right exterior rearview mirror camera, etc.

[0066] The electronic rearview mirror chip 102 can be a dedicated integrated circuit that integrates image processing, logic control, and communication functions to perform image data processing tasks in the electronic rearview mirror system. In this embodiment, the electronic rearview mirror chip 102 serves as the execution carrier of the image processing method, coordinating the data interaction between the image acquisition device 101 and the vehicle's infotainment system 104, and completing core calculations such as liquid occlusion judgment and defogging.

[0067] In this application, the electronic rearview mirror chip 102 acquires first image data acquired by the image acquisition device 101; in response to determining that there is liquid obstruction in the image acquisition device based on the first image data, it acquires first status data from the vehicle system 104, the first status data including the driving status data of the first vehicle adjacent to the vehicle; based on the first status data, it determines whether the liquid obstruction belongs to a first situation, the first situation being that the liquid obstruction is caused by the first vehicle running over the water; in response to the liquid obstruction not belonging to the first situation, it performs de-raining and defogging processing on the first image data to obtain second image data.

[0068] While liquid obstruction of the image acquisition device 101 can also occur when adjacent vehicles drive over puddles, this obstruction is temporary compared to rain or fog, thus requiring less complex defogging processing. This application identifies whether the liquid obstruction on the image acquisition device 101 is caused by adjacent vehicles driving over puddles, and performs the corresponding defogging process only after ruling out this possibility. This not only overcomes the impact of rain and fog on the rear view image but also avoids unnecessary defogging processing when adjacent vehicles cause liquid obstruction, reducing the burden on image processing.

[0069] Based on this, the embodiments of this application provide an image processing method, and the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0070] Example 1: The main flow of the image processing method will be described below.

[0071] Please see Figure 2 , Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. The method is applied to the aforementioned electronic rearview mirror chip. Figure 2 As shown, the method includes the following steps.

[0072] Step S201: Acquire first image data from the image acquisition device.

[0073] The first image data can be the raw image information acquired by the image acquisition device at the current moment and transmitted to the electronic rearview mirror chip. This image information serves as input for liquid obstruction detection and subsequent processing, reflecting the current state of the camera's field of view. In this embodiment, the image acquisition device can continuously transmit image data to the electronic rearview mirror chip via a video interface or serial communication protocol.

[0074] Step S202: In response to determining from the first image data that the image acquisition device is obstructed by liquid, first state data is acquired.

[0075] The first state data includes the driving state data of the first vehicle adjacent to this vehicle. Liquid occlusion can be a phenomenon where water droplets, fog, or other liquids adhere to the surface of the camera lens, causing a decrease in image clarity. In this embodiment, liquid occlusion can be determined by image analysis algorithms (such as contrast evaluation and edge blur detection) on the first image data.

[0076] The first state data can be a dataset provided by the vehicle's infotainment system containing dynamic information about adjacent vehicles, including but not limited to position, speed, acceleration, and direction of travel. This data is used to infer the likelihood that adjacent vehicles are driving over puddles and splashing liquid onto the vehicle's camera. The vehicle can be the main body of a vehicle equipped with an electronic rearview mirror system. The first vehicle can be another vehicle adjacent to the vehicle that may cause liquid obstruction; this can be used as the object for determining the behavior of driving over puddles.

[0077] The vehicle infotainment system (VIS) serves as the core computing unit for the in-vehicle infotainment and control system. It possesses multi-sensor data fusion and vehicle status monitoring capabilities, providing driving status data for adjacent vehicles and supporting the electronic rearview mirror chip in determining the cause of obstruction. In this embodiment, the VIS collects data from various vehicle subsystems (such as Advanced Driving Assistance Systems (ADAS), radar, and cameras) via a Controller Area Network (CAN) bus or Ethernet, integrates and processes the data, and then outputs it.

[0078] Step S203: Determine whether liquid obstruction belongs to the first situation based on the first state data.

[0079] The first scenario refers to a situation where adjacent vehicles drive over water on the road surface, causing liquid to splash onto the lens of the vehicle's camera. This scenario serves as a criterion for excluding rain and fog processing, preventing invalid image enhancement in non-rain and foggy environments. Determining whether liquid obstruction falls under the first scenario based on the first state data can be achieved by analyzing the driving status of the first vehicle in the first state data and combining it with preset rules or machine learning models to determine whether there is any behavior of driving over water. This allows for accurate differentiation between real rain / fog and splashing water scenarios, avoiding misjudgments that lead to unnecessary image processing.

[0080] In step S204, in response to the fact that liquid occlusion does not belong to the first case, the first image data is subjected to de-raining and defogging processing to obtain the second image data.

[0081] The second image data can be de-fogging processed image data, which has higher clarity and contrast, used to drive the monitor display and improve the driver's visual perception of the rear environment. Responding to the fact that liquid occlusion is not part of the first scenario, the second image data is obtained by de-fogging the first image data. This can be achieved by using a physics-based de-fogging algorithm, such as the dark channel prior method or guided filtering; or by using deep learning networks (such as Convolutional Neural Networks (CNNs) or Transformer structures) for end-to-end image enhancement. This allows for high-computational image enhancement to be performed only when necessary, reducing system power consumption and latency while ensuring image quality in rainy and foggy environments.

[0082] Optionally, after obtaining the second image data, it is necessary to transmit the second image data to the display for the driver. In the case of liquid obstruction, which is the first scenario, the first image data can be directly transmitted to the display for the driver.

[0083] In this embodiment, after detecting liquid obstruction, the electronic rearview mirror chip actively calls the vehicle's driving status data to determine whether the obstruction is caused by adjacent vehicles running over water (i.e., the first scenario). If this scenario is ruled out, defogging processing is triggered; otherwise, it is skipped. This mechanism distinguishes the causes of liquid obstruction, avoiding redundant image enhancement calculations in non-rainy / foggy environments (vehicle splashing water), thus significantly reducing the computational load and power consumption of the image processing module. Simultaneously, by activating the defogging algorithm only in real rainy / foggy environments, processing efficiency and system response speed are improved, ensuring that the output image maintains high clarity and contrast even under rainy / foggy conditions, effectively overcoming the impact of rain and fog on the rearview image and ensuring driving safety.

[0084] Example 2: The image processing method will be described in detail below based on the details of determining whether liquid occlusion belongs to the first case.

[0085] Please see Figure 3 , Figure 3 This is a flowchart illustrating another image processing method provided in an embodiment of this application. This method is applied to the aforementioned electronic rearview mirror chip, such as... Figure 3 As shown, the method includes the following steps.

[0086] Step S301: Acquire first image data from the image acquisition device.

[0087] Step S302: In response to determining from the first image data that the image acquisition device is obstructed by liquid, first state data is acquired.

[0088] The first state data includes the driving state data of the first vehicle adjacent to this vehicle. The first state data includes vehicle speed and first relative position, where the first relative position is the position of the first vehicle relative to this vehicle.

[0089] Step S303: Determine the lateral distance threshold based on the vehicle speed.

[0090] The higher the vehicle speed, the larger the lateral distance threshold. Vehicle speed can be the current speed of adjacent vehicles, serving as the basis for setting the lateral distance threshold and reflecting the impact of vehicle dynamics on the splash range. The lateral distance threshold can be a critical distance value dynamically adjusted based on vehicle speed, used to determine whether adjacent vehicles are likely to cause splashing. In this embodiment, vehicle speed can be mapped to the corresponding lateral distance threshold using a preset function, such as a piecewise linear function or a lookup table.

[0091] Step S304: Determine the lateral distance based on the first relative position.

[0092] The lateral distance refers to the lateral separation between this vehicle and the first vehicle. The first relative position can be a set of spatial coordinates of adjacent vehicles relative to this vehicle, usually expressed as lateral and longitudinal offsets, which can be used to calculate the lateral distance between this vehicle and the first vehicle to determine whether it is within the splashing water's influence range.

[0093] Lateral distance can be the actual distance between this vehicle and the first vehicle in the lateral direction. It can be compared with a lateral distance threshold to determine whether the vehicle is within the water splashing area. For example, the lateral distance can be obtained by directly extracting the lateral component after obtaining the relative position through an onboard positioning system (such as Global Positioning System (GPS) or millimeter-wave radar), or by estimating the relative position and calculating the lateral distance through Simultaneous Localization and Mapping (SLAM) or multi-sensor fusion algorithms, thereby quantifying the spatial relationship between the vehicle and the adjacent vehicles.

[0094] Step S305: In response to the lateral distance threshold being not less than the lateral distance, determine whether the first vehicle has run over the accumulated water.

[0095] The act of driving over puddles can be defined as the action of an adjacent vehicle passing through a puddled area, causing water to splash onto the lens of the vehicle's image acquisition device. This can be used as a core feature of the first scenario to distinguish between brief splashes and continuous rain or fog. In this embodiment, the act of driving over puddles can be determined by considering vehicle speed and lateral distance in conjunction with the context.

[0096] Optionally, determining whether the first vehicle has driven over accumulated water includes: in response to identifying water on the road surface during a first time period before the first moment, determining the second relative position of the water on the road surface at the first moment, wherein the second relative position is the position of the water on the road surface relative to the vehicle, and the first moment is the moment when the image acquisition device acquires the first image data; in response to the existence of coordinate overlap between the second relative position and the first relative position, determining that the first vehicle has driven over accumulated water; in response to the absence of coordinate overlap between the second relative position and the first relative position, determining that the first vehicle has not driven over accumulated water.

[0097] Road surface water can be any area of ​​water accumulation on the road surface caused by rainfall or poor drainage. For example, road surface water can be detected by identifying water reflections, color features, or depth information in images captured by other cameras on the vehicle. When the first vehicle drives over the water, the presence of the water may be difficult to detect because the vehicle body is covered by the water. Therefore, it is necessary to identify the road surface water within a first time period before the first moment, that is, before the first vehicle drives over the water. After identifying the road surface water, its second relative position at the first moment (i.e., the moment when the image acquisition device is obscured by liquid) should be determined.

[0098] The second relative position can be a set of spatial coordinates of the water on the road surface relative to the vehicle. It can be determined based on the relative position of the vehicle at the second moment when the water on the road surface is detected (for example, after identifying the water accumulation area through image processing algorithms (such as edge detection and reflectivity analysis), the relative position is calculated by combining the camera calibration parameters), and the movement of the vehicle between the second moment and the first moment.

[0099] Since both the first relative position and the second relative position are sets of spatial coordinates, when it is determined that there are overlapping coordinates between the two, it can be determined that the first vehicle has driven over the water. When it is determined that there are no overlapping coordinates between the two, it can be determined that the first vehicle has not driven over the water.

[0100] Alternatively, the determination can be made based on the distance between the center positions of the first and second relative positions; if the distance between the center positions of the first and second relative positions is less than or equal to a preset distance, it is determined that the first vehicle has driven over accumulated water; if the distance between the center positions of the first and second relative positions is greater than the preset distance, it is determined that the first vehicle has not driven over accumulated water. This distance can be the distance between two coordinates.

[0101] In this embodiment, by introducing correlation analysis between the road surface water and the second relative position and the first relative position of the first vehicle at the first moment, the accuracy of judging the vehicle's behavior of running over the water is improved. Furthermore, based on the dual judgment of time and space, the accuracy of subsequently determining that the liquid obstruction belongs to the first situation is improved.

[0102] For example, please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of a vehicle, as shown in the embodiment of the present application. Figure 4 The diagram includes the vehicle 410 and the first vehicle 420, as well as the lateral distance 401 between the vehicle 410 and the first vehicle 420, and the water accumulation on the road surface 402. It can be seen that there is a coordinate overlap between the second relative position of the water accumulation on the road surface 402 and the first relative position of the first vehicle 420. At this time, it can be considered that the first vehicle 420 is driving over the water accumulation.

[0103] Step S306: In response to the first vehicle's act of running over the accumulated water, it is determined that the liquid obstruction belongs to the first situation.

[0104] The first scenario involves the first vehicle running over the accumulated water, causing liquid blockage.

[0105] In this embodiment, by adaptively setting a lateral distance threshold based on vehicle speed, the rationality of the threshold setting is improved. By comparing the actual lateral distance between the vehicle and the first vehicle with the threshold, it is initially determined whether the vehicle is within the splashing water influence range. Furthermore, by determining whether the first vehicle has run over the water, it is further confirmed whether the liquid obstruction is directly caused by running over the water, which can improve the accuracy of the first scenario judgment.

[0106] In step S307, in response to the fact that liquid occlusion does not belong to the first case, the first image data is subjected to de-raining and defogging processing to obtain the second image data.

[0107] Example 3: The image processing method will be described in detail below based on the details of determining whether there is liquid obstruction in the image acquisition device.

[0108] Please see Figure 5 , Figure 5 This is a flowchart illustrating another image processing method provided in an embodiment of this application. This method is applied to the aforementioned electronic rearview mirror chip, such as... Figure 5 As shown, the method includes the following steps.

[0109] Step S501: Acquire first image data from the image acquisition device.

[0110] Step S502: Convert the color space of the first image data to obtain the third image data.

[0111] The first image data can be mathematically transformed at each pixel using a color space conversion algorithm to generate the third image data. This conversion process can employ the standard HSV (Hue Saturation Value) conversion formula for each pixel, or it can be implemented by calling a built-in color space conversion function in an image processing library. This allows for the extraction of independent features such as hue, saturation, and brightness, providing structured input for subsequent clustering processing. For example, the third image data can include, but is not limited to, one or more of HSV image data, YUV image data, etc.

[0112] Hue can be an attribute describing the type of color, representing the visual perception corresponding to different wavelengths in the spectrum. It can be used to distinguish different color regions in an image and assist in semantic segmentation in clustering processing. Saturation can be an attribute describing the purity or vividness of a color, reflecting the proportion of gray components in the color. It can be used to identify color intensity variations in an image and enhance sensitivity to water droplets or fog areas. Brightness can be a numerical value describing the brightness of pixels in an image, reflecting light intensity. It can serve as a key indicator for judging liquid occlusion; water droplets or fog typically exhibit localized high brightness or low contrast. In this embodiment, saturation, hue, and brightness are used together for image clustering to improve the accuracy of region segmentation.

[0113] Step S503: Cluster the third image data based on hue, saturation and brightness to obtain multiple regions in the third image data.

[0114] This process involves clustering the third image data based on hue, saturation, and brightness to obtain multiple regions. This can be achieved by grouping the color features of each pixel in the image using a clustering algorithm to form semantic regions. Furthermore, this operation can be implemented using the K-means algorithm for clustering based on color feature vectors, or by using a mean-shift algorithm for density estimation and region segmentation in the image space. This allows for the regional segmentation of image content, improving the ability to identify local occlusion phenomena.

[0115] Step S504: Obtain the fourth image data acquired by the image acquisition device in the frame before acquiring the first image data, and perform color space conversion on the fourth image data to obtain the fifth image data.

[0116] The fourth image data can be image data acquired by the image acquisition device in the frame preceding the acquisition of the first image data. It is used to compare the brightness of the current frame with the first image data to detect any changes in occlusion. In this embodiment, the fourth image data is obtained by reading the previous frame image data from the frame buffer of the image acquisition device. The fifth image data can be implemented by reusing the color space conversion parameters and algorithms of the first image data, or by synchronously processing the current frame and the previous frame in the image processing pipeline. This ensures that the preceding and following frames are compared in the same color space, eliminating the influence of color mapping differences.

[0117] Step S505: Compare the brightness of multiple regions in the third image data with the corresponding regions in the fifth image data to determine whether there is liquid obstruction in the image acquisition device.

[0118] Specifically, the brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether liquid occlusion exists in the image acquisition device. This can be achieved by calculating the brightness difference between the current frame and the previous frame for each region, and determining occlusion if the difference exceeds a threshold. Alternatively, this operation can be performed by calculating the average brightness difference for each region and setting a dynamic threshold for judgment.

[0119] In this embodiment, by extracting independent features in the color space and dividing the region, and combining the brightness comparison analysis of the previous and next frames, the presence of liquid occlusion can be identified. This color space-based clustering process can not only effectively divide the region where liquid occlusion may exist, but also enhance the robustness to complex background interference. Furthermore, by comparing the brightness of the different regions after division to identify whether liquid occlusion exists, the accuracy of liquid occlusion detection is improved.

[0120] Optionally, the brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether the image acquisition device is obstructed by liquid. This includes: in response to the fact that the brightness difference between the first region and the second region in the multiple regions of the third image data is greater than a first preset threshold, obtaining the morphological feature parameters of the first region, including roundness, the first region including at least one region, the ratio between the area of ​​the first region and the area of ​​the multiple regions being greater than a second preset threshold, and the second region being the corresponding region of the first region in the fifth image data; in response to the fact that the roundness of the first region is greater than a third preset threshold, determining that the image acquisition device is obstructed by liquid.

[0121] The first region can be at least one candidate object identified as having significant brightness changes among multiple regions. That is, the first region may include only a single region from multiple regions, or it may include two or more regions from multiple regions. When the first region includes only a single region, its area is the area of ​​that single region; when the first region includes two or more regions, its area is the sum of the areas of those two or more regions. The second region can be the corresponding region of the first region in the fifth image data, or it can be an image region at the same location in the previous frame.

[0122] The first preset threshold can be a numerical threshold used to determine whether the brightness difference has reached the standard of significant change, and can be dynamically adjusted according to the actual scenario to improve robustness.

[0123] Morphological feature parameters can be a set of quantitative indicators describing the geometric shape attributes of an image region, which can be used to distinguish liquid occlusion from other interference factors (such as reflection and noise). Furthermore, morphological feature parameters can include, but are not limited to, one or more of roundness, compactness, aspect ratio, and convexity, with key parameters selected based on the morphological characteristics of the target occlusion.

[0124] Circularity is a dimensionless parameter that measures how close an image region is to a circle. The closer the value is to 1, the more round it is. It can be used to identify typical circular spots formed by water droplets and eliminate non-circular interference sources.

[0125] The second preset threshold can be a numerical threshold used to determine whether the ratio between the area of ​​the liquid-blocked region and the area of ​​all regions reaches a significant threshold. It can be used to filter out areas that are too small, ensuring that only physically significant blockages are analyzed.

[0126] The third preset threshold can be a numerical threshold used to determine whether the roundness meets the typical morphological standard of a water droplet. It can be used as a morphological basis for the final liquid occlusion determination, improving recognition accuracy. It is understood that the relevant thresholds in this application can include empirical thresholds, machine learning training thresholds, multi-scene adaptive thresholds, etc. When the roundness of the first region exceeds the third preset threshold, a judgment result confirming liquid occlusion in the image acquisition device is output.

[0127] In this embodiment, high-precision identification of liquid occlusion is achieved by introducing dual criteria of brightness difference and morphological features. Simultaneously, the area proportion condition eliminates the influence of minor noise or localized reflections, ensuring that only significant areas are evaluated. This multi-dimensional, layered screening mechanism improves the accuracy of identifying whether an image acquisition device is obstructed by liquid.

[0128] Step S506: In response to determining from the first image data that the image acquisition device is obstructed by liquid, first state data is acquired.

[0129] The first state data includes the driving state data of the first vehicle adjacent to this vehicle.

[0130] Step S507: Determine whether liquid obstruction belongs to the first situation based on the first state data.

[0131] The first scenario involves the first vehicle running over the accumulated water, causing liquid blockage.

[0132] In step S508, in response to the fact that liquid occlusion does not belong to the first case, the first image data is subjected to de-raining and defogging processing to obtain the second image data.

[0133] Optionally, the first image data is subjected to dehazing to obtain the second image data, including: downsampling the first image data to obtain the seventh image data; calculating the brightness difference between the first image data and the fourth image data to obtain the first image feature, wherein the fourth image data is the image data acquired by the image acquisition device in the frame before acquiring the first image data; inputting the seventh image data and the first image feature into a first model for processing to obtain the first parameter set, wherein the first parameter set is used to characterize the state of rain and fog; inputting the first image data and the first parameter set into a second model for processing to obtain the second parameter set, wherein the second parameter set includes the image parameters for dehazing the first image data; and performing dehazing on the first image data according to the second parameter set to obtain the second image data.

[0134] The seventh image data can be low-resolution image data obtained by downsampling the first image data. This can reduce the computational load of subsequent model processing while retaining sufficient information for rain and fog state modeling. Downsampling the first image data to obtain the seventh image data can be achieved by using image scaling algorithms (such as bilinear interpolation or nearest neighbor interpolation) to reduce the spatial resolution of the first image data. This reduces the amount of data processed by subsequent models, lowers computational complexity, and improves processing efficiency.

[0135] The first image feature can be a dynamic feature vector extracted by comparing the brightness difference between the first image data and the fourth image data. It can be used to reflect the changing trend of rain, fog, or motion artifacts in the time dimension, thereby enhancing the ability to recognize rain and fog. In this embodiment, the first image feature can be generated by performing brightness difference and normalization on each pixel, and then extracting statistical features such as mean and variance after generating a difference map. Alternatively, it can be used to perform gradient analysis on the difference map using an edge detection operator (such as Sobel) to extract motion texture features. This can capture brightness changes caused by rain, fog, or motion, thereby enhancing the ability to perceive dynamic occlusion.

[0136] The first model can be a deep learning model used to extract global parameters of rain and fog states from low-resolution images and dynamic features. It can generate a first parameter set representing characteristics such as rain and fog intensity and distribution patterns, providing a decision-making basis for subsequent defogging processing. In this embodiment, the first model receives the seventh image data and the first image features as input and outputs the first parameter set for use by the second model. Furthermore, the first model can be one or more of the following, depending on the network structure: convolutional neural network model, Transformer model, U-Net structure model, etc.

[0137] The first parameter set can include rain / fog type, scattering coefficient, rain line mixing coefficient, mean global transmittance, and background complexity. Rain / fog type includes advection fog, radiation fog, light rain, and heavy rain; the larger the scattering coefficient value, the denser the fog; the larger the rain line mixing coefficient, the denser the rain; the smaller the mean global transmittance, the denser the fog; the greater the background complexity, the more complex the background, for example, dense streetlights indicate a high background complexity.

[0138] The seventh image data and the first image features are input into the first model for processing to obtain the first parameter set. Alternatively, the low-resolution image and dynamic features can be stitched or fused together and then input into the first model, and the parameter set can be output through forward propagation.

[0139] The second model can be a deep learning model that combines the original image with rain and fog state parameters to generate specific dehazing operation parameters. It can be used to generate a second parameter set containing information such as contrast enhancement, color correction, and detail restoration to guide the dehazing process.

[0140] Specifically, the second parameter set may include contrast adjustment coefficients, color restoration parameters, detail enhancement factors, etc., which are directly applied to the first image data for dehazing.

[0141] In this embodiment, the second image data can be one or more of the following, depending on the processing algorithm: dehazing image based on physical model, enhanced image based on deep learning, and repaired image based on frequency domain filtering.

[0142] The first image data is dehazed according to the second parameter set to obtain the second image data. This can be achieved by the image signal processing (ISP) unit in the electronic rearview mirror chip adjusting the first image data pixel by pixel based on the contrast adjustment coefficient, color restoration parameters, detail enhancement factors, etc. contained in the second parameter set.

[0143] In this embodiment, the computational complexity of subsequent model processing is reduced by downsampling the first image data. Brightness difference calculation is performed based on the first and fourth image data to capture dynamically changing rain and fog textures and motion artifacts, enhancing the perception of rain and fog events. The seventh image data and the features of the first image are input into the first model for processing to obtain the first parameter set, realizing global modeling of the rain and fog state, extracting corresponding rain and fog characterization parameters, and then generating targeted image restoration parameters. Finally, rain and fog interference is effectively removed while maintaining the integrity of the image structure, improving visual clarity. This achieves efficient, accurate, and robust rain and fog removal processing technology.

[0144] In step S509, in response to the liquid occlusion belonging to the first case, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data.

[0145] Since the fifth image data is obtained by color space conversion based on the fourth image data, and the third image data is also obtained by color space conversion based on the first image data, the second region corresponding to the fourth image data can be determined based on the relative position of the second region corresponding to the fifth image data, and the first region corresponding to the first image data can be determined based on the relative position of the first region corresponding to the third image data.

[0146] Image reconstruction is a technique for restoring content from damaged areas in the current frame based on historical frames or neighborhood information. It can be used to generate clear images in scenarios with temporary occlusion, avoiding the need for complex de-raining and de-fogging algorithms. Furthermore, image reconstruction can include, but is not limited to, one or more of the following: interpolation-based reconstruction, texture synthesis-based reconstruction, and deep learning-based inpainting.

[0147] In response to the liquid occlusion being classified as the first scenario, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data. Alternatively, when liquid occlusion is determined to be the first scenario, the content of the second region in the fourth image data can be extracted and mapped to fill the first region in the first image data to generate the sixth image data. Furthermore, this operation can be achieved by using a block matching algorithm to directly copy the pixels of the second region to the first region, combined with edge smoothing processing, or by using a deep learning-based image inpainting model to generate high-fidelity content of the first region based on the second region. This achieves lightweight image restoration in transient occlusion scenarios, avoids unnecessary de-fogging processing, reduces computational load, and maintains visual continuity.

[0148] In this embodiment, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data. This allows for effective recovery of visual information without the need to initiate complex de-fogging algorithms. It leverages the characteristic that the background environment remains largely unchanged in the time domain, replacing computationally expensive de-fogging processing with image reconstruction technology, thus significantly reducing system latency and resource consumption while ensuring image clarity.

[0149] For embodiments consistent with those shown above, please refer to... Figure 6 , Figure 6 This is a functional unit block diagram of an image processing device provided in an embodiment of this application. The image processing device is the aforementioned electronic rearview mirror chip or a part thereof, such as... Figure 6 As shown, the image processing apparatus 60 includes:

[0150] Communication unit 601 is used to acquire first image data from image acquisition device;

[0151] In response to the processing unit 602 determining, based on the first image data, that the image acquisition device is obstructed by liquid, the communication unit 601 is further configured to acquire first status data, which includes the driving status data of a first vehicle adjacent to the vehicle.

[0152] The processing unit 602 is also used to determine whether the liquid obstruction belongs to the first situation based on the first state data. The first situation is that the liquid obstruction is caused by the first vehicle running over the water.

[0153] The processing unit 602 is further configured to perform de-raining processing on the first image data in response to the fact that liquid occlusion does not belong to the first case, to obtain the second image data.

[0154] In one feasible embodiment, the first state data includes vehicle speed and a first relative position, where the first relative position is the position of the first vehicle relative to itself. In determining whether liquid obstruction falls under the first condition based on the first state data, the processing unit 602 is specifically configured to:

[0155] The lateral distance threshold is determined based on vehicle speed; the higher the vehicle speed, the larger the lateral distance threshold.

[0156] The lateral distance is determined based on the first relative position, and the lateral distance is the lateral interval between this vehicle and the first vehicle.

[0157] In response to a lateral distance threshold not being less than the lateral distance, determine whether the first vehicle has driven over accumulated water;

[0158] In response to the fact that the first vehicle ran over the accumulated water, it was determined that the liquid obstruction was the first scenario.

[0159] In one feasible embodiment, in determining whether the first vehicle has driven over accumulated water, the processing unit 602 is specifically configured to:

[0160] In response to the identification of road surface water within a first time period before the first moment, the second relative position of the road surface water at the first moment is determined, the second relative position being the position of the road surface water relative to the vehicle, and the first moment being the moment when the image acquisition device acquires the first image data.

[0161] In response to the coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle has driven over the accumulated water.

[0162] Since there is no coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle did not run over the accumulated water.

[0163] In one feasible embodiment, the processing unit 602 is further configured to:

[0164] The first image data is converted to a different color space to obtain the third image data;

[0165] Clustering is performed on the third image data based on hue, saturation, and brightness to obtain multiple regions in the third image data;

[0166] The communication unit 601 is also used to acquire the fourth image data acquired by the image acquisition device in the frame before the acquisition of the first image data;

[0167] The processing unit 602 is also used to perform hue space conversion on the fourth image data to obtain the fifth image data;

[0168] The brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether there is liquid obstruction in the image acquisition device.

[0169] In a feasible embodiment, in determining whether the image acquisition device is obstructed by liquid by comparing the brightness of multiple regions in the third image data with the corresponding regions in the fifth image data, the processing unit 602 is further configured to:

[0170] In response to the fact that the brightness difference between the first region and the second region in multiple regions of the third image data is greater than a first preset threshold, the morphological feature parameters of the first region are obtained. The morphological feature parameters include roundness. The first region includes at least one region. The ratio between the area of ​​the first region and the area of ​​multiple regions is greater than a second preset threshold. The second region is the corresponding region of the first region in the fifth image data.

[0171] If the circularity of the first region is greater than a third preset threshold, it is determined that there is liquid obstruction in the image acquisition device.

[0172] In one feasible embodiment, the processing unit 602 is further configured to: in response to the liquid occlusion belonging to the first situation, perform image reconstruction on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data.

[0173] In one feasible embodiment, in order to perform rain and fog removal processing on the first image data to obtain the second image data, the processing unit 602 is further configured to:

[0174] The first image data is downsampled to obtain the seventh image data;

[0175] The brightness difference is calculated based on the first image data and the fourth image data to obtain the first image features. The fourth image data is the image data acquired by the image acquisition device in the frame before the acquisition of the first image data.

[0176] The seventh image data and the first image features are input into the first model for processing to obtain the first parameter set, which is used to characterize the state of rain and fog.

[0177] The first image data and the first parameter set are input into the second model for processing to obtain the second parameter set, which includes the image parameters of the first image data after dehazing.

[0178] The first image data is dehazed based on the second parameter set to obtain the second image data.

[0179] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0180] When using integrated units, such as Figure 7 As shown, Figure 7 This is a block diagram of the functional units of another image processing apparatus provided in an embodiment of this application. Figure 7 In this document, the image processing apparatus 60 includes a processing module 712 and a communication module 711. The processing module 712 controls and manages the operation of the image processing apparatus 60, such as the steps of the processing unit 602, and / or performs other processes according to the techniques described herein. The communication module 711 supports interaction between the image processing apparatus 60 and other devices, such as the steps of the communication unit 601. Figure 7 As shown, the image processing apparatus 60 may further include a storage module 713, which is used to store the program code and data of the image processing apparatus 60.

[0181] The processing module 712 can be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The communication module 711 can be a transceiver, RF circuitry, or a communication interface, etc. The storage module 713 can be a memory.

[0182] All relevant content in each scenario involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here. The above image processing device 60 can all perform the above... Figure 2 , Figure 3 as well as Figure 5 The image processing method shown.

[0183] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0184] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Figure 8As shown, the electronic device 800 may include one or more of the following components: processor 801, memory 802 and communication interface 803. The processor 801, memory 802 and communication interface 803 are interconnected and perform communication between them. The memory 802 may store one or more computer programs, which may be configured to implement the methods described in the above embodiments when executed by one or more processors 801.

[0185] Processor 801 may include one or more processing cores. Processor 801 connects to various parts within the electronic device 800 using various interfaces and lines, and performs various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets, or instruction sets stored in memory 802, and by calling data stored in memory 802. Optionally, processor 801 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 801 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. It is understood that the aforementioned modem may also not be integrated into processor 801, but may be implemented separately through a communication chip.

[0186] The memory 802 may include random access memory (RAM) or read-only memory (ROM). The memory 802 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 802 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the electronic device 800 during use.

[0187] It is understood that the electronic device 800 may include more or fewer structural elements than those shown in the above block diagram, such as a power module, physical buttons, WiFi (Wireless Fidelity) module, speaker, Bluetooth module, sensor, etc., without limitation.

[0188] The aforementioned electronic device 800 may be an electronic rearview mirror chip or a part of an electronic rearview mirror chip.

[0189] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements some or all of the steps of any of the image processing methods described in the above method embodiments.

[0190] This application also provides a computer program product, including a computer program that, when executed by a processor, implements some or all of the steps of any of the image processing methods described in the above method embodiments. This computer program product can be a software installation package.

[0191] This application provides an electronic rearview mirror chip, which is coupled to a memory and used to read and execute program instructions in the memory so that the device containing the electronic rearview mirror chip can implement any of the above-described methods.

[0192] Optional, please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic rearview mirror chip provided in an embodiment of this application, as shown below. Figure 9 As shown, the electronic rearview mirror chip (LQ560) includes at least 12 pins, and the correspondence of these 12 pins is represented by Table 1, as shown below:

[0193] Table 1

[0194]

[0195] Understandable. Figure 9 This only lists part of the structure of the electronic rearview mirror chip. It should also include other pins used to implement the method of this application, as well as general pins of the chip, such as the reset pin (RESET), the reference voltage pin (VREF), etc.

[0196] For example, please refer to Figure 10 , Figure 10 A schematic diagram of the appearance of another electronic rearview mirror chip provided in the embodiments of this application is shown below. Figure 10 As shown, Figure 10 This is a schematic diagram of the electronic rearview mirror chip (LQ560), including its name (LQ560) and corresponding serial number (200M1120324050 101500263502010933-CN).

[0197] In addition, the electronic rearview mirror chip may also include a corresponding control board, which will be discussed below. Figure 11 Explain it:

[0198] For example, please refer to Figure 11 , Figure 11 This is a schematic diagram of the structure of an electronic rearview mirror chip control board provided in an embodiment of this application, as shown below. Figure 11 As shown, it includes a camera deserializer connected to the LQ560 (electronic rearview mirror chip), a CAN Transceiver (CAN transceiver), a video serializer / video bridge IC, LPDDR4 (memory), eMMC (storage), and a PMIC (power management IC).

[0199] The system includes a camera deserializer that converts serial camera data into parallel data; it connects to the electronic rearview mirror chip via an LVDS input interface. A CAN Transceiver enables CAN bus communication, exchanging data with the vehicle's infotainment system; it connects to the electronic rearview mirror chip via a CAN bus interface. A video serializer / video bridge IC converts the chip's LVDS output into a display-compatible signal; it connects to the electronic rearview mirror chip via its LVDS output. LPDDR4 memory is used for high-speed caching of image data, supporting frame-to-frame comparison; it connects to the electronic rearview mirror chip via a memory interface. An eMMC (emulated metal memory module) stores system software, configuration files, and processing results; it connects to the electronic rearview mirror chip via an eMMC interface. A PMIC manages the power supply for the entire system, ensuring stable voltage for all components; it connects to the electronic rearview mirror chip via VCC and GND interfaces.

[0200] It should be noted that, for the sake of simplicity, each of the aforementioned image processing method embodiments is described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0201] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce a good effect.

[0202] Those skilled in the art will understand that all or part of the steps in the various method embodiments of any of the above image processing methods can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0203] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of an image processing method, apparatus, electronic device, and storage medium of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas of this application. At the same time, for those skilled in the art, based on the ideas of an image processing method, apparatus, electronic device, and storage medium of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

[0204] This application is described with reference to flowchart illustrations and / or block diagrams of methods, hardware products, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0205] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0206] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0207] It is understood that any product that is controlled or configured to perform the processing method described in the flowchart of an embodiment of an image processing method of this application, such as the terminal and computer program product of the above flowchart, falls within the scope of the related products described in this application.

[0208] Obviously, those skilled in the art can make various modifications and variations to the image processing method, apparatus, electronic device, and storage medium provided in this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. An image processing method, characterized in that, The method is applied to an electronic rearview mirror chip in an electronic rearview mirror system, which further includes an image acquisition device for acquiring rear-view images of a vehicle. The method includes: Acquire first image data from the image acquisition device; In response to determining that the image acquisition device is obstructed by liquid based on the first image data, first state data is acquired, the first state data including the driving state data of a first vehicle adjacent to this vehicle; Based on the first state data, it is determined whether the liquid obstruction belongs to the first situation, where the first situation is that the liquid obstruction is caused by the first vehicle running over the water. In response to the fact that the liquid obstruction does not fall under the first case, the first image data is subjected to de-raining and defogging processing to obtain the second image data; Since the liquid obstruction falls under the first scenario, no defogging treatment is required. The first state data includes vehicle speed and a first relative position, where the first relative position is the position of the first vehicle relative to the vehicle itself. Determining whether the liquid obstruction falls under the first scenario based on the first state data includes: A lateral distance threshold is determined based on the vehicle speed; the higher the vehicle speed, the larger the lateral distance threshold. The lateral distance is determined based on the first relative position, and the lateral distance is the lateral interval between the vehicle and the first vehicle. In response to the lateral distance threshold being not less than the lateral distance, it is determined whether the first vehicle has driven over accumulated water. In response to the first vehicle's action of running over the accumulated water, it is determined that the liquid obstruction belongs to the first scenario.

2. The method according to claim 1, characterized in that, Determining whether the first vehicle has driven over accumulated water includes: In response to the detection of road surface water within a first time period before the first moment, the second relative position of the road surface water at the first moment is determined, the second relative position being the position of the road surface water relative to the vehicle, and the first moment being the moment when the image acquisition device acquires the first image data; In response to the coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle has driven over the accumulated water. In response to the absence of coordinate overlap between the second relative position and the first relative position, it is determined that the first vehicle did not drive over the accumulated water.

3. The method according to claim 1, characterized in that, The method further includes: The first image data is converted to a different color space to obtain the third image data; Based on the hue, saturation, and brightness of the third image data, clustering processing is performed on the third image data to obtain multiple regions in the third image data; The fourth image data acquired by the image acquisition device in the frame before the acquisition of the first image data is obtained, and the fourth image data is converted in color space to obtain the fifth image data; The brightness of multiple regions in the third image data is compared with the corresponding regions in the fifth image data to determine whether the image acquisition device has liquid obstruction.

4. The method according to claim 3, characterized in that, The step of comparing the brightness of multiple regions in the third image data with the corresponding regions in the fifth image data to determine whether the image acquisition device has liquid obstruction includes: In response to the fact that the brightness difference between the first region and the second region in multiple regions of the third image data is greater than a first preset threshold, the morphological feature parameters of the first region are obtained. The morphological feature parameters include roundness. The first region includes at least one region. The ratio between the area of ​​the first region and the area of ​​the multiple regions is greater than a second preset threshold. The second region is the corresponding region of the first region in the fifth image data. In response to the circularity of the first region being greater than a third preset threshold, it is determined that the image acquisition device is obstructed by liquid.

5. The method according to claim 4, characterized in that, The method further includes: In response to the liquid occlusion belonging to the first situation, image reconstruction is performed on the first region corresponding to the first image data based on the second region corresponding to the fourth image data to obtain the sixth image data.

6. The method according to claim 1, characterized in that, The step of performing rain and fog removal processing on the first image data to obtain the second image data includes: The first image data is downsampled to obtain the seventh image data; The brightness difference is calculated based on the first image data and the fourth image data to obtain the first image feature. The fourth image data is the image data acquired by the image acquisition device in the frame before acquiring the first image data. The seventh image data and the first image features are input into the first model for processing to obtain a first parameter set, which is used to characterize the state of rain and fog. The first image data and the first parameter set are input into the second model for processing to obtain the second parameter set, which includes image parameters of the first image data after dehazing. The first image data is dehazed according to the second parameter set to obtain the second image data.

7. An image processing apparatus, characterized in that, The device is applied to the electronic rearview mirror chip in the electronic rearview mirror system, which also includes an image acquisition device for acquiring rear-view images of the vehicle. The device includes: A communication unit is used to acquire first image data from the image acquisition device; In response to the processing unit determining, based on the first image data, that the image acquisition device is obstructed by liquid, the communication unit is further configured to acquire first status data, the first status data including the driving status data of a first vehicle adjacent to the vehicle. The processing unit is further configured to determine whether the liquid obstruction belongs to a first situation based on the first state data, wherein the first situation is that the liquid obstruction is caused by the first vehicle running over the water. The processing unit is further configured to, in response to the liquid obstruction not belonging to the first situation, perform rain and fog removal processing on the first image data to obtain the second image data; Since the liquid obstruction falls under the first scenario, no defogging treatment is required. The first state data includes vehicle speed and a first relative position, where the first relative position is the position of the first vehicle relative to the vehicle itself. In determining whether the liquid obstruction falls under the first scenario based on the first state data, the processing unit is specifically used for: A lateral distance threshold is determined based on the vehicle speed; the higher the vehicle speed, the larger the lateral distance threshold. The lateral distance is determined based on the first relative position, and the lateral distance is the lateral interval between the vehicle and the first vehicle. In response to the lateral distance threshold being not less than the lateral distance, it is determined whether the first vehicle has driven over accumulated water. In response to the first vehicle's action of running over the accumulated water, it is determined that the liquid obstruction belongs to the first scenario.

8. An electronic device, the device comprising a processor, a memory, and executable program code stored in the memory, characterized in that, The processor is configured to retrieve the executable program code stored in the memory to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images