Automatic driving camera cleaning method and system

Abstract

The invention provides an automatic driving camera cleaning method and system. According to the method, when the vehicle is in the automatic driving mode, the environment image of the monitoring area corresponding to the camera is obtained, the first cleaning state parameter of the camera is determined according to the environment image, and then the cleaning device is controlled to spray the cleaning fluid to the camera according to the first cleaning state parameter, so that the cleaning fluid is sprayed to the camera under the complex and changeable automatic driving working condition. The imaging definition of a monitoring view field in front of the camera is continuously maintained, and the reliability and safety of an automatic driving perception function are ensured.

Description

Technical Field

[0001] This application relates to data processing technology, and more particularly to a method and system for cleaning cameras for autonomous driving. Background Technology

[0002] In autonomous vehicles, environmental perception relies heavily on cameras installed at the front, rear, left, and right of the vehicle to achieve key functions such as lane recognition, obstacle detection, and traffic sign recognition.

[0003] However, during actual road driving, the monitoring field of view in front of the camera is easily affected by various complex conditions such as rain, mud, dust, fog, frost, and sewage splashed by vehicles. This causes dirt to adhere to the outer surface of the camera cover or form a water film or frost layer, resulting in problems such as reduced image contrast, blurred edges, and loss of high-frequency details.

[0004] In existing technologies, vehicles typically only trigger the cleaning nozzles at regular intervals or based on simple wiper linkage logic to spray and clean the camera in a fixed pattern. This results in redundant cleaning actions and wasted cleaning fluid when the pollution is light, and insufficient cleaning and unsatisfactory image clarity recovery when the pollution is heavy. It is impossible to reliably guarantee the camera image quality requirements of the autonomous driving system under all operating conditions. Summary of the Invention

[0005] This application provides a method and system for cleaning cameras for autonomous driving, which controls the cleaning device to spray cleaning fluid onto the camera, thereby continuously maintaining the image clarity of the monitoring field of view in front of the camera under complex and ever-changing autonomous driving conditions, and ensuring the reliability and safety of autonomous driving perception functions.

[0006] In a first aspect, this application provides a method for cleaning a camera used in autonomous driving, comprising: When the vehicle is in autonomous driving mode, an environmental image of the monitoring area corresponding to the camera is acquired, and the environmental image is used to indicate the current imaging clarity information of the camera; Determine the first cleaning status parameter of the camera based on the environmental image; Based on the first cleaning status parameter, the cleaning device is controlled to spray cleaning fluid onto the camera.

[0007] Secondly, this application provides an autonomous driving camera cleaning system, comprising: Cameras are used to acquire environmental images of the corresponding monitored area; A cleaning device is used to store a wax-containing cleaning solution and spray the wax-containing cleaning solution onto the camera to clean the camera. An electronic control unit, electrically connected to the camera and the cleaning device, is used for: Determine the first cleaning status parameter of the camera based on the environmental image; Based on the first cleaning status parameter, the cleaning device is controlled to spray the wax-containing cleaning liquid onto the camera.

[0008] Thirdly, this application provides an electronic device, comprising: Processor; and, Memory for storing the executable instructions of the processor; The processor is configured to perform any of the possible methods described in the first aspect by executing the executable instructions.

[0009] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the possible methods described in the first aspect.

[0010] The autonomous driving camera cleaning method and system provided in this application acquires environmental images of the monitoring area corresponding to the camera when the vehicle is in autonomous driving mode, determines the first cleaning state parameters of the camera based on the environmental images, and then controls the cleaning device to spray cleaning fluid onto the camera based on the first cleaning state parameters. This ensures the continuous maintenance of the imaging clarity of the monitoring field of view in front of the camera under complex and ever-changing autonomous driving conditions, thereby ensuring the reliability and safety of the autonomous driving perception function. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] Figure 1 This is a schematic diagram of the structure of an autonomous driving camera cleaning system according to an example embodiment of this application; Figure 2 This is a schematic flowchart illustrating an autonomous driving camera cleaning method according to an example embodiment of this application; Figure 3 This is a flowchart illustrating how the control parameters of the injection assembly are determined according to an example embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application.

[0013] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0014] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0015] To address the aforementioned issues, the embodiments provided in this application first acquire environmental images of the monitoring area corresponding to the camera in real time while the vehicle is in autonomous driving mode. These environmental images are then analyzed at the pixel level, and a comprehensive clarity index is constructed to characterize the imaging quality of the monitoring field of view by calculating pixel contrast or sharpness evaluation metrics. Next, the clarity index of the current environmental image is compared with the corresponding index of a pre-calibrated first calibrated image under camera cleaning conditions to obtain the degree of local and overall imaging degradation, thereby generating a first cleaning state parameter that includes at least the degree of imaging degradation. Based on this, the first cleaning state parameter is used as input and mapped to control variables of the cleaning device, including whether cleaning is triggered and execution parameters such as the target spray direction, target spray angle, target spray pressure, and target spray flow rate of the spray component. The processor performs fine control of the cleaning device to match the spray cleaning process with the degree of imaging degradation. After cleaning, environmental images are re-acquired to evaluate imaging clarity, achieving closed-loop maintenance of the clarity of the monitoring field of view in front of the camera, thereby realizing dynamic coupling and adaptive adjustment between cleaning behavior and image quality changes.

[0016] Correspondingly, the specific technical effects achieved include, but are not limited to: First, the cleaning action corresponds to the actual degree of image degradation, and the spray is only triggered when the image clarity drops to a certain level, effectively avoiding unnecessary cleaning and reducing cleaning fluid consumption.

[0017] Secondly, by matching appropriate spraying strategies according to different degrees of degradation, suitable spraying intensity and coverage can be selected under different working conditions such as light pollution, heavy dirt adhesion, rain or frost, thereby improving the imaging restoration effect of a single cleaning and reducing the number of repeated cleanings.

[0018] Third, by continuously monitoring and adjusting the image clarity of the monitoring field of view after cleaning, the monitoring field of view in front of the camera can maintain a preset clarity level for a long time during autonomous driving, thereby improving the reliable perception capability and safety redundancy of the autonomous driving system of the road environment.

[0019] Figure 1This is a schematic diagram of the structure of an autonomous driving camera cleaning system according to an example embodiment of this application. Figure 1 As shown, the system provided in this embodiment includes: Camera 110 is used to acquire environmental images of the corresponding monitored area. The environmental images are used to indicate the image clarity information of the camera.

[0020] The cleaning device 120 is used to store wax-containing cleaning fluid and spray the wax-containing cleaning fluid onto the camera to clean the camera 110.

[0021] Electronic control unit 130, electrically connected to camera 110 and cleaning device 120, is used for: The first cleaning status parameter of the camera is determined based on the environmental image. The first cleaning status parameter includes at least the degree of image degradation in the monitoring field of view in front of the camera. Based on the first cleaning status parameter, the cleaning device is controlled to spray wax-containing cleaning fluid onto the camera.

[0022] Specifically, the cleaning device 120 includes an air compressor 121, a liquid reservoir 122, a solenoid valve 123, and a nozzle 124. The liquid reservoir 122 stores the wax-containing cleaning fluid. The air compressor 121 is connected to the liquid reservoir 122 and supplies compressed air to it, creating pressure within the reservoir to propel the wax-containing cleaning fluid. The solenoid valve 123 is located on the cleaning fluid delivery pipeline connecting the liquid reservoir 122 and the nozzle 124, and controls the flow of the wax-containing cleaning fluid under the control of the electronic control unit 130. The nozzle 124 is connected to the outlet of the solenoid valve 123 and, under the compressed air pressure provided by the air compressor 121 and with the solenoid valve 123 in the open state, sprays the wax-containing cleaning fluid from the liquid reservoir 122 onto the camera 110.

[0023] Figure 2 This is a schematic flowchart illustrating an autonomous driving camera cleaning method according to an example embodiment of this application. Figure 2 As shown, the method provided in this embodiment includes: S110. Acquire an environmental image of the monitoring area corresponding to the camera. The environmental image is used to indicate the current imaging clarity information of the camera.

[0024] This can be done while the vehicle is in autonomous driving mode, by acquiring environmental images of the monitored area corresponding to the camera, and the environmental images are used to indicate the current image clarity information of the camera.

[0025] Specifically, during the continuous autonomous driving mode, the camera can be periodically activated according to a preset sampling period or based on the image acquisition trigger command issued by the autonomous driving control system to collect raw image data of the monitoring area corresponding to the camera.

[0026] Optionally, the original image data may be preprocessed with at least one of the following: denoising, white balance correction, gamma correction, and brightness equalization, to obtain an environmental image for image sharpness assessment.

[0027] Then, based on the camera's mounting pose parameters in the vehicle coordinate system and the field of view parameters of the monitored area, the distortion of the environmental image is corrected so that the environmental image accurately reflects the current imaging clarity information of the camera.

[0028] S120. Determine the first cleaning status parameters of the camera based on the environmental image.

[0029] In this step, a first cleaning status parameter of the camera may be determined based on the environmental image. The first cleaning status parameter includes at least the degree of image degradation in the monitoring field of view in front of the camera.

[0030] Specifically, the degree of image degradation in the monitoring field of view in front of the camera can be determined based on the contrast or sharpness evaluation indicators of pixels in the environmental image. Then, the first cleanliness state parameter of the camera is determined based on the degree of image degradation.

[0031] Optionally, the degree of image degradation in the monitoring field of view in front of the camera can be dimensionlessly processed to obtain an image degradation characterization value that represents the overall image quality of the monitoring field of view in front of the camera. This image degradation characterization value is then compared with a reference image characterization value obtained during camera calibration in a clean state, and the difference between the two values ​​is calculated. Based on this difference, a cleanliness coefficient is determined to characterize the cleanliness of the outer surface of the camera housing. The cleanliness coefficient decreases as the difference increases, reaching its maximum value when the difference is zero. Finally, the cleanliness coefficient is used as one of the first cleanliness state parameters of the camera to characterize the cleanliness of the outer surface of the camera housing.

[0032] Furthermore, the determination of the aforementioned contrast or sharpness evaluation metrics can also be implemented using the existing OpenCV image processing library. Specifically, in the implementation of processing environmental images based on the OpenCV image processing library, the environmental image is acquired by calling the image reading interface provided by OpenCV, and then converted into a grayscale environmental image by calling the color space conversion interface. Next, the grayscale environmental image is calculated by calling the statistical operation interface, and the grayscale standard deviation is used as the contrast evaluation metric for the environmental image.

[0033] Furthermore, gradient or Laplacian operator operations can be performed on grayscale environment images by calling the convolution and filtering interface to obtain the corresponding gradient image or Laplacian response image. Element-wise squaring and summation operations can then be performed on the gradient image or Laplacian response image by calling the fundamental matrix operation interface to obtain the sharpness evaluation index of the environment image in the sense of gradient energy or Laplacian variance.

[0034] Furthermore, to determine the degree of image degradation, one can select the ratio between one of the contrast or sharpness evaluation indicators and its corresponding standard value, or the ratio between the difference between the two and its corresponding standard value, to characterize the degree of image degradation. Alternatively, the contrast and sharpness evaluation indicators can be normalized separately to obtain normalized contrast and sharpness evaluation indicators within the same numerical range. The normalized contrast and sharpness evaluation indicators are then summed to obtain a normalized fusion evaluation indicator used to characterize the degree of image degradation in the front monitoring field of view of the camera, and this normalized fusion evaluation indicator is used to characterize the degree of image degradation in the front monitoring field of view of the camera.

[0035] It is worth noting that one of the aforementioned contrast or sharpness evaluation metrics and its corresponding standard value, or the base value used in the aforementioned normalization process, can be determined through calibration. Specifically, a first calibration image, pre-calibrated under clean camera conditions, corresponding to a standard environmental scene, can be acquired first. Then, the contrast standard value and sharpness index standard value between the environmental image and the first calibration image are determined using the OpenCV image processing library, serving as one of the aforementioned contrast or sharpness evaluation metrics and its corresponding standard value, or the base value used in the aforementioned normalization process.

[0036] It is worth noting that under dynamic conditions such as heavy rain, snowfall, and splashing mud and water, raindrops and snowflakes may briefly pass through the camera's field of view or momentarily adhere to the surface of the camera's protective cover. In a very small number of frames, these may create areas with significantly reduced sharpness or highlight occlusion. This can cause the imaging degradation judgment based on a single frame or a short time segment to easily mistake such transient occlusion for continuous dirt degradation, thus frequently triggering cleaning actions. This results in excessively rapid consumption of cleaning fluid, excessively high frequency of spray component operation, and even further interference with camera imaging due to the sprayed liquid curtain itself, thereby deteriorating the perception effect of autonomous driving.

[0037] To address this, multiple frames of environmental images corresponding to the camera can be continuously acquired within a preset time window. Then, these multiple frames are divided into several sub-regions, and a sharpness evaluation index sequence for each sub-region within the time window is calculated. Next, based on the sharpness evaluation index sequence for each sub-region, the percentage of time during which the sharpness evaluation index for each sub-region is below a preset sharpness threshold is determined.

[0038] When the time proportion of a sub-region within the time window is greater than a first preset proportion threshold and the difference between the minimum and average values ​​of its sharpness evaluation index is less than a first preset difference threshold, the corresponding sub-region is determined as a sub-region with continuous imaging degradation.

[0039] When the time proportion of a sub-region within the time window is less than the second preset proportion threshold and the difference between the minimum and average values ​​of its sharpness evaluation index is greater than the second preset difference threshold, the corresponding sub-region is determined as a transient occlusion sub-region, and the contribution of the transient occlusion sub-region to the overall imaging degradation degree is suppressed in the subsequent imaging degradation degree calculation.

[0040] It is worth understanding that, firstly, multiple frames of environmental images output by the camera are cumulatively observed within a preset time window, each frame of image is spatially divided into multiple sub-regions with the same layout, and a sequence of clarity evaluation indicators that change over time is constructed for each sub-region.

[0041] Secondly, for each sub-region, by statistically analyzing the proportion of time when its sharpness evaluation index is lower than the preset sharpness threshold and calculating the difference between the average and minimum values ​​of the sequence, etc., when the proportion of time is large and the difference between the minimum and average values ​​is small, it indicates that the sub-region is in a state of continuous degradation for most of the time, and its sharpness index shows a stable low value distribution, which can be identified as a sub-region with continuous imaging degradation.

[0042] When the time percentage is small and the difference between the minimum and average values ​​is large, it indicates that the sub-region experiences a significant sharp drop in clarity only in a very few frames, exhibiting a sharp, transient anomaly, which can be identified as a transient occlusion sub-region. By accumulating only the contribution of the continuous image degradation sub-region in the image degradation degree calculation and suppressing the influence of the transient occlusion sub-region, the continuous characteristics in the time dimension and the statistical distribution characteristics of the clarity index are utilized to effectively distinguish between continuous dirt degradation and transient occlusion degradation. This achieves time-dimensional filtering and smoothing evaluation of the degree of image degradation in the monitoring field of view in front of the camera under dynamic and complex environments. It can reduce false cleaning triggers caused by short-term occlusion, effectively reduce the number of invalid actions of the cleaning device and cleaning fluid consumption, and improve the detection accuracy for true continuous dirt degradation, thereby significantly improving the reliability and economy of camera cleaning control in autonomous driving scenarios.

[0043] Furthermore, in another possible implementation, the determination of the aforementioned sharpness index difference can be achieved by dividing the environmental image and the first calibration image into multiple corresponding sub-regions according to the same image partitioning rules. These image partitioning rules include at least one or more of fixed-pixel grid partitioning and equal-area partitioning. Then, for each corresponding sub-region, the sharpness index is calculated for both the sub-region in the environmental image and the sub-region in the first calibration image. This can be done by referring to the determination method based on the OpenCV image processing library described above, which will not be elaborated upon here.

[0044] Next, based on the sharpness index of a sub-region in the environmental image and the sharpness index of the corresponding sub-region in the first calibration image, the difference in sharpness index is calculated according to a preset difference calculation method, which includes one or more of absolute difference calculation, normalized difference calculation, and weighted difference calculation.

[0045] Finally, the sharpness index difference corresponding to each sub-region is compared with the preset sharpness difference threshold to determine the degree of local imaging degradation of the corresponding sub-region.

[0046] It is worth noting that because the image degradation caused by dirt on the outer surface of the camera cover is similar in appearance to the overall image degradation caused by atmospheric media such as fog, rain, and snow, in severe weather scenarios such as heavy fog, high humidity, or heavy rain, the decrease in distant contrast caused by atmospheric media may be mistaken for dirt contamination on the camera surface. This leads to frequent and ineffective cleaning of the camera, which not only wastes cleaning fluid and energy but may also form a water film on the lens surface, further deteriorating the image clarity and affecting the environmental perception reliability and driving safety of the autonomous driving system.

[0047] To address this, prior to S120, environmental state parameters from the vehicle's corresponding environmental sensors can be acquired. These parameters include at least one or more of the following: rainfall intensity, ambient humidity, and temperature. Then, based on the environmental state parameters and the degree of image degradation in the monitoring field of view in front of the camera, the probability of image degradation caused by atmospheric media is determined. When the probability of image degradation caused by atmospheric media exceeds a preset threshold and the image degradation in the monitoring field of view in front of the camera exhibits a layered characteristic that increases with distance, the image degradation is determined to be caused by non-camera housing surface contamination, and the cleaning trigger command for the camera is suppressed; that is, no subsequent cleaning trigger command is triggered.

[0048] Specifically, to improve the assessment of imaging degradation probability caused by atmospheric medium, the monitored area can be divided into multiple horizontal bands along the vertical direction in the environmental image. Then, the average sharpness evaluation index and contrast index for each horizontal band are determined. When the average sharpness evaluation index of the horizontal band near the upper part of the image is lower than that of the horizontal band near the lower part of the image, and the environmental humidity information reaches a preset high humidity threshold, the imaging degradation is determined to be caused by atmospheric medium, and the probability of imaging degradation caused by atmospheric medium is increased.

[0049] It is worth noting that in the imaging optical path, atmospheric media, such as haze and drizzle, are distributed in the long-distance propagation path in front of the camera. Their impact on imaging is mainly manifested as scattering and attenuation that accumulates and intensifies with the increase of target distance. This causes the contrast and sharpness of distant target areas to decrease preferentially, thus forming a layered degradation feature on the image plane from bottom to top and from near to far. Meanwhile, dirt on the outer surface of the camera protective cover adheres to the near-field position in front of the lens, causing local obstruction or scattering of light. Its imaging degradation is mostly manifested as fixed low-resolution patches that are highly localized in space and independent of distance.

[0050] The above steps acquire rainfall intensity, ambient humidity, and temperature information through environmental sensors to indicate whether the current condition is high humidity or heavy precipitation. The environmental image is divided into multiple horizontal zones along the vertical direction, and the average sharpness evaluation index and contrast index of each horizontal zone are statistically analyzed. When a significant decrease in sharpness is detected in the upper region relative to the lower region and the ambient humidity reaches the high humidity threshold, the image degradation can be attributed to the atmospheric medium based on atmospheric scattering theory. At the same time, the probability of image degradation caused by the atmospheric medium is calculated and improved.

[0051] When the probability exceeds a preset threshold and the layering characteristics are significant, the control logic suppresses the camera cleaning trigger command. This enables the system to utilize the optical path degradation location (i.e., the far-field atmospheric path and the near-field lens surface), spatial distribution characteristics (i.e., layered uniform attenuation and local patch degradation), and environmental state parameters (i.e., humidity and rainfall intensity) to reliably distinguish between lens surface contaminant degradation that can be removed by cleaning and atmospheric medium degradation that cannot be removed by cleaning. This effectively avoids misjudging overall image degradation caused by atmospheric medium as local degradation caused by contaminant on the outer surface of the camera cover in foggy, hazy, and high-humidity environments. It also reduces ineffective or harmful cleaning actions, lowers cleaning fluid and energy consumption, and prevents water curtains or films from forming during spray cleaning that further obstruct the camera's field of view. As a result, under complex weather conditions, it effectively improves the imaging clarity maintenance effect and environmental perception reliability of the autonomous driving system for the forward monitoring field of view.

[0052] S130. Based on the first cleaning status parameters, control the cleaning device to spray cleaning fluid onto the camera.

[0053] In this step, the cleaning device can be controlled to spray cleaning fluid onto the camera according to the first cleaning status parameter in order to clean the camera and maintain the imaging clarity of the monitoring field of view in front of the camera after cleaning.

[0054] Specifically, the target spray direction and angle of the spray assembly can be determined based on the camera's installation location and the field of view of the monitored area. Furthermore, the target spray pressure and flow rate can be determined based on the degree of image degradation. Then, the spray assembly is controlled to clean the camera according to the target spray direction, angle, pressure, and flow rate.

[0055] in, Figure 3 This is a flowchart illustrating the method for determining the control parameters of the injection assembly according to an example embodiment of this application. Figure 3 As shown, the above-mentioned method for determining the control parameters of the injection assembly includes: S131. Obtain the installation posture parameters of the camera, the field of view parameters of the monitored area, and the installation posture parameters of the spraying assembly.

[0056] Specifically, this can involve acquiring the camera's mounting pose parameters, which at least include the camera's mounting position coordinates in the vehicle coordinate system and its optical axis direction vector. It can also involve acquiring the field of view parameters of the monitored area, which at least include the horizontal field of view angle, the vertical field of view angle, and the boundary information of the effective imaging area. Finally, it can involve acquiring the injection assembly's mounting pose parameters, which at least include the injection assembly's mounting position coordinates in the vehicle coordinate system and its injection reference direction vector.

[0057] Optionally, the camera's mounting pose parameters can be obtained during vehicle production or maintenance calibration stages, based on calibration data from the vehicle assembly fixture, in the vehicle coordinate system, by acquiring the camera's nominal mounting position coordinates and nominal optical axis direction vector. The actual mounting position of the camera is measured and calibrated using at least one of a calibration fixture, laser rangefinder, or 3D measuring equipment to obtain the camera's actual mounting position coordinates. Then, based on preset spatial feature points or a calibration checkerboard pattern in the calibration scene, the camera's intrinsic and extrinsic parameter matrices are calculated, thereby determining the camera's actual optical axis direction vector. Finally, the actual mounting position coordinates and the actual optical axis direction vector are stored as the camera's mounting pose parameters in the vehicle coordinate system for retrieval during vehicle operation.

[0058] Optionally, the field-of-view parameters for the aforementioned monitoring area can be constructed in the vehicle coordinate system based on the camera's installation position coordinates and optical axis direction vector in the vehicle coordinate system, combined with the camera's horizontal and vertical field-of-view angles. Then, based on the camera's effective imaging resolution, image boundary pixel coordinates, and lens distortion parameters, the field-of-view cone model is geometrically corrected to obtain the corrected field-of-view boundary. The projection area of ​​the corrected field-of-view boundary onto the image plane is then determined as the effective imaging area, and its boundary information is extracted. Finally, the horizontal and vertical field-of-view angles, along with the boundary information of the effective imaging area, are determined as the field-of-view parameters for the monitoring area and stored in the vehicle control system for subsequent use.

[0059] Optionally, the mounting pose parameters of the aforementioned injection assembly can be obtained in the vehicle coordinate system based on the reference holes and reference surfaces of the vehicle body structural components or mounting brackets after the injection assembly is completed. Alternatively, the actual spatial position of the injection assembly nozzle can be measured and calibrated using at least one of the following methods: 3D measurement equipment, robotic arm teaching, or vision calibration plate measurement, to obtain the actual mounting pose coordinates of the injection assembly. Then, during the factory testing or vehicle calibration stage, injection tests are performed on the injection assembly at different drive angles. Based on the detection results of the injection trajectory or injection coverage area, the injection reference direction vector of the injection assembly is fitted and obtained. Finally, the actual mounting pose coordinates and the injection reference direction vector of the injection assembly are stored as the mounting pose parameters of the injection assembly in the vehicle coordinate system for subsequent calculation of the target injection direction and target injection angle based on the vehicle coordinate system.

[0060] S132. Determine the target coverage area for covering the outer surface of the camera protective cover without obstructing the effective imaging area of ​​the camera.

[0061] In this step, the target coverage area can be determined in the vehicle coordinate system based on the camera's installation pose parameters and the field of view parameters of the monitored area, to cover the outer surface of the camera's protective cover without obstructing the camera's effective imaging area.

[0062] Specifically, it is possible to construct a field-of-view spatial model of the camera in the vehicle coordinate system and the corresponding effective imaging space area based on the camera's installation position coordinates and optical axis direction vector in the vehicle coordinate system, combined with the horizontal field of view, vertical field of view, and boundary information of the effective imaging area.

[0063] Then, obtain the shape contour parameters of the camera protective cover. The shape contour parameters include at least the spatial contour line, surface equation and installation boundary position of the outer surface of the camera protective cover.

[0064] Next, based on the spatial relationship between the field of view model and the outer surface of the camera cover, a visibility analysis of the outer surface of the camera cover is performed in the vehicle coordinate system. Surface areas that would obstruct the effective imaging area are eliminated, and candidate coverage areas that do not obstruct the effective imaging area are obtained.

[0065] Finally, within the candidate coverage area, the cleaning coverage efficiency of different candidate coverage areas is evaluated based on the diffusion range of the sprayed droplets on the outer surface of the camera cover, the direction of gravity, and the wind field conditions during vehicle travel. Target coverage areas that meet the cleaning coverage requirements are then selected based on a preset coverage efficiency threshold.

[0066] S133. Determine the target spray direction vector between the installation position coordinates of the connecting spray assembly and the centroid position of the target coverage area.

[0067] In this step, the target injection direction vector connecting the installation position coordinates of the injection assembly and the centroid position of the target coverage area can be determined in the vehicle coordinate system based on the installation pose parameters of the injection assembly and the target coverage area.

[0068] Specifically, the spatial position of the nozzle of the injection assembly can be determined based on the installation position coordinates of the injection assembly in the vehicle coordinate system.

[0069] Then, based on the spatial distribution range of the target coverage area in the vehicle coordinate system, the target coverage area is discretized according to the preset area weighting rules, and the area weight coefficient of the discrete sub-region is calculated.

[0070] Next, based on the spatial coordinates of the discrete sub-regions and the corresponding area weighting coefficients, the centroid coordinates of the target coverage area are obtained.

[0071] Finally, the spatial vector starting from the coordinates of the injection component installation position and ending at the coordinates of the centroid of the target coverage area is determined as the unnormalized target injection direction vector. The unnormalized target injection direction vector is then normalized to obtain the target injection direction vector.

[0072] S134. Determine the target deflection angle and map the target deflection angle to the target spray angle of the spray assembly.

[0073] In this step, the target spray direction vector can be solved by calculating the vector angle between the target spray direction vector and the spray reference direction vector to obtain the target deflection angle of the spray assembly around the preset rotation axis, and the target deflection angle can be mapped to the target spray angle of the spray assembly.

[0074] Specifically, the cosine of the angle between the injection reference direction vector and the target injection direction vector of the injection component can be calculated using vector dot product in the vehicle coordinate system, and the initial angle value can be obtained by solving the inverse trigonometric function.

[0075] Then, based on the mechanical degrees of freedom of the injection assembly and the preset rotation axis direction, the target injection direction vector is decomposed in a plane perpendicular to the preset rotation axis, and the target deflection angle of the injection assembly around the preset rotation axis is calculated based on the components of the injection reference direction vector in the same plane.

[0076] Next, based on the drive structure parameters and angle encoding rules of the injection assembly, the target deflection angle is converted into the corresponding drive control angle value through one or more methods, such as lookup table mapping, linear conversion, or multi-segment piecewise function.

[0077] Finally, the drive control angle value is determined as the target injection angle of the injection assembly, which is used to drive the injection assembly to adjust the injection attitude.

[0078] S135. Obtain the first mapping parameters corresponding to different degrees of image degradation in the monitoring field of view in front of the camera.

[0079] In this step, the first mapping parameter may be obtained corresponding to different degrees of image degradation in the monitoring field of view in front of the camera. The first mapping parameter may include at least the correspondence between different degrees of image degradation and injection pressure and injection flow rate.

[0080] Specifically, when the camera is in a clean state, a first reference environmental image of the corresponding monitoring area is acquired, and based on the contrast and sharpness evaluation indicators of the pixels in the first reference environmental image, the reference sharpness index of the monitoring field of view in front of the camera is determined.

[0081] Then, under different adhesion conditions such as simulated rain, mud, dust, fog and frost, multiple sets of second environment images were acquired, and the corresponding sharpness degradation was calculated based on the contrast and sharpness evaluation index of the pixels in each second environment image.

[0082] Next, based on the amount of resolution degradation, and according to the preset degradation level classification rules, the degree of image degradation of the monitoring field of view in front of the camera is divided into several discrete degradation levels, and the degradation level includes at least one or more of mild degradation, moderate degradation and severe degradation.

[0083] Then, cleaning experiments were conducted multiple times under different degradation levels, with different combinations of spray pressure and spray flow rate. The cleaning effect evaluation results under each combination of conditions were recorded. The cleaning effect evaluation results should include at least one or more of the following: image clarity restoration rate after cleaning, cleaning fluid consumption, and cleaning time.

[0084] Finally, based on the cleaning effect evaluation results, the combination of injection pressure and injection flow rate that has a better cleaning effect than the preset threshold and whose cleaning fluid consumption and cleaning time meet the constraints at each degradation level is selected. The degradation level is then associated with the corresponding combination of injection pressure and injection flow rate to form the first mapping parameter, which is then stored in the vehicle control system for later use.

[0085] S136. Based on the degree of imaging degradation, determine the corresponding target injection pressure and target injection flow rate from the first mapping parameters.

[0086] When the vehicle is in autonomous driving mode, the degree of image degradation of the monitoring field of view in front of the camera can be determined based on the environmental image, and the degree of image degradation can be mapped to the corresponding degradation level label.

[0087] When the degree of image degradation falls exactly within a certain discrete degradation level range preset in the first mapping parameters, a preset recommended combination is directly selected from the combination of injection pressure and injection flow rate corresponding to that degradation level range as the target injection pressure and target injection flow rate.

[0088] When the degree of image degradation is between two adjacent degradation levels, based on the combination of jet pressure and jet flow rate corresponding to the adjacent degradation levels recorded in the first mapping parameter, the interpolated jet pressure and interpolated jet flow rate corresponding to the current degree of image degradation are calculated according to the linear interpolation or piecewise interpolation rules, and the interpolated jet pressure and interpolated jet flow rate are determined as the target jet pressure and target jet flow rate.

[0089] After determining the target injection pressure and target injection flow rate, optionally, the target injection pressure and target injection flow rate can be compensated and corrected by combining the current ambient temperature, vehicle speed and wind direction information, so as to ensure the cleaning effect on the camera under low temperature, high speed or crosswind conditions. The compensated and corrected injection pressure and injection flow rate are used as the final target injection pressure and target injection flow rate to control the injection components to perform spray cleaning.

[0090] It is worth noting that this could involve acquiring the vehicle's current ambient temperature information, which includes at least one or more of the following: ambient temperature values ​​collected by external temperature sensors and road segment forecast temperature information provided by meteorological services or vehicle-to-everything (V2X) systems. It could also involve acquiring the vehicle's current speed and wind direction information, where the wind direction information includes at least one of the following: external airflow direction information acquired by an onboard wind direction sensor, a barometric pressure sensor, and an onboard communication unit, as well as relative airflow direction information calculated based on the vehicle speed and driving direction.

[0091] Then, based on the current ambient temperature information, the temperature compensation coefficient is determined. When the ambient temperature is lower than the preset low temperature threshold, the temperature compensation coefficients of the target injection pressure and the target injection flow rate are increased to compensate for the decrease in injection range and atomization effect caused by the increase in the viscosity of the cleaning fluid under low temperature conditions.

[0092] Next, based on the current vehicle speed information, the vehicle speed compensation coefficient is determined. When the vehicle speed is higher than the preset high speed threshold, the vehicle speed compensation coefficient of the target injection pressure and the target injection flow rate is increased to compensate for the influence of relative wind pressure caused by the high speed of the vehicle on the deflection and fall of the injection flow.

[0093] Based on wind direction information, a wind direction compensation coefficient is determined. When there is a preset angle deviation between the relative incoming flow direction and the optical axis direction of the camera or the spray reference direction of the spray component, the target spray pressure and target spray flow rate are increased without significantly increasing the consumption of cleaning fluid. The target spray angle of the spray component can also be finely adjusted to compensate for the deviation of the spray trajectory under crosswind or headwind conditions.

[0094] The temperature compensation coefficient, vehicle speed compensation coefficient, and wind direction compensation coefficient are then combined according to a preset weighting rule to obtain the total compensation coefficient corresponding to the current operating condition. Based on the total compensation coefficient, the target injection pressure and target injection flow rate are scaled and corrected to obtain the compensated and corrected injection pressure and injection flow rate.

[0095] After the compensation correction is completed, the corrected injection pressure and injection flow rate are checked for safety range constraints. When any parameter exceeds the preset safety upper limit or falls below the preset safety lower limit, it is limited to the corresponding safety range to avoid damage to the camera cover or injection components.

[0096] Finally, the compensated and corrected injection pressure and compensated and corrected injection flow rate, after being verified by the safety range constraints, are determined as the final target injection pressure and target injection flow rate. The final target injection pressure and final target injection flow rate are then sent to the drive control module of the injection assembly to control the injection assembly to perform spray cleaning.

[0097] S137. Control the spray assembly to spray according to the determined spray assembly control parameters.

[0098] In this step, the target injection direction vector and the target injection angle can be determined as the target injection direction and the target injection angle of the injection component, and the injection component can be controlled to inject according to the determined target injection pressure and the target injection flow rate.

[0099] In this embodiment, by acquiring environmental images of the monitoring area corresponding to the camera when the vehicle is in autonomous driving mode, determining the first cleaning status parameter of the camera based on the environmental images, and then controlling the cleaning device to spray cleaning fluid onto the camera based on the first cleaning status parameter, the imaging clarity of the monitoring field of view in front of the camera is continuously maintained under complex and ever-changing autonomous driving conditions, thereby ensuring the reliability and safety of the autonomous driving perception function.

[0100] Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. For example... Figure 4 As shown, the electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein: Memory 402 is used to store computer programs, and the memory may also be flash memory.

[0101] Processor 401 is used to execute the execution instructions stored in the memory to implement the various steps in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0102] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0103] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include: Bus 403 is used to connect the memory 402 and the processor 401.

[0104] This embodiment also provides a readable storage medium storing a computer program, which, when executed by at least one processor of an electronic device, enables the electronic device to perform the methods provided in the various embodiments described above.

[0105] This embodiment also provides a program product including a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods provided in the various embodiments described above.

[0106] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0107] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for cleaning a camera used in autonomous driving, characterized in that, include: When the vehicle is in autonomous driving mode, an environmental image of the monitoring area corresponding to the camera is acquired, and the environmental image is used to indicate the current imaging clarity information of the camera; Determining the first cleaning status parameter of the camera based on the environmental image includes: determining the degree of image degradation of the monitoring field of view in front of the camera based on the contrast or sharpness evaluation index of the pixels in the environmental image; including: acquiring a first calibration image pre-calibrated in the camera cleaning state. Calculate the difference in sharpness index between the environmental image and the first calibration image in multiple sub-regions; Based on the difference in the sharpness index, the degree of local imaging degradation in the corresponding sub-region is determined, and the degree of overall imaging degradation is calculated based on the degree of local imaging degradation. The first cleaning status parameter of the camera is determined based on the degree of image degradation.

2. The method for cleaning an autonomous driving camera according to claim 1, characterized in that, The step of controlling the cleaning device to spray cleaning fluid onto the camera based on the first cleaning status parameter includes: Based on the installation location of the camera and the field of view of the monitored area, determine the target spray direction and target spray angle of the spraying component; The target injection pressure and target injection flow rate are determined based on the degree of image degradation. The spraying assembly is controlled to clean the camera according to the target spraying direction, the target spraying angle, the target spraying pressure, and the target spraying flow rate.

3. The method for cleaning an autonomous driving camera according to claim 2, characterized in that, Determining the target spray direction and target spray angle of the spraying component based on the installation location of the camera and the field of view of the monitored area includes: Based on the installation pose parameters of the camera and the field of view parameters of the monitoring area, a target coverage area is determined in the vehicle coordinate system to cover the outer surface of the camera protective cover without obstructing the effective imaging area of ​​the camera. Based on the installation pose parameters of the injection assembly and the target coverage area, a target injection direction vector connecting the installation position coordinates of the injection assembly and the centroid position of the target coverage area is determined in the vehicle coordinate system. The target deflection angle of the injection component around a preset rotation axis is obtained by solving the vector angle between the target injection direction vector and the injection reference direction vector, and the target deflection angle is mapped to the target injection angle of the injection component. The target injection direction vector and the target injection angle are determined as the target injection direction and target injection angle of the injection component, which are used to control the injection component to perform injection.

4. The method for cleaning an autonomous driving camera according to claim 3, characterized in that, Before the installation pose parameters of the camera and the field of view parameters of the monitored area, the following are included: Obtain the installation pose parameters of the camera, which include at least the installation position coordinates of the camera in the vehicle coordinate system and the optical axis direction vector; Obtain the field of view parameters of the monitored area, wherein the field of view parameters include at least the horizontal field of view angle, the vertical field of view angle, and the boundary information of the effective imaging area; The installation orientation parameters of the injection assembly are obtained, including at least the installation position coordinates of the injection assembly in the vehicle coordinate system and the injection reference direction vector.

5. The method for cleaning an autonomous driving camera according to claim 2, characterized in that, Determining the target jet pressure and target jet flow rate based on the degree of image degradation includes: Obtain the first mapping parameter corresponding to different imaging degradation levels in the monitoring field of view in front of the camera, wherein the first mapping parameter includes at least the correspondence between different imaging degradation levels and injection pressure and injection flow rate; Based on the degree of imaging degradation, the corresponding target jet pressure and target jet flow rate are determined from the first mapping parameters.

6. An autonomous driving camera cleaning system, characterized in that, Use the method according to any one of claims 1 to 5.

7. An electronic device, characterized in that, include: processor; as well as, Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 5.

Images