Camera lens surface cleaning method and system and storage medium

Abstract

The embodiment of the invention provides a camera lens surface cleaning method and system and a storage medium, the method and system are applied to an automobile, the automobile comprises a camera assembly with camera cleaning equipment, and the camera assembly comprises a plurality of cameras, and the method comprises the steps that vehicle information of the automobile and image information of all the cameras are obtained, based on the trained pollution prediction model and pollutant identification model, respectively identifying the image information to obtain pollution conditions and pollutant types of each camera; when the pollutant type of the camera is the first pollutant and the current weather is abnormal, lens surface cleaning operation is carried out on each camera according to the vehicle information of the vehicle, the current weather environment and the pollution condition of each camera; and when the pollutant type of the camera is the second pollutant type or when the pollutant type of the camera is the first pollutant type and the current weather environment of the automobile is not abnormal, lens surface cleaning operation is carried out on each camera according to the vehicle information of the automobile and the pollution condition of each camera.

Description

Technical Field

[0001] The present invention relates to the technical field of cleaning equipment, and in particular to a method and system for cleaning the surface of a camera lens, and a storage medium. Background Art

[0002] With the development of the automobile industry and the advancement of science and technology, most vehicles today provide information to the driver to actively or passively assist the driver. The driving assistance system processes the information captured by the sensors on the vehicle to assist the driver's driving operations. For example, vehicle optical sensors are integrated on the outside of the car body and used as electronic driving assistance to sense objects near the vehicle. Common vehicle optical sensors include cameras integrated on the outside of the vehicle, which are used to record relevant images of the space around the vehicle. However, when the car is in use, it is inevitable to encounter various road conditions: muddy road conditions, rainy road conditions, dusty road conditions, etc. Under these road conditions, the camera located on the outside of the car will inevitably be contaminated by dirt, affecting the function of the camera and thus affecting the driver's user experience. Summary of the invention

[0003] The present application provides a camera lens surface cleaning method and system, and a storage medium, which can solve the problem in the related art that the camera lens surface is contaminated by dirt and stains, thereby affecting its function and the driver's user experience.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect, according to an embodiment of the present application, a method for cleaning a surface of a camera lens is provided, which is applied to a car, wherein the car includes a camera assembly having a camera cleaning device, wherein the camera assembly includes a plurality of cameras, and the method for cleaning the surface of the camera lens includes:

[0006] Acquire vehicle information of the vehicle; the vehicle information includes vehicle speed information and wiper switch information;

[0007] Obtain image information of each camera, and identify the image information collected by each camera based on the trained pollution prediction model to obtain the pollution status of each camera; the pollution status includes the type of pollutant and the degree of pollution;

[0008] Based on the trained pollutant identification model, the image information collected by each camera is identified to determine the type of pollutant of each camera; the pollutant type includes a first pollutant and a second pollutant;

[0009] When the pollutant type detected by the camera is the first pollutant, the image information collected by the camera is identified based on the trained weather prediction model to determine whether the current weather environment of the car is abnormal;

[0010] If the current weather environment of the car is abnormal, the camera cleaning device is used to clean the lens surface of each camera according to the vehicle information of the car, the current weather environment and the pollution condition of each camera;

[0011] When the type of pollutant in the camera is the second type of pollutant or when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is not abnormal, the camera cleaning equipment is used to clean the lens surface of each camera according to the vehicle information of the car and the pollution condition of each camera.

[0012] It can be seen from the above scheme that the embodiment of the present application adopts a combination of a pollution prediction model, a pollutant identification model and a weather prediction model, uses the pollution prediction model to identify the pollution situation of each camera, uses the pollutant identification model to identify the type of pollutant of each camera, and uses the weather prediction model to identify the current weather environment of the car. In combination with the pollution situation of each camera, the type of pollutant, the current weather environment and the vehicle information of the car, the camera cleaning equipment adopts corresponding cleaning modes to clean the lens surface of the camera according to different situations, thereby achieving effective cleaning of the lens surface of the camera, ensuring a high degree of cleanliness of the lens surface of the camera. At the same time, by combining the pollution prediction model with the pollutant identification model, the identification results of the two can also be verified, thereby improving the accuracy of identification and the efficiency of cleaning operations.

[0013] Optionally, the camera cleaning device includes:

[0014] A fixing frame, wherein a fixing frame is respectively arranged on the outer wall of the housing of each camera,

[0015] A nozzle device, wherein the nozzle device is arranged on a first surface of the fixing frame close to the lens surface of the camera along a first direction, and the nozzle of the nozzle device faces the lens surface of the camera,

[0016] A nozzle adapter, wherein the nozzle adapter is disposed on a second surface of the fixing frame away from the lens surface of the camera along the first direction, and an inlet of the nozzle adapter is connected to an outlet of a delivery pipe for delivering a cleaning medium, and an outlet of the nozzle adapter is connected to the nozzle device,

[0017] A three-way joint, the outlet end of which is connected to the inlet end of the delivery pipe,

[0018] A water storage tank, wherein the water storage tank stores cleaning liquid, and a water outlet end of the water storage tank is connected to the first inlet end of the three-way joint through a water pipe,

[0019] a first solenoid valve, wherein the first solenoid valve is arranged on the water delivery pipe,

[0020] An air storage tank, wherein the air storage tank stores compressed air, and an air outlet end of the air storage tank is connected to the second inlet end of the three-way joint through an air delivery pipe.

[0021] a second solenoid valve, wherein the second solenoid valve is arranged on the gas delivery pipe,

[0022] A filter is provided on the water pipe between the water storage tank and the first solenoid valve and on the gas pipe between the gas storage tank and the second solenoid valve.

[0023] It can be seen from the above scheme that the embodiment of the present application adopts a nozzle adapter to connect with the nozzle device, which has strong compatibility and can adapt to media in different physical states and media with different pressures. It only needs to directly replace the corresponding delivery pipe. The replacement is simple and practical. At the same time, the water storage tank and the gas storage tank are respectively connected to the nozzle device using a three-way joint, so that the nozzle device can switch the required cleaning medium according to specific needs to adapt to the cleaning mode required by the camera.

[0024] Further optionally, the first pollutant includes rain, snow, and dust, and if the weather environment is rain, snow, or dust, then the weather environment is determined to be abnormal;

[0025] The camera cleaning device is used to clean the lens surface of each camera according to the vehicle information of the car, the current weather environment and the pollution condition of each camera, including:

[0026] When the current weather environment of the automobile is rainy and snowy and the wiper switch is turned on, the second solenoid valve corresponding to the camera whose pollution degree is not greater than the pollution threshold is controlled to start, the compressed air is delivered to the nozzle device corresponding to the camera whose pollution degree is not greater than the pollution threshold, and the camera whose pollution degree is not greater than the pollution threshold is cleaned by the compressed air, and the first solenoid valve corresponding to the camera whose pollution degree is greater than the pollution threshold is controlled to start, the cleaning fluid is delivered to the nozzle device corresponding to the camera whose pollution degree is greater than the pollution threshold, and the camera whose pollution degree is greater than the pollution threshold is cleaned by the cleaning fluid;

[0027] When the current weather environment of the car is rainy or snowy and the wiper switch is not turned on, a cleaning operation prompt information is generated, and the driver chooses whether to perform a lens surface cleaning operation on each camera according to the cleaning operation prompt information; wherein the cleaning operation prompt information includes the degree of contamination of each camera;

[0028] When the current weather environment of the automobile is dusty weather, each of the first solenoid valves is controlled to start, the cleaning fluid is delivered to each of the nozzle devices, and each of the cameras is cleaned with the cleaning fluid; after a preset cleaning time, each of the first solenoid valves is closed, and each of the second solenoid valves is opened to deliver the compressed air to each of the nozzle devices, and each of the cameras is cleaned with the compressed air;

[0029] Wherein, the pressure of the cleaning fluid and the compressed air are set according to the vehicle speed information.

[0030] It can be seen from the above scheme that in the embodiment of the present application, when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is abnormal, a cleaning mode for each camera cleaning operation is formulated in combination with the vehicle information of the car, the current weather environment and the pollution condition of each camera. Then, the camera cleaning equipment performs lens surface cleaning operations on each camera according to the cleaning mode of each camera, thereby saving cleaning fluid, reducing cleaning costs, and preventing rain, snow or dust from blocking the camera's line of sight.

[0031] Further optionally, when the speed of the vehicle is 0-20 km / h, the pressure of the cleaning fluid and the compressed air is 60 kPa;

[0032] When the speed of the automobile is 20-80 km / h, the pressure of the cleaning fluid and the compressed air is 150 kPa.

[0033] It can be seen from the above scheme that the embodiment of the present application can set the pressure of the cleaning medium accordingly according to the vehicle speed information, thereby ensuring that the camera cleaning equipment can effectively and efficiently clean the lens surface of the camera, thereby avoiding the phenomenon of ineffective cleaning due to excessive vehicle speed.

[0034] Optionally, the cleaning operation of the lens surface of each camera is performed by using the camera cleaning device according to the vehicle information of the automobile and the pollution condition of each camera, including:

[0035] If the contamination degree of the camera belongs to the first contamination degree range, controlling the second solenoid valve corresponding to the camera to start, delivering the compressed air to the nozzle device corresponding to the camera, and using the compressed air to clean the camera;

[0036] If the contamination degree of the camera belongs to the second contamination degree range, the first solenoid valve corresponding to the camera is controlled to open for a preset operation time, and during the preset operation time, the cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the cleaning liquid. After the first solenoid valve is closed, the second solenoid valve corresponding to the camera is controlled to start, and the compressed air is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the compressed air;

[0037] If the contamination degree of the camera belongs to the third contamination degree range, the first solenoid valve corresponding to the camera is controlled to start, and the cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned by the cleaning liquid.

[0038] It can be seen from the above scheme that the embodiment of the present application, when the type of pollutants in the camera is the second type of pollutants or the type of pollutants in the camera is the first type of pollutants and the current weather environment of the car is not abnormal, combines the vehicle information of the car and the pollution conditions of each camera to formulate a cleaning mode for each camera cleaning operation, sets the cleaning pressure of the cleaning medium according to the speed information of the car, and sets different cleaning modes according to the degree of pollution of the camera, so as to ensure the effective removal of dirt and the high efficiency of the cleaning operation.

[0039] Optionally, the method for training the pollution prediction model includes:

[0040] Acquire a plurality of pollution training sample images taken by each of the cameras; wherein each of the pollution training sample images includes a pollution target mark; wherein the pollution target mark is set according to the type and degree of pollution of the camera;

[0041] Based on the contaminated target label, extracting the positive contaminated sample image and the negative contaminated sample image corresponding to the contaminated training sample image from the contaminated image data;

[0042] Based on the initial pollution prediction model, the pollution situation of the polluted training sample image is predicted to obtain the training pollution situation corresponding to the polluted training sample image, and the pollution situation of the positive polluted sample image and the negative polluted sample image are predicted to obtain the positive pollution situation corresponding to the positive polluted sample image and the negative pollution situation corresponding to the negative polluted sample image;

[0043] Constructing a contrastive learning loss based on the difference between the training contamination situation and the positive contamination situation and the difference between the training contamination situation and the negative contamination situation;

[0044] Based on the contrastive learning loss, the initial pollution prediction model is optimized to obtain the pollution prediction model after pre-training.

[0045] It can be seen from the above scheme that the embodiment of the present application adopts the optimization method of positive and negative pollution conditions to optimize the pollution prediction model, which can effectively improve the pollution condition prediction ability of the model and ensure the accuracy of pollution condition identification using the camera's image information.

[0046] Optionally, the method for training the pollutant identification model includes:

[0047] Acquire multiple pollution images taken by each of the cameras, pre-mark the type of pollution to which each pollution image belongs, and obtain training samples with the pollution image as an input target and the type of pollution described in the marked pollution image as a target output value;

[0048] The pollutant identification model to be trained obtained after deep learning of several training samples is used as the initial pollutant identification model;

[0049] Acquire a plurality of test samples for testing the initial pollutant identification model; wherein the test samples are test images marked with pollutant types;

[0050] Based on the initial pollutant identification model, multiple test samples are identified to obtain the pollutant type of each test sample. If the pollutant type identified by the initial pollutant identification model is the same as the pollutant type marked by the test sample, the identification result of the initial pollutant identification model is normal; otherwise, the identification result of the initial pollutant identification model is wrong.

[0051] The initial pollutant identification model is used to identify the pollutant types of several test samples, and the accuracy of the pollutant type identification by the initial pollutant identification model is calculated. If the accuracy is greater than a preset accuracy threshold, the initial pollutant identification model is used as the pollutant identification model after training. Otherwise, the initial pollutant identification model continues to be trained through test samples.

[0052] It can be seen from the above scheme that the embodiment of the present application uses the accuracy of the pollutant types identified by the training model to verify the training model. Only when the accuracy is greater than the set preset accuracy threshold, the training model is used as the pollutant identification model after training to identify the pollutant types to which the polluted image belongs. Otherwise, the initial pollutant identification model is continuously trained through training samples until the accuracy of the initial pollutant identification model in identifying the pollutant types to which the polluted image belongs is greater than the preset accuracy threshold, so as to ensure the accuracy of the pollutant identification model and improve the recognition ability of the model.

[0053] Optionally, the method for training the weather prediction model includes:

[0054] Acquire a plurality of weather forecast training sample images taken by each of the cameras; wherein each of the weather forecast training sample images includes a weather forecast target mark, and the weather forecast target mark includes a true depth value of a target pixel point;

[0055] In the process of training the initial weather prediction model based on the plurality of weather prediction training sample images, a depth prediction value of a target pixel point of each weather prediction training sample image is obtained;

[0056] According to the true depth value and the predicted depth value of the target pixel point, the target loss value of the initial weather prediction model is calculated, and the model parameters of the initial weather prediction model are adjusted according to the target loss value until the target loss value meets the preset loss condition, thereby obtaining the weather prediction model after training.

[0057] It can be seen from the above scheme that the embodiment of the present application trains the weather prediction training sample images including the depth true value of the target pixel point, and calculates the target loss value according to the depth true value and the depth prediction value output by the model, and then adjusts the model parameters of the initial weather prediction model based on the target loss value, so as to obtain a weather prediction model that meets the preset loss conditions through continuous training of the model, so as to be applied to the weather environment judgment of the car.

[0058] In a second aspect, according to an embodiment of the present application, a camera lens surface cleaning system is provided, which is used to implement the camera lens surface cleaning method as described in the first aspect, including:

[0059] An acquisition unit, used to acquire vehicle information of a vehicle and image information of each camera on the vehicle; the vehicle information includes vehicle speed information and wiper switch information;

[0060] A pollution prediction unit, used to identify the image information collected by each camera based on the trained pollution prediction model, and obtain the pollution status of each camera; the pollution status includes the type of pollutant and the degree of pollution;

[0061] A pollutant identification unit, used to identify the image information collected by each camera based on the trained pollutant identification model, and determine the type of pollutant of each camera; the pollutant type includes a first pollutant and a second pollutant;

[0062] A weather prediction unit, configured to identify the image information collected by the camera based on the trained weather prediction model when the pollutant type detected by the camera is the first pollutant, and determine whether the current weather environment of the vehicle is abnormal;

[0063] A cleaning control unit, used for, when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is abnormal, using the camera cleaning device to clean the lens surface of each camera respectively according to the vehicle information of the car, the current weather environment and the pollution condition of each camera; and used for, when the type of pollutant in the camera is the second type of pollutant or when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is not abnormal, using the camera cleaning device to clean the lens surface of each camera respectively according to the vehicle information of the car and the pollution condition of each camera.

[0064] The camera lens surface cleaning system provided in the embodiment of the present application utilizes a pollution prediction unit, a pollutant identification unit, and a weather prediction unit to respectively identify the pollution conditions of each camera, the types of pollutants, and the current weather environment of the car, so that the cleaning control unit combines the pollution conditions of each camera, the types of pollutants, the current weather environment, and the vehicle information of the car, and adopts corresponding cleaning modes for different situations to clean the camera lens surface, thereby achieving effective cleaning of the camera lens surface and ensuring a high degree of cleanliness of the camera lens surface.

[0065] In a third aspect, according to an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for cleaning the surface of a camera lens as described in the first aspect is implemented.

[0066] In a fourth aspect, according to an embodiment of the present application, an electronic device is provided, the electronic device comprising a processor and a memory;

[0067] The processor is coupled to the memory, the memory is used to store a computer program, and the computer program is executed by the processor to enable the electronic device to implement the method for cleaning the surface of a camera lens as described in the first aspect.

[0068] In a fifth aspect, according to an embodiment of the present application, a vehicle is provided, comprising a cleaning system for a camera lens surface as described in the second aspect, or comprising an electronic device as described in the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1A schematic flow chart of a method for cleaning the surface of a camera lens provided in an embodiment of the present application;

[0071] Figure 2 A block diagram of a camera cleaning device provided in an embodiment of the present application;

[0072] Figure 3 A partial structural schematic diagram of a camera equipped with a camera cleaning device provided in an embodiment of the present application;

[0073] Figure 4 for Figure 3 A partial enlarged view of

[0074] Figure 5 An exploded cross-sectional view of a fixing frame, a nozzle device and a nozzle adapter provided in the first embodiment of the present application;

[0075] Figure 6 A top view of a fixing frame provided in the first embodiment of the present application;

[0076] Figure 7 A schematic diagram of the connection between a fixing frame, a nozzle device and a nozzle adapter provided in the first embodiment of the present application;

[0077] Figure 8 An exploded cross-sectional view of a fixing frame, a nozzle device and a nozzle adapter provided in the second embodiment of the present application;

[0078] Fig. 9 A top view of a fixing frame provided in the second embodiment of the present application;

[0079] Fig.10 A schematic diagram of the connection between a fixing frame, a nozzle device and a nozzle adapter provided in the second embodiment of the present application;

[0080] Fig.11 An exploded cross-sectional view of a fixing frame, a nozzle device and a nozzle adapter provided in the third embodiment of the present application;

[0081] Fig.12 A top view of a fixing frame provided in the third embodiment of the present application;

[0082] Fig.13 A top cross-sectional view of a fixing frame provided in the third embodiment of the present application;

[0083] Fig.14 A schematic diagram of the connection between a fixing frame, a nozzle device and a nozzle adapter provided in the third embodiment of the present application;

[0084] Fig.15 A block diagram of a camera lens surface cleaning system provided in an embodiment of the present application;

[0085] Fig.16 A block diagram of the composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without paying creative work are within the scope of protection of the present application. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The terms "including" and "having" in the embodiments of the present application and the drawings and any of their variations are intended to cover non-exclusive inclusions. For example, the process, method, system, product or device of a series of steps or units included is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0087] The embodiment of the present application discloses a method for cleaning the surface of a camera lens, which can be applied to a car, wherein the car includes a camera assembly with a camera cleaning device, the camera assembly includes a plurality of cameras, and the camera cleaning device is used to clean each camera separately.

[0088] In one embodiment, if Figure 2 – Fig.14As shown, the camera cleaning device includes a fixing frame 101, a nozzle device 102, a nozzle adapter 103, a three-way joint 104, a water tank 105, a filter 109, a first solenoid valve 106, an air tank 107 and a second solenoid valve 108. The camera cleaning device is equipped with a nozzle device 102 on each camera 111 through multiple fixing frames 101. The nozzle device 102 is connected to a delivery pipe for conveying a cleaning medium through the nozzle adapter 103. The delivery pipe is connected to two cleaning medium paths through the three-way joint 104. One path is connected to the water tank 105 through the first solenoid valve 106 and the filter 109 to supply cleaning liquid to the nozzle device 102. The other path is connected to the air tank 107 through the second solenoid valve 108 and the filter 109 to supply compressed air to the nozzle device 102, so as to use the cleaning liquid or compressed air to clean the lens surface 112 of the camera 111. Among them, it is necessary to note and understand that the camera cleaning device is provided with a water tank 105 and an air tank 107, but since a plurality of cameras 111 are provided on the car, a camera 111 needs to include a set of cleaning devices including a fixing frame 101, a nozzle device 102, a nozzle adapter 103, a three-way connector 104, a first solenoid valve 106 and a second solenoid valve 108, so the camera cleaning device is provided with a plurality of cleaning devices corresponding to each camera 111, and each set of cleaning devices is respectively connected to the water tank 105 and the air tank 107, and in order to briefly and clearly indicate the connection relationship between the various components, Figure 2 Only one cleaning device is shown.

[0089] Specifically, in the first embodiment of the present application, Figure 3 As shown, the fixing frame 101 is a fixing mounting member for the nozzle device 102 on the camera 111. A fixing frame 101 is respectively arranged on the outer wall 114 of the housing 113 of each camera 111. In a specific embodiment, as shown in FIG. Figure 6 As shown, one side of the fixing bracket 101 is arranged on the outer wall 114 of the housing 113 of the camera 111, and one or more mounting ears 115 are respectively arranged on the two sides adjacent to the one side of the fixing bracket 101 and the outer wall 114, and the mounting ears 115 are fixedly mounted on the housing 113 of the camera 111 by connecting members 116 (for example, bolts), thereby fixing the fixing bracket 101 on the housing 113 of the camera 111.

[0090] like Figure 3 , Figure 5 and Figure 7As shown, the fixing frame 101 includes a first surface 117 and a second surface 118 which are arranged opposite to each other in a first direction, wherein the first surface 117 is a surface of the fixing frame 101 close to the lens surface 112 of the camera 111 along the first direction, and the second surface 118 is a surface of the fixing frame 101 away from the lens surface 112 of the camera 111 along the first direction.

[0091] In the first embodiment of the present application, Figure 3 , Figure 5 and Figure 7 As shown, the nozzle device 102 is the main component of the camera cleaning device for cleaning operations. The nozzle device 102 includes a spray port 119 for spraying for cleaning operations. The nozzle device 102 is arranged on the first surface 117 of the fixing frame 101, and the spray port 119 faces the lens surface 112 of the camera 111, so that the cleaning medium is sprayed to the lens surface 112 of the camera 111 through the spray port 119 of the nozzle device 102, and the lens surface 112 of the camera 111 is cleaned. The camera cleaning device can ensure that the dirt and water droplets on the lens surface 112 of the camera 111 are effectively cleaned by designing the spray area of ​​the spray port 119 of the nozzle device 102, thereby avoiding the limitation of the camera function due to dirt or water droplets and improving the driving safety. In a specific implementation process, as shown in FIG. Figure 4 As shown, the jet port 119 of the nozzle device 102 is a cylindrical through hole, and the spray area of ​​the jet port 119 of the nozzle device 102 can be adjusted by changing the angle between the central axis of the cylindrical through hole and the second direction to ensure that it can effectively clean the lens surface 112 of the camera 111. Among them, the first direction in the present application is the direction of the central axis of the lens 120 of the camera 111, and at the same time, the central axis of the nozzle device 102 is parallel to the central axis of the lens 120 of the camera 111, and the second direction is perpendicular to the first direction, and the second direction is the direction of the vertical line segment between the central axis of the lens 120 of the camera 111 and the central axis of the nozzle device 102. It should be noted that the vertical in the present application is not absolutely vertical, and can be 90°±5°. Similarly, parallel is not absolutely parallel, and can be 180°±5°.

[0092] In a specific embodiment, Figure 5 and Figure 7 As shown, the fixing frame 101 is provided with a through hole 121 extending in a first direction, the nozzle device 102 includes a nozzle 122, and the nozzle 122 includes a first end 123 and a second end 124, wherein the second end 124 is an end of the nozzle 122 away from the fixing frame 101, and the injection port 119 of the nozzle 122 is provided on the second end 124 of the nozzle 122, and the first end 123 is an end of the nozzle 122 without the injection port 119, and at the same time, the nozzle 122 is provided with a nozzle 119. Figure 3As shown, the jet port 119 of the nozzle 122 faces the lens surface 112 of the camera 111, and the nozzle 122 is located on the side of the camera 111 away from the ground along the second direction, so that the spraying area of ​​the jet port 119 of the nozzle 122 covers the upper part of the lens surface 112 of the camera 111 away from the ground, so that the cleaning medium sprayed by the jet port 119 of the nozzle 122 is sprayed to the upper part of the lens surface 112 of the camera 111 away from the ground, and the cleaning medium will flow along the lens surface 112 of the camera 111 to the lower part of the lens surface 112 close to the ground, so as long as the upper area of ​​the lens surface 112 of the camera 111 away from the ground is all located in the spraying area of ​​the jet port 119 of the nozzle 122, the lens surface 112 of the camera 111 can be effectively cleaned. Further, as Figure 5 and Figure 7 As shown, the first end 123 of the nozzle 122 is arranged in the through hole 121, and the shape and size of the first end 123 of the nozzle 122 are the same as the shape and size of the through hole 121. The first end 123 of the nozzle 122 is embedded in the through hole 121, so that the radial positioning of the nozzle 122 is achieved through the through hole 121 to prevent it from being displaced in the second direction.

[0093] In the first embodiment of the present application, Figure 3 As shown, the nozzle adapter 103 is the connecting medium between the nozzle 122 in the camera cleaning device and the delivery pipe for conveying the cleaning medium. The nozzle adapter 103 in the present application adopts a standard nozzle adapter, which can be connected to an 8mm tube or a larger tube, so as to realize the delivery of gas cleaning medium or liquid cleaning medium. The camera cleaning device adopts the nozzle adapter 103 to be connected with the nozzle device 102, so that it is more compatible and can be adapted to media in different physical states and media with different pressures. It is also simple to repair and replace. When there is a problem with the delivery pipe, it can be directly replaced, which is more convenient and practical.

[0094] Specifically, Figure 5 and Figure 7 As shown, the nozzle adapter 103 is arranged on the second surface 118 of the fixing frame 101, and includes an inlet 125 and an outlet 126. The inlet 125 of the nozzle adapter 103 is connected to a delivery pipe for delivering a cleaning medium, and the outlet 126 of the nozzle adapter 103 is connected to the nozzle device 102, so that the delivery pipe is connected to the nozzle device 102 through the nozzle adapter 103, and the cleaning medium is sprayed to the lens surface 112 of the camera 111 through the injection port 119 of the nozzle device 102 to clean the lens surface 112 of the camera 111.

[0095] In a specific embodiment, Figure 5As shown in FIG. 7 , the number of nozzle adapters 103 is the same as the number of nozzles 122, that is, the camera cleaning device is provided with one nozzle 122 and one nozzle adapter 103. In detail, the nozzle 122 is provided with a spray channel 127 extending along a first direction, and the injection port 119 of the nozzle device 102 is connected to the spray channel 127. The first end face 128 of the nozzle 122 close to the nozzle adapter 103 is provided with an opening 129 connecting the spray channel 127 and the outside of the nozzle 122, and the opening 129 is provided with a first internal thread 130. At the same time, the nozzle adapter 103 is a stepped cylinder, which includes a large diameter cylinder 131 and a small diameter cylinder 132 connected to each other, wherein the diameter of the small diameter cylinder 132 of the nozzle adapter 103 is the same as the diameter of the opening 129, and the outer side of the small diameter cylinder 132 of the nozzle adapter 103 A first external thread 133 matched with the first internal thread 130 is provided on the surface, and the nozzle adapter 103 and the nozzle 122 are connected through the first internal thread 130 and the first external thread 133, and the outlet 126 of the nozzle adapter 103 is connected with the spray channel 127, so that the cleaning medium flows into the spray channel 127 through the nozzle adapter 103, so as to be further sprayed from the injection port 119 of the nozzle device 102 to the lens surface 112 of the camera 111. In addition, the diameter of the large diameter cylinder 131 of the nozzle adapter 103 is larger than the diameter of the through hole 121, so that when the nozzle adapter 103 and the nozzle 122 are connected, the large diameter cylinder 131 of the nozzle adapter 103 is close to the first end face 134 of the fixing frame 101 and fits with the second surface 118 of the fixing frame 101, so as to realize the axial positioning of the nozzle adapter 103 and the axial positioning of the nozzle 122.

[0096] Further, such as Figure 3 , Figure 5 and Figure 7 As shown, the camera cleaning device also includes a first sealing ring 135, which is sleeved on the small diameter cylinder 132 of the nozzle adapter 103. When the nozzle adapter 103 and the nozzle 122 are connected, the first sealing ring 135 is arranged between the nozzle adapter 103 and the nozzle 122. The nozzle adapter 103 and the nozzle 122 are fixed by means of threads and sealing rings, which is not only convenient for disassembly and assembly, but also can ensure the firmness of fixation, and has good sealing performance, and the cleaning medium is not easy to leak, which is safer.

[0097] In another specific implementation process, such as Figure 3 , Figure 5 and Figure 7As shown, a positioning block 136 is provided on the outer surface of the nozzle 122, and a positioning groove 137 corresponding to the positioning block 136 is provided on the first surface 117 of the fixing frame 101, that is, the positioning groove 137 is provided at one end of the through hole 121 close to the first surface 117 of the fixing frame 101, and when the nozzle 122 is fixed on the fixing frame 101, the positioning block 136 is clamped in the positioning groove 137, thereby further axially positioning the nozzle 122 to ensure the accuracy of the distance between the injection port 119 of the nozzle 122 and the lens surface 112 of the camera 111.

[0098] In the first embodiment of the present application, Figure 3 , Figure 5 and Figure 7 As shown, in the first direction, the nozzle 122 is inserted into the through hole 121 of the fixing frame 101 from the side of the first surface 117 of the fixing frame 101 to realize the radial positioning of the nozzle 122, and the positioning block 136 is clamped in the positioning groove 137. At the same time, the nozzle 122 is connected to the nozzle adapter 103 through the first internal thread 130 and the first external thread 133, so that the nozzle 122 and the nozzle adapter 103 are both fixed on the fixing frame 101, and the axial positioning of the nozzle 122 is realized by the positioning block 136 being clamped in the positioning groove 137 and the first end face 134 of the large-diameter cylinder 131 of the nozzle adapter 103 being fitted with the second surface 118 of the fixing frame 101, thereby realizing the precise positioning of the nozzle 122. The device is accurately installed to ensure that the cleaning medium (gas or liquid) is transported to the nozzle 122 through the delivery pipe via the nozzle adapter 103, and is sprayed to the lens surface 112 of the camera 111 through the injection port 119 of the nozzle 122. In addition, during the design stage of the equipment, the distance between the injection port 119 of the nozzle 122 and the lens surface 112 of the camera 111, the diameter of the injection port 119 of the nozzle 122, and the angle between the central axis of the injection port 119 of the nozzle 122 and the second direction can be adjusted to set the position of the spraying area of ​​the nozzle 122 relative to the lens surface 112 of the camera 111 and the spraying area, so as to adapt to the cleaning operation of the lens 120 of the camera 111 in different positions, and the adaptation is more flexible. It should be noted and understood that adjusting the distance between the nozzle 119 of the nozzle head 122 and the lens surface 112 of the camera 111 is to adjust the installation position of the nozzle head 122 relative to the lens surface 112 of the camera 111 and the installation height of the nozzle 119 of the nozzle head 122 relative to the lens surface 112 of the camera 111.

[0099] In the second embodiment of the present application, Figure 8 – Fig.10As shown, a plurality of through holes 121 extending along a first direction are provided on the fixing frame 101, and the plurality of through holes 121 are equidistantly distributed along the circumferential direction of the lens 120 of the camera 111. Correspondingly, the nozzle device 102 includes a plurality of nozzles 122. Similarly, the number of nozzle adapters 103 is the same as the number of nozzles 122, that is, the camera cleaning device is provided with a plurality of nozzles 122 and a plurality of nozzle adapters 103, and a nozzle 122 is respectively provided in each through hole 121. The shape and size of the first end 123 of the nozzle 122 provided in the through hole 121 are the same as the shape and size of the through hole 121, and the first end 123 of the nozzle 122 is embedded in the through hole 121, and the injection port 119 of the nozzle 122 is provided at the second end 124 of the nozzle 122 away from the fixing frame 101 and facing the lens surface 112 of the camera 111.

[0100] It is important to note and understand that if Figure 5 As shown in FIG. 10 , the fixing frame 101, the nozzle device 102 and the nozzle adapter 103 in the second embodiment of the present application are identical to those in the first embodiment, except that the number of nozzles 122 and nozzle adapters 103 and the structure of the fixing frame 101 and the number of through holes 121 provided thereon are different. For matters not mentioned in the second embodiment, reference may be made to the corresponding parts in the first embodiment, which will not be repeated here.

[0101] In the third embodiment of the present application, Fig.11 – Fig.14 As shown, the fixing frame 101 is annularly arranged on the outer wall 114 of the housing 113 of the camera 111, and a plurality of mounting holes 138 extending along the first direction are arranged on the first surface 117 of the fixing frame 101. The plurality of mounting holes 138 are equally spaced along the circumferential direction of the lens 120 of the camera 111, and a coherent channel 139 is arranged in the fixing frame 101. The coherent channel 139 is an annular shape having the same center as the fixing frame 101. The nozzle device 102 includes a plurality of nozzles 122. A nozzle 122 is respectively arranged in each mounting hole 138. The shape and size of the first end 123 of the nozzle 122 arranged in the mounting hole 138 are similar to the first end 123 of the nozzle 122. The mounting holes 138 have the same shape and size. A second external thread 140 is provided on the outer surface of the first end 123 of the nozzle 122. A second internal thread 141 matching the second external thread 140 is provided in the mounting hole 138. The nozzle 122 is fixed in the mounting hole 138 through the second external thread 140 and the second internal thread 141. Thus, the first end 123 of the nozzle 122 is embedded in the mounting hole 138 to position the nozzle 122 in the radial direction. The nozzle 122 is fixed to the fixing frame 101 by the threaded connection between the second external thread 140 and the second internal thread 141 to position the nozzle 122 in the axial direction. At the same time, Fig.14As shown, each spray head 122 is provided with a spray channel 127 extending along the first direction, the spray channel 127 is connected to the connecting channel 139 , and the injection port 119 is connected to the spray channel 127 .

[0102] Further, such as Fig.11 , Fig.12 and Fig.14 As shown, a positioning block 136 is provided on the outer surface of the nozzle 122, and a positioning groove 137 corresponding to the positioning block 136 is provided on the first surface 117 of the fixing frame 101, that is, the positioning groove 137 is provided at one end of the mounting hole 138 close to the first surface 117 of the fixing frame 101. When the nozzle 122 is fixed on the fixing frame 101, the positioning block 136 is clamped in the positioning groove 137, thereby further axially positioning the nozzle 122 to ensure the accuracy of the distance between the injection port 119 of the nozzle 122 and the lens surface 112 of the camera 111.

[0103] In a specific implementation process, Fig.11 and Fig.14 As shown, a second sealing ring 142 is provided in the mounting hole 138. When the nozzle 122 is fixed in the mounting hole 138, the second sealing ring 142 is provided between the first end surface 134 of the nozzle 122 close to the nozzle adapter 103 and the mounting hole 138 to seal the nozzle 122 and the fixing frame 101, thereby ensuring the firmness of the connection between the nozzle 122 and the fixing frame 101 and preventing the cleaning medium from leaking.

[0104] In the third embodiment of the present application, Fig.11 and Fig.14 As shown, a fixing hole 143 is provided on the second surface 118 of the fixing frame 101, and a third internal thread 144 is provided in the fixing hole 143. At the same time, the nozzle adapter 103 is a stepped cylinder. The diameter of the small diameter cylinder 132 of the nozzle adapter 103 is the same as the diameter of the fixing hole 143, and a third external thread 145 adapted to the third internal thread 144 is provided on the outer surface of the small diameter cylinder 132 of the nozzle adapter 103. The nozzle adapter 103 is fixed to the fixing frame 101 through the third internal thread 144 and the third external thread 145. At the same time, the nozzle adapter The diameter of the large-diameter cylinder 131 of the head 103 is larger than the diameter of the fixing hole 143. The large-diameter cylinder 131 of the nozzle adapter 103 is close to the first end face 134 of the fixing frame 101 and is in contact with the second surface 118 of the fixing frame 101, and the outlet 126 of the nozzle adapter 103 is connected to the connecting channel 139, so that the cleaning medium enters the connecting channel 139 of the fixing frame 101 through the nozzle adapter 103, and is then transported to the spray channel 127 of each nozzle 122 through the connecting channel 139, thereby realizing the structural design of one nozzle adapter 103 corresponding to multiple nozzles 122.

[0105] Furthermore, a third sealing ring 146 is sleeved on the small diameter cylinder 132 of the nozzle adapter 103. When the nozzle adapter 103 is fixed on the fixing frame 101, the third sealing ring 146 is arranged between the nozzle adapter 103 and the fixing frame 101 to seal the nozzle adapter 103 and the fixing frame 101. Similarly, the firmness of the connection between the nozzle adapter 103 and the fixing frame 101 is ensured, and the cleaning medium is prevented from leaking.

[0106] The three design structures of the fixing frame, the nozzle device and the nozzle adapter are introduced above through the first embodiment, the second embodiment and the third embodiment. In addition, in other embodiments, the nozzle device 102 can be fixed on the fixing frame 101 by a rotating mechanism, and the cleaning medium is delivered to the nozzle device 102 through the nozzle adapter 103. The nozzle device 102 is rotated and moved on the fixing frame 101 by the rotating mechanism to realize rotation around the lens surface 112 of the camera 111, so as to perform circular spray cleaning on the lens surface 112 of the camera 111.

[0107] In one embodiment, if Figure 1 As shown, the water tank 105 and the gas tank 107 are used as components for storing cleaning media in the camera cleaning device, wherein the water tank 105 is used to store liquid cleaning media (i.e., cleaning liquid), the gas tank 107 is used to store gas cleaning media (i.e., compressed air), and the three-way joint 104 is used as a flow dividing and connecting component of the nozzle device 102, so that the nozzle device 102 is connected to the water tank 105 and the gas tank 107 respectively. Specifically, the outlet end of the three-way joint 104 is connected to the inlet of the delivery pipe, and the water outlet end of the water tank 105 is connected to the three-way joint 104 through the water delivery pipe. The first inlet end of the head 104 is connected, and the outlet end of the gas tank 107 is connected to the second inlet end of the three-way joint 104 through the gas pipe. In addition, a first solenoid valve 106 is provided on the water pipe, and the first solenoid valve 106 is used to control the delivery of the cleaning liquid so that the cleaning liquid is delivered according to the required flow rate, outlet pressure, and cleaning time. Similarly, a second solenoid valve 108 is provided on the gas pipe, and the second solenoid valve 108 is used to control the delivery of compressed air so that the compressed air is delivered according to the required flow rate, outlet pressure, and cleaning time. Further, the camera cleaning device is also provided with a filter 109. A filter 109 is provided on the water pipe between the water tank 105 and the first solenoid valve 106 and on the gas pipe between the gas tank 107 and the second solenoid valve 108 to ensure the cleanliness of the delivered cleaning medium, avoid the phenomenon of pipeline blockage or valve body damage caused by impurities in the cleaning medium, improve the service life of the first solenoid valve 106 and the second solenoid valve 108, and also ensure the safety and stability of the cleaning medium delivery.

[0108] In the embodiments of the present application, Figure 1 As shown, the method for cleaning the surface of the camera lens may include the following steps:

[0109] S110: Obtain vehicle information of the car.

[0110] Among them, the vehicle information includes but is not limited to vehicle speed information and wiper switch information. The vehicle speed information specifically refers to the gear information of the car and the current driving speed. The wiper switch information specifically refers to whether the wiper switch is turned on.

[0111] S120: Obtain image information of each camera, and identify the image information collected by each camera based on the trained pollution prediction model to obtain the pollution status of each camera.

[0112] In one embodiment, the image information captured by each camera is obtained respectively, and the image information captured by each camera is used as the input of the pollution prediction model after training, and the pollution situation of each camera is output, wherein the pollution situation includes the type of pollutant and the degree of pollution. In a specific embodiment, the type of pollutant may include rain, snow, dust, oil stains, sludge, flying catkins, etc. The degree of pollution is set according to factors such as the area occupied by the pollutant on the camera. For example, the larger the area occupied by the pollutant, the greater the degree of pollution.

[0113] The following is an introduction to the method of training the pollution prediction model:

[0114] a. Obtain multiple pollution training sample images taken by each camera.

[0115] In the embodiment of the present application, the pollution training sample image is an image obtained by taking pictures of non-predetermined scenes using various cameras to be tested on the car. Each pollution training sample image includes a pollution target mark, and the pollution target mark is set according to the type and degree of pollution of the camera. The specific setting method includes but is not limited to: weighted processing based on the type and degree of pollution. In a specific embodiment, the weighted calculation formula satisfies:

[0116] Dirt=αT+βW

[0117] Wherein, T represents the pollutant type, W represents the pollution degree, α and β represent the weight coefficients of the pollutant type and pollution degree respectively. The above formula indicates that the pollution conditions of different pollutant types can be represented by assigning values.

[0118] It should be noted that the obtained multiple pollution training sample images should contain all pollutant types as much as possible, and the pollutant type in each pollution training sample image can be any one or a combination of several.

[0119] b. Based on the contaminated target label, the positive contaminated sample images and negative contaminated sample images corresponding to the contaminated training sample images are extracted from the contaminated image data.

[0120] Among them, there are a large number of polluted sample images in the polluted image data, each of which has a pollution target label set according to the pollutant type and pollution degree. The positive polluted sample image corresponding to the polluted training sample image refers to an image with a pollutant type and pollution degree similar to that of the polluted training sample image, and the negative polluted sample image refers to an image with a pollutant type and pollution degree not similar to that of the polluted training sample image.

[0121] c. Based on the initial pollution prediction model, the pollution situation of the polluted training sample image is predicted to obtain the training pollution situation corresponding to the polluted training sample image, and the pollution situation of the positive polluted sample image and the negative polluted sample image are predicted respectively to obtain the positive pollution situation corresponding to the positive polluted sample image and the negative pollution situation corresponding to the negative polluted sample image.

[0122] In one embodiment, the contaminated training sample image, the positive contaminated sample image and the negative contaminated sample image are respectively used as the input of the initial contamination prediction model to obtain the output of the initial contamination prediction model: training contamination situation, positive pollution situation and negative pollution situation. In the specific implementation process, the pollution situation is represented by an assignment method, that is, the training contamination situation, positive pollution situation and negative pollution situation output by the initial contamination prediction model can be a specific value.

[0123] d. Construct a contrastive learning loss based on the difference between the training contamination and the positive contamination, and the difference between the training contamination and the negative contamination.

[0124] In one embodiment, the formula for calculating contrastive learning loss satisfies:

[0125]

[0126] Among them, L is the contrastive learning loss, Dirt0 is the training pollution situation, Dirt1 is the positive pollution situation, and Dirt 2i is the negative example contamination situation, and M is the number of negative example contaminated sample images.

[0127] e. Based on contrastive learning loss, the initial pollution prediction model is optimized to obtain the pollution prediction model after pre-training.

[0128] According to the above formula for calculating the contrastive learning loss, when the distance between the positive pollution situation and the training pollution situation is small enough, and the distance between each negative pollution situation and the training pollution situation is large enough, the contrastive learning loss can converge, and then the pollution prediction model updated based on the contrastive learning loss has the ability to tighten the distance between the training pollution situation and the positive pollution situation as the positive example, and to increase the distance between the training pollution situation and the negative pollution situation as the negative example. Then, the pollution prediction model can generate different pollution situations based on sample images of different sample image types (positive or negative), so that the generated pollution situation has the sample image category information, improves the ability to predict the pollution situation, and ensures the accuracy of the camera's image information in identifying the pollution situation.

[0129] S130: Identify the image information collected by each camera based on the trained pollutant identification model to determine whether the pollutant type of each camera is the first pollutant.

[0130] In one embodiment, the image information captured by each camera is used as the input of a pollutant identification model after training, and the pollutant type of each camera is output, wherein the pollutant identification model is a model trained using pollution images that are pre-labeled with the type of pollutant to which they belong as samples for deep learning, and the pollutant types include a first pollutant and a second pollutant. In a specific embodiment, the first pollutant is rain, snow, and dust, and the second pollutant is other pollutants except rain, snow, and dust.

[0131] The following is an introduction to the method of training the pollutant identification model:

[0132] A. Obtain multiple pollution images taken by various cameras, pre-mark the type of pollutants to which each pollution image belongs, and obtain training samples with the pollution image as the input target and the pollutant type of the marked pollution image as the target output value.

[0133] In the embodiment of the present application, the pollution image is an image taken by each camera to be tested on the car, and the pollution image is an image whose pollutant type has been marked. It should be noted and understood that the multiple pollution images obtained should include all the first pollutants and as many pollutant types of the second pollutants as possible. The pollutant type in each pollution image can be any one or a combination of several. When the pollutant type in the pollution image includes one or two of the pollutant types of the first pollutant, the pollutant type to which the pollution image belongs is determined to be the first pollutant. When the pollutant type in the pollution image includes one or two of the pollutant types of the second pollutant, the pollutant type to which the pollution image belongs is determined to be the second pollutant. When the pollutant type in the pollution image includes both the pollutant type of the first pollutant and the pollutant type of the second pollutant, the pollutant type to which the pollution image belongs is determined to be the first pollutant.

[0134] B. The pollutant identification model to be trained obtained after deep learning of several training samples is used as the initial pollutant identification model.

[0135] The initial pollutant identification model is a model obtained by deep learning using polluted images that have been marked with the type of pollutants they belong to as training samples. The initial pollutant identification model trained with a large number of samples can accurately identify the type of pollutants to which the polluted image belongs. It should be understood that the pollutant identification model determines the type of pollutant to which the polluted image belongs by identifying the type of pollutant to which it belongs.

[0136] C. Obtain multiple test samples for testing the initial pollutant identification model.

[0137] The detection samples are detection images marked with pollutant types.

[0138] D. Based on the initial pollutant identification model, multiple test samples are identified to obtain the pollutant type of each test sample. If the pollutant type identified by the initial pollutant identification model is the same as the pollutant type marked by the test sample, the identification result of the initial pollutant identification model is normal; otherwise, the identification result of the initial pollutant identification model is wrong.

[0139] The test sample is used as the input of the initial pollutant identification model, and the output is the type of pollutant to which the test sample belongs. The obtained type of pollutant to which the test sample belongs is compared with the marked type of pollutant. If the two are the same, the identification result of the initial pollutant identification model is determined to be correct. If the two are different, the identification result of the initial pollutant identification model is determined to be wrong.

[0140] E. The initial pollutant identification model identifies the pollutant types of several test samples, and calculates the accuracy of the initial pollutant identification model in identifying the pollutant types. If the accuracy is greater than the preset accuracy threshold, the initial pollutant identification model is used as the pollutant identification model after training. Otherwise, the initial pollutant identification model continues to be trained through test samples.

[0141] In the embodiment of the present application, in order to further ensure the accuracy of the model in identifying the type of pollutants to which the polluted image belongs, after obtaining the training model (i.e., the initial pollutant identification model) through deep learning, the accuracy of the type of pollutants identified by the training model is calculated. Only when the accuracy is greater than the preset accuracy threshold, the training model is used as the pollutant identification model after training to identify the type of pollutants to which the polluted image belongs. Otherwise, the initial pollutant identification model obtained continues to be trained through training samples until the accuracy of the initial pollutant identification model in identifying the type of pollutants to which the polluted image belongs is greater than the preset accuracy threshold. Among them, the accuracy of the type of pollutants identified by the training model is the ratio of the number of test samples with correct identification results of the training model to the total number of test samples.

[0142] In an embodiment of the present application, the method for cleaning the surface of the camera lens uses a pollution prediction model and a pollutant identification model to respectively identify the pollution conditions and types of pollutants of the camera. The pollution prediction model is used to identify the type and degree of pollution of the camera, while the pollutant identification model focuses on identifying the type of pollutants (i.e., pollutant type) of the camera. Using two different models for identification can avoid the problem of large errors caused by one model identification and ensure the accuracy of identification. In the specific implementation process, the results of the pollutant type identified by the pollution prediction model and the pollutant identification model can also be used to verify the results of the two model identifications. If the two identification results are the same, it is determined that the identification of the two models is correct. If the two identification results are different, it is determined that one or both of the two models are incorrectly identified and need to be re-identified, thereby further improving the accuracy of model identification.

[0143] When the type of the pollutant in the camera is the first type of pollutant, the process proceeds from step S130 to step S140 ; ​​when the type of the pollutant in the camera is the second type of pollutant, the process proceeds from step S130 to step S160 .

[0144] S140: Identify the image information collected by the camera based on the trained weather prediction model to determine whether the current weather environment of the car is abnormal.

[0145] When the camera's pollutant type is identified as the first pollutant based on the trained pollutant identification model, step S130 is used to enter step S140 to further identify the image information collected by the camera, and to determine whether the current weather environment of the car is abnormal based on the trained weather prediction model, that is, to determine whether the pollutants on the camera are caused by the current weather environment around the car. For example, when the camera's pollutant type is the first pollutant, it is determined whether the current weather environment is rainy, snowy or dusty. If so, it is determined that the current weather environment of the car is abnormal.

[0146] The following is an introduction to the method of training the weather prediction model:

[0147] 1) Obtain multiple weather forecast training sample images taken by each camera.

[0148] Each weather forecast training sample image includes a weather forecast target tag, and the weather forecast target tag includes the depth truth value of the target pixel point, and the depth truth value refers to the distance from the camera to the object in the weather forecast training sample image. In the embodiment of the present application, the depth truth value can be manually added to the weather forecast training sample image by manual measurement, that is, marked, or added automatically to improve the efficiency of depth truth value marking.

[0149] The method for automatically adding the true depth value includes: obtaining multiple weather forecast training sample images taken by the camera and the radar point cloud corresponding to each weather forecast training sample image, fitting the target plane in the radar point cloud, mapping the radar point cloud in the target plane to the corresponding weather forecast training sample image, obtaining the true depth value of each target pixel in the weather forecast training sample image, and marking it accordingly, wherein the target plane is the plane obtained when fitting points with the same characteristics in the radar point cloud. Among them, the camera lens can be a monocular lens, and the correspondence between the weather forecast training sample image and the radar point cloud can be determined according to the acquisition time, that is, when the acquisition time is the same or the acquisition time difference is ≦ a preset time difference threshold, it is determined that the weather forecast training sample image and the radar point cloud have a corresponding relationship.

[0150] In the specific implementation process, when marking the depth truth value of the weather forecast training sample image, the depth truth value of only the polluted area can be marked, or the depth truth value of the entire weather forecast training sample image can be marked, so the target pixel point can only include the pixel point of the polluted area, or any pixel point in the weather forecast training sample image. When the target pixel point includes any pixel point in the weather forecast training sample image, the radar point cloud can be directly mapped to the corresponding weather forecast training sample image. When the target pixel point only includes the pixel point of the polluted area, the target plane can be fitted first, and then the radar point cloud in the target plane can be mapped to the corresponding weather forecast training sample image.

[0151] Among them, the method of mapping the radar point cloud to the weather forecast training sample image includes: converting the radar point cloud from the radar coordinate system to the image coordinate system according to the laser radar calibration parameters. After mapping the radar point cloud to the weather forecast training sample image, there may be a situation where the mapping points jump out of the weather forecast training sample image or there are vacancies in the weather forecast training sample image. Therefore, the mapping points can be cropped according to the size of the weather forecast training sample image to solve the problem of mapping point jumping out, and the mapping points can be added by performing nearest neighbor interpolation on the mapping points to solve the problem of vacancies in mapping points and sparse mapping points. In addition, the method of fitting the target plane includes but is not limited to the RANSAC (Random Sample Consensus) algorithm.

[0152] 2) In the process of training the initial weather forecast model based on multiple weather forecast training sample images, the depth prediction value of the target pixel point of each weather forecast training sample image is obtained.

[0153] The weather forecast training sample image is used as the input of the initial weather forecast model to obtain the depth prediction value of the target pixel point of the weather forecast training sample image, that is, the output result of the initial weather forecast model.

[0154] 3) According to the true depth value and the predicted depth value of the target pixel point, the target loss value of the initial weather prediction model is calculated, and the model parameters of the initial weather prediction model are adjusted according to the target loss value until the target loss value meets the preset loss condition, and the weather prediction model after training is obtained.

[0155] There are many loss functions for calculating the target loss value, for example:

[0156]

[0157] Among them, loss represents the target loss value, and d represents the difference between the true depth value and the predicted depth value.

[0158] The preset loss condition may be that the target loss value is less than a preset loss threshold, or that the target loss value of the loss function reaches a minimum value.

[0159] By training the weather prediction training sample images including the true depth value of the target pixel point, and calculating the target loss value according to the true depth value and the depth prediction value output by the model, and then adjusting the model parameters of the initial weather prediction model based on the target loss value, a weather prediction model that meets the preset loss conditions is obtained through continuous training of the model, so as to be applied to the judgment of the weather environment of the car, and by judging the distance between the rain, snow or sand and dust on the weather prediction training sample image and the lens surface of the camera, it is judged whether the rain, snow or sand and dust in the weather prediction training sample image taken are all on the lens surface of the camera. If all are located on the lens surface of the camera, it is judged that the current weather environment of the car is not abnormal, otherwise it is judged that the current weather environment of the car is abnormal. In this embodiment, a high-quality weather prediction model can be trained only by relying on images taken by a monocular lens, which is not only accurate but also low in cost.

[0160] If the current weather environment of the car is abnormal, the process proceeds from step S140 to step S150 ; if the current weather environment of the car is not abnormal, the process proceeds from step S140 to step S160 .

[0161] S150: Based on the vehicle information, current weather conditions and pollution conditions of each camera, the camera cleaning device is used to clean the lens surface of each camera.

[0162] When the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is abnormal, a cleaning mode for each camera cleaning operation is formulated in combination with the vehicle information of the car, the current weather environment and the pollution condition of each camera, and then the camera cleaning equipment performs lens surface cleaning operations on each camera according to the cleaning mode of each camera.

[0163] In a specific embodiment, when the current weather environment of the car is rainy and snowy, the wiper switch information in the vehicle information is extracted. If the obtained wiper switch information shows that the wiper switch is on, it means that the current rain and snow have affected the driver's driving operation. Therefore, in this case, the camera lens surface also needs to be cleaned to avoid rain and snow blocking the camera lens, affecting the camera's function, and further affecting the driver's driving safety. However, a corresponding cleaning mode needs to be formulated according to the degree of contamination of the camera. When the degree of contamination of the camera is not greater than a preset contamination threshold, the gas cleaning mode is turned on, that is, the second solenoid valve corresponding to the camera is controlled to start, and compressed air is delivered to the nozzle device corresponding to the camera, and then the camera is cleaned with compressed air. While removing the contaminated stains, the cleanliness of the camera lens surface is ensured to prevent rain and snow from dripping on the camera lens surface and affecting the camera function; when the degree of contamination of the camera is greater than the preset contamination threshold, the liquid cleaning mode is turned on, that is, the first solenoid valve corresponding to the camera is controlled to start, and cleaning liquid is delivered to the nozzle device corresponding to the camera, and then the cleaning liquid is used to clean the contaminated stains on the camera lens surface until the degree of contamination of the camera is no greater than the preset contamination threshold. When the degree of contamination of the camera in the liquid cleaning mode is no greater than the preset contamination threshold, the liquid cleaning mode is changed to the gas cleaning mode to save cleaning liquid and reduce cleaning costs. At the same time, compressed air is used to prevent rain and snow from dripping on the camera lens surface and blocking the camera's line of sight as much as possible.

[0164] If the acquired wiper switch information indicates that the wiper switch is not turned on, it means that the current rain and snow have not affected the driver's driving operation, and there is no need to consider the problem of rain and snow blocking the camera's line of sight. In this case, cleaning operation prompt information can be generated, and the driver chooses whether to clean the lens surface of each camera according to the cleaning operation prompt information. The cleaning operation prompt information includes the degree of pollution of each camera, that is, the driver can decide whether to start the cleaning operation according to the degree of pollution of the camera and the current driving environment, for the driver to choose independently. In addition, the cleaning operation prompt information may also include cleaning operation suggestions for each camera. If the driver does not make any choice within the preset time threshold, the operation is performed with the default settings. For example, when the driver does not make any choice within the preset time threshold, each camera is cleaned according to the cleaning operation suggestions of each camera. The cleaning operation of each camera can be set according to the degree of pollution of the camera and the current driving environment.

[0165] When the current weather environment of the car is dusty, it is necessary to avoid dust blocking the camera's line of sight. In the specific implementation process, each camera can first turn on the liquid cleaning mode to remove the dust and dirt on the camera lens surface. After the preset cleaning time, turn off the liquid cleaning mode, and turn on the gas cleaning mode to use compressed air to clean the camera lens surface to avoid dust falling on the camera lens surface and blocking the camera's line of sight. Specifically, control each first solenoid valve to start, deliver cleaning liquid to each nozzle device, and use the cleaning liquid to clean each camera. After the preset cleaning time, turn off each first solenoid valve, and turn on each second solenoid valve to deliver compressed air to each nozzle device to use compressed air to clean each camera. The mode of first liquid cleaning and then gas cleaning can not only effectively clean the camera lens surface and ensure that the camera lens surface has a high degree of cleanliness, thereby ensuring the function of each camera and the driver's user experience, but also save cleaning fluid and reduce costs.

[0166] In the embodiment of the present application, the pressure of the cleaning liquid in the above-mentioned liquid cleaning mode and the pressure of the compressed air in the gas cleaning mode can be set accordingly according to the vehicle speed information. In a specific implementation process, when the vehicle speed is 0-20km / h, the pressure of the cleaning liquid and compressed air is 60kpa; when the vehicle speed is 20-80km / h, the pressure of the cleaning liquid and compressed air is 150kpa, so as to ensure that the camera cleaning device can effectively and efficiently clean the lens surface of the camera, and avoid the phenomenon of ineffective cleaning due to excessive speed.

[0167] S160: Based on the vehicle information of the automobile and the pollution condition of each camera, the lens surface of each camera is cleaned by using a camera cleaning device.

[0168] When the camera's pollutant type is the second pollutant or when the camera's pollutant type is the first pollutant and the current weather environment of the car is not abnormal, a cleaning mode for each camera cleaning operation is formulated based on the vehicle information of the car and the pollution conditions of each camera.

[0169] In a specific implementation process, different cleaning modes can be set according to the different pollution levels of the camera. Specifically, the pollution level of the camera can be specifically divided into a first pollution level range, a second pollution level range and a third pollution level range. The pollution level of the third pollution level range is the heaviest, the pollution level of the second pollution level range is the second, and the pollution level of the first pollution level range is the lightest. If the pollution level of the camera belongs to the first pollution level range, the gas cleaning mode is selected, that is, the second solenoid valve corresponding to the camera is controlled to start, and compressed air is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with compressed air; if the pollution level of the camera belongs to the second pollution level range, a mode combining liquid cleaning and gas cleaning is selected, that is, the first solenoid valve corresponding to the camera is controlled to start for a preset operation time, and within the preset operation time, cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the cleaning liquid. After the first solenoid valve is closed, the second solenoid valve corresponding to the camera is controlled to start, and compressed air is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with compressed air; if the pollution level of the camera belongs to the third pollution level range, the liquid cleaning mode is selected, that is, the first solenoid valve corresponding to the camera is controlled to start, and cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the cleaning liquid. The preset operation time may be set according to the degree of contamination of the camera. For example, the heavier the degree of contamination of the camera, the longer the preset operation time.

[0170] In addition, in an embodiment of the present application, the camera lens surface cleaning method can monitor the degree of contamination of the camera in real time, and adjust the camera cleaning mode accordingly or shut down the camera cleaning equipment to clean the camera according to the degree of contamination of the camera after the cleaning operation.

[0171] The camera lens surface cleaning method provided in the embodiment of the present application adopts a combination of a pollution prediction model, a pollutant identification model and a weather prediction model to respectively identify the pollution conditions of each camera, the type of pollutants and the current weather environment of the car. In combination with the pollution conditions of each camera, the type of pollutants, the current weather environment and the vehicle information of the car, the camera cleaning equipment adopts corresponding cleaning modes to clean the camera lens surface according to different situations, thereby achieving effective cleaning of the camera lens surface and ensuring a high degree of cleanliness of the camera lens surface. At the same time, the combination of the pollution prediction model and the pollutant identification model can also verify the identification results of the two, thereby improving the accuracy of identification and the efficiency of cleaning operations.

[0172] Corresponding to the above-mentioned camera lens surface cleaning method embodiment, another embodiment of the present application provides a camera lens surface cleaning system, such as Fig.15 As shown, the cleaning system includes:

[0173] The acquisition unit 210 is used to acquire vehicle information of the vehicle and image information of each camera on the vehicle, wherein the vehicle information includes vehicle speed information and wiper switch information;

[0174] The pollution prediction unit 220 is used to identify the image information collected by each camera based on the trained pollution prediction model to obtain the pollution situation of each camera, wherein the pollution situation includes the type of pollutants and the degree of pollution;

[0175] The pollutant identification unit 230 is used to identify the image information collected by each camera based on the trained pollutant identification model, and determine the type of pollutant of each camera, wherein the pollutant type includes a first pollutant and a second pollutant;

[0176] The weather prediction unit 240 is used to identify the image information collected by the camera based on the trained weather prediction model when the pollutant type detected by the camera is the first pollutant, and determine whether the current weather environment of the vehicle is abnormal;

[0177] The cleaning control unit 250 is used to perform lens surface cleaning operations on each camera using a camera cleaning device according to vehicle information of the car, current weather environment and pollution conditions of each camera when the type of pollutants in the camera is the first type of pollutants and the current weather environment of the car is abnormal; and is used to perform lens surface cleaning operations on each camera using a camera cleaning device according to vehicle information of the car and pollution conditions of each camera when the type of pollutants in the camera is the second type of pollutants or when the type of pollutants in the camera is the first type of pollutants and the current weather environment of the car is not abnormal.

[0178] The camera lens surface cleaning system provided in the embodiment of the present application utilizes a pollution prediction unit, a pollutant identification unit, and a weather prediction unit to respectively identify the pollution conditions of each camera, the types of pollutants, and the current weather environment of the car, so that the cleaning control unit combines the pollution conditions of each camera, the types of pollutants, the current weather environment, and the vehicle information of the car, and adopts corresponding cleaning modes for different situations to clean the camera lens surface, thereby achieving effective cleaning of the camera lens surface and ensuring a high degree of cleanliness of the camera lens surface.

[0179] Based on the above method embodiment, another embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the method for cleaning the surface of a camera lens as described in the above embodiment is implemented.

[0180] Based on the above method embodiment, another embodiment of the present application provides an electronic device, such as Fig.16As shown, the electronic device includes a processor 310 and a memory 320. The processor 310 is coupled to the memory 320. The memory 320 is used to store a computer program. The computer program is executed by the processor 310 to enable the electronic device to implement the camera lens surface cleaning method described in the above embodiment.

[0181] Based on the above method embodiment, another embodiment of the present application provides a vehicle, which includes the cleaning system for the surface of a camera lens as described in the above embodiment, or includes the electronic device as described in the above embodiment.

[0182] The above-mentioned camera lens surface cleaning system embodiment corresponds to the method embodiment, and has the same technical effect as the camera lens surface cleaning method embodiment. For specific descriptions, please refer to the method embodiment. The camera lens surface cleaning system embodiment is obtained based on the camera lens surface cleaning method embodiment. For specific descriptions, please refer to the camera lens surface cleaning method embodiment part, which will not be repeated here. A person of ordinary skill in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0183] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for cleaning the surface of a camera lens, characterized in that: Applied in an automobile, the automobile includes a camera assembly with a camera cleaning device, the camera assembly includes a plurality of cameras, and the method for cleaning the surface of the camera lens includes: Acquire vehicle information of the vehicle; the vehicle information includes vehicle speed information and wiper switch information; Obtain image information of each camera, and identify the image information collected by each camera based on the trained pollution prediction model to obtain the pollution status of each camera; the pollution status includes the type of pollutant and the degree of pollution; Based on the trained pollutant identification model, the image information collected by each camera is identified to determine the type of pollutant of each camera; the pollutant type includes a first pollutant and a second pollutant; When the pollutant type detected by the camera is the first pollutant, the image information collected by the camera is identified based on the trained weather prediction model to determine whether the current weather environment of the car is abnormal; If the current weather environment of the car is abnormal, the camera cleaning device is used to clean the lens surface of each camera according to the vehicle information of the car, the current weather environment and the pollution condition of each camera; When the type of pollutant in the camera is the second type of pollutant or when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is not abnormal, the camera cleaning equipment is used to clean the lens surface of each camera according to the vehicle information of the car and the pollution condition of each camera.

2. The method for cleaning the surface of a camera lens according to claim 1, characterized in that: The camera cleaning device comprises: A fixing frame, wherein a fixing frame is respectively arranged on the outer wall of the housing of each camera, a nozzle device, wherein the nozzle device is disposed on a first surface of the fixing frame close to the lens surface of the camera along a first direction, and a nozzle of the nozzle device faces the lens surface of the camera, A nozzle adapter, wherein the nozzle adapter is disposed on a second surface of the fixing frame away from the lens surface of the camera along the first direction, and an inlet of the nozzle adapter is connected to an outlet of a delivery pipe for delivering a cleaning medium, and an outlet of the nozzle adapter is connected to the nozzle device, A three-way joint, the outlet end of which is connected to the inlet end of the delivery pipe, A water storage tank, wherein the water storage tank stores cleaning liquid, and a water outlet end of the water storage tank is connected to the first inlet end of the three-way joint through a water pipe, a first solenoid valve, wherein the first solenoid valve is arranged on the water delivery pipe, An air storage tank, wherein the air storage tank stores compressed air, and an air outlet end of the air storage tank is connected to the second inlet end of the three-way joint through an air delivery pipe. a second solenoid valve, wherein the second solenoid valve is arranged on the gas delivery pipe, A filter is provided on the water pipe between the water storage tank and the first solenoid valve and on the gas pipe between the gas storage tank and the second solenoid valve.

3. The method for cleaning the surface of a camera lens according to claim 2, characterized in that: The first type of pollutants include rain, snow, and dust. If the weather environment is rain, snow, or dust, then the weather environment is determined to be abnormal. The camera cleaning device is used to clean the lens surface of each camera according to the vehicle information of the car, the current weather environment and the pollution condition of each camera, including: When the current weather environment of the automobile is rainy and snowy and the wiper switch is turned on, the second solenoid valve corresponding to the camera whose pollution degree is not greater than the pollution threshold is controlled to start, the compressed air is delivered to the nozzle device corresponding to the camera whose pollution degree is not greater than the pollution threshold, and the camera whose pollution degree is not greater than the pollution threshold is cleaned by the compressed air, and the first solenoid valve corresponding to the camera whose pollution degree is greater than the pollution threshold is controlled to start, the cleaning fluid is delivered to the nozzle device corresponding to the camera whose pollution degree is greater than the pollution threshold, and the camera whose pollution degree is greater than the pollution threshold is cleaned by the cleaning fluid; When the current weather environment of the automobile is rainy or snowy and the wiper switch is not turned on, a cleaning operation prompt information is generated, and the driver chooses whether to perform a lens surface cleaning operation on each of the cameras according to the cleaning operation prompt information; wherein the cleaning operation prompt information includes the degree of contamination of each of the cameras; When the current weather environment of the automobile is dusty weather, each of the first solenoid valves is controlled to start, the cleaning fluid is delivered to each of the nozzle devices, and each of the cameras is cleaned with the cleaning fluid; after a preset cleaning time, each of the first solenoid valves is closed, and each of the second solenoid valves is opened to deliver the compressed air to each of the nozzle devices, and each of the cameras is cleaned with the compressed air; Wherein, the pressure of the cleaning fluid and the compressed air are set according to the vehicle speed information.

4. The method for cleaning the surface of a camera lens according to claim 3, characterized in that: When the speed of the vehicle is 0–20 km / h, the pressure of the cleaning fluid and the compressed air is 60 kPa; When the speed of the automobile is 20-80 km / h, the pressure of the cleaning fluid and the compressed air is 150 kPa.

5. The method for cleaning the surface of a camera lens according to claim 1, characterized in that: The method of using the camera cleaning device to clean the lens surface of each camera according to the vehicle information of the automobile and the pollution condition of each camera includes: If the contamination degree of the camera belongs to the first contamination degree range, controlling the second solenoid valve corresponding to the camera to start, delivering the compressed air to the nozzle device corresponding to the camera, and using the compressed air to clean the camera; If the contamination degree of the camera belongs to the second contamination degree range, the first solenoid valve corresponding to the camera is controlled to open for a preset operation time, and during the preset operation time, the cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the cleaning liquid. After the first solenoid valve is closed, the second solenoid valve corresponding to the camera is controlled to start, and the compressed air is delivered to the nozzle device corresponding to the camera, and the camera is cleaned with the compressed air; If the contamination degree of the camera belongs to the third contamination degree range, the first solenoid valve corresponding to the camera is controlled to start, and the cleaning liquid is delivered to the nozzle device corresponding to the camera, and the camera is cleaned by the cleaning liquid.

6. The method for cleaning the surface of a camera lens according to claim 1, characterized in that: The method for training the pollution prediction model includes: Acquire a plurality of pollution training sample images taken by each of the cameras; wherein each of the pollution training sample images includes a pollution target mark; wherein the pollution target mark is set according to the type and degree of pollution of the camera; Based on the contaminated target label, extracting the positive contaminated sample image and the negative contaminated sample image corresponding to the contaminated training sample image from the contaminated image data; Based on the initial pollution prediction model, the pollution situation of the polluted training sample image is predicted to obtain the training pollution situation corresponding to the polluted training sample image, and the pollution situation of the positive polluted sample image and the negative polluted sample image are predicted to obtain the positive pollution situation corresponding to the positive polluted sample image and the negative pollution situation corresponding to the negative polluted sample image; Constructing a contrastive learning loss based on the difference between the training contamination situation and the positive contamination situation and the difference between the training contamination situation and the negative contamination situation; Based on the contrastive learning loss, the initial pollution prediction model is optimized to obtain the pollution prediction model after pre-training.

7. The method for cleaning the surface of a camera lens according to claim 1, characterized in that: The method for training the pollutant identification model includes: Acquire multiple pollution images taken by each of the cameras, pre-mark the type of pollution to which each pollution image belongs, and obtain training samples with the pollution image as an input target and the type of pollution described in the marked pollution image as a target output value; The pollutant identification model to be trained obtained after deep learning of several training samples is used as the initial pollutant identification model; Acquire a plurality of test samples for testing the initial pollutant identification model; wherein the test samples are test images marked with pollutant types; Based on the initial pollutant identification model, multiple test samples are identified to obtain the pollutant type of each test sample. If the pollutant type identified by the initial pollutant identification model is the same as the pollutant type marked by the test sample, the identification result of the initial pollutant identification model is normal; otherwise, the identification result of the initial pollutant identification model is wrong. The initial pollutant identification model is used to identify the pollutant types of several test samples, and the accuracy of the pollutant type identification by the initial pollutant identification model is calculated. If the accuracy is greater than a preset accuracy threshold, the initial pollutant identification model is used as the pollutant identification model after training. Otherwise, the initial pollutant identification model continues to be trained through test samples.

8. The method for cleaning the surface of a camera lens according to claim 1, characterized in that: The method for training the weather prediction model includes: Acquire a plurality of weather forecast training sample images taken by each of the cameras; wherein each of the weather forecast training sample images includes a weather forecast target mark, and the weather forecast target mark includes a true depth value of a target pixel point; In the process of training the initial weather prediction model based on the plurality of weather prediction training sample images, a depth prediction value of a target pixel point of each weather prediction training sample image is obtained; According to the true depth value and the predicted depth value of the target pixel point, the target loss value of the initial weather prediction model is calculated, and the model parameters of the initial weather prediction model are adjusted according to the target loss value until the target loss value meets the preset loss condition, thereby obtaining the weather prediction model after training.

9. A camera lens surface cleaning system, characterized in that: A method for cleaning the surface of a camera lens according to any one of claims 1 to 8, comprising: An acquisition unit, used to acquire vehicle information of a vehicle and image information of each camera on the vehicle; the vehicle information includes vehicle speed information and wiper switch information; A pollution prediction unit, used to identify the image information collected by each camera based on the trained pollution prediction model, and obtain the pollution status of each camera; the pollution status includes the type of pollutant and the degree of pollution; A pollutant identification unit, used to identify the image information collected by each camera based on the trained pollutant identification model, and determine the type of pollutant of each camera; the pollutant type includes a first pollutant and a second pollutant; A weather prediction unit, configured to identify the image information collected by the camera based on the trained weather prediction model when the pollutant type detected by the camera is the first pollutant, and determine whether the current weather environment of the vehicle is abnormal; A cleaning control unit, used for, when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is abnormal, using the camera cleaning device to clean the lens surface of each camera respectively according to the vehicle information of the car, the current weather environment and the pollution condition of each camera; and used for, when the type of pollutant in the camera is the second type of pollutant or when the type of pollutant in the camera is the first type of pollutant and the current weather environment of the car is not abnormal, using the camera cleaning device to clean the lens surface of each camera respectively according to the vehicle information of the car and the pollution condition of each camera.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for cleaning the surface of a camera lens according to any one of claims 1 to 8 is implemented.

Images