Camera lens dirt detection and cleaning method, device and equipment and readable storage medium
By segmenting and comparing the brightness of the camera lens images, positioning dirty areas, and using methods such as water spray, jetting and heating for targeted cleaning, the problem of low efficiency of traditional methods is solved, and the efficiency and accuracy of lens cleaning is improved. Reduced manual intervention.
Patent Information
- Application Number
- CN202411089622.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-06
AI Technical Summary
Camera lenses are susceptible to contamination in the external environment, resulting in a decrease in imaging quality. Traditional dirty detection and cleaning methods are inefficient and require manual intervention to affect their functionality.
By dividing the acquired images into multiple areas, comparing the brightness value of each area, positioning the dirty area, and cleaning it using a combination of water spray, jet and heating methods, combining the dirty model database to identify the dirty type and area. Achieve targeted cleaning.
It improves the efficiency and accuracy of lens cleaning, reduces manual intervention, and can automatically adjust the cleaning method according to different dirt types and environmental conditions, improving the functionality and reliability of the camera.
Smart Images

Figure CN119094905A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of lens cleaning, and in particular to a method, device, equipment and readable storage medium for detecting and cleaning dirt on a camera lens. Background Art
[0002] In some application areas, the imaging quality of the camera is particularly critical, for example, in the field of visual sensing used in smart cars. However, since this part of the camera is directly exposed to the external environment, the camera lens is easily contaminated and dirty, affecting the quality of the image it obtains and the accuracy of subsequent steps such as image recognition. Although traditional lens dirt detection can detect lens dirt, the external environment of the camera may switch multiple times, resulting in the complexity and diversity of the causes of dirt. The mechanical cleaning efficiency of the camera lens is low and often requires manual intervention, which greatly affects and limits the functionality of the camera. Summary of the invention
[0003] The purpose of the present disclosure is to provide a camera lens dirt detection and cleaning method, device, equipment and readable storage medium to improve the above-mentioned problems.
[0004] In order to achieve the above objectives, the present disclosure provides the following technical solutions:
[0005] In a first aspect, the present disclosure provides a camera lens dirt detection and cleaning method, the method comprising:
[0006] The method comprises:
[0007] Divide a first image into at least two first unit areas, wherein the first image is an image acquired by the camera;
[0008] Comparing the image brightness values of the first unit areas with each other, acquiring at least one of the first unit areas, and defining each of the acquired first unit areas as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area;
[0009] dividing each of the second unit regions into at least two third unit regions;
[0010] Comparing the image brightness values of the third unit areas in the second unit area with each other, obtaining at least one third unit area, and defining each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area;
[0011] According to the fourth unit area, obtaining a dirty area on the camera lens;
[0012] A first cleaning operation is performed on the soiled area.
[0013] In a second aspect, the present disclosure provides a camera lens dirt detection and cleaning device, the device comprising:
[0014] A first segmentation module, used to segment a first image into at least two first unit areas, wherein the first image is an image acquired by the camera;
[0015] A first comparison module is used to compare the image brightness values of the first unit areas with each other, obtain at least one of the first unit areas, and define each of the obtained first unit areas as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area;
[0016] A second segmentation module, used for segmenting each of the second unit areas into at least two third unit areas;
[0017] a second comparison module, configured to compare the image brightness values of the third unit areas in the second unit area with each other, obtain at least one third unit area, and define each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area;
[0018] A dirt positioning module, used for acquiring a first dirt area on the camera lens according to the fourth unit area;
[0019] The cleaning module is used to perform a first cleaning operation on the first dirty area.
[0020] In a third aspect, the present disclosure provides an electronic device, including:
[0021] a memory storing execution instructions; and
[0022] A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the camera lens dirt detection and cleaning method as described in the first aspect.
[0023] In a fourth aspect, the present disclosure provides a readable storage medium, wherein the readable storage medium stores execution instructions, and when the execution instructions are executed by a processor, they are used to implement the camera lens dirt detection and cleaning method as described in the first aspect.
[0024] The beneficial effects of the present disclosure are:
[0025] 1. The present disclosure provides a method for detecting and cleaning camera lens dirtiness, wherein the image brightness values of the first unit area obtained by segmenting the first image acquired by the camera are compared with each other to obtain a second unit area with a low image brightness value. Further, the second unit area is segmented into a third unit area and the image brightness values of the third unit area are compared with each other to determine whether the camera lens is dirty and the dirty area, so as to clean the dirty area.
[0026] 2. The method cleans the camera lens by a free combination of three optional operations: water spraying, air jet and heating, which greatly improves the cleanliness.
[0027] 3. The method identifies the type of dirt based on multiple dimensions such as the image brightness value of the image acquired by the camera, the area and change of the dirty area in the image, the color, the surface temperature of the camera lens, and the dirt model database, thereby improving the recognition rate of dirt, thereby performing targeted cleaning and improving cleaning efficiency and cleanliness.
[0028] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present disclosure. The purposes and other advantages of the present disclosure can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0030] Figure 1 It is a flow chart of a camera lens dirt detection and cleaning method provided by an embodiment of the present disclosure;
[0031] Figure 2 It is a structural schematic diagram of a camera lens dirt detection and cleaning device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.
[0033] It should be noted that similar reference numerals or letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0034] Embodiment 1
[0035] The disclosed embodiment provides a camera lens dirt detection and cleaning method.
[0036] refer to Figure 1 , a camera lens dirt detection and cleaning method S100, the method comprising:
[0037] S102, dividing a first image into at least two first unit areas, wherein the first image is an image acquired by the camera;
[0038] S104, comparing the image brightness values of the first unit areas with each other, acquiring at least one of the first unit areas, and defining each acquired first unit area as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area;
[0039] S106, dividing each of the second unit regions into at least two third unit regions;
[0040] S108, comparing the image brightness values of the third unit areas in the second unit area with each other to obtain at least one third unit area, and defining each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area;
[0041] S110, acquiring a first dirty area on the camera lens according to the fourth unit area;
[0042] S112: performing a first cleaning operation on the first dirty area.
[0043] In step S102, the number of first unit regions may be a fixed value, for example, the first image is divided into N×M first unit regions. The first image may also be divided according to a certain pixel size, for example, the pixel size of each first unit region is fixed to P×Q.
[0044] The brightness values of the image area can be identified, counted and compared through the ISP (Image Signal Processor).
[0045] In some embodiments, the brightness value of an image region is the average brightness value of all image unit regions in the region. For example, the region is divided into a plurality of image unit regions with a pixel size of 3×3, and the brightness value of each image unit region is obtained and then averaged.
[0046] Of course, the pixel size of the image unit area may also be other values.
[0047] Step S104 can be considered to include the following two steps:
[0048] Sort the image brightness values of all first unit areas.
[0049] Each first unit area is classified into a second unit area and a non-second unit area.
[0050] In some implementations, in step S104, comparing the image brightness values of the first unit regions with each other to obtain at least one of the first unit regions includes:
[0051] If the image brightness value of a first unit area is lower than the product of the average image brightness value of the first high-brightness value unit area and the first threshold, the first unit area is obtained, wherein each of the first unit areas having an image brightness value greater than that of the first unit area is defined as the first high-brightness value unit area.
[0052] The first threshold may be 80%, that is, if the image brightness value of the first unit area is lower than 20% of the average image brightness value of the first high brightness value unit area, it is determined to be the second unit area.
[0053] In some other embodiments, the acquisition of the second unit area may also be, according to the sorting, a comparison between a first unit area and one or more first unit areas having higher brightness values than the first unit area in the sorting from high to low of the image.
[0054] The comparison with a plurality of items may be a one-to-one comparison or an average comparison of a plurality of items.
[0055] For example, there are 10 first unit regions, with the one ranked 10 being compared with the one ranked 9.
[0056] Or the one ranked 10 is compared with the ones ranked 7, 8, and 9.
[0057] Or the comparison of ranking 10 with ranking 1, 2, 3, 4.
[0058] Or the comparison between the ranking 10 and the rankings 1, 3, and 5. That is, the plurality of first unit regions may be ranked continuously or discontinuously.
[0059] It can also be considered that if a first unit region has been determined to be a second unit region, it does not participate in the comparison with other first unit regions.
[0060] In some embodiments, the second unit area is obtained by obtaining a plurality of first images by a camera within a preset time, and counting the number of times the difference between each first unit area and other first unit areas reaches a preset value; if the number of times for a first unit area reaches a preset threshold, it is determined to be a second unit area.
[0061] In particular, when the dirt is floating, the determination of the second unit area may be different in the first images acquired at different times, but the dirt continues to exist.
[0062] Furthermore, the number of first images may depend on the sampling rate of the camera.
[0063] Furthermore, a step length is used to slide the window of the preset time on the time axis.
[0064] In step S104, the image brightness values of the first unit areas are compared with each other, which can be regarded as a dynamic process rather than a comparison and determination based on fixed parameters, such as the brightness value of the ambient light.
[0065] In step S104, if the image brightness values of all the first unit areas are determined to be consistent, or the difference is small and does not reach the preset value, and the second unit area cannot be acquired, it is necessary to consider the situation that the entire camera lens is covered with dirt.
[0066] At this time, it can be combined with external sensors or dirt model databases, for example, the brightness of ambient light can be obtained through an ambient light sensor to determine the difference between the brightness value of the surrounding environment and the first image, or the time can be obtained through a clock to determine whether it is dark, or by obtaining weather information to determine whether it is raining, snowing, etc.
[0067] The dirt model database may be a model database used for comparison in the prior art to identify dirt types.
[0068] If no abnormality is found, it is determined that the camera lens does not need to be cleaned.
[0069] The method can determine whether the area where a specific scene in the first image is located has a low brightness value based on the comparison of brightness values between the second unit areas obtained by segmentation in the first image, thereby determining whether the specific scene in the first image corresponds to an object blocked on the camera lens or an object far away from the camera. This method can avoid the need to use an image recognition algorithm to make corresponding judgments and consume computing power.
[0070] The method obtains the second unit area with low image brightness value, and then performs local segmentation to obtain the third unit area for comparison, which can avoid the segmentation operation of the first unit area (not the second unit area) with higher image brightness value and the comparison of the segmented areas therein, and can improve the efficiency of dirt recognition.
[0071] In step S106, the segmentation method is similar to that in step S102, which may be to obtain a fixed number of third unit areas, or to segment with third unit areas of a fixed pixel size. The segmentation is grid segmentation.
[0072] In step S108, by comparing areas of smaller dimensions, if the fourth unit area is obtained, it can be determined that there is a dirty image area on the first image, and the dirty image area can be further located and cleaned, which can improve the accuracy of the cleaning operation and avoid blindly cleaning the entire camera lens, wasting resources and affecting efficiency.
[0073] If in the second unit area with low image brightness values, the image brightness values of all the third unit areas are judged to be consistent, or the difference is small and does not reach the preset value, and the fourth unit area cannot be acquired, it is possible that the dirt level of the area has not reached the level that requires cleaning, which can avoid excessive cleaning of the camera lens and waste of resources.
[0074] In some embodiments, in step S108, the image brightness values of the third unit areas in the second unit areas are compared with each other. The third unit areas in all the second unit areas may be compared with each other, or the third unit areas may be compared with each other limited to each second unit area.
[0075] In some embodiments, in step S108, the fourth unit area can be set to a smaller area, for example, a pixel size of 3×3, and then each fourth unit area can be regarded as a dirty unit image area as a whole, so as to avoid the need to further identify the outline of the dirty area in the fourth unit area or the area ratio occupied.
[0076] A plurality of continuous dirty unit image regions form a first dirty image region. The first dirty image region corresponds to a first dirty region on the camera lens.
[0077] Therefore, the first dirty area can be one or more. For example, a first dirty area can be a mud spot, or a large piece of ice and snow covering the camera lens.
[0078] The area of each first dirty image region is the product of the number of continuous dirty unit image regions constituting the first dirty image region and the area of the dirty unit image region.
[0079] In some embodiments, comparing the image brightness values of the third unit areas in the second unit areas to obtain at least one third unit area includes:
[0080] If the image brightness value of one of the third unit areas is lower than the product of the average of the image brightness values of the second high brightness value unit area and the second threshold, the third unit area is obtained, wherein each of the third unit areas in the same second unit area whose image brightness value is greater than that of the third unit area is defined as the second high brightness value unit area.
[0081] Similarly, in some other embodiments, the fourth unit area may be obtained by comparing a third unit area with one or more first unit areas whose image brightness values are higher than the third unit area in the order from high to low.
[0082] In some embodiments, the fourth unit area may be obtained in a manner similar to the above-mentioned second unit area, based on statistics within the same or different preset time.
[0083] In some embodiments, in step S110, obtaining a first dirty area on the camera lens according to the fourth unit area includes:
[0084] The dirt level of the fourth unit area, ie, the impact on the image acquired by the camera, is determined according to the ratio of the image brightness value of each fourth unit area to the average image brightness value of the second high brightness value unit area.
[0085] Further, the dirt level of the first dirty area is determined according to the dirt level of the fourth unit area and the number of fourth unit areas of each dirt level in the first dirty image area.
[0086] In some embodiments, in step S110, obtaining a dirty area on the camera lens according to the fourth unit area includes:
[0087] In some embodiments, in step S110, obtaining a first dirty area on the camera lens according to the fourth unit area includes:
[0088] According to the position of the fourth unit area on the first image, the position of the dirty area on the camera lens is obtained, that is, the dirty area is located, so as to perform a cleaning operation on the area.
[0089] In some embodiments, in step S110, obtaining a first dirty area on the camera lens according to the fourth unit area includes:
[0090] According to the distribution of the fourth unit area on the first image, the dirt distribution type of the first dirt area is determined, for example, block distribution, point distribution, etc., or there are multiple distribution types of dirt.
[0091] The distribution also needs to take into account the number of fourth unit areas in each first dirty image area.
[0092] In some embodiments, in step S112, the number of all fourth unit areas in the first image needs to be considered. If the number is small, less than a preset threshold, such as one or two dots of dirt, the first cleaning operation is not required.
[0093] In some embodiments, the method further comprises:
[0094] Obtaining the color of the fourth unit area;
[0095] Obtaining the temperature of the camera lens;
[0096] If the color of at least one of the fourth unit regions is white, comparing the at least one of the fourth unit regions with the dirt model database to obtain a comparison result;
[0097] If, in the comparison result, the matching rate with ice and snow is greater than a third threshold, and the temperature is not greater than 0° C., it is determined that the dirty area is ice and snow.
[0098] The comparison result is specifically a result of comparing the at least one fourth unit area with the model data of various types of dirt in the dirt model database.
[0099] Furthermore, the third threshold may be 90%.
[0100] Furthermore, the surface temperature of the camera lens is obtained by a temperature sensor of the lens.
[0101] In some embodiments, the color of the first dirty image region is determined according to a color average of dirty unit image regions in the first dirty image region.
[0102] In some embodiments, the first cleaning operation for ice and snow dirt includes:
[0103] Heating the snow-covered area of the camera lens will melt the snow and dirt into water, reducing its adhesion to the lens and making it easier to remove.
[0104] Optionally, continue heating the area, or identify areas of water damage, including adjusting the intensity of the heating to directly evaporate the melted water.
[0105] Alternatively, spray air toward the camera lens to blow the melted water and other dirt away from the lens to avoid water stains interfering with camera shooting.
[0106] In some embodiments, the method further comprises:
[0107] Obtaining the color of the fourth unit area;
[0108] Obtaining the temperature of the camera lens;
[0109] If the color of at least one of the fourth unit regions is white, comparing the at least one of the fourth unit regions with the dirt model database to obtain a comparison result;
[0110] If, in the comparison result, the matching rate with fog is greater than a fourth threshold, and the temperature of the dirty area corresponding to the fourth unit area on the camera lens is greater than 0° C., it is determined that the dirty area is fog.
[0111] In some embodiments, the first cleaning operation for fog dirt includes: spraying air toward the fog-covered area on the camera lens to blow the attached fog away from the lens. This cleaning operation saves heating operations and water spraying operations, and improves cleaning efficiency.
[0112] In some embodiments, the method further comprises:
[0113] Obtaining a change in the number of the fourth unit areas in the second unit area within a preset time interval;
[0114] If the change amount is greater than the third threshold, it is determined that the dirty area on the camera lens corresponding to the fourth unit area in the second unit area is a floating object, that is, a floating object attached to the camera lens, such as leaves, paper, etc.
[0115] In some embodiments, further, when a floating object is determined, the attached part of the floating object (i.e., the non-floating part, which together with the floating part constitutes the floating object as a whole) corresponding to the dirty image area in the first image is obtained, and the dirtiness level of the floating object is determined based on the number of fourth unit areas therein.
[0116] In some embodiments, the first cleaning operation for floating animal dirt includes: spraying air toward the area covered by floating animal dirt on the camera lens. If the floating animal can be blown away from the lens directly, the cleaning efficiency is greatly improved, saving other optional operations such as spraying water.
[0117] If this mode fails, for example, the floating animals are too attached and cannot be removed by the jet operation, or some of the floating animals remain and have not been completely removed, try other optional operations such as spraying water.
[0118] Furthermore, by integrating the recognition process of multiple dimensions such as the color, distribution type, individual areas or total areas, area change, camera lens surface temperature and dirt reference model of the first dirty area in the first image mentioned above, four large models of ice and snow, fog, mud spots and floating animals can be established to directly identify these specific dirt, thereby improving the recognition rate and speed.
[0119] In some embodiments, in step S112, performing a first cleaning operation on the first dirty area includes: at least one of spraying water on the first dirty area, spraying air on the first dirty area, and heating the first dirty area.
[0120] Spraying water to the first dirty area on the camera lens, spraying air to the first dirty area, and heating the first dirty area are three optional operations of the cleaning operation. The free combination of the three optional operations can greatly improve the cleanliness of the camera lens.
[0121] The direction of the water spray or air jet can be from one direction or multiple directions to improve the cleanliness.
[0122] The heating may be performed by heating the entire camera lens or by heating a local area.
[0123] In some embodiments, water is sprayed onto the first dirty area, specifically, water is sprayed at a preset water pressure and for a preset time.
[0124] The spraying of air toward the first dirty area may be performed by spraying air at a preset spraying air pressure, and the spraying time may be a preset time which is the same as or different from the water spraying time.
[0125] The first dirty area is heated by using a preset heating power, and the heating time is a preset time which is the same as or different from the water spraying time and the air spraying time.
[0126] In some embodiments, spraying water onto the first dirty area at least includes: spraying water onto the first dirty area at a first water spraying intensity and spraying water onto the first dirty area at a second water spraying intensity;
[0127] Jetting the dirty area at least includes: jetting the dirty area at a first jet intensity and jetting the dirty area at a second jet intensity;
[0128] Heating the dirty region includes at least: heating the first dirty region with a first heating intensity and heating the first dirty region with a second heating intensity.
[0129] In some embodiments, the dirt type of the first dirty image area is obtained based on at least one of the temperature of the first dirty image area on the camera lens surface, the area of each first dirty image area in the first image, the area of all first dirty image areas in the first image or the proportion of all first dirty image areas in the first image, the area change of the first dirty image area in the first image, the distribution type of the first dirty image area in the first image, and the color of the first dirty image area in the first image.
[0130] In some embodiments, the first dirty image region is compared with a dirt model database, and if in the comparison result, the matching rate between the first dirty image region and a certain type of dirt is greater than a preset threshold, the first dirty region is determined to be the type of dirt.
[0131] The dirt type may specifically be a specific type of dirt, or a classification of dirt.
[0132] In some embodiments, the method further includes: inputting the first image into a dirt reference model, wherein the model is capable of identifying pixel point areas of specific dirt in the first image through training, for example, the first image is identified using an image recognition algorithm based on deep learning, so as to obtain the specific type of dirt corresponding to the first dirty image area in the first image, such as ice and snow, or leaves, or mud blocks, or mud spots, etc.
[0133] Alternatively, the dirt may be pre-classified and then the corresponding dirt classification in the first image may be obtained through model recognition.
[0134] In some embodiments, the dirt level of the first dirty image area is obtained based on at least one of the temperature of the first dirty image area on the camera lens surface, the area of each first dirty image area in the first image, the area of all first dirty image areas in the first image or the proportion of all first dirty image areas in the first image, the area change of the first dirty image area in the first image, the distribution type of the first dirty image area in the first image, and the color of the first dirty image area in the first image.
[0135] In some embodiments, according to the dirt level or dirt type of the first dirt area, or the combination of the dirt level and the dirt type, the combination may be weighted calculation, using different combinations of the three optional operations, and each optional operation may adopt different strengths. Alternatively, the same optional operation may adopt a combination of different strengths.
[0136] For example, when the dirt is leaves, and the leaves are attached to the camera lens due to rain, a high-level jet pressure is first used to blow away the leaves, and then a low-level jet pressure is used to blow away the remaining rain.
[0137] When the dirt is mud spots, first use the medium-range water spray operation to wash away the mud spots, and then use the low-range jet air pressure to blow away the residual water on the camera lens.
[0138] In some embodiments, each optional operation adopts a different intensity, and the water pressure for spraying water to the first dirty area may include multiple preset intensities, for example, three levels of high, medium and low, or the water spraying time may include multiple preset intensities, or different combinations of the intensity of the water pressure and the water spraying time.
[0139] The air jet pressure for jetting air toward the first dirty area includes a plurality of preset intensities, or the air jet time includes a plurality of preset intensities, or different combinations of the water jet pressure and the water jet time.
[0140] The heating power for heating the first dirty area includes a plurality of preset intensities, or the heating time includes a plurality of preset intensities, or different combinations of the intensities of the heating water pressure and the heating time.
[0141] In some embodiments, the first cleaning operation is a specific combination of three optional operations preset based on one or more information of the first dirty area, such as the above-mentioned dirt level, dirt distribution type, etc., that is, a preset mode. When the specific information of the first dirty area is identified and obtained to meet the conditions corresponding to the preset mode, a cleaning operation of the predetermined mode is performed, which can improve the cleaning efficiency.
[0142] Furthermore, the combination of the three optional operations in the preset mode can be obtained and optimized by counting the previous dirt cleaning effects.
[0143] In some embodiments, the method further comprises:
[0144] Acquire a second dirty area on the camera lens according to the second image, wherein the second image is an image acquired by the camera after a first cleaning operation is performed on the first dirty area according to the first image;
[0145] A second cleaning operation is performed on the second dirty area.
[0146] That is, after the camera is cleaned once, it is determined whether the cleaning operation effectively removes dirt and makes the camera lens suitable for continued use. If the lens cannot meet the use requirements, that is, the brightness of the acquired image is still affected, a second cleaning operation is performed.
[0147] Specifically, the cleaning operation may be performed multiple times until the dirt is removed.
[0148] The user may also be reminded to perform manual cleaning after a predetermined number of cleaning operations.
[0149] Alternatively, the user can manually start all three optional operations, and the manual start method can be manual control, voice control, etc. After receiving the reminder of cleaning failure, the user can manually confirm the dirtiness and manually start the optional operation for cleaning.
[0150] In some embodiments, the secondary cleaning operation may use a combination of optional operations with greater intensity, or switch the combination of optional operations to improve the cleaning success rate.
[0151] Specifically, the method can be applied to a vehicle-mounted camera, wherein a speed-controlled motor is used to spray water and jet air to support multiple gear strengths. The water supply of the water pump can pass through the glass water pool of the vehicle body windshield.
[0152] The method is performed by an ECU (Electronic Control Unit), which may include an MCU (Microcontroller Unit) for controlling cleaning operations, an ISP for dirt detection, and a display screen for human-computer interaction.
[0153] Furthermore, the ECU stores the dirt model database.
[0154] Embodiment 2
[0155] The disclosed embodiment provides a camera lens dirt detection and cleaning device.
[0156] The device may include a corresponding module for performing each or several steps in the above flowchart. Therefore, each step or several steps in the above flowchart may be performed by a corresponding module, and the device may include one or more modules in these modules. The module may be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.
[0157] Specifically, refer to Figure 2 , a camera lens dirt detection and cleaning device 1000, comprising:
[0158] A first segmentation module 1002, configured to segment a first image into at least two first unit areas, wherein the first image is an image acquired by the camera;
[0159] A first comparison module 1004 is used to compare the image brightness values of the first unit areas with each other, obtain at least one of the first unit areas, and define each of the obtained first unit areas as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area;
[0160] A second segmentation module 1006, configured to segment each of the second unit regions into at least two third unit regions;
[0161] A second comparison module 1008 is used to compare the image brightness values of the third unit areas in the second unit area with each other, obtain at least one third unit area, and define each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area;
[0162] A dirt positioning module 1010, configured to obtain a first dirt area on the camera lens according to the fourth unit area;
[0163] The cleaning module 1012 is used to perform a first cleaning operation on the first dirty area.
[0164] The disclosed embodiment also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executes the execution instructions stored in the memory, so that the processor or other hardware module executes the above-mentioned camera lens dirt detection and cleaning method.
[0165] The embodiments of the present disclosure also provide a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above-mentioned camera lens dirt detection and cleaning method.
[0166] Among them, the hardware structure adopted by the camera lens dirt detection and cleaning device implemented by the hardware implementation method using a processor disclosed in the present invention can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus connects various circuits including one or more processors, memories and / or hardware modules together. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuits, external antennas, etc.
[0167] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one connecting line is used in this figure, but it does not mean that there is only one bus or one type of bus.
[0168] Any process or method description in the flowchart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by a person skilled in the art of the present disclosure. The processor performs the various methods and processes described above. For example, the method implementation in the present disclosure can be implemented as a software program, which is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).
[0169] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions on an instruction execution system, apparatus, or device), or in combination with such instruction execution systems, apparatuses, or devices.
[0170] For the purposes of this specification, a "readable storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples of readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable read-only memory (CDROM). In addition, the readable storage medium may even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a memory.
[0171] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0172] A person skilled in the art may understand that all or part of the steps of implementing the above-mentioned implementation method may be completed by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the implementation method.
[0173] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a readable storage medium. The storage medium may be a read-only memory, a disk or an optical disk, etc.
[0174] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.
Claims
1. A camera lens dirt detection and cleaning method, characterized in that: The method comprises: Divide a first image into at least two first unit areas, wherein the first image is an image acquired by the camera; Comparing the image brightness values of the first unit areas with each other, acquiring at least one of the first unit areas, and defining each of the acquired first unit areas as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area; dividing each of the second unit regions into at least two third unit regions; Comparing the image brightness values of the third unit areas in the second unit area with each other, obtaining at least one third unit area, and defining each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area; Acquire a first dirty area on the camera lens according to the fourth unit area; A first cleaning operation is performed on the first dirty area.
2. The method according to claim 1, characterized in that Comparing the image brightness values of the first unit areas with each other to obtain at least one of the first unit areas includes: If the image brightness value of one of the first unit areas is lower than the product of the average image brightness value of the first high brightness value unit area and the first threshold, the first unit area is acquired, wherein each of the first unit areas with an image brightness value greater than that of the first unit area is defined as the first high brightness value unit area; Comparing the image brightness values of the third unit areas in the second unit areas with each other to obtain at least one third unit area includes: If the image brightness value of one of the third unit areas is lower than the product of the average of the image brightness values of the second high brightness value unit area and the second threshold, the third unit area is obtained, wherein each of the third unit areas in the same second unit area whose image brightness value is greater than that of the third unit area is defined as the second high brightness value unit area.
3. The method according to claim 1, characterized in that Performing a first cleaning operation on the dirty area includes at least one of spraying water to the dirty area, spraying air to the dirty area, and heating the dirty area.
4. The method according to claim 2, characterized in that Spraying water to the dirty area at least includes: spraying water to the dirty area with a first water spraying intensity and spraying water to the dirty area with a second water spraying intensity; Jetting the dirty area at least includes: jetting the dirty area at a first jet intensity and jetting the dirty area at a second jet intensity; Heating the dirty area includes at least: heating the dirty area with a first heating intensity and heating the dirty area with a second heating intensity.
5. The method according to claim 1, characterized in that The method further comprises: Acquire a second dirty area on the camera lens according to the second image, wherein the second image is an image acquired by the camera after a first cleaning operation is performed on the first dirty area according to the first image; A second cleaning operation is performed on the second dirty area.
6. The method according to claim 1, characterized in that The method further comprises: Get the color of the fourth unit area; Obtaining the temperature of the camera lens; If the color of at least one of the fourth unit regions is white, comparing the at least one of the fourth unit regions with the dirt model database to obtain a comparison result; If, in the comparison result, the matching rate with ice and snow is greater than a third threshold, and the temperature is not greater than 0° C., it is determined that the dirty area is ice and snow.
7. The method according to claim 1, characterized in that The method further comprises: Obtaining a change in the number of the fourth unit areas in the second unit area within a preset time interval; If the change amount is greater than a third threshold, it is determined that the dirty area on the camera lens corresponding to the fourth unit area in the second unit area is a floating object.
8. A camera lens dirt detection and cleaning device, characterized in that: The device comprises: A first segmentation module, used to segment a first image into at least two first unit areas, wherein the first image is an image acquired by the camera; A first comparison module is used to compare the image brightness values of the first unit areas with each other, obtain at least one of the first unit areas, and define each of the obtained first unit areas as a second unit area, wherein the image brightness value of the second unit area is lower than the image brightness value of each of the first unit areas except the second unit area; A second segmentation module, used for segmenting each of the second unit areas into at least two third unit areas; a second comparison module, configured to compare the image brightness values of the third unit areas in the second unit area with each other, obtain at least one third unit area, and define each of the obtained third unit areas as a fourth unit area, wherein the image brightness value of the fourth unit area is lower than the image brightness value of each of the third unit areas except the fourth unit area in the same second unit area; A dirt positioning module, used for acquiring a first dirt area on the camera lens according to the fourth unit area; The cleaning module is used to perform a first cleaning operation on the first dirty area.
9. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the camera lens dirt detection and cleaning method according to any one of claims 1 to 8.
10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the camera lens dirt detection and cleaning method as described in any one of claims 1 to 7.
Citation Information
Cited By
Camera lens stain detection method and device suitable for mine environment
CN119810028A
