Camera device cleaning evaluation method, device, equipment, medium and vehicle

Abstract

The invention relates to a camera device cleaning evaluation method and device, equipment, a medium and a vehicle. The method comprises the steps that at least two frames of target images collected in the camera device cleaning process are acquired; and according to the target image and a reference image of the target image, a cleanliness evaluation result of the camera device is determined, and the reference image is an image collected by the camera device in a clean state. By comparing the target image in the cleaning process of the camera device with the reference image collected in the clean state of the camera device, the cleaning evaluation result of the camera device is determined, manual subjective cleaning evaluation on the camera device is avoided, and the accuracy of cleaning evaluation on the camera device is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a camera device cleanliness evaluation method, device, equipment, medium and vehicle. Background Art

[0002] Since the vehicle reversing camera is used outdoors, it is very easy to be contaminated by stains in the external environment, thus affecting the display effect of the camera. In order to clean the common stains such as water, ice, mud, dust, etc. on the surface of the camera, there are many different cleaning solutions, and each stain condition will adopt the cleaning solution with the best evaluation.

[0003] However, currently, most of the existing evaluation methods for the cleaning of camera devices are based on subjective human judgment, and there is no objective and complete evaluation standard for the cleaning process. Human subjective judgment may believe that the cleaning has achieved the best cleaning effect, but from the shooting angle of the camera device, there are still stains on the image captured, resulting in that the cleaning effect of the camera device is not actually optimal, resulting in low accuracy of the cleaning evaluation of the camera device. Summary of the invention

[0004] The embodiment of the present application provides a method for evaluating the cleanliness of a camera device, which can improve the accuracy of the cleanliness evaluation of the camera device to solve the above-mentioned technical problems.

[0005] In order to achieve the above-mentioned object, according to a first aspect of the present application, a method for evaluating the cleanliness of an imaging device is provided, the method comprising:

[0006] Acquire at least two frames of target images collected by the camera device during the cleaning process;

[0007] A cleanliness evaluation result of the imaging device is determined according to the target image and a reference image of the target image, wherein the reference image is an image captured when the imaging device is in a clean state.

[0008] Optionally, the target image is an image captured for the calibration image when the camera device is in a dirty state;

[0009] The reference image is an image acquired by the camera device in a clean state for the calibration image;

[0010] The calibration image includes at least one of a corner point image, a grid image, a stripe image, and a circular pattern image.

[0011] Optionally, the calibration image is a corner point image; and determining the cleanliness evaluation result of the camera device according to the target image and a reference image of the target image includes:

[0012] The number of target corner points of the target image is compared with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the imaging device.

[0013] Optionally, before comparing the number of target corner points of the target image with the number of standard corner points of the reference image, the method further comprises:

[0014] Acquire a first pixel characteristic value corresponding to each pixel point of the target image, and determine the number of target corner points of the target image according to the first pixel characteristic value;

[0015] A second pixel characteristic value corresponding to each pixel point of the reference image is obtained, and the number of standard corner points of the reference image is determined according to the second pixel characteristic value.

[0016] Optionally, obtaining a first pixel feature value corresponding to each pixel point of the target image includes:

[0017] Performing gradient calculation on the target image to obtain a horizontal gradient and a vertical gradient corresponding to the target image;

[0018] Based on the horizontal gradient and the vertical gradient, a first pixel feature value corresponding to each pixel point in the target image is extracted.

[0019] Optionally, comparing the number of target corner points of the target image with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the camera device includes:

[0020] Comparing the number of target corner points of the target image with the number of standard corner points of the reference image to determine the degree of contamination corresponding to the target image;

[0021] An evaluation result of the imaging device is determined based on the contamination level.

[0022] Optionally, determining an evaluation result of the camera device based on the pollution degree includes:

[0023] generating a pollution degree change curve based on the acquisition time of the target image and the pollution degree;

[0024] Based on the pollution degree variation curve, an evaluation result of the camera device is determined.

[0025] Optionally, determining a cleanliness evaluation result of the imaging device according to the target image and a reference image of the target image includes:

[0026] The target pixel value matrix of the target image is compared with the standard pixel value matrix of the reference image to obtain a cleanliness evaluation result of the imaging device.

[0027] Optionally, comparing the target pixel value matrix of the target image with the standard pixel value matrix of the reference image to obtain the cleanliness evaluation result of the imaging device includes:

[0028] Comparing a target pixel value matrix of the target image with a standard pixel value matrix of the reference image to determine a transmittance difference between the target image and the reference image;

[0029] An evaluation result of the camera device is determined according to the transmittance difference.

[0030] Optionally, determining an evaluation result of the camera device according to the transmittance difference includes:

[0031] generating a transmittance variation curve based on the acquisition time of the target image and the transmittance difference;

[0032] Based on the transmittance variation curve, an evaluation result of the camera device is determined.

[0033] Optionally, acquiring at least two frames of target images captured by the camera device during the cleaning process includes:

[0034] Acquire the target image collected by the camera device during the cleaning process;

[0035] The target image is sampled to obtain at least two frames of target images.

[0036] Optionally, the method further comprises:

[0037] If the camera device is in a dirty state, the camera device is cleaned by at least two cleaning schemes, and at least two frames of target images captured during the cleaning process of the camera device are acquired;

[0038] The step of determining a cleanliness evaluation result of the imaging device according to the target image and a reference image of the target image includes:

[0039] A cleaning evaluation result of the cleaning scheme on the imaging device is determined according to the target image and a reference image of the target image.

[0040] Optionally, after determining the cleanliness evaluation result of the camera device according to the target image and the reference image of the target image, the method includes:

[0041] Determining a dirt type corresponding to the dirt state;

[0042] A target cleaning scheme corresponding to the optimal target cleaning evaluation result is obtained, and the dirt type and the target cleaning scheme are associated and saved.

[0043] Optionally, after obtaining the target cleaning solution corresponding to the optimal target cleaning evaluation result and associating and saving the dirt type and the target cleaning solution, the method includes:

[0044] If the camera device is in a dirty state, obtaining the dirt type of the dirty state;

[0045] Acquire a target cleaning solution corresponding to the dirt type in a pre-created target cleaning solution database;

[0046] The imaging device is cleaned based on the target cleaning scheme.

[0047] According to a second aspect of the present application, a camera device cleaning evaluation device is provided, the camera device cleaning evaluation device comprising:

[0048] An acquisition module, used to acquire at least two frames of target images collected by the camera device during the cleaning process;

[0049] A determination module is used to determine a cleanliness evaluation result of the camera device according to the target image and a reference image of the target image, wherein the reference image is an image captured when the camera device is in a clean state.

[0050] According to the third aspect of the present application, an embodiment of the present application also provides a camera device cleanliness evaluation device, which includes a processor and a memory, and the memory stores a computer program; when the computer program is executed by the processor, the processor executes the steps of the above-mentioned camera device cleanliness evaluation method.

[0051] According to the fourth aspect of the present application, an embodiment of the present application also provides a computer-readable storage medium, which stores multiple computer programs. The computer programs are suitable for loading by a processor to execute the steps of any camera device cleaning evaluation method provided in the embodiment of the present application.

[0052] According to the fifth aspect of the present application, an embodiment of the present application also provides a computer program product, including a computer program or a computer program, which, when executed by a processor, implements the steps of any camera device cleaning evaluation method provided in the embodiment of the present application.

[0053] According to the sixth aspect of the present application, an embodiment of the present application also provides a vehicle, the vehicle includes a processor and a memory, and the memory stores a computer program; when the computer program is executed by the processor, the processor executes the steps of the above-mentioned camera device cleaning evaluation method.

[0054] The camera device cleanliness evaluation method of the embodiment of the present application obtains at least two frames of target images collected during the cleaning process of the camera device; and determines the cleanliness evaluation result of the camera device based on the target image and the reference image of the target image, wherein the reference image is an image collected when the camera device is in a clean state. By comparing the target image during the cleaning process of the camera device with the reference image collected when the camera device is in a clean state, the cleanliness evaluation result of the camera device is determined, thereby avoiding subjective human cleanliness evaluation of the camera device and improving the accuracy of the cleanliness evaluation of the camera device.

[0055] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0057] In order to more completely understand the present application and its beneficial effects, the following description will be made in conjunction with the accompanying drawings.

[0058] Figure 1 It is a schematic flow chart of a first embodiment of a camera device cleaning evaluation method provided by the present application;

[0059] Figure 2 It is a flow chart of a second embodiment of the cleaning evaluation method of a camera device provided by the present application;

[0060] Figure 3 is a schematic diagram of the positional relationship between the calibration image and the camera device provided in this application;

[0061] Figure 4 1 is a flow chart of a third embodiment of a camera device cleaning evaluation method provided by the present application;

[0062] Figure 5 4 is a flowchart of a fourth embodiment of a camera device cleaning evaluation method provided by the present application;

[0063] Figure 6 is a structural schematic diagram of a camera device cleaning evaluation system provided in an embodiment of the present application;

[0064] Figure 7 It is a schematic diagram of the structure of the vehicle provided in the embodiment of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0066] Embodiments of the present application provide a camera device cleanliness evaluation method, device, equipment, medium and vehicle.

[0067] The camera device cleanliness evaluation method provided in the embodiment of the present application can be applied to the rear-wheel steering system in a vehicle, and the vehicle can be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and the present application does not make specific limitations on this; it can also be applied to camera device cleanliness evaluation equipment, and the camera device cleanliness evaluation equipment includes smart terminals, smart PCs, etc., and the present application does not make specific limitations on this; it should be noted that the camera device cleanliness evaluation equipment can be integrated into the vehicle or exist independently of the vehicle, which is not limited here.

[0068] The following is a detailed description in conjunction with the accompanying drawings. It should be noted that, for ease of description, the following embodiments are based on the cleaning evaluation device of the camera device as the execution subject, and the description order of the following embodiments is not used as a limitation on the preferred order of the embodiments. Although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order than that shown in the accompanying drawings.

[0069] Please refer to Figure 1 , a first embodiment of the camera device cleaning evaluation method is proposed, and the camera device cleaning evaluation method of the first embodiment includes the following steps:

[0070] Step 101, obtaining at least two frames of target images captured by a camera device during a cleaning process;

[0071] In this step, when the camera device is working normally, contaminants are added to the lens of the camera device and the contaminants are cleaned at the same time. During the cleaning process, the camera device captures at least two frames of target images, and the camera device cleaning evaluation equipment obtains at least two frames of target images captured during the cleaning process of the camera device.

[0072] Specifically, step 101 includes:

[0073] Step 1011, obtaining a target image captured by the camera device during the cleaning process;

[0074] In this step, during the cleaning process, the camera device captures a target image, which records the entire process of contaminants on the lens of the camera device from none to presence and then to none, and the camera device cleaning evaluation equipment obtains the target image collected during the camera device cleaning process.

[0075] Step 1012: sampling the target image to obtain at least two frames of target images.

[0076] In this step, after acquiring the target image, the camera device cleanliness evaluation device samples the target image to obtain at least two frames of target images. Preferably, for the target image, the camera device cleanliness evaluation device samples the target image once every preset number of frames to obtain a set of target images, wherein the preset number of frames is related to the shooting frame rate of the camera device and is not limited here.

[0077] Exemplarily, the shooting frame rate of the camera device is 30 frames. Considering that the difference between adjacent frames in the target image is not obvious and will cause the image processing process to take too long, the camera device cleanliness evaluation device samples the target image once every 3 frames to obtain a target image set. It can be understood that when the duration of the target image is 9 seconds, the target image includes 270 frames of images, and the camera device cleanliness evaluation device samples the target image once every 3 frames to obtain a target image set including 90 frames of target images.

[0078] Step 102: determining a cleanliness evaluation result of the camera device according to the target image and a reference image of the target image, wherein the reference image is an image captured when the camera device is in a clean state.

[0079] In this step, after obtaining the target image, the camera device cleanliness evaluation device compares the target image with a reference image of the target image to determine the cleanliness evaluation result of the camera device, wherein the reference image is an image captured when the camera device is in a clean state. Specifically, the cleanliness evaluation result is a curve showing the change of the cleanliness degree of the camera device over time, wherein the cleanliness degree can be reflected by the pollution degree of the target image or by the light transmittance of the target image.

[0080] The camera device cleanliness evaluation device of this embodiment obtains at least two frames of target images collected during the cleaning process of the camera device; determines the cleanliness evaluation result of the camera device based on the target image and the reference image of the target image, where the reference image is an image collected when the camera device is in a clean state. By comparing the target image during the cleaning process of the camera device with the reference image collected when the camera device is in a clean state, the cleanliness evaluation result of the camera device is determined, thereby avoiding subjective human cleanliness evaluation of the camera device and improving the accuracy of the cleanliness evaluation of the camera device.

[0081] Further, refer to Figure 2 , a second embodiment of the present application is proposed, the difference between the second embodiment and the first embodiment is that the cleaning evaluation result of the camera device is determined according to the target image and the reference image of the target image, including:

[0082] Step 1021 : Compare the number of target corner points of the target image with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the camera device.

[0083] It should be noted that before acquiring the target image and the reference image, the camera cleaning evaluation device places the calibration image vertically in the test environment as the image acquisition background of the camera. The lens of the camera is facing the center of the calibration image to ensure that the interface of the camera image acquisition is basically covered by the calibration image, and the calibration image is within the focus range of the camera, wherein the calibration image includes: at least one of a corner image, a grid image, a stripe image, and a circular image; at this time, the camera collects a reference image for the calibration image in a clean state. Afterwards, the camera collects a target image for the calibration image during the cleaning process in a dirty state, thereby obtaining a target image.

[0084] In the specific implementation, the calibration image is a corner point image, such as Figure 3 As shown, the corner point image is placed vertically in the test environment as the image acquisition background of the camera device. The lens of the camera device is facing the center of the corner point image, ensuring that the interface of the camera device to collect images is basically covered by the corner point image screen, and the corner point image is within the focus range of the camera device. At this time, the camera device collects reference images for the corner point image in a clean state. After that, the camera device collects target images for the corner point image in the cleaning process of the dirty state, and then obtains the target image.

[0085] In this step, after obtaining the target image and the reference image, the camera device cleanliness evaluation equipment extracts the number of target corner points of the target image and the number of standard corner points of the reference image, compares the number of target corner points of the target image with the number of standard corner points of the reference image, and obtains the cleanliness evaluation result of the camera device.

[0086] Further, step 1021 includes:

[0087] Step 10211, comparing the number of target corner points of the target image with the number of standard corner points of the reference image to determine the pollution degree corresponding to the target image;

[0088] In this step, the camera device cleanliness evaluation equipment obtains multiple frames of target images, compares the number of target corner points in each frame of the target image with the number of standard corner points in the reference image, and obtains a corresponding ratio, which is used to describe the degree of contamination of the target image.

[0089] Specifically, as the cleaning process proceeds, the distribution and area of ​​contaminants on the lens of the camera device will change in real time. Part of the area of ​​the corner point image blocked by contaminants cannot detect corner points, so the corner point detection results presented will also change. The number of target corner points of the target image of the i-th frame is recorded as ni, and the ratio of target images with different frame numbers, that is, the degree of contamination, is recorded as α=ni / m, where m is the standard number of corner points of the reference image, and the value range of α is (0,1). When the degree of contamination of the camera device is greater, the area covered by the contaminants on the lens is larger, the number of corner points detected is smaller, and the value of α is smaller at this time; when the degree of contamination of the camera device is smaller, the area covered by the contaminants on the lens is smaller, the number of corner points detected is more, and the value of α is larger at this time.

[0090] Step 10212: Determine an evaluation result of the camera device based on the pollution level.

[0091] In this step, the imaging device cleaning evaluation apparatus determines an evaluation result of the imaging device based on the contamination degree of each frame of the target image.

[0092] Specifically, step 10212 includes:

[0093] Step 102121, generating a pollution degree change curve based on the acquisition time of the target image and the pollution degree;

[0094] In this step, the camera device cleanliness evaluation equipment generates an evaluation curve of the ratio changing over time according to the acquisition time and ratio corresponding to multiple frames of target images. The evaluation curve is used to represent the changing trend of the contamination degree of the target image over time. Therefore, the evaluation curve is a contamination degree change curve.

[0095] Step 102122: Determine the evaluation result of the camera device based on the pollution degree change curve.

[0096] In this step, the camera device cleanliness evaluation device determines the evaluation result of the camera device based on the pollution degree change curve. Specifically, the camera device cleanliness evaluation device determines the total time taken for the pollution degree to drop to the preset pollution degree threshold according to the pollution degree change curve, and uses the total time as the evaluation result of the camera device.

[0097] The camera device cleanliness evaluation device of this embodiment compares the number of target corner points of the target image with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the camera device. The degree of contamination is determined by comparing the number of corner points, and then the evaluation result of the camera device is determined, thereby avoiding subjective cleanliness evaluation of the camera device and improving the accuracy of the cleanliness evaluation of the camera device.

[0098] In one embodiment, before comparing the number of target corner points of the target image with the number of standard corner points of the reference image, the method includes:

[0099] Step a1, obtaining a first pixel feature value corresponding to each pixel point of the target image, and determining the number of target corner points of the target image according to the first pixel feature value;

[0100] In this step, after acquiring the target image, the camera device cleanliness evaluation device acquires the first pixel characteristic value corresponding to each pixel point of the target image for each frame of the target image, and determines the number of target corner points of the target image according to the first pixel characteristic value.

[0101] Specifically, obtaining a first pixel feature value corresponding to each pixel point of the target image includes:

[0102] Step a11, performing gradient calculation on the target image to obtain the horizontal gradient and vertical gradient corresponding to the target image;

[0103] Step a12: extracting a first pixel feature value corresponding to each pixel point in the target image based on the horizontal gradient and the vertical gradient.

[0104] In this step, for each frame of the target image, the camera device cleanliness evaluation device performs gradient calculation on the target image to obtain the horizontal gradient and vertical gradient corresponding to the target image, and extracts the first pixel feature value corresponding to each pixel point in the target image based on the horizontal gradient and the vertical gradient. Specifically, the camera device cleanliness evaluation device performs grayscale processing and binarization processing on the target image, and based on the gradients of the grayscale and binarized image in the x and y directions, i.e., I x and I y And build the covariance matrix based on the gradient. Consider the change of pixel grayscale value in a local window in the image. Harris corner detection uses the following covariance matrix: By calculating the covariance matrix, the first pixel eigenvalues ​​λ1 and λ2 corresponding to each pixel point can be obtained.

[0105] Furthermore, the specific process of the camera device cleanliness evaluation device determining the number of target corner points of the target image according to the first pixel characteristic value is as follows: for each pixel point, the camera device cleanliness evaluation device uses the first pixel characteristic value to calculate the corner point response function R=det(M)-k(trace(M)) 2, where det(M) = λ1·λ2 is the determinant of the covariance matrix, trace(M) = λ1+λ2 is the trace of the covariance matrix, k is a constant, and R is the target value. If it is determined that the target value is greater than the set threshold, it is determined that there is a corner point in the pixel. The camera device cleanliness evaluation device performs the above processing on each pixel point, and then determines whether there is a corner point in the pixel point, and then counts the number of target corner points of the target image.

[0106] Step a2: obtaining a second pixel characteristic value corresponding to each pixel point of the reference image, and determining the number of standard corner points of the reference image according to the second pixel characteristic value.

[0107] In this step, the camera device cleanliness evaluation device performs gradient calculation on the reference image to obtain the horizontal gradient and vertical gradient corresponding to the reference image, and extracts the second pixel characteristic value corresponding to each pixel point in the reference image based on the horizontal gradient and the vertical gradient; for each pixel point, the camera device cleanliness evaluation device uses the second pixel characteristic value to calculate the corner point response function R = det (M) - k (trace (M)) 2 . Wherein det(M)=λ1·λ2 is the determinant of the covariance matrix, trace(M)=λ1+λ2 is the trace of the covariance matrix, k is a constant, and R is a target value. If it is determined that the target value is greater than the set threshold, it is determined that there is a corner point in the pixel. The camera device cleanliness evaluation device performs the above processing on each pixel point, and then determines whether there is a corner point in the pixel point, and then counts the number of standard corner points of the reference image.

[0108] The camera device cleanliness evaluation device of this embodiment obtains the first pixel characteristic value corresponding to each pixel point of the target image, and determines the number of target corner points of the target image according to the first pixel characteristic value; obtains the second pixel characteristic value corresponding to each pixel point of the reference image, and determines the number of standard corner points of the reference image according to the second pixel characteristic value. By obtaining the pixel characteristic value corresponding to each pixel point of the image, the number of corner points is determined, and the accuracy of determining the number of corner points is improved, which in turn helps to improve the accuracy of the subsequent cleanliness evaluation of the camera device determined based on the number of corner points.

[0109] Further, refer to Figure 4 , a third embodiment of the present application is proposed. The difference between the third embodiment and the first embodiment and the second embodiment is that the cleaning evaluation result of the camera device is determined according to the target image and the reference image of the target image, including:

[0110] Step 1022 , comparing the target pixel value matrix of the target image with the standard pixel value matrix of the reference image to obtain a cleanliness evaluation result of the camera device.

[0111] In this step, after obtaining the target image and the reference image, the camera device cleanliness evaluation equipment extracts the target pixel value matrix of the target image and the standard pixel value matrix of the reference image, compares the target pixel value matrix of the target image with the standard pixel value matrix of the reference image, and obtains the cleanliness evaluation result of the camera device.

[0112] It should be noted that the evaluation result of the camera device determined by the degree of pollution cannot further evaluate the degree of pollution for pollutants of the same size but different types in the occlusion area. Therefore, this embodiment uses the target pixel value matrix of the target image and the standard pixel value matrix of the reference image for comparison to obtain the cleanliness evaluation result of the camera device.

[0113] Specifically, step 1022 includes:

[0114] Step 10221, comparing the target pixel value matrix of the target image with the standard pixel value matrix of the reference image to determine the transmittance difference between the target image and the reference image;

[0115] In this step, the camera device cleaning evaluation equipment obtains multiple frames of target images, compares the target pixel value matrix of each frame of the target image with the standard pixel value matrix of the reference image, and obtains the corresponding correlation coefficient, which is used to describe the transmittance difference between the target image and the reference image.

[0116] Specifically, as the cleaning process proceeds, the distribution and area of ​​the contaminants on the lens of the camera device will change in real time, so the transmittance difference between the target image and the reference image will also change. Assuming that the image resolution captured by the camera device is u×v, for each frame of the target image, the target pixel value matrix of the target image and the standard pixel value matrix of the reference image are correlated, and the specific calculation formula is as follows:

[0117]

[0118] Where P is the standard pixel value matrix of the reference image, is the pixel average of the reference image, Q is the target pixel value matrix of the target image of the i-th frame during the cleaning process, is the pixel average of the target image in the i-th frame. The pixel values ​​of the corresponding image positions of stains with different transmittances are different. For example, the pixel values ​​of water and ice stains with high transmittance are larger than those of mud and dust stains with low transmittance. The more obvious the difference in transmittance, the smaller the correlation.

[0119] Step 10222: Determine an evaluation result of the camera device according to the transmittance difference.

[0120] In this step, the imaging device cleaning evaluation apparatus determines an evaluation result of the imaging device based on a transmittance difference between each frame of the target image and the reference image.

[0121] Specifically, step 10222 includes:

[0122] Step 102221, generating a transmittance variation curve based on the acquisition time of the target image and the transmittance difference;

[0123] In this step, the camera device cleanliness evaluation equipment generates an evaluation curve of transmittance difference changing with time according to the acquisition time and transmittance difference corresponding to multiple frames of target images. The evaluation curve is used to represent the change trend of the transmittance of the target image over time. Therefore, the evaluation curve is a transmittance change curve.

[0124] Step 102222: Determine an evaluation result of the camera device based on the transmittance variation curve.

[0125] In this step, the camera device cleanliness evaluation device determines the evaluation result of the camera device based on the transmittance change curve. Specifically, the camera device cleanliness evaluation device determines the total time taken for the transmittance to rise to the preset transmittance threshold according to the transmittance change curve, and uses the total time as the evaluation result of the camera device.

[0126] The camera device cleanliness evaluation device of this embodiment compares the target pixel value matrix of the target image with the standard pixel value matrix of the reference image to obtain the cleanliness evaluation result of the camera device. The degree of contamination is determined by comparing the pixel value matrix, and then the evaluation result of the camera device is determined, thereby avoiding subjective cleanliness evaluation of the camera device and avoiding the inability to further evaluate the degree of contamination of contaminants of the same size but different types in the cleanliness evaluation of the camera device based on the number of corner points, thereby improving the accuracy of the cleanliness evaluation of the camera device.

[0127] Further, refer to Figure 5 , a fourth embodiment of the present application is proposed. The difference between the fourth embodiment and the first to third embodiments is that the camera device cleaning evaluation method further includes:

[0128] Step a, if the camera device is in a dirty state, cleaning the camera device by at least two cleaning schemes, and acquiring at least two frames of target images captured during the cleaning process of the camera device;

[0129] In this step, if the camera device cleaning evaluation device determines that the camera device is in a dirty state, the camera device is cleaned by at least two cleaning schemes, and at least two frames of target images collected during the cleaning process of the camera device are obtained. Specifically, if the camera device cleaning evaluation device determines that the camera device is in a dirty state, the camera device is cleaned by three cleaning schemes, and for each cleaning scheme, the camera device collects a target image during the cleaning process, and the target image is sampled to obtain at least two frames of target images.

[0130] Step b: determining a cleaning evaluation result of the cleaning scheme on the imaging device according to the target image and a reference image of the target image.

[0131] In this step, after obtaining the target image, the camera device cleaning evaluation device compares the target image with the reference image of the target image to determine the cleaning evaluation result of each cleaning scheme for the camera device, wherein the reference image is an image captured when the camera device is in a clean state. Specifically, the cleaning evaluation result is a curve showing the change of the cleaning degree of the camera device over time, wherein the cleaning degree can be reflected by the pollution degree of the target image or by the light transmittance of the target image.

[0132] Further, after step b, the method further comprises:

[0133] Step c, determining the dirt type corresponding to the dirt state;

[0134] Step d: obtaining a target cleaning solution corresponding to the optimal target cleaning evaluation result, and associating and saving the dirt type and the target cleaning solution.

[0135] In steps c to d, after evaluating each cleaning scheme to obtain a cleaning rating result, the camera device cleaning evaluation equipment determines the type of dirt corresponding to the dirt state of the camera device, obtains the target cleaning scheme corresponding to the optimal target cleaning evaluation result, and associates and saves the dirt type and the target cleaning scheme.

[0136] Optionally, the cleaning evaluation result is a curve showing how the degree of contamination of the imaging device changes over time. For each cleaning scheme, the imaging device cleaning evaluation device determines the total time taken for the degree of contamination to drop to a preset contamination threshold value based on the contamination change curve, uses the total time as the evaluation result of the cleaning scheme, determines the cleaning scheme with the smallest total time as the target cleaning scheme, and associates and saves the dirt type with the target cleaning scheme. Optionally, the cleaning evaluation result is a curve showing how the transmittance of the imaging device changes over time. For each cleaning scheme, the imaging device cleaning evaluation device determines the total time taken for the transmittance to rise to a preset transmittance threshold value based on the transmittance change curve, uses the total time as the evaluation result of the cleaning scheme, determines the cleaning scheme with the smallest total time as the target cleaning scheme, and associates and saves the dirt type with the target cleaning scheme.

[0137] Further, after step d, the method further comprises:

[0138] Step e, if the camera device is in a dirty state, obtaining the dirt type of the dirty state;

[0139] Step f, obtaining a target cleaning solution corresponding to the dirt type from a pre-created target cleaning solution database;

[0140] Step g: cleaning the imaging device based on the target cleaning solution.

[0141] In steps e to g, the camera device cleaning evaluation equipment associates and saves the dirt type and the target cleaning solution, and then sends the associated and saved dirt type and target cleaning solution to the vehicle. During the actual operation of the vehicle, if it is determined that the camera device is in a dirty state, the dirt type of the dirty state is obtained, and the target cleaning solution corresponding to the dirt type is obtained from a pre-created target cleaning solution database, and the camera device is cleaned based on the target cleaning solution.

[0142] The camera device cleaning evaluation device of this embodiment obtains at least two frames of target images collected during the cleaning process of the camera device by each cleaning scheme; and determines the cleaning evaluation result of the camera device by the cleaning scheme based on the target image and the reference image of the target image. The cleaning evaluation result of the camera device is determined by comparing the target image of the camera device during the cleaning process of the cleaning scheme with the reference image collected when the camera device is in a clean state, thereby avoiding subjective cleaning evaluation of the cleaning effect of the cleaning scheme and improving the accuracy of the cleaning evaluation of the camera device by the cleaning scheme.

[0143] Accordingly, reference Figure 6 The embodiment of the present application further provides a camera device cleaning evaluation device, the camera device cleaning evaluation device comprising:

[0144] An acquisition module 1001 is used to acquire at least two frames of target images collected by the camera device during the cleaning process;

[0145] The determination module 1002 is used to determine the cleanliness evaluation result of the camera device according to the target image and a reference image of the target image, where the reference image is an image captured by the camera device in a clean state.

[0146] Optionally, the determination module is further configured to:

[0147] The target image is an image captured for the calibration image when the camera device is dirty;

[0148] The reference image is an image acquired by the camera device in a clean state for the calibration image;

[0149] The calibration image includes at least one of a corner point image, a grid image, a stripe image, and a circular pattern image.

[0150] Optionally, the determination module is further configured to:

[0151] The number of target corner points of the target image is compared with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the imaging device.

[0152] Optionally, the determination module is further configured to:

[0153] Acquire a first pixel characteristic value corresponding to each pixel point of the target image, and determine the number of target corner points of the target image according to the first pixel characteristic value;

[0154] A second pixel characteristic value corresponding to each pixel point of the reference image is obtained, and the number of standard corner points of the reference image is determined according to the second pixel characteristic value.

[0155] Optionally, the determination module is further configured to:

[0156] Performing gradient calculation on the target image to obtain a horizontal gradient and a vertical gradient corresponding to the target image;

[0157] Based on the horizontal gradient and the vertical gradient, a first pixel feature value corresponding to each pixel point in the target image is extracted.

[0158] Optionally, the determination module is further configured to:

[0159] Comparing the number of target corner points of the target image with the number of standard corner points of the reference image to determine the degree of contamination corresponding to the target image;

[0160] An evaluation result of the imaging device is determined based on the contamination level.

[0161] Optionally, the determination module is further configured to:

[0162] generating a pollution degree change curve based on the acquisition time of the target image and the pollution degree;

[0163] Based on the pollution degree variation curve, an evaluation result of the camera device is determined.

[0164] Optionally, the determination module is further configured to:

[0165] The target pixel value matrix of the target image is compared with the standard pixel value matrix of the reference image to obtain a cleanliness evaluation result of the imaging device.

[0166] Optionally, the determination module is further configured to:

[0167] Comparing a target pixel value matrix of the target image with a standard pixel value matrix of the reference image to determine a transmittance difference between the target image and the reference image;

[0168] An evaluation result of the camera device is determined according to the transmittance difference.

[0169] Optionally, the determination module is further configured to:

[0170] generating a transmittance variation curve based on the acquisition time of the target image and the transmittance difference;

[0171] Based on the transmittance variation curve, an evaluation result of the camera device is determined.

[0172] Optionally, the acquisition module is also used to:

[0173] Acquire the target image collected by the camera device during the cleaning process;

[0174] The target image is sampled to obtain at least two frames of target images.

[0175] Optionally, the acquisition module is also used to:

[0176] If the camera device is in a dirty state, the camera device is cleaned by at least two cleaning schemes, and at least two frames of target images collected during the cleaning process of the camera device are obtained.

[0177] Optionally, the determination module is further configured to:

[0178] A cleaning evaluation result of the cleaning scheme on the imaging device is determined according to the target image and a reference image of the target image.

[0179] Optionally, the camera device cleaning evaluation device further includes an association module, and the association module is used to:

[0180] Determining a dirt type corresponding to the dirt state;

[0181] A target cleaning scheme corresponding to the optimal target cleaning evaluation result is obtained, and the dirt type and the target cleaning scheme are associated and saved.

[0182] Optionally, the camera device cleaning evaluation device further includes a cleaning module, and the cleaning module is used to:

[0183] If the camera device is in a dirty state, obtaining the dirt type of the dirty state;

[0184] Acquire a target cleaning solution corresponding to the dirt type in a pre-created target cleaning solution database;

[0185] The imaging device is cleaned based on the target cleaning scheme.

[0186] This embodiment acquires at least two frames of target images captured during the cleaning process of the camera device; determines the cleanliness evaluation result of the camera device based on the target image and a reference image of the target image, wherein the reference image is an image captured when the camera device is in a clean state. By comparing the target image during the cleaning process of the camera device with the reference image captured when the camera device is in a clean state, the cleanliness evaluation result of the camera device is determined, thereby avoiding subjective human cleanliness evaluation of the camera device and improving the accuracy of the cleanliness evaluation of the camera device.

[0187] Accordingly, the embodiment of the present application further provides a vehicle, which is arranged in the vehicle, such as Figure 7 As shown, Figure 7 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle 1100 includes a processor 1101 having one or more processing cores, a memory 1102 having one or more computer-readable storage media, and a computer program stored in the memory 1102 and executable on the processor. The processor 1101 is electrically connected to the memory 1102. It will be understood by those skilled in the art that the vehicle structure shown in the figure does not constitute a limitation on the vehicle, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0188] The processor 1101 is the control center of the vehicle 1100, and uses various interfaces and lines to connect various parts of the entire vehicle 1100. By running or loading software programs and / or units stored in the memory 1102, and calling data stored in the memory 1102, it executes various functions of the vehicle 1100 and processes data, thereby monitoring the vehicle 1100 as a whole. The processor 1101 can be a processor CPU, a graphics processor GPU, a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0189] In an embodiment of the present application, the processor 1101 in the vehicle 1100 will load the computer program corresponding to the process of one or more applications into the memory 1102 according to the following steps, and the processor 1101 will run the application stored in the memory 1102 to execute the camera device cleaning evaluation method.

[0190] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0191] Optional, such as Figure 7 As shown, the vehicle 1100 further includes: a touch screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107, respectively. Those skilled in the art will appreciate that Figure 7 The vehicle structure shown in the figure does not constitute a limitation on the vehicle, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0192] The touch display screen 1103 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 1103 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the vehicle, which can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-Emitting Diode) and the like. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or accessory such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute the corresponding program. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 1101, and can receive the command sent by the processor 1101 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event, and then the processor 1101 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 1103 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 1103 can also be used as a part of the input unit 1106 to realize the input function.

[0193] The RF circuit 1104 may be used to send and receive RF signals to establish wireless communication with network devices or other vehicles through wireless communication, and to send and receive signals with network devices or other vehicles.

[0194] The audio circuit 1105 can be used to provide an audio interface between the user and the vehicle through a speaker and a microphone. The audio circuit 1105 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1105 and converted into audio data, and then the audio data is output to the processor 1101 for processing, and then sent to another vehicle through the radio frequency circuit 1104, or the audio data is output to the memory 1102 for further processing. The audio circuit 1105 may also include an earplug jack to provide communication between an external headset and the vehicle.

[0195] The input unit 1106 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0196] The power supply 1107 is used to supply power to various components of the vehicle 1100. Optionally, the power supply 1107 can be logically connected to the processor 1101 through a power management device, so that the power management device can manage charging, discharging, and power consumption. The power supply 1107 can also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0197] although Figure 7 Not shown, the vehicle 1100 may also include a camera device, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0198] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0199] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0200] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0201] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0202] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0203] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the descriptions of each embodiment in all the embodiments of the present application have different focuses, for the parts not described in detail in a certain embodiment, reference can be made to the relevant embodiments of other embodiments. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A camera device cleanliness evaluation method, characterized in that: The method comprises: Acquire at least two frames of target images collected by the camera device during the cleaning process; A cleanliness evaluation result of the imaging device is determined according to the target image and a reference image of the target image, wherein the reference image is an image captured when the imaging device is in a clean state.

2. The imaging device cleaning evaluation method according to claim 1, characterized in that: The target image is an image acquired for the calibration image when the camera device is dirty; The reference image is an image acquired by the camera device in a clean state for the calibration image; The calibration image includes at least one of a corner point image, a grid image, a stripe image, and a circular pattern image.

3. The imaging device cleaning evaluation method according to claim 2, characterized in that: The calibration image is a corner point image; and determining the cleanliness evaluation result of the camera device according to the target image and the reference image of the target image includes: The number of target corner points of the target image is compared with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the imaging device.

4. The imaging device cleaning evaluation method according to claim 3, characterized in that: Before comparing the number of target corner points of the target image with the number of standard corner points of the reference image, the method includes: Acquire a first pixel characteristic value corresponding to each pixel point of the target image, and determine the number of target corner points of the target image according to the first pixel characteristic value; A second pixel characteristic value corresponding to each pixel point of the reference image is obtained, and the number of standard corner points of the reference image is determined according to the second pixel characteristic value.

5. The imaging device cleaning evaluation method according to claim 4, characterized in that: The obtaining of a first pixel characteristic value corresponding to each pixel point of the target image includes: Performing gradient calculation on the target image to obtain a horizontal gradient and a vertical gradient corresponding to the target image; Based on the horizontal gradient and the vertical gradient, a first pixel feature value corresponding to each pixel point in the target image is extracted.

6. The imaging device cleaning evaluation method according to claim 3, characterized in that: The step of comparing the number of target corner points of the target image with the number of standard corner points of the reference image to obtain a cleanliness evaluation result of the imaging device includes: Comparing the number of target corner points of the target image with the number of standard corner points of the reference image to determine the degree of contamination corresponding to the target image; An evaluation result of the imaging device is determined based on the contamination level.

7. The imaging device cleaning evaluation method according to claim 6, characterized in that: The step of determining an evaluation result of the camera device based on the pollution degree includes: generating a pollution degree change curve based on the acquisition time of the target image and the pollution degree; Based on the pollution degree variation curve, an evaluation result of the camera device is determined.

8. The imaging device cleaning evaluation method according to claim 1, characterized in that: The step of determining a cleanliness evaluation result of the imaging device according to the target image and a reference image of the target image includes: The target pixel value matrix of the target image is compared with the standard pixel value matrix of the reference image to obtain a cleanliness evaluation result of the imaging device.

9. The imaging device cleaning evaluation method according to claim 8, characterized in that: The step of comparing the target pixel value matrix of the target image with the standard pixel value matrix of the reference image to obtain a cleanliness evaluation result of the imaging device includes: Comparing a target pixel value matrix of the target image with a standard pixel value matrix of the reference image to determine a transmittance difference between the target image and the reference image; An evaluation result of the camera device is determined according to the transmittance difference.

10. The imaging device cleaning evaluation method according to claim 9, characterized in that: The step of determining the evaluation result of the camera device according to the transmittance difference includes: generating a transmittance variation curve based on the acquisition time of the target image and the transmittance difference; Based on the transmittance variation curve, an evaluation result of the camera device is determined.

11. The imaging device cleaning evaluation method according to claim 1, characterized in that: The step of acquiring at least two frames of target images collected by the camera device during the cleaning process includes: Acquire the target image collected by the camera device during the cleaning process; The target image is sampled to obtain at least two frames of target images.

12. The camera device cleaning evaluation method according to any one of claims 1 to 11, characterized in that: The method further comprises: If the camera device is in a dirty state, the camera device is cleaned by at least two cleaning schemes, and at least two frames of target images captured during the cleaning process of the camera device are acquired; The step of determining a cleanliness evaluation result of the imaging device according to the target image and a reference image of the target image includes: A cleaning evaluation result of the cleaning scheme on the imaging device is determined according to the target image and a reference image of the target image.

13. The imaging device cleaning evaluation method according to claim 12, characterized in that: After determining the cleanliness evaluation result of the imaging device according to the target image and the reference image of the target image, the method includes: Determining a dirt type corresponding to the dirt state; A target cleaning scheme corresponding to the optimal target cleaning evaluation result is obtained, and the dirt type and the target cleaning scheme are associated and saved.

14. The imaging device cleaning evaluation method according to claim 13, characterized in that: After obtaining the target cleaning solution corresponding to the optimal target cleaning evaluation result and associating and saving the dirt type and the target cleaning solution, the method includes: If the camera device is in a dirty state, obtaining the dirt type of the dirty state; Acquire a target cleaning solution corresponding to the dirt type in a pre-created target cleaning solution database; The imaging device is cleaned based on the target cleaning scheme.

15. A camera device cleaning evaluation device, characterized in that: The camera device cleaning evaluation device comprises: An acquisition module, used to acquire at least two frames of target images collected by the camera device during the cleaning process; A determination module is used to determine a cleanliness evaluation result of the camera device according to the target image and a reference image of the target image, wherein the reference image is an image captured when the camera device is in a clean state.

16. A camera device cleaning evaluation device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of computer programs, and the computer programs are suitable for being loaded by a processor to execute the steps of the method according to any one of claims 1 to 14.

18. A vehicle, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 14.

Images