A windshield dirt detection method based on texture features

Abstract

The present invention relates to a windshield dirt detection method based on texture features, comprising: step S1, using threshold judgment based on Sobel edge detection algorithm to divide the original image into texture area and non-texture area; step S2, for transparent or semi-transparent dirt, calculating the edge ratio of two images with different edge strengths extracted from the original image to obtain pixel blur, and then determining dirt candidates according to the threshold division of different texture areas; for opaque dirt, threshold judgment is performed in weak texture areas according to RGB values ​​and average edge strength of pixel blocks to calibrate dirt candidates; step S3, removing false pixels from dirt candidates by multi-frame superposition to determine the dirt area and dirt degree. Compared with the prior art, the present invention has the advantages of high efficiency and convenience and can quantitatively analyze the dirt situation of the windshield.

Description

Technical Field

[0001] The invention relates to the technical field of security monitoring, and in particular to a windshield dirt detection method based on texture features. Background Art

[0002] There are currently two main approaches for detecting dirt on windshields or lenses: (1) using the sensor value to determine whether there is dirt and the amount of dirt; (2) determining based on the visual information of the image, which is more convenient and cost-effective than the former.

[0003] After searching, Chinese patent CN103434487B discloses a method and device for detecting rain on a windshield, which uses at least one sensor to detect the wetness of the windshield. However, if the entire windshield is used as a detection surface, a large number of sensors are required to fully cover it and achieve high accuracy, and other external auxiliary equipment is required, which is costly.

[0004] Chinese patent CN109598706A discloses a camera lens occlusion detection method, which determines whether the camera lens is blocked based on the clarity of the video image, does not require additional time for background modeling, and does not require additional memory to establish long and short buffer areas, and has made great progress in detection accuracy and detection time. However, it can only qualitatively analyze whether occlusion occurs.

[0005] Chinese patent CN109241977A discloses a camera lens occlusion detection method, which uses a depth histogram to statistically determine the foreground depth and background depth based on depth information, effectively improving the accuracy of camera lens occlusion detection. However, this method generally uses a binocular camera, that is, the depth of the object is determined by the information difference between the two cameras, and the evaluation algorithm is based on the difference between the depth histogram of the foreground image frame and the depth histogram of the background model, and can only qualitatively analyze whether occlusion occurs.

[0006] In view of the above defects, it is urgent to design an efficient, convenient and quantitative method to analyze the dirtiness of the windshield. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a windshield dirt detection method based on texture features that is efficient, convenient and can quantitatively analyze the dirtiness of the windshield.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to a first aspect of the present invention, a method for detecting windshield dirt based on texture features is provided, the method comprising the following steps:

[0010] Step S1, using threshold judgment based on Sobel edge detection algorithm to divide the original image into texture area and non-texture area;

[0011] Step S2: For transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain pixel blur, and then the dirt candidate is determined based on the threshold division of different texture areas; for opaque dirt, the threshold is judged based on the RGB value and the average edge strength of the pixel block in the weak texture area to calibrate the dirt candidate;

[0012] Step S3: Remove false pixels from the dirty candidates by superimposing multiple frames to determine the dirty area and the degree of dirtiness.

[0013] Preferably, the step S1 specifically comprises:

[0014] The original image is divided into grid blocks of the same size, and the edge strength of each grid block is calculated using the Sobel edge detection operator; the texture block and non-texture block are divided by comparing the sum of the edge strengths with a preset threshold.

[0015] Preferably, in step S2, for transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain pixel blur, and then the dirt candidate is determined according to the threshold division of different texture areas, specifically:

[0016] 1) Input the original image into two Gaussian filters with different variances to satisfy δ 2 >δ 1 , and obtain two smooth images;

[0017] 2) The two smooth images are processed using the Sobel edge detection algorithm to obtain two different edge intensity images I 2 ,I 1 ;

[0018] 3) Calculate edge intensity image I 2 ,I 1 The edge ratio of the pixel is obtained, and the expression is:

[0019]

[0020] 4) Based on pixel blur, the dirt candidates are determined according to the threshold division of different texture areas.

[0021] Preferably, the determination of dirt candidates based on pixel blur and threshold division of different texture regions is specifically as follows: the condition that pixel p(i, j) is used as a candidate semi-transparent dirt satisfies:

[0022] When the edge intensity image I(i, j) is in the non-texture area and satisfies

[0023]

[0024] When the edge intensity image I(i, j) is in the texture area and satisfies

[0025]

[0026] Among them, T n , T m is the preset threshold; D(i, j) is the pixel blur of pixel p(i, j).

[0027] Preferably, in step S2, for opaque dirt, threshold judgment is performed in the weak texture area according to the RGB value and the average edge strength of the pixel block to calibrate the dirt candidate, specifically:

[0028] In the non-texture area, a 3*3 block centered on the current pixel is selected, and the dirt candidate is calibrated based on the RGB value and whether the average edge strength of the pixel block is greater than a preset threshold.

[0029] Preferably, the R, G, B thresholds corresponding to the RGB value are 110.

[0030] Preferably, the dirt candidates in step S3 include semi-transparent dirt candidates, transparent dirt candidates and opaque dirt candidates.

[0031] Preferably, removing false pixels from dirty semi-transparent candidates by superimposing multiple frames is specifically as follows:

[0032] Set the semi-transparent dirty pixels to 1 and the rest to 0 to get a binary image;

[0033] Take multiple frames of binary images and add them together, and regard the pixels larger than the threshold as semi-transparent dirt, and the rest as non-dirty;

[0034] The degree of dirtiness of the windshield within the field of view of the driving recorder is determined based on the number of calibration points.

[0035] According to a second aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, any one of the methods described above is implemented.

[0036] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, any of the methods described in the foregoing is implemented.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1) The method of the present invention can be used to detect when the front windshield in the field of view of the vehicle-mounted driving recorder is blocked by dirt, and give the driver a timely warning to prevent the recorder from failing. Only visual information is used, and no auxiliary equipment other than the driving recorder is required, which is efficient and convenient;

[0039] 2) The present invention can calibrate the location of dirt, facilitating quantitative analysis;

[0040] 3) The method of the present invention can control the intensity of the warning according to the degree of dirtiness by setting a threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Flow chart of the method of the present invention

[0042] Figure 2 A block diagram of a method for detecting transparent or semi-transparent dirt;

[0043] Figure 3 A schematic diagram showing the distinction between textured areas and non-textured areas corresponding to transparent or semi-transparent dirt;

[0044] Figure 4 Schematic diagram of Sobel edge detection intensity corresponding to transparent or semi-transparent dirt;

[0045] Figure 5 is a comparison diagram of the original image of the driving recorder corresponding to transparent or semi-transparent dirt and the dirt calibration image; wherein, Figure 5 a is the original image, Figure 5 b is the image after dirt calibration;

[0046] Figure 6 It is a block diagram of the detection method for opaque dirt;

[0047] Figure 7 A schematic diagram showing the distinction between the texture area and the non-texture area corresponding to the opaque dirt;

[0048] Figure 8 Schematic diagram of Sobel edge detection intensity corresponding to opaque dirt;

[0049] Fig. 9 is a comparison diagram of the original image of the driving recorder corresponding to the opaque dirt and the dirt calibration image; wherein, Fig. 9 a is the original image, Fig. 9 b is the image after dirt calibration. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0051] Example

[0052] This embodiment provides a windshield dirt detection method based on texture features, which includes the following steps:

[0053] Step S1, dividing the original image into grid blocks of the same size, using the Sobel edge detection operator to calculate the edge strength of each grid block; dividing the texture block and the non-texture block by comparing the sum of the edge strengths with a preset threshold;

[0054] Step S2: For transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain pixel blur, and then the dirt candidate is determined based on the threshold division of different texture areas; for opaque dirt, the threshold is judged based on the RGB value and the average edge strength of the pixel block in the weak texture area to calibrate the dirt candidate;

[0055] A. For transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain the pixel blurriness, and then the dirt candidate is determined based on the threshold division of different texture areas, specifically:

[0056] 1) Input the original image into two Gaussian filters with different variances to satisfy δ 2 >δ 1 , and obtain two smooth images;

[0057] 2) The two smooth images are processed using the Sobel edge detection algorithm to obtain two different edge intensity images I 2 ,I 1 ;

[0058] 3) Calculate edge intensity image I 2 ,I 1 The edge ratio of the pixel is obtained, and the expression is:

[0059]

[0060] 4) Based on pixel blur, the dirt candidates are determined according to the threshold division of different texture areas.

[0061] The method is based on pixel fuzziness and determines the dirt candidate according to the threshold division of different texture areas. Specifically, the condition that the pixel p(i, j) is used as a candidate semi-transparent dirt satisfies:

[0062] When the edge intensity image I(i, j) is in the non-texture area and satisfies

[0063]

[0064] When the edge intensity image I(i, j) is in the texture area and satisfies

[0065]

[0066] Among them, T n , T m is the preset threshold; D(i, j) is the pixel blur of pixel p(i, j).

[0067] B. For opaque dirt, threshold judgment is performed in the weak texture area based on the RGB value and the average edge strength of the pixel block to calibrate the dirt candidate, specifically:

[0068] A 3*3 block centered on the current pixel is taken in the non-texture area, and the dirt candidate is calibrated based on the RGB value and whether the average edge strength of the pixel block is greater than a preset threshold; wherein the R, G, and B thresholds corresponding to the RGB value are all set to 110.

[0069] Step S3: Remove false pixels from the dirt candidates by stacking multiple frames to determine the dirt area and dirt degree. The dirt candidates include semi-transparent dirt candidates, transparent dirt candidates and opaque dirt candidates.

[0070] For the candidate semi-transparent dirt, the false pixel removal process is as follows: 1) set the pixels with semi-transparent dirt to 1 and the rest to 0 to obtain a binary image; 2) add multiple frames of binary images forward, and regard the pixels greater than the threshold as semi-transparent dirt, and the rest as non-dirty; 3) judge the degree of dirtiness of the windshield within the field of view of the driving recorder based on the number of calibration points.

[0071] The detection process of the Sobel edge detection algorithm adopted in this embodiment is specifically as follows: firstly, the Sobel operator is used to extract grayscale difference information, that is, the image gradient value, and then a single threshold is used to further extract boundary information.

[0072] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0073] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0074] The processing unit performs the various methods and processes described above, such as methods S1 to S3. For example, in some embodiments, methods S1 to S3 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S3 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S3 in any other appropriate manner (e.g., by means of firmware).

[0075] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0076] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0077] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0078] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A windshield dirt detection method based on texture features, characterized in that: The method comprises the following steps: Step S1, using threshold judgment based on Sobel edge detection algorithm to divide the original image into texture area and non-texture area; Step S2: For transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain pixel blur, and then the dirt candidate is determined based on the threshold division of different texture areas; for opaque dirt, the threshold is judged based on the RGB value and the average edge strength of the pixel block in the weak texture area to calibrate the dirt candidate; Step S3, removing false pixels from the dirty candidates by superimposing multiple frames, and determining the dirty area and the degree of dirtiness; In step S2, for transparent or semi-transparent dirt, the edge ratio of two images with different edge strengths extracted from the original image is calculated to obtain pixel blur, and then the dirt candidate is determined according to the threshold division of different texture areas, specifically: 1) Input the original image into two Gaussian filters with different variances, satisfying δ2>δ1, and obtain two smoothed images; 2) Using the Sobel edge detection algorithm to process the two smoothed images, two different edge intensity images I2 and I1 are obtained; 3) Calculate the edge ratio of edge intensity images I2 and I1 to obtain pixel blur, which is expressed as: 4) Based on pixel blur, the dirt candidates are determined according to the threshold division of different texture areas.

2. The method for detecting windshield dirt based on texture features according to claim 1, characterized in that: The step S1 is specifically as follows: The original image is divided into grid blocks of the same size, and the edge strength of each grid block is calculated using the Sobel edge detection operator; the texture block and non-texture block are divided by comparing the sum of the edge strengths with a preset threshold.

3. The method for detecting windshield dirt based on texture features according to claim 1, characterized in that: The method is based on pixel fuzziness and determines the dirt candidate according to the threshold division of different texture areas. Specifically, the condition that the pixel p(i, j) is used as a candidate for semi-transparent dirt satisfies: When the edge intensity image I(i,j) is in the non-texture area and satisfies When the edge intensity image I(i,j) is in the texture area and satisfies Among them, T n , T m is the preset threshold; D(i,j) is the pixel blur of pixel p(i,j).

4. The method for detecting windshield dirt based on texture features according to claim 1, characterized in that: In step S2, for opaque dirt, threshold judgment is performed in the weak texture area based on the RGB value and the average edge strength of the pixel block to calibrate the dirt candidate, specifically: In the non-texture area, a 3*3 block centered on the current pixel is selected, and the dirt candidate is calibrated based on the RGB value and whether the average edge strength of the pixel block is greater than a preset threshold.

5. The method for detecting windshield dirt based on texture features according to claim 1, characterized in that: The R, G, and B thresholds corresponding to the RGB value are all set to 110.

6. The method for detecting windshield dirt based on texture features according to claim 1, characterized in that: The dirt candidates in step S3 include semi-transparent dirt candidates, transparent dirt candidates and opaque dirt candidates.

7. The method for detecting windshield dirt based on texture features according to claim 6, characterized in that: The method of removing false pixels from semi-transparent dirty candidates by superimposing multiple frames is specifically as follows: Set the semi-transparent dirty pixels to 1 and the rest to 0 to get a binary image; Take multiple frames of binary images and add them together, and regard the pixels larger than the threshold as semi-transparent dirt, and the rest as non-dirty; The degree of dirtiness of the windshield within the field of view of the driving recorder is determined based on the number of calibration points.

8. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images