Water stain detection method and device for vehicle-mounted camera, medium and equipment

Abstract

The invention provides a water stain detection method and device for a vehicle-mounted camera, a medium and equipment. The water stain detection method comprises the following steps: shooting by adopting the vehicle-mounted camera to obtain a detection image; edge detection is carried out on the detection image to obtain an edge pixel point set, and edge pixel points enclose a first candidate area; performing semantic segmentation on the detection image to obtain a second candidate region; fusing the first candidate region and the second candidate region to obtain a water stain region in the detection image; according to the method, a first candidate area and a second candidate area containing a water stain area are obtained through edge detection and semantic segmentation respectively and fused to obtain the water stain area in a detection image, so that whether water stains exist on a lens of the vehicle-mounted camera or not is determined, and respective advantages of edge detection and semantic segmentation are utilized for mutual verification; the accuracy of water spot area detection is improved, so that the accuracy of water spot detection on a vehicle-mounted camera is improved, and a more accurate data basis is provided for subsequent intelligent driving or auxiliary driving.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, device, medium and equipment for detecting water stains in a vehicle-mounted camera. Background Art

[0002] With the continuous development of automobile intelligence, more and more cars have adopted intelligent driving or assisted driving systems, and cameras, as one of the environmental data collection devices for intelligent driving and assisted driving, are also widely used in intelligent driving cars and assisted driving systems. In order to collect environmental image data outside the car, the camera needs to be exposed to the outside of the car, and the moisture in the environment will form water stains such as water droplets or water mist on the lens of the camera. The presence of water stains will affect the picture quality and blurriness of the image obtained by the camera, thereby affecting the calculation results of the subsequent algorithm model and causing misjudgment. Therefore, timely detection of water droplets and water mist on the camera lens and clearing them is conducive to improving the accuracy and safety of intelligent driving and assisted driving. Since lane lines or other obstacles will affect the detection results of water stains, resulting in inaccurate water stain detection results, if misidentification occurs, a clearing operation will be performed, thereby increasing system energy consumption. If missed detection occurs, it may affect subsequent calculation results. Therefore, a method that can improve the accuracy of camera lens water stain recognition is urgently needed. Summary of the invention

[0003] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a water stain detection method, device, medium and equipment for a vehicle-mounted camera.

[0004] According to one aspect of the present application, a method for detecting water stains using a vehicle-mounted camera is provided, comprising: obtaining a detection image by photographing with the vehicle-mounted camera; performing edge detection on the detection image to obtain a set of edge pixel points; wherein the set of edge pixel points includes a plurality of edge pixel points, and the edge pixel points form a first candidate area; performing semantic segmentation on the detection image to obtain a second candidate area; wherein the second candidate area is a region semantically identified as containing water stains in the detection image; and fusing the first candidate area and the second candidate area to obtain a water stain area in the detection image.

[0005] In one embodiment, the fusing of the first candidate area and the second candidate area to obtain the water stain area in the detection image includes: calculating the degree of overlap between the first candidate area and the second candidate area; if the degree of overlap is greater than a preset overlap threshold, determining that the corresponding first candidate area and the second candidate area are successfully matched; and fusing the successfully matched first candidate area and the second candidate area to obtain the water stain area in the detection image.

[0006] In one embodiment, the fusing of the successfully matched first candidate area and the second candidate area to obtain the water stain area in the detection image includes: taking the intersection area of ​​the successfully matched first candidate area and the second candidate area as the water stain area in the detection image.

[0007] In one embodiment, after the first candidate area and the second candidate area are fused to obtain the water stain area in the detection image, the water stain detection method of the vehicle-mounted camera further includes: correcting the water stain area in the detection image to obtain a corrected water stain area.

[0008] In one embodiment, the detection image includes a continuous multi-frame image; wherein, the correction of the water stain area in the detection image to obtain the corrected water stain area includes: taking the water stain area in the continuous multi-frame image as the target area, binarizing the continuous multi-frame image to obtain a continuous multi-frame binary image; adding pixel values ​​of corresponding pixels in the continuous multi-frame binary image to obtain a combined image; and taking the area where the pixel points in the combined image whose pixel values ​​are equal to the first preset value are located as the corrected water stain area.

[0009] In one embodiment, the detected image includes a continuous plurality of frames of images; wherein, the correction of the water stain area in the detected image to obtain the corrected water stain area includes: calculating the pixel variance of the water stain area of ​​each image in the continuous plurality of frames of images; calculating the difference between the pixel variances of the same water stain area in adjacent frames of images; if there is a difference between two of the differences that is greater than a second preset value, then the water stain area corresponding to the difference that is greater than the second preset value is used as the corrected water stain area.

[0010] In one embodiment, after edge detection is performed on the detection image to obtain a set of edge pixel points, the water stain detection method of the vehicle-mounted camera further includes: calculating the gradient amplitude of the edge pixel points; wherein the gradient amplitude represents the amplitude of change of the pixel difference between the edge pixel point and the adjacent pixel points; selecting low-amplitude edge pixel points in a strong texture area and high-amplitude edge pixel points in a weak texture area as candidate edge pixel points; wherein the detection image includes multiple texture areas, the strong texture area represents a texture area in which the variance between the pixel points in the area is greater than a preset variance threshold, the weak texture area represents a texture area in which the variance between the pixel points in the area is less than or equal to the variance threshold, the low-amplitude edge pixel points represent edge pixel points in which the gradient amplitude is less than the preset amplitude threshold, and the high-amplitude edge pixel points represent edge pixel points in which the gradient amplitude is greater than or equal to the amplitude threshold; the candidate edge pixel points surround the first candidate area.

[0011] According to another aspect of the present application, a water stain detection device for a vehicle-mounted camera is provided, comprising: a detection image acquisition module, used to obtain a detection image captured by the vehicle-mounted camera; a first region acquisition module, used to perform edge detection on the detection image to obtain a set of edge pixel points; wherein the set of edge pixel points includes a plurality of edge pixel points, and the edge pixel points form a first candidate region; a second region acquisition module, used to perform semantic segmentation on the detection image to obtain a second candidate region; wherein the second candidate region is a region in the detection image that contains water stains through semantic identification; and a water stain region acquisition module, used to fuse the first candidate region and the second candidate region to obtain a water stain region in the detection image.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute any of the above methods.

[0013] According to another aspect of the present application, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and the processor is used to execute any of the above-described methods.

[0014] The present application provides a method, device, medium and equipment for detecting water stains of a vehicle-mounted camera, which obtains a detection image by shooting with a vehicle-mounted camera; performs edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes multiple edge pixel points, and the edge pixel points surround a first candidate area; performs semantic segmentation on the detection image to obtain a second candidate area; wherein the second candidate area is an area containing water stains in the semantic recognition detection image; and fuses the first candidate area and the second candidate area to obtain a water stain area in the detection image; that is, shoots the detection image with a vehicle-mounted camera, obtains a first candidate area and a second candidate area containing the water stain area by edge detection and semantic segmentation respectively, and fuses the first candidate area and the second candidate area to obtain the water stain area in the detection image, thereby determining whether there is water stain on the lens of the vehicle-mounted camera, and using the respective advantages of edge detection and semantic segmentation to verify each other to improve the accuracy of water stain area detection, thereby improving the accuracy of water stain detection on the vehicle-mounted camera, and then providing a more accurate data basis for subsequent intelligent driving or assisted driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 It is a flowchart of a water stain detection method for a vehicle-mounted camera provided by an exemplary embodiment of the present application.

[0017] Figure 2 It is a schematic structural diagram of a water stain detection device for a vehicle-mounted camera provided by an exemplary embodiment of the present application.

[0018] Figure 3 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] Figure 1 FIG. 1 is a flow chart of a water stain detection method for a vehicle-mounted camera provided by an exemplary embodiment of the present application. Figure 1 As shown, the water stain detection method of the vehicle-mounted camera includes the following steps: Step 110: Use the vehicle-mounted camera to capture and obtain a detection image.

[0021] The present application uses a vehicle-mounted camera that needs to detect water stains to obtain a detection image, and analyzes the detection image obtained by the vehicle-mounted camera to determine whether there is a water stain area image in the detection image, thereby determining whether there are water droplets or water mist on the lens of the vehicle-mounted camera.

[0022] Step 120: Perform edge detection on the detection image to obtain a set of edge pixel points.

[0023] Among them, the edge pixel point set includes multiple edge pixel points, and the edge pixel points surround the first candidate area. The present application performs edge detection on the detection image to obtain an edge pixel point set including multiple edge pixel points, and the edge pixel points surround the first candidate area, wherein the first candidate area may include multiple, such as water stains, white lane lines in the background, curbs, railings, vehicles, etc., which may appear as an area surrounded by edge pixels in the detection image. Optionally, the present application smoothes the detection image through Gaussian filtering to remove noise while retaining the details and edges in the detection image, and then performs edge detection through operators such as Canny or Sobel to obtain an edge pixel point set, that is, to obtain the first candidate area containing the water stain area, so as to narrow the detection range for subsequent identification of the water stain area.

[0024] Step 130: Perform semantic segmentation on the detection image to obtain a second candidate region.

[0025] The second candidate region is the region containing water stains in the semantic recognition detection image. The present application performs semantic segmentation on the detection image to obtain the sky, road, water stains and other backgrounds (including lane lines, street lights, guardrails, etc.), and then extracts the pixel point region with the semantic attribute of water stains to obtain the second candidate region.

[0026] Step 140: The first candidate region and the second candidate region are fused to obtain a water stain region in the detection image.

[0027] Since the first candidate area may include white lane lines, curbs, railings, vehicles, etc., and semantic segmentation also has certain errors, the present application obtains the water stain area in the detection image by fusing the first candidate area and the second candidate area after obtaining the first candidate area and the second candidate area through edge detection and semantic segmentation respectively, thereby improving the accuracy of water stain area detection.

[0028] The present application provides a method for detecting water stains of a vehicle-mounted camera, which obtains a detection image by photographing with a vehicle-mounted camera; performs edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes multiple edge pixel points, and the edge pixel points surround a first candidate area; performs semantic segmentation on the detection image to obtain a second candidate area; wherein the second candidate area is an area containing water stains in the semantic recognition detection image; and fuses the first candidate area and the second candidate area to obtain a water stain area in the detection image; that is, the detection image is photographed with a vehicle-mounted camera, and the first candidate area and the second candidate area containing the water stain area are obtained by edge detection and semantic segmentation respectively, and the first candidate area and the second candidate area are fused to obtain the water stain area in the detection image, so as to determine whether there is water stain on the lens of the vehicle-mounted camera, and use the respective advantages of edge detection and semantic segmentation to verify each other to improve the accuracy of water stain area detection, thereby improving the accuracy of water stain detection on the vehicle-mounted camera, and then providing a more accurate data basis for subsequent intelligent driving or assisted driving.

[0029] In one embodiment, the specific implementation method of the above-mentioned step 140 may be: calculating the degree of overlap between the first candidate area and the second candidate area; if the degree of overlap is greater than a preset overlap threshold, determining that the corresponding first candidate area and the second candidate area are successfully matched; and fusing the successfully matched first candidate area and the second candidate area to obtain the water stain area in the detection image.

[0030] The present application determines whether the first candidate area and the second candidate area match by calculating the degree of overlap between the first candidate area and the second candidate area, that is, by calculating the degree of overlap between the first candidate area and the second candidate area, to determine whether the first candidate area and the second candidate area are the same water stain area. If the degree of overlap between the first candidate area and the second candidate area is greater than the overlap threshold, it means that the first candidate area and the second candidate area are the same water stain area. At this time, a water stain area is obtained by merging the first candidate area and the second candidate area.

[0031] Optionally, the present application may use an IOU algorithm to calculate the degree of overlap between the first candidate region and the second candidate region, wherein the calculation formula for the degree of overlap is: IOU= ; in, represents the area of ​​the intersection of the first candidate area and the second candidate area, The IOU, which represents the area of ​​the union of the first candidate region and the second candidate region, represents the degree of overlap between the first candidate region and the second candidate region. The range of IOU is [0,1]. The larger the IOU, the larger the overlap area between the first candidate region and the second candidate region, that is, the higher the matching degree.

[0032] It should be understood that the first candidate area in the present application may include multiple, and the second candidate area may also include multiple. If there are multiple first candidate areas and / or multiple second candidate areas, a single first candidate area or a single second candidate area is matched and fused respectively. For example, if multiple first candidate areas are included, the degree of overlap between the first candidate area and each second candidate area (which may be one or more) can be calculated for each first candidate area. If the degree of overlap between the first candidate area and a second candidate area is greater than the overlap threshold, it means that the first candidate area and the second candidate area are the same water stain area. At this time, a water stain area is obtained by fusing the first candidate area and the corresponding second candidate area. Similarly, the matching relationship between all first candidate areas and all second candidate areas can be obtained, and then the corresponding water stain area is obtained by fusing.

[0033] In one embodiment, a specific implementation of the above step 140 may be: using the intersection area of ​​the successfully matched first candidate area and the second candidate area as the water stain area in the detection image.

[0034] If the above IOU value is greater than the overlap threshold, it is determined that the corresponding first candidate area and the second candidate area are matched successfully, that is, the first candidate area and the second candidate area are the same water stain area. At this time, the intersection area of ​​the first candidate area and the second candidate area is selected as the water stain area to ensure the credibility of the water stain area.

[0035] In one embodiment, after step 140, the water stain detection method for the vehicle-mounted camera may further include: correcting the water stain area in the detection image to obtain a corrected water stain area.

[0036] After fusing the first candidate area and the second candidate area to obtain the water stain area in the detection image, the present application corrects the water stain area to further filter out the interference area therein, thereby further improving the accuracy of water stain area detection.

[0037] In one embodiment, the detection image includes a continuous multi-frame image; wherein, the specific implementation method of correcting the water stain area in the detection image may be: taking the water stain area in the continuous multi-frame image as the target area, binarizing the continuous multi-frame image to obtain a continuous multi-frame binary image; adding the pixel values ​​of corresponding pixels in the continuous multi-frame binary image to obtain a combined image; and taking the area where the pixel points in the combined image whose pixel values ​​are equal to the first preset value are located as the corrected water stain area.

[0038] The present application can use a vehicle-mounted camera that needs to detect water stains to continuously shoot to obtain continuous multi-frame images, and analyze the continuous multi-frame images to filter out interference areas (such as areas where street lamps, railings, etc. are located). Since street lamps, railings, etc. will move continuously in continuous multi-frame images, while lane lines, curbs and water stains are relatively fixed in continuous multi-frame images, the present application can first perform binarization processing on continuous multi-frame images, that is, the water stain area obtained by fusing the first candidate area and the second candidate area is used as the target area (pixel value is set to 1), and other areas are used as background areas (pixel values ​​are set to 0), and then the pixel values ​​of corresponding pixels of continuous multi-frame images are added, and the area where the pixel point with a pixel value equal to the first preset value (number of image frames) is located is selected as the corrected water stain area, so as to further exclude interference areas such as street lamps and railings.

[0039] In one embodiment, the detection image includes multiple consecutive frame images; wherein, the specific implementation method of correcting the water stain area in the detection image may be: calculating the pixel variance of the water stain area of ​​each image in the multiple consecutive frame images; calculating the difference between the pixel variances of the same water stain area in adjacent frame images; if there is a difference between two differences greater than a second preset value, then the water stain area corresponding to the difference greater than the second preset value is used as the corrected water stain area.

[0040] The present application can use a vehicle-mounted camera that needs to detect water stains to continuously shoot to obtain continuous multi-frame images, and analyze the continuous multi-frame images to filter out interference areas (such as lane lines, curbs, etc.). Due to the uneven distribution of water droplets and water mist, the direction of their optical flow is unstable, while lane lines and curbs extend in a straight line, and their optical flow direction is relatively stable. Therefore, the present application calculates the pixel variance of the water stain area in the continuous multi-frame images to determine the flow direction of the area (that is, the direction in which the pixel variance of the area changes along the same trend, such as the direction in which the pixel variance decreases or increases), and determines whether the flow direction of each area is continuous. If the pixel variance of the same water stain in adjacent frame images changes continuously (that is, the difference is small), the water stain area with continuously changing variance is determined to be an interference area to further exclude interference areas such as lane lines and curbs.

[0041] In one embodiment, after step 120, the water stain detection method of the above-mentioned vehicle-mounted camera may further include: calculating the gradient amplitude of edge pixels; wherein the gradient amplitude represents the amplitude of change of pixel difference between the edge pixel and adjacent pixels; selecting low-amplitude edge pixels in the strong texture area and high-amplitude edge pixels in the weak texture area as candidate edge pixels; wherein the detection image includes multiple texture areas, the strong texture area represents a texture area in which the variance between pixels in the area is greater than a preset variance threshold, the weak texture area represents a texture area in which the variance between pixels in the area is less than or equal to the variance threshold, the low-amplitude edge pixels represent edge pixels with a gradient amplitude less than a preset amplitude threshold, and the high-amplitude edge pixels represent edge pixels with a gradient amplitude greater than or equal to the amplitude threshold; the candidate edge pixels surround a first candidate area.

[0042] After obtaining the edge pixel set, the present application calculates the gradient of each edge pixel in the edge pixel set in two mutually perpendicular directions (for example, horizontal direction and vertical direction) G x and G x , calculate the gradient amplitude of the edge pixel point based on the gradient of the edge pixel point in two perpendicular directions G ,in, . When water stains appear in a strong texture area, the water stain area will be more blurred than other areas of the strong texture area, and when water stains appear in a weak texture area, the water stain area will have a stronger texture than other areas of the weak texture area. Therefore, the present application selects low-amplitude edge pixels in the strong texture area and high-amplitude edge pixels in the weak texture area to improve the accuracy of edge detection to obtain water stain areas. Among them, texture intensity represents the prominence or clarity of the texture. Strong texture appears obvious and high contrast, while weak texture appears blurred or not obvious. The present application can use a 10×10 pixel frame as a texture area, calculate the variance of each texture area, and use the maximum texture value of the sky and the road as the variance threshold. The area with a variance greater than the variance threshold is defined as a strong texture area, and vice versa.

[0043] Figure 2 FIG. 1 is a schematic diagram of a water stain detection device for a vehicle-mounted camera provided by an exemplary embodiment of the present application. Figure 2 As shown, the water stain detection device 20 of the vehicle-mounted camera includes: a detection image acquisition module 21, which is used to obtain a detection image by taking a vehicle-mounted camera; a first area acquisition module 22, which is used to perform edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes multiple edge pixel points, and the edge pixel points surround a first candidate area; a second area acquisition module 23, which is used to perform semantic segmentation on the detection image to obtain a second candidate area; wherein the second candidate area is an area containing water stains in the semantic recognition detection image; a water stain area acquisition module 24, which is used to fuse the first candidate area and the second candidate area to obtain a water stain area in the detection image.

[0044] The present application provides a method for detecting water stains of a vehicle-mounted camera, which uses a vehicle-mounted camera to shoot a detection image through a detection image acquisition module 21; a first area acquisition module 22 performs edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes multiple edge pixel points, and the edge pixel points surround a first candidate area; a second area acquisition module 23 performs semantic segmentation on the detection image to obtain a second candidate area; wherein the second candidate area is an area containing water stains in the semantic recognition detection image; a water stain area acquisition module 24 fuses the first candidate area and the second candidate area to obtain a water stain area in the detection image; that is, a vehicle-mounted camera is used to shoot a detection image, and a first candidate area and a second candidate area containing water stains are obtained by edge detection and semantic segmentation respectively, and the first candidate area and the second candidate area are fused to obtain the water stain area in the detection image, so as to determine whether there is water stain on the lens of the vehicle-mounted camera, and the respective advantages of edge detection and semantic segmentation are used for mutual verification to improve the accuracy of water stain area detection, thereby improving the accuracy of water stain detection on the vehicle-mounted camera, and then providing a more accurate data basis for subsequent intelligent driving or assisted driving.

[0045] In one embodiment, the water stain area acquisition module 24 can be further configured to: calculate the degree of overlap between the first candidate area and the second candidate area; if the degree of overlap is greater than a preset overlap threshold, determine that the corresponding first candidate area and the second candidate area are successfully matched; and fuse the successfully matched first candidate area and the second candidate area to obtain the water stain area in the detection image.

[0046] In one embodiment, the water stain area acquisition module 24 may be further configured to use the intersection area of ​​the successfully matched first candidate area and the second candidate area as the water stain area in the detection image.

[0047] In one embodiment, the water stain detection device 20 of the vehicle-mounted camera may be further configured to correct the water stain area in the detection image to obtain a corrected water stain area.

[0048] In one embodiment, the detection image includes a continuous multi-frame image; wherein the water stain detection device 20 of the above-mentioned vehicle-mounted camera can be further configured as: taking the water stain area in the continuous multi-frame image as the target area, binarizing the continuous multi-frame image to obtain a continuous multi-frame binary image; adding the pixel values ​​of corresponding pixels in the continuous multi-frame binary image to obtain a combined image; and taking the area where the pixel points in the combined image whose pixel values ​​are equal to the first preset value are located as the corrected water stain area.

[0049] In one embodiment, the detected image includes multiple consecutive frames of images; wherein the water stain detection device 20 of the above-mentioned vehicle-mounted camera can be further configured to: calculate the pixel variance of the water stain area of ​​each image in the multiple consecutive frames of images; calculate the difference between the pixel variances of the same water stain area in adjacent frame images; if there is a difference between two differences greater than a second preset value, then the water stain area corresponding to the difference greater than the second preset value is used as the corrected water stain area.

[0050] In one embodiment, the water stain detection device 20 of the above-mentioned vehicle-mounted camera can be further configured as: calculating the gradient amplitude of edge pixel points; wherein the gradient amplitude represents the amplitude of change of pixel difference between edge pixel points and adjacent pixel points; selecting low-amplitude edge pixel points in strong texture areas and high-amplitude edge pixel points in weak texture areas as candidate edge pixel points; wherein the detection image includes multiple texture areas, the strong texture area represents a texture area in which the variance between pixel points in the area is greater than a preset variance threshold, the weak texture area represents a texture area in which the variance between pixel points in the area is less than or equal to the variance threshold, the low-amplitude edge pixel points represent edge pixel points with a gradient amplitude less than a preset amplitude threshold, and the high-amplitude edge pixel points represent edge pixel points with a gradient amplitude greater than or equal to the amplitude threshold; the candidate edge pixel points surround a first candidate area.

[0051] Below, reference Figure 3 The electronic device according to the embodiment of the present application is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signal from them.

[0052] Figure 3 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0053] like Figure 3 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0054] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0055] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0056] In one example, the electronic device 10 may further include: an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0057] When the electronic device is a stand-alone device, the input device 13 may be a communication network connector, which is used to receive the collected input signals from the first device and the second device.

[0058] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0059] The output device 14 can output various information to the outside, including the determined distance information, direction information, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0060] Of course, to simplify, Figure 3 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0061] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0062] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0063] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0064] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0065] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0066] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0067] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0068] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0069] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A water stain detection method for a vehicle-mounted camera, characterized in that: include: The vehicle-mounted camera is used to capture a detection image; Performing edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes a plurality of edge pixel points, and the edge pixel points surround a first candidate area; Perform semantic segmentation on the detection image to obtain a second candidate region; wherein the second candidate region is a region in the detection image that contains water stains through semantic recognition; The first candidate region and the second candidate region are fused to obtain the water stain region in the detection image.

2. The water stain detection method for a vehicle-mounted camera according to claim 1, characterized in that: The fusing the first candidate region and the second candidate region to obtain the water stain region in the detection image comprises: Calculating the overlap degree between the first candidate area and the second candidate area; If the overlap degree is greater than a preset overlap degree threshold, it is determined that the corresponding first candidate region and the second candidate region are matched successfully; The first candidate region and the second candidate region that are successfully matched are fused to obtain the water stain region in the detection image.

3. The water stain detection method for a vehicle-mounted camera according to claim 2, characterized in that: The step of fusing the successfully matched first candidate region and the second candidate region to obtain the water stain region in the detection image comprises: The intersection area of ​​the successfully matched first candidate area and the second candidate area is used as the water stain area in the detection image.

4. The water stain detection method for a vehicle-mounted camera according to claim 1, characterized in that: After the first candidate area and the second candidate area are fused to obtain the water stain area in the detection image, the water stain detection method of the vehicle-mounted camera further includes: The water stain area in the detection image is corrected to obtain a corrected water stain area.

5. The water stain detection method for a vehicle-mounted camera according to claim 4, characterized in that: The detection image includes a plurality of continuous frames of images; wherein the step of correcting the water stain area in the detection image to obtain the corrected water stain area includes: Taking the water stain area in the continuous multi-frame images as the target area, performing binarization processing on the continuous multi-frame images to obtain continuous multi-frame binary images; Adding pixel values ​​of corresponding pixels in the continuous multiple frames of binary images to obtain a combined image; The area where the pixel points in the combined image have pixel values ​​equal to the first preset value is used as the corrected water stain area.

6. The water stain detection method for a vehicle-mounted camera according to claim 4, characterized in that: The detection image includes a plurality of continuous frames of images; wherein the step of correcting the water stain area in the detection image to obtain the corrected water stain area includes: Calculating the pixel variance of the water stain area of ​​each image in the continuous multiple frames of images; Calculate the difference between the pixel variances of the same water stain area in adjacent frame images; If there are two differences whose difference is greater than a second preset value, the water stain area corresponding to the difference whose difference is greater than the second preset value is used as the corrected water stain area.

7. The water stain detection method for a vehicle-mounted camera according to claim 1, characterized in that: After edge detection is performed on the detection image to obtain a set of edge pixel points, the water stain detection method of the vehicle-mounted camera further includes: Calculating the gradient amplitude of the edge pixel point; wherein the gradient amplitude represents the amplitude of the pixel difference change between the edge pixel point and the adjacent pixel point; Low-amplitude edge pixel points in a strong texture area and high-amplitude edge pixel points in a weak texture area are selected as candidate edge pixel points; wherein the detection image includes multiple texture areas, the strong texture area represents a texture area in which the variance between pixel points in the area is greater than a preset variance threshold, the weak texture area represents a texture area in which the variance between pixel points in the area is less than or equal to the variance threshold, the low-amplitude edge pixel points represent edge pixel points in which the gradient amplitude is less than a preset amplitude threshold, and the high-amplitude edge pixel points represent edge pixel points in which the gradient amplitude is greater than or equal to the amplitude threshold; the candidate edge pixel points surround the first candidate area.

8. A water stain detection device for a vehicle-mounted camera, characterized in that: include: A detection image acquisition module, used to obtain a detection image by taking the vehicle-mounted camera; A first region acquisition module is used to perform edge detection on the detection image to obtain an edge pixel point set; wherein the edge pixel point set includes a plurality of edge pixel points, and the edge pixel points surround a first candidate region; A second region acquisition module is used to perform semantic segmentation on the detection image to obtain a second candidate region; wherein the second candidate region is a region semantically identified as containing water stains in the detection image; The water stain area acquisition module is used to fuse the first candidate area and the second candidate area to obtain the water stain area in the detection image.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the method described in any one of claims 1 to 7.

Images