A boundary propagation method and device for image raindrop removal
By identifying raindrop outlines and filling them with surrounding colors, the image is divided into blocks for raindrop removal, solving the problem of limited computing power of onboard ADCUs and achieving efficient and fast image raindrop removal.
Patent Information
- Application Number
- CN202310222706.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-09
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2043-03-09
AI Technical Summary
In existing technologies, vehicle-mounted ADCUs, due to their limited computing power, cannot effectively process complex images containing raindrops and cannot recognize untrained scenes, resulting in poor raindrop removal performance.
By recognizing the outlines of raindrops in an image and filling them with the colors of the surrounding environment, the system divides regions into blocks and fills them with color until the raindrops are completely filled, reducing computational load and making it suitable for various scenarios.
It achieves efficient and fast raindrop removal in images with low computational cost, no scene limitations, and is suitable for a variety of scenarios.
Smart Images
Figure CN116245976B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of data optimization processing of visual sensors, in particular to a boundary propagation method and device for image raindrop removal. BACKGROUND
[0002] The vehicle-mounted camera continuously captures images in real time when the vehicle is running. In rainy weather, raindrops in the image will interfere with the line of sight of human passengers and will also interfere with the target recognition of the image processing algorithm, so image raindrop removal is a relatively active research branch.
[0003] The prior art uses a generative adversarial network (GAN) to recover an image without raindrops from an image containing raindrops. The basic principle is to establish a corresponding relationship between "with raindrops" and "without raindrops" for the same or similar pair of scenes by training a neural network. For example, take a picture of the window outside the study with and without raindrops, establish a corresponding relationship between the two images, and then input the image with raindrops into the trained neural network to output (generate) the corresponding image without raindrops. However, this method has a large amount of neural network calculations, and the vehicle-mounted ADCU (Automated Driving Control Unit) has limited computing power due to cost, weight, power supply, etc., and cannot handle more complex pictures, cannot recognize scenes that did not appear during training, and has limited application scenarios. SUMMARY
[0004] In view of the defects in the prior art, the purpose of the present application is to provide a boundary propagation method and device for image raindrop removal, which uses the color of the surrounding environment immediately around each raindrop to extend to the inside of the raindrop and combines the color of the internal color block of the raindrop to fill the color of the raindrop until the entire raindrop is filled, achieving the effect of image raindrop removal. The required amount of calculation is small, the speed of processing pictures is fast and efficient, and there is no scene restriction, making it suitable for a variety of scenarios.
[0005] To achieve the above purpose, the technical solution adopted by the present application is:
[0006] The contour line of the raindrop in the image is identified, and the color near the outside of the raindrop contour line is identified, and based on the identified color:
[0007] If the color near the outside of the raindrop contour line is not a single color, the raindrop is divided into multiple region blocks based on the boundary lines between the colors, and the region block color is filled based on the color near the region block.
[0008] If the color near the outside of the raindrop contour line is a single color, the inside of the raindrop is filled with the color.
[0009] On the basis of the above technical solutions, if the color near the raindrop contour line is not a single color, the raindrop is divided into multiple region blocks based on the boundary lines between the colors, and the color of the region block is filled based on the color near the region block, and the specific steps include:
[0010] When the color near the raindrop contour line is not a single color, the color boundary lines between the colors are identified;
[0011] Based on the identified color boundary lines, the color boundary lines are extended to divide the raindrop into multiple region blocks;
[0012] For the divided region block, if the current region block contains the raindrop contour line, the current region block is filled with the color outside the raindrop contour line, and if the current region block does not contain the raindrop contour line, the current region block is filled with the color of other region blocks near the current region block.
[0013] On the basis of the above technical solutions, based on the identified color boundary lines, the color boundary lines are extended to divide the raindrop into multiple region blocks, and the specific steps include:
[0014] According to the identified color boundary lines:
[0015] If the color boundary line has an intersection with the raindrop contour line, the color boundary line is defined as a shooting line;
[0016] The shooting line is extended to the inside of the raindrop and out of the raindrop, and the raindrop is regionally divided to obtain region blocks.
[0017] On the basis of the above technical solutions, the shooting line is extended to the inside of the raindrop and out of the raindrop, and the specific steps include:
[0018] Based on the intersection of the current shooting line and the raindrop contour line, a tangent line of the current shooting line is made, and the tangent line is extended to the inside of the raindrop and out of the raindrop.
[0019] On the basis of the above technical solutions, if the current region block contains the raindrop contour line, the current region block is filled with the color outside the raindrop contour line, and if the current region block does not contain the raindrop contour line, the current region block is filled with the color of other region blocks near the current region block, and the specific steps include:
[0020] If the current region block contains the raindrop contour line, the boundary of the current region block is identified, and according to the identified boundary:
[0021] If the boundary of the region block contains only one raindrop contour line, the region block is filled with the color outside the raindrop contour line;
[0022] if the two raindrop contour lines are contained in the boundary of the region block, all colors outside the two raindrop contour lines are obtained, the obtained colors are pooled, and the pooled colors are used for color filling of the current region block;
[0023] if the current region block does not contain a raindrop contour line, other region blocks near the current region block are used for color filling of the current region.
[0024] On the basis of the above technical scheme, if the two raindrop contour lines are contained in the boundary of the region block, all colors outside the two raindrop contour lines are obtained, the obtained colors are pooled, and the pooled colors are used for color filling of the current region block, and the specific steps include:
[0025] When the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and it is judged whether the obtained colors are the same:
[0026] If the same, the obtained color is used for color filling of the region block;
[0027] If not, the obtained colors are pooled, and the pooled colors are used for color filling of the region block.
[0028] On the basis of the above technical scheme, if the current region block does not contain a raindrop contour line, other region blocks near the current region block are used for color filling of the current region, and the specific steps include:
[0029] When the current region block does not contain a raindrop contour line, the colors of all adjacent region blocks around the region block are obtained, the obtained colors are pooled, and the pooled colors are used for color filling of the current region block.
[0030] On the basis of the above technical scheme, the specific steps of pooling the obtained colors include:
[0031] The obtained colors are averaged and pooled to obtain a single color, and the color is used for uniform filling of the region block.
[0032] On the basis of the above technical scheme, before identifying the colors near the raindrop contour line, it further includes:
[0033] All color blocks inside the raindrop are obtained, and according to the obtained color blocks:
[0034] If the color block inside the raindrop has no intersection with the raindrop contour line, the ratio of the area of the current color block to the area of the raindrop where the current color block is located is calculated:
[0035] When the ratio is greater than a preset ratio, the region where the current color block is located is not subjected to color filling processing.
[0036] when the ratio is not greater than the preset ratio, the region where the current color block is located is subjected to color filling processing.
[0037] If there is an intersection between the color block inside the raindrop and the raindrop contour line, the region where the current color block is located is subjected to color filling processing.
[0038] The application further provides a device for boundary spreading of raindrop-removed images, comprising:
[0039] a recognition module for recognizing a contour line of a raindrop in an image, recognizing colors near the raindrop contour line, and recognizing demarcation lines between the colors near the raindrop contour line;
[0040] an execution module for filling the raindrop inside with a color if the colors near the raindrop contour line are single colors according to the colors near the raindrop contour line recognized by the recognition module, and dividing the raindrop into a plurality of region blocks based on the demarcation lines between the colors and filling the colors of the region blocks based on the colors near the region blocks if the colors near the raindrop contour line are non-single colors.
[0041] Compared with the prior art, the application has the advantages that the colors of the raindrops are filled by extending the colors of the surrounding environment near each raindrop to the inside of the raindrop and combining the colors of the color blocks inside the raindrop until the entire raindrop is filled, thereby achieving the effect of raindrop-removed images. The application requires a small amount of calculation, has a high speed and efficiency in processing pictures, and has no scene limitation and is suitable for various scenes. BRIEF DESCRIPTION OF DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.
[0043] Figure 1 The flowchart of the method for boundary spreading of raindrop-removed images in an embodiment of the present application is shown.
[0044] Figure 2 The diagram for dividing region blocks by extending the color demarcation lines in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0045] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments.
[0046] Referring to Figure 1 As shown in the drawings, the embodiment of the present application provides a boundary propagation method for removing raindrops from an image, comprising the following steps:
[0047] S1: identifying the contour line of the raindrop in the image, and identifying the color near the contour line of the raindrop, according to the identified color, if the color near the contour line of the raindrop is not a single color, go to S2, if the color near the contour line of the raindrop is a single color, go to S3;
[0048] S2: dividing the raindrop into multiple region blocks based on the boundary line between each color, and filling the color of the region block based on the color near the region block;
[0049] S3: filling the inside of the raindrop with the color.
[0050] That is, first, the contour of the raindrop in the image is identified, the contour line of the raindrop in the image is identified, and the color near the contour line of the raindrop is identified, it is judged whether the identified color near the contour line of the raindrop is a single color, if the color near the contour line of the raindrop is not a single color, the raindrop is divided into multiple region blocks based on the boundary line between each color near the contour line of the raindrop, and the color of all region blocks is filled according to the color near the region block, if the color near the contour line of the raindrop is a single color, the color of the region block is filled with the color of the single color, and the raindrop in the image is removed.
[0051] In the present application, if the color near the contour line of the raindrop is not a single color, the raindrop is divided into multiple region blocks based on the boundary line between each color, and the color of the region block is filled based on the color near the region block, the specific steps comprising:
[0052] When the color near the contour line of the raindrop is not a single color, the color boundary line between each color is identified;
[0053] Based on the identified color boundary line, the color boundary line is extended to divide the raindrop into multiple region blocks;
[0054] For the divided region blocks, if the current region block contains the contour line of the raindrop, the current region block is filled with the color outside the contour line of the raindrop, if the current region block does not contain the contour line of the raindrop, the current region block is filled with the color of other region blocks near the current region block.
[0055] As Figure 2 shown, the color near the contour line of the raindrop is not only one, Figure 2The colors near the raindrop contour line outside are two, which are distinguished by gray and white in the figure; if the colors near the raindrop contour line outside are not the same, the raindrop is divided into multiple area blocks based on the color boundary lines between the colors, and the specific steps of color filling of the area blocks according to the colors near the area blocks are as follows: if the colors near the raindrop contour line outside are more than one, the color boundary lines between the colors are identified, and the raindrop is divided into multiple area blocks by extending the color boundary lines, Figure 2 The extension lines of the color boundary lines divide the raindrop into two area blocks, namely area block 1 and area block 2; based on the divided area blocks, it is judged whether the current area block contains the raindrop contour line, if the current area block contains the contour line, the color near the raindrop contour line outside contained in the current area block is used to fill the current area block, if the current area block does not contain the raindrop contour line, that is, the area block is an internal block, the edges of the area block are all composed of extension lines, then the color of the other area blocks near the current area block is used to fill the color of the current area block. Figure 2 The edges of the area block 1 and the area block 2 in the figure both contain the raindrop contour line, so the color near the raindrop contour line outside contained in the area block 1, that is, gray, is used to fill the area block 1, and the color near the raindrop contour line outside contained in the area block 2, that is, white, is used to fill the area block 2.
[0056] In the present application, the specific steps of extending the color boundary lines to divide the raindrop into multiple area blocks based on the identified color boundary lines include:
[0057] According to the identified color boundary lines:
[0058] If the color boundary line intersects with the raindrop contour line, the color boundary line is defined as a shooting line;
[0059] The shooting line is extended to the inside of the raindrop and penetrates out of the raindrop, and the raindrop is divided into area blocks.
[0060] That is, the specific steps of extending the color boundary lines to divide the raindrop into multiple area blocks according to the identified color boundary lines are as follows: first, identify the color boundary lines near the raindrop contour line outside, and then judge according to the identified color boundary lines; if the color boundary line intersects with the raindrop contour line, the color boundary line is set as a shooting line, as shown in Figure 2 , and Figure 2 If the two color boundary lines in the figure both intersect with the raindrop contour line, the two color boundary lines are set as shooting lines; then the shooting line is extended to the inside of the raindrop and penetrates through the raindrop, and the extension line of the shooting line divides the raindrop into area blocks.
[0061] In the present application, the incident line is extended to the inside of the raindrop and out of the raindrop, and the specific steps include:
[0062] Based on the intersection of the current incident line and the raindrop contour line, a tangent line of the current incident line is made, and the tangent line is extended to the inside of the raindrop and out of the raindrop.
[0063] That is, the specific steps of extending the incident line are based on the intersection of the current incident line and the raindrop contour line, making a tangent line of the current incident line, and extending the tangent line to the inside of the raindrop and out of the raindrop. The incident line can be a straight line or a curve. The curved incident line is extended in the form of a straight line according to its original trend, which is a first-order differential approximation.
[0064] In the present application, if the current region block contains a raindrop contour line, the current region block is filled with the color outside the raindrop contour line, and if the current region block does not contain a raindrop contour line, the current region block is filled with the color of other region blocks near the current region block. The specific steps include:
[0065] If the current region block contains a raindrop contour line, the boundary of the current region block is identified, and according to the identified boundary:
[0066] If the boundary of the region block contains only one raindrop contour line, the region block is filled with the color outside the raindrop contour line;
[0067] If the boundary of the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and the obtained colors are pooled, and the pooled colors are used to fill the current region block with color;
[0068] If the current region block does not contain a raindrop contour line, the current region is filled with the color of other region blocks near the current region block.
[0069] That is, if the current region block contains a raindrop contour line, the color near the raindrop contour line outside the current region block is used to fill the current region block with color, and if the current region block does not contain a raindrop contour line, the color of other region blocks near the current region block is used to fill the current region block with color. The specific steps are as follows: If the current region block contains a raindrop contour line, the boundary of the current region block is identified, and according to the identified boundary of the region block, if the boundary of the current region block contains only one raindrop contour line, the color near the raindrop contour line outside the region block is used to fill the region block with color, and if the boundary of the current region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and the obtained colors are pooled, and the pooled colors are used to fill the current region block with color; Figure 2As shown, the boundary of the region block 1 is composed of a raindrop contour line and an extension line, and when the region block 1 is filled with color, the color near the raindrop contour line contained in the region block 1 is used to fill the region block 1 with color; if the boundary of the current region block contains two raindrop contour lines, all colors near the two raindrop contour lines are obtained and pooled, and the pooled color is used to fill the current region block with color; if the current region block does not contain a raindrop contour line, that is, the boundary of the current region block is composed of extension lines, the color of other region blocks near the current region block is used to fill the current region block with color.
[0070] In the present application, if the boundary of the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and the obtained colors are pooled, and the pooled color is used to fill the current region block with color, and the specific steps include:
[0071] When the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and it is judged whether the obtained colors are the same:
[0072] If the same, the obtained color is used to fill the region block with color;
[0073] If not the same, the obtained colors are pooled, and the pooled color is used to fill the region block with color.
[0074] That is, if the boundary of the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained and pooled, and the pooled color is used to fill the current region block with color, and the specific steps are that when the boundary of the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and it is judged whether the obtained colors are the same color, if the same color, the obtained color is directly used to fill the current region block with color, if not the same color, the colors are first pooled, and then the pooled color is used to fill the current region block with color.
[0075] In the present application, if the current region block does not contain a raindrop contour line, the color of other region blocks near the current region is used to fill the current region, and the specific steps include:
[0076] When the current region block does not contain a raindrop contour line, the colors of all adjacent region blocks around the region block are obtained, the obtained colors are pooled, and the pooled color is used to fill the current region block with color.
[0077] If the current region block does not contain the raindrop contour line in the boundary, the specific steps of color filling of the current region block using the color of other region blocks near the current region block are as follows: if the current region block does not contain the raindrop contour line in the boundary, the color of the region block adjacent to the current region block is obtained, the obtained color is pooled, and the current region block is color filled using the pooled color.
[0078] In the present application, the specific steps of pooling the obtained color include:
[0079] The obtained color is average-pooled to obtain a single color, and the region block is uniformly filled using the color.
[0080] That is, the specific steps of pooling the obtained color in the present application include color average-pooling the obtained two or more colors to obtain the mean value (the mean value of each color channel) of the color, and uniformly filling the region block using the color. Filling the region block in the raindrop with uniform color is also a differential approximation. The raindrop itself is very small, and the region block inside is smaller. It can be considered that each region block is a uniform color.
[0081] In the present application, before identifying the color near the raindrop contour line, it further includes:
[0082] All color blocks inside the raindrop are obtained, and according to the obtained color blocks:
[0083] If the color block inside the raindrop has no intersection with the raindrop contour line, the ratio of the area of the current color block to the area of the raindrop where the current color block is located is calculated:
[0084] When the ratio is greater than a preset ratio, the region where the current color block is located is not subjected to color filling processing.
[0085] When the ratio is not greater than the preset ratio, the region where the current color block is located is subjected to color filling processing.
[0086] If the color block inside the raindrop has intersection with the raindrop contour line, the region where the current color block is located is subjected to color filling processing.
[0087] That is, before identifying the raindrop contour line in the image, the color block inside the raindrop also needs to be identified, the identified color block is obtained and judged, if the color block inside the raindrop has no intersection with the raindrop contour line of the raindrop, the ratio of the area of the current color block to the area of the raindrop is calculated, when the calculated area ratio is greater than a preset ratio, the area covered by the current color block is not subjected to subsequent color filling, and the final display color of the current color block covered area is the original color; when the calculated area ratio is not greater than the preset ratio, the area covered by the current color block is subjected to subsequent color filling, and the final display color of the current color block covered area is the color after color filling.
[0088] When all the region blocks in a single raindrop are filled, the raindrop disappears from the image, and is filled with the surrounding environment color, realizing the image optimization of raindrop removal.
[0089] In a possible implementation, the embodiment of the present application also provides a readable storage medium, which is located in a PLC (Programmable Logic Controller, programmable logic controller) controller, and a computer program is stored on the readable storage medium, the program is executed by a processor to realize the steps of the image raindrop removal boundary spreading method described below:
[0090] The contour line of the raindrop in the image is identified, and the color near the raindrop contour line is identified, and the color identified is used to:
[0091] If the color near the raindrop contour line is not a single color, the raindrop is divided into multiple region blocks based on the boundary line between each color, and the color filling of the region block is performed based on the color near the region block;
[0092] If the color near the raindrop contour line is a single color, the inside of the raindrop is filled with the color.
[0093] The storage medium can be one or any combination of computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus or device.
[0094] The computer readable signal medium can include a computer readable program code in a baseband or propagated as a carrier wave in a propagation medium. The propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.
[0095] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).
[0096] The embodiment of the present application also provides a boundary propagation device for image raindrop removal, comprising:
[0097] The identification module is used for identifying the contour line of the raindrop in the image, identifying the color near the raindrop contour line, and identifying the boundary line between each color near the raindrop contour line.
[0098] The execution module is used for performing the following operation according to the color near the raindrop contour line identified by the identification module: if the color near the raindrop contour line is a single color, the execution module fills the raindrop with the color; if the color near the raindrop contour line is a non-single color, the execution module divides the raindrop into a plurality of region blocks based on the boundary line between each color, and fills the region blocks with the color based on the color near the region blocks.
[0099] The execution module is used for performing the following operation according to the color near the raindrop contour line identified by the identification module: if the color near the raindrop contour line is a single color, the execution module fills the raindrop with the color; if the color near the raindrop contour line is a non-single color, the execution module divides the raindrop into a plurality of region blocks based on the boundary line between each color, and fills the region blocks with the color based on the color near the region blocks.
[0100] The above merely provides the specific implementation of the present application, and enables the person skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be subject to the widest scope consistent with the principles and novel features shown herein.
[0101] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or a plurality of flows and / or blocks Figure 1 one flow or a plurality of flows and / or blocks
Claims
1. A boundary propagation method for image deraining, characterized in that, The method comprises the following steps: The contour line of the raindrop in the image is identified, and the color near the contour line of the raindrop is identified, and the color near the contour line of the raindrop is identified according to the identified color: If the color near the contour line of the raindrop is not a single color, the raindrop is divided into multiple region blocks based on the boundary line between the colors, and the color of the region block is filled based on the color near the region block: If the color near the contour line of the raindrop is a single color, the inside of the raindrop is filled with the color. Before identifying the color near the contour line of the raindrop, all color blocks inside the raindrop are obtained, and the color blocks obtained are: If the color blocks inside the raindrop have no intersection with the contour line of the raindrop, the ratio of the area of the current color block to the area of the raindrop where the current color block is located is calculated: -When the ratio is greater than a preset ratio, the region where the current color block is located is not subjected to color filling processing; -When the ratio is not greater than the preset ratio, the region where the current color block is located is subjected to color filling processing; If the color blocks inside the raindrop have intersection with the contour line of the raindrop, the region where the current color block is located is subjected to color filling processing.
2. The boundary propagation method for image raindrop removal according to claim 1, wherein, If the color near the contour line of the raindrop is not a single color, the raindrop is divided into multiple region blocks based on the boundary line between the colors, and the color of the region block is filled based on the color near the region block, and the specific steps comprise: When the color near the contour line of the raindrop is not a single color, the color boundary line between the colors is identified; Based on the identified color boundary line, the color boundary line is extended to divide the raindrop into multiple region blocks; For the divided region blocks, if the current region block contains the contour line of the raindrop, the current region block is filled with the color outside the contour line of the raindrop, and if the current region block does not contain the contour line of the raindrop, the current region block is filled with the color of other region blocks near the current region block.
3. The boundary propagation method for image raindrop removal of claim 2, wherein, Based on the identified color boundary line, the color boundary line is extended to divide the raindrop into multiple region blocks, and the specific steps comprise: According to the identified color boundary line: If the color boundary line has intersection with the contour line of the raindrop, the color boundary line is defined as a shooting line; The shooting line is extended to the inside of the raindrop and out of the raindrop, and the raindrop is regionally divided to obtain region blocks.
4. The boundary propagation method for image raindrop removal of claim 3, wherein, The specific steps of extending the shooting line to the inside of the raindrop and out of the raindrop comprise: Based on the intersection of the current shooting line and the contour line of the raindrop, a tangent line of the current shooting line is made, and the tangent line is extended to the inside of the raindrop and out of the raindrop.
5. The method of claim 4, wherein the boundary propagation is performed by: determining a first boundary of the raindrop candidate; determining a second boundary of the raindrop candidate; and determining a third boundary of the raindrop candidate based on the first and second boundaries. If the current region block contains the contour line of the raindrop, the contour of the current region block is identified, and according to the identified contour: -If the contour of the region block contains only one contour line of the raindrop, the region block is filled with the color outside the contour line of the raindrop; -If the contour of the region block contains two contour lines of the raindrop, all colors outside the two contour lines of the raindrop are obtained, and the obtained colors are pooled, and the current region block is filled with the pooled color; If the current region block does not contain the contour line of the raindrop, the color of other region blocks near the current region block is used to fill the current region block. If the current region block does not contain a raindrop contour line, the current region is filled with the color of other region blocks near the current region.
6. A boundary propagation method for image deraining as claimed in claim 5, wherein, If the boundary of the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, the obtained colors are pooled, and the current region block is filled with the pooled color. When the region block contains two raindrop contour lines, all colors outside the two raindrop contour lines are obtained, and it is determined whether the obtained colors are the same. If the colors are the same, the region block is filled with the obtained color. If the colors are not the same, the obtained colors are pooled, and the region block is filled with the pooled color.
7. The method of claim 5, wherein the boundary propagation is performed by using a morphological closing operation. If the current region block does not contain a raindrop contour line, the current region is filled with the color of other region blocks near the current region, and the specific steps include: When the current region block does not contain a raindrop contour line, the colors of all adjacent region blocks around the region block are obtained, the obtained colors are pooled, and the current region block is filled with the pooled color.
8. A method of boundary propagation for image deraining according to claim 6 or 7, wherein, The specific steps of pooling the obtained colors include: The obtained colors are averaged to obtain a single color, and the region block is uniformly filled with the color.
9. A boundary propagation apparatus for image deraining, characterized by, The specific steps include: An identification module is configured to identify the contour line of a raindrop in an image, identify the color near the contour line outside the raindrop, and identify the boundary between each color near the contour line outside the raindrop. An execution module is configured to fill the inside of the raindrop with the color near the contour line outside the raindrop based on the color identified by the identification module: if the color near the contour line outside the raindrop is a single color, the raindrop is divided into multiple region blocks based on the boundary between each color, and the color of each region block is filled based on the color near the region block. The identification module is further configured to obtain all color blocks inside the raindrop The execution module is further configured to obtain all color blocks inside the raindrop based on the obtained color blocks. If the color block inside the raindrop has no intersection with the contour line of the raindrop, the ratio of the area of the current color block to the area of the raindrop where the current color block is located is calculated: When the ratio is greater than a preset ratio, the region where the current color block is located is not filled with color. When the ratio is not greater than the preset ratio, the region where the current color block is located is filled with color. If the color block inside the raindrop has an intersection with the contour line of the raindrop, the region where the current color block is located is filled with color.