Rain line processing method and system for low-illumination image

Abstract

The invention relates to a rain line processing method and system for a low-illumination image. The method comprises the following steps: collecting a rain line-containing image of which the illumination is lower than a threshold value; extracting an image feature of the rain-line-containing image, and extracting a first feature based on a rain line region part in the image feature; extracting rain line priori knowledge of the rain line-containing image, fusing the first feature with the priori knowledge to obtain a first fusion result, and extracting a second feature based on the first fusion result; and extracting rain line semantic information of the rain line-containing image, fusing the second feature with the semantic information to obtain a second fusion result, and reconstructing a rain line-free image based on the second fusion result. Compared with the prior art, the rain lines in the low-illumination image can be accurately removed, details are prevented from being lost and artifacts are prevented from being generated in the rain line removing process of the image, and the method has the advantages of being high in accuracy and definition, high in adaptability under the low-illumination condition and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for processing rain lines in low-illuminance images. Background Art

[0002] Rainy weather is the most common bad weather. Images collected outdoors on rainy days are often affected by rain lines, resulting in almost complete loss of background information in the blocked area, which greatly reduces the quality of the image. At the same time, the illumination in rainy weather is low, which further increases image noise and reduces image detail information. Therefore, the study of rain line removal methods for low-illumination rainy day images, improving the accuracy of rain line removal and the authenticity of images after rain line removal, is of great significance to promoting the implementation of many smart transportation application technologies.

[0003] There are currently two main methods for processing low-light rainy day images: one is to first increase the image brightness through low-light enhancement, and then use the traditional image rain line removal method for processing. This method will cause color distortion while increasing the brightness, and is not suitable for scenes with high requirements for image color accuracy. The other method is to directly cut in from the perspective of rainy day images, and introduce deep learning methods to remove rain line images based on their characteristics. However, due to the impact of low light on image information, this method will cause artifacts or loss of image details, and the quality and visual effects of the processed image will be affected.

[0004] Therefore, it is necessary to develop a rain line processing method and system that can ensure the accuracy and authenticity of low-light rainy day image processing results. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for processing rain lines in low-illuminance images in order to overcome the defects of the prior art that artifacts and detail loss are prone to occur.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for processing rain lines in a low-light image comprises the following steps:

[0008] Collect rain line images with illumination below the threshold;

[0009] Extracting image features of the image containing rain lines, and obtaining a first feature based on the rain line region in the image features;

[0010] Extracting rain line prior knowledge of the rain line-containing image, fusing the first feature with the prior knowledge to obtain a first fusion result, and extracting a second feature based on the first fusion result;

[0011] The rain line semantic information of the rain line-containing image is extracted, the second feature is fused with the semantic information to obtain a second fusion result, and the rain line-free image is reconstructed based on the second fusion result.

[0012] Furthermore, the first feature is obtained based on the partial extraction of the rain line area in the image features: the rain line area in the image features is re-weighted by using the coordinate attention mechanism, and the first feature is obtained through pixel adaptive convolution according to the shape, size and direction of the rain line, and after refinement and enhancement.

[0013] Furthermore, extracting rain line prior knowledge of the rain line-containing image is specifically to capture the rain line through dynamic convolution, and the first feature is fused with the prior knowledge through a U-net network.

[0014] Furthermore, the semantic information of rain lines in the image containing rain lines is extracted and captured through the ResNet network, and the second feature and the semantic information of rain lines are fused through the FPN network.

[0015] Furthermore, the rain-line image with illumination lower than a threshold includes a rain-line image and a corresponding true value image, the rain-line image is used to extract image features, rain-line prior knowledge and rain-line semantic information, and when the second feature is fused with the semantic information, the corresponding true value image is used as a constraint.

[0016] A rain line processing system for low-illuminance images comprises an image acquisition unit, a first image processing unit, a second image processing unit and a third image processing unit. The image acquisition unit acquires rain line images with illumination lower than a threshold value, and the image acquisition unit outputs them to the first image processing unit, the second image processing unit and the third image processing unit respectively. The second image processing unit comprises a first fusion module, and the third image processing unit comprises a second fusion module. The first image processing unit outputs to the first fusion module, and the second image processing unit outputs to the second fusion module. The third image processing unit outputs a rain line-free image. When the rain line processing system is in operation, any of the above rain line processing methods is executed.

[0017] Furthermore, the image acquisition unit includes an image acquisition module and an image processing module. The image acquisition module acquires rain line images with illumination lower than a threshold value. The preprocessing module preprocesses the rain line images and transmits them to the first image processing unit, the second image processing unit and the third image processing unit respectively.

[0018] Furthermore, the first image processing unit includes an image primary feature extraction module, a rain line shallow feature generation module and a shallow feature refinement and enhancement module which are connected in sequence. The image primary feature extraction module extracts image features of the image containing rain lines. The rain line shallow feature generation module extracts image features to obtain rain line shallow features. The shallow feature refinement and enhancement module refines and enhances the rain line shallow features to obtain a first feature, and outputs the first feature to the first fusion module.

[0019] Furthermore, the second image processing unit includes a rain line prior knowledge learning module, a first fusion module, a rain line deep feature generation module and a deep feature refinement and enhancement module which are connected in sequence. The rain line prior knowledge learning module obtains rain line prior knowledge based on the rain line-containing image. The first fusion module fuses the rain line prior knowledge and the first feature to obtain a first fusion result. The rain line deep feature generation module extracts the rain line deep features based on the first fusion result. The deep feature refinement and enhancement module refines and enhances the rain line deep features to obtain second features, and outputs the second features to the second fusion module.

[0020] Furthermore, the third image processing unit includes a rain line semantic information supplement module, a second fusion module, a low-illumination rain-free line image reconstruction module and an output module which are connected in sequence. The rain line semantic information supplement module obtains rain line semantic information based on the rain line-containing image. The second fusion module fuses the rain line semantic information with the second feature to obtain a second fusion result. The low-illumination rain-free line image reconstruction module reconstructs the rain-free line image based on the second fusion result. The output module outputs the rain-free line image.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1) The present invention uses a three-stage rain line processing method and system for low-illumination images, which can separate rain line features more accurately, reduce error accumulation, and significantly improve the accuracy of rain line removal. Through the image features, prior knowledge and semantic information of rain line images, without changing the original brightness of the image, the network's ability to identify and remove rain lines in complex low-illumination environments is enhanced, the adaptability under low-light conditions is enhanced, the loss of image details is avoided, and the authenticity and clarity of the rain line removed image are ensured.

[0023] 2) The present invention can be applied to traffic monitoring systems to improve their recognition capabilities in severe weather, provide higher quality image data support for intelligent monitoring systems, and help improve the efficiency and accuracy of intelligent applications such as violation detection and traffic monitoring, while avoiding dependence on high-end hardware and reducing application costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a system block diagram of the present invention.

[0025] Figure 2 It is a workflow diagram of the first image processing unit.

[0026] Figure 3 It is a workflow diagram of the second image processing unit.

[0027] Figure 4 It is a workflow diagram of the third image processing unit.

[0028] Explanation of the marks in the figure: 1. Image acquisition unit, 11. Image acquisition module, 12. Image processing module, 2. First image processing unit, 21. Image primary feature extraction module, 22. Rain line shallow feature generation module, 23. Shallow feature refinement and enhancement module, 3. Second image processing unit, 31. Rain line prior knowledge learning module, 32. First fusion module, 33. Rain line deep feature generation module, 34. Deep feature refinement and enhancement module, 4. Third image processing unit, 41. Rain line semantic information supplement module, 42. Second fusion module, 43. Low-illumination rain-free image reconstruction module, 44. Output module. DETAILED DESCRIPTION

[0029] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0030] Example 1

[0031] The purpose of the present invention is to address the problem that existing image rain line removal methods are prone to artifacts and detail loss when processing low-illuminance rainy images, and propose a three-stage image rain line removal technology for low-illuminance rainy images.

[0032] Specifically, the present invention is a method for processing rain lines in a low-light image, comprising the following steps:

[0033] Collect rain line images with illumination below the threshold;

[0034] The rain line images with illumination below a threshold include rain line images and corresponding true value images. The rain line images are used to extract image features, rain line prior knowledge and rain line semantic information. When the second feature is fused with the semantic information, the corresponding true value image is used as a constraint.

[0035] Extracting image features of the image containing rain lines, and obtaining a first feature based on the rain line region in the image features;

[0036] The first feature is obtained based on the partial extraction of the rain line area in the image features: the coordinate attention mechanism is used to re-weight the rain line area in the image features, and the first feature is obtained through pixel adaptive convolution according to the shape, size and direction of the rain line, and after refinement and enhancement.

[0037] Extracting rain line prior knowledge of the rain line-containing image, fusing the first feature with the prior knowledge to obtain a first fusion result, and extracting a second feature based on the first fusion result;

[0038] The extraction of rain line prior knowledge of images containing rain lines is specifically carried out by capturing rain lines through dynamic convolution, fusing the first feature with the prior knowledge through the U-net network, and generating the deep features of rain lines through the rain line deep feature deep module; finally, the deep features of rain lines are refined and enhanced through the deep feature refinement and enhancement module to obtain the second feature, so as to better capture and process the local rain line features in the image.

[0039] Extracting rain line semantic information from the rain line-containing image, fusing the second feature with the semantic information to obtain a second fusion result, and reconstructing a rain line-free image based on the second fusion result;

[0040] The rain line semantic information of the rain line-containing image is captured by the ResNet network, and the second feature and the rain line semantic information are fused by the FPN network.

[0041] Through three-stage processing steps, this technology improves the accuracy of rain line removal and the authenticity of the image after rain line removal, which helps to improve the reliability of practical applications of smart transportation such as vehicle identification, traffic detection and intelligent monitoring systems.

[0042] Example 2

[0043] The present invention also provides a rain line processing system for low illumination images, such as Figure 1 As shown, it includes an image acquisition unit 1, a first image processing unit 2, a second image processing unit 3 and a third image processing unit 4. The image acquisition unit 1 acquires a rain line image with an illumination lower than a threshold value, and the image acquisition unit 1 outputs to the first image processing unit 2, the second image processing unit 3 and the third image processing unit 4 respectively. The second image processing unit 3 includes a first fusion module 32, and the third image processing unit 4 includes a second fusion module 42. The first image processing unit 2 outputs to the first fusion module 32, the second image processing unit 3 outputs to the second fusion module 42, and the third image processing unit 4 outputs a rain line-free image. When the rain line processing system is running, a rain line processing method based on Example 1 is executed.

[0044] The image acquisition unit 1 includes an image acquisition module 11 and an image processing module 12. The image acquisition module 11 acquires rain line images with illumination below a threshold. The preprocessing module preprocesses the rain line images and transmits them to the first image processing unit 2, the second image processing unit 3 and the third image processing unit 4 respectively.

[0045] Preferably, the image acquisition module 11 captures rain line images with illumination below a threshold, including rain line images and corresponding true value images. The processing method of the rain line images remains unchanged, and the true value images are used as pixel-level constraints in the process of fusing the second feature with the rain line semantic information, thereby effectively improving the accuracy of the fusion of the second feature with the rain line semantic information, ensuring accurate separation and removal of rain line features, reducing error accumulation, and significantly enhancing detail retention and image restoration quality of the processing results.

[0046] The first image processing unit 2 includes an image primary feature extraction module 21, a rain line shallow feature generation module 22 and a shallow feature refinement and enhancement module 23 which are connected in sequence. The image primary feature extraction module 21 extracts image features of an image containing rain lines. The rain line shallow feature generation module 22 extracts image features to obtain rain line shallow features. The shallow feature refinement and enhancement module 34 refines and enhances the rain line shallow features to obtain a first feature, and outputs the first feature to the first fusion module 32.

[0047] like Figure 2 As shown, the image primary feature extraction module 21 extracts the image features of the image containing rain lines; the rain line area in the primary features is re-weighted through the coordinate attention mechanism to further emphasize the importance of the rain lines; the shallow features of the rain lines are generated by the rain line shallow feature generation module 22; the shallow features are adjusted through pixel adaptive convolution according to the shape, size and direction of different rain lines; finally, the shallow features of the rain lines are refined and enhanced through the shallow feature refinement and enhancement module 23 to better capture and process the local rain line features in the image, obtain the first feature and output it to the first fusion module 32.

[0048] The second image processing unit 3 includes a rain line prior knowledge learning module 31, a first fusion module 32, a rain line deep feature generation module 33 and a deep feature refinement and enhancement module 34 which are connected in sequence. The rain line prior knowledge learning module 31 obtains rain line prior knowledge based on the rain line-containing image. The first fusion module 32 fuses the rain line prior knowledge and the first feature to obtain a first fusion result. The rain line deep feature generation module 33 extracts the rain line deep features based on the first fusion result. The deep feature refinement and enhancement module 34 refines and enhances the rain line deep features to obtain second features, and outputs the second features to the second fusion module 42.

[0049] like Figure 3As shown, the rain line prior knowledge learning module 31 captures the diversity of rain lines through dynamic convolution, learns the prior knowledge of rain lines, and reduces the confusion between rain lines and backgrounds; the first fusion module 32 fuses the shallow features of rain lines with the prior knowledge through the U-net network; the deep features of rain lines are generated through the rain line deep feature generation module 33; finally, the deep features of rain lines are refined and enhanced through the deep feature refinement and enhancement module 34 to better capture and process the local rain line features in the image, and output as the second feature to the second fusion module 42.

[0050] The third image processing unit 4 includes a rain line semantic information supplement module 41, a second fusion module 42, a low-illumination rain-free line image reconstruction module 43 and an output module 44 which are connected in sequence. The rain line semantic information supplement module 41 obtains rain line semantic information based on the rain line-containing image. The second fusion module 42 fuses the rain line semantic information with the second feature to obtain a second fusion result. The low-illumination rain-free line image reconstruction module 43 reconstructs the rain-free line image based on the second fusion result. The output module 44 outputs the rain-free line image.

[0051] like Figure 4 As shown, the rain line semantic information supplementation module 41 captures the rain line semantic information of the rain line-containing image through the ResNet network, and the second fusion module 42 fuses the rain line semantic information and the second feature through the FPN network. The accuracy of the fusion of the second feature and the rain line semantic information is improved through pixel-level constraints to ensure the accurate separation and removal of rain line features; according to the fused features after constraints, a low-illumination rain-free image is generated through the low-illumination rain-free image reconstruction module; finally, the final image is output through the low-illumination rain-free image output module 44.

[0052] Example 3

[0053] On the basis of Example 2, the image acquisition unit 1 is respectively connected to the first image processing unit 2, the second image processing unit 3 and the third image processing unit 4 in a wired or wireless manner, the first image processing unit 2 and the second image processing unit 3 are connected in a wired or wireless manner, and the second image processing unit 3 and the third image processing unit 4 are connected in a wired or wireless manner.

[0054] The three-stage low-light image rain line processing method and system of the present invention are of great value in the field of smart transportation. By accurately separating the rain line features and retaining the image details, this technology significantly improves the image clarity and authenticity in rainy and low-light environments. Its application can effectively improve the recognition ability of traffic monitoring systems in bad weather, optimize the environmental perception effect of automatic driving systems, and ensure operational safety under complex conditions. At the same time, the present invention provides higher-quality image data support for intelligent monitoring systems, which helps to improve the efficiency and accuracy of intelligent applications such as violation detection and traffic monitoring. In addition, this method reduces the dependence on high-end hardware, while reducing the cost of equipment upgrades, it improves the environmental adaptability of the smart transportation system, and provides strong support for achieving efficient operation around the clock.

[0055] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for processing rain lines in low-light images, characterized in that: The following steps are involved: Collect rain line images with illumination below the threshold; Extracting image features of the image containing rain lines, and obtaining a first feature based on the rain line region in the image features; Extracting rain line prior knowledge of the rain line-containing image, fusing the first feature with the prior knowledge to obtain a first fusion result, and extracting a second feature based on the first fusion result; Extracting rain line semantic information from the rain line-containing image, fusing the second feature with the semantic information to obtain a second fusion result, and reconstructing the rain line-free image based on the second fusion result.

2. The method for processing rain lines in low-light images according to claim 1, characterized in that: The first feature is obtained by extracting the rain line area part in the image feature in detail: the rain line area part in the image feature is re-weighted by using the coordinate attention mechanism, and the first feature is obtained by pixel adaptive convolution according to the shape, size and direction of the rain line, and after refinement and enhancement.

3. The method for processing rain lines in low-light images according to claim 1, characterized in that: The extraction of rain line prior knowledge of the rain line-containing image is specifically to capture the rain line through dynamic convolution, and the fusion of the first feature and the prior knowledge is performed through a U-net network.

4. The method for processing rain lines in low-light images according to claim 1, characterized in that: The rain line semantic information of the extracted rain line-containing image is captured by a ResNet network, and the second feature and the rain line semantic information are fused by an FPN network.

5. The method for processing rain lines in low-light images according to claim 1, characterized in that: The rain line image with illumination lower than a threshold value includes a rain line image and a corresponding true value image, wherein the rain line image is used to extract image features, rain line prior knowledge and rain line semantic information, and when the second feature is fused with the semantic information, the corresponding true value image is used as a constraint.

6. A rain line processing system for low-light images, characterized in that: The system comprises an image acquisition unit (1), a first image processing unit (2), a second image processing unit (3) and a third image processing unit (4); the image acquisition unit (1) acquires a rain line image with an illumination lower than a threshold value; the image acquisition unit (1) outputs the image to the first image processing unit (2), the second image processing unit (3) and the third image processing unit (4); the second image processing unit (3) comprises a first fusion module (32); the third image processing unit (4) comprises a second fusion module (42); the first image processing unit (2) outputs the image to the first fusion module (32); the second image processing unit (3) outputs the image to the second fusion module (42); the third image processing unit (4) outputs a rain line-free image; when the rain line processing system is in operation, a rain line processing method according to any one of claims 1 to 5 is executed.

7. The rain line processing system for low-light images according to claim 6, characterized in that: The image acquisition unit (1) comprises an image acquisition module (11) and an image processing module (12); the image acquisition module (11) acquires rain line images with illumination below a threshold; the preprocessing module preprocesses the rain line images and transmits the preprocessed images to a first image processing unit (2), a second image processing unit (3), and a third image processing unit (4), respectively.

8. The rain line processing system for low-light images according to claim 6, characterized in that: The first image processing unit (2) comprises an image primary feature extraction module (21), a rain line shallow feature generation module (22) and a shallow feature refinement and enhancement module (23) which are connected in sequence, wherein the image primary feature extraction module (21) extracts image features of an image containing rain lines, the rain line shallow feature generation module (22) extracts image features to obtain rain line shallow features, and the shallow feature refinement and enhancement module (23) refines and enhances the rain line shallow features to obtain a first feature, and outputs the first feature to a first fusion module (32).

9. The rain line processing system for low-light images according to claim 6, characterized in that: The second image processing unit (3) comprises a rain line prior knowledge learning module (31), a first fusion module (32), a rain line deep feature generation module (33) and a deep feature refinement and enhancement module (34) which are connected in sequence, wherein the rain line prior knowledge learning module (31) obtains rain line prior knowledge based on the rain line-containing image, the first fusion module (32) fuses the rain line prior knowledge and the first feature to obtain a first fusion result, the rain line deep feature generation module (33) extracts rain line deep features based on the first fusion result, and the deep feature refinement and enhancement module (34) refines and enhances the rain line deep features to obtain second features, and outputs the second features to the second fusion module (42).

10. The rain line processing system for low-light images according to claim 6, characterized in that: The third image processing unit (4) comprises a rain line semantic information supplement module (41), a second fusion module (42), a low-illumination rain-free line image reconstruction module (43) and an output module (44) which are connected in sequence, wherein the rain line semantic information supplement module (41) obtains rain line semantic information based on the rain line-containing image, the second fusion module (42) fuses the rain line semantic information with the second feature to obtain a second fusion result, the low-illumination rain-free line image reconstruction module (43) reconstructs the rain-free line image based on the second fusion result, and the output module (44) outputs the rain-free line image.

Images