A method and device for constructing a real rain line layer data set and a storage medium

Abstract

This invention discloses a method, apparatus, and storage medium for constructing a real rainline layer dataset, belonging to the field of computer vision data processing and image enhancement. It involves acquiring a sequence of real rainline images under black background conditions, obtaining a rainless black field or a background image after dark / flat field correction, separating the rainline layer, performing upper quantile truncation and normalization, and adaptively determining the gamma coefficient based on saturation pixel ratio, etc., to obtain the final rainline layer. The final rainline layer is then quality-checked based on indicators such as background residue, overexposure saturation, fog dominance, and the proportion of slender rainline structures, and tags such as rainfall intensity level, exposure gain, and batch number are generated and stored to form a reusable rainline layer sample library.

Description

Technical Field

[0001] This invention relates to the fields of computer vision data processing, image enhancement, and dataset construction technology, and particularly to a method, apparatus, and storage medium for constructing a real rain line layer dataset. Background Technology

[0002] Rain removal from single images and visual perception of rainy weather tasks are highly dependent on high-quality training and evaluation data. Existing data construction methods mainly include artificial synthesis, real-world shooting of natural scenes, and extraction based on video filtering. Artificial synthesis methods are easy to scale, but the rain line texture and intensity distribution have statistical biases compared to the real physical process, which can easily lead to data domain shift. Real-world shooting of natural scenes is affected by uncontrollable weather, unpredictable rain intensity, and lighting and exposure drift, making it difficult to obtain stable and reproducible rain line samples. Although video filtering extraction can obtain realistic rain line layers, it is sensitive to scene motion and exposure changes, and it is difficult to form a unified labeling system and quality control standards.

[0003] Meanwhile, directly using "rainy day images with background" as data samples results in strong coupling between rain lines and background content, making it difficult to extract rain line layers in a background-independent manner. This hinders the reuse of rainy day images in different background scenarios, limiting rainy day image synthesis, cross-domain training, and fair evaluation. Therefore, there is an urgent need for a scheme to construct a real rain line layer dataset that is reproducible, quality-checkable, intensity-annotated, and stored in a background-independent format.

[0004] It should be noted that this invention is not intended for rainy day image restoration (rain removal / fog removal / visibility measurement), but rather to provide a method for constructing and quality controlling a sample library of background-independent real rain line layers as a basis for training and evaluation data. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a method for constructing a real rainline layer dataset, so that rainline layers can be stably extracted from image sequences and subjected to intensity correction, quality control and labeling storage, thereby forming a background-independent, reusable and traceable rainline layer sample library.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A method for constructing a real rainline layer dataset includes the following steps: S1. Acquire at least one set of image sequences containing real rain lines, wherein the image sequences correspond to the same or approximately the same background imaging conditions; S2. Perform background estimation on the image sequence to obtain a background image. And based on the background image The image sequence is decoupled and separated to obtain the initial image of the rainline layer. ; S3. Initial image of the rainline layer Intensity normalization was performed, and gamma correction was applied to obtain a gamma-corrected rainline layer image. Gamma correction is used to suppress high-brightness saturation and enhance the controllable expression of the rainline layer; S4. The gamma-corrected rainline layer image Quality control was carried out to remove samples with background residue, fog dominance, overexposure, low contrast, or texture abnormalities, so as to obtain a qualified rainline layer sample set. S5. Establish label information associated with the acquisition environment parameters and / or imaging parameters for each qualified rainline sample, and store it according to a preset data organization structure to form a real rainline dataset.

[0007] Furthermore, the background estimation in step S2 includes any of the following methods: (1) Obtain a black field reference image under rainless conditions as background image B, or use a preset constant black field as background image B. (2) After performing dark field correction and / or flat field correction on the image sequence, the correction result is used as background image B; (3) In a non-pure black background scene, the background image B is obtained by performing median statistics or quantile statistics on the image sequence in the time dimension.

[0008] (4) In non-pure black background scenes, low-rank sparse decomposition is used to separate the background component from the rain line component to obtain background image B.

[0009] Furthermore, the decoupling separation in step S2 satisfies any of the following forms: (1) Where I is the rainline image, This is a truncation function; (2) Under fixed camera and fixed background conditions, steady-state segment screening is performed on the image sequence, and rain line candidate components are obtained based on time dimension difference or quantile difference, and then thresholding and connected component constraints are applied to obtain... In cases of camera shake or changes in viewing angle, image registration is performed on the image sequence before the differential extraction is performed.

[0010] Further, step S3 includes: right After performing upper quantile truncation and normalization, gamma mapping is performed to obtain... ; where the gamma coefficient The gamma map is adaptively determined based on the saturated pixel ratio, rainline highlight ratio, and / or histogram distribution; the gamma map satisfies: .

[0011] Furthermore, step S3 further includes: i that step's pair or Perform denoising and morphological constraint processing, wherein the morphological constraint processing includes any one or more of opening operations, closing operations, elongated structure preservation, or isolated noise point removal.

[0012] Furthermore, the quality control in step S4 includes at least one of the following indicators: (1) The background residual rate is less than the preset threshold; (2) The proportion of saturated pixels is less than the preset threshold; (3) The density of the rain line edge is within the preset range; (4) The proportion of fog obstruction is less than the preset threshold; (5) Texture similarity deduplication: When the similarity with an existing sample is greater than the threshold, it is judged as a near duplicate and removed or downweighted; (6) The proportion of the elongated structure of the rain line is within a preset range. The proportion of the elongated structure is calculated by any one or more of the following: directional consistency, linear filter response, or connected domain aspect ratio statistics.

[0013] Furthermore, the label information in step S5 includes at least any combination of the following: rain line intensity level, exposure time, gain, color temperature / illuminance, resolution, collection batch number, and sample quality score.

[0014] Furthermore, the real rainline layer dataset is stored in a background-independent manner and includes at least one of the following data representations: (1) Single-channel rain line intensity map; (2) Rain line transparency map alpha; (3) Binary rain line mask; (4) Combined expression of “intensity map + mask” or “intensity map + alpha”.

[0015] Furthermore, step S5 further includes: generating an index file and a data partitioning file, wherein the index file is used to support retrieval and sampling by rain line intensity level, exposure parameters, or acquisition batch; the data partitioning file is used to generate a training set, a validation set, and a test set, and restricts samples of the same acquisition segment from crossing sets.

[0016] Furthermore, it also includes the step of generating a synthetic image of rainy weather using the real rainline layer dataset, including: A1. Obtain a clean background image ; A2. The clean background image Depth estimation is performed to obtain a depth map. ; A3. Regarding the depth map Perform inverse transformation and normalization to obtain the depth weight map. or transmittance-related characterization ; A4. Select gamma-corrected rainline images from the real rainline dataset. And based on the depth weight map The gamma-corrected rain line layer image Deep correlation modulation was performed to obtain the rain line superposition term. ; A5. Based on the atmospheric scattering model, the clean background image is... Overlay item with the rain line Coupled, outputting a composite image of rainy days. .

[0017] This invention also proposes an apparatus for constructing a real rainline layer dataset, comprising: The acquisition module is used to acquire at least one image sequence containing real rain lines; The separation module is used to perform background estimation and decoupling separation to obtain the initial image of the rainline layer. ; The correction module is used to correct the initial image of the rainline layer. Normalization and gamma correction were performed to obtain gamma-corrected rainline images. ; The quality control module is used to process gamma-corrected rainline layer images. Perform quality control and screen qualified samples; The labeling and storage module is used to generate label information for qualified samples and store them according to a preset data organization structure to form a real rainline layer dataset. The functions of the above modules correspond to the methods described in this invention.

[0018] The present invention also proposes an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the method described in the present invention.

[0019] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention.

[0020] This invention constructs a real rain line layer dataset through a process of "background estimation and decoupling separation - normalization and gamma correction - quality control - label association - data storage organization"; furthermore, the real rain line layer dataset can be used to synthesize rainy days from clean background images to support downstream training and evaluation.

[0021] Compared with the prior art, the present invention has the following beneficial effects: a) Background-independent: Rain line layers are obtained through background estimation and decoupling, decoupling the rain line texture from the background and supporting reuse on any background; b) Stable intensity: Normalization and gamma correction suppress overexposure and highlight while enhancing rain line details, making intensity expression more stable and controllable; c) Quality Consistency: Improve sample consistency by removing samples with background residue, fog dominance, or texture anomalousness through quantifiable quality indicators; d) Traceable and reproducible: By associating acquisition conditions and imaging parameters with tag information and index files, stratified sampling and reproducible experiments are supported; e) Support for application expansion: The rain line layer library can be used for rainy day image synthesis, rain removal model training and evaluation, as well as robustness verification for downstream detection, segmentation and other tasks. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the process of constructing and synthesizing a real rainline layer dataset according to the present invention.

[0023] Figure 2 This is a structural block diagram of a device for constructing a real rainline layer dataset according to the present invention.

[0024] Explanation of reference numerals in the attached figures 100 - Clean background image; 110 - Depth estimation; 120 - Depth map; 130 - Reciprocal transformation and normalization; 140 - Rainy day image generation module (rainy day imaging model / atmospheric scattering coupling module); 300 - Rainy day composite image; 200 - Rain and fog simulation chamber (rain line acquisition environment); 210 - Rain line image; 220 - Initial image of the rain line layer ;230-gamma correction;240-gamma correction rainline layer image .

[0025] 400 - Input rainline image sequence; 410 - Acquisition module; 420 - Background estimation module; 430 - Decoupling and separation module; 440 - Normalization and gamma correction module; 450 - Denoising and morphological constraint module; 460 - Quality control module; 470 - Label generation module; 480 - Data organization and storage module; 490 - Output of real rainline layer dataset; 500 - Synthesis application module; 510 - Synthesized rainy day image. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0027] refer to Figure 2 A structural block diagram of a device for constructing a real rainline layer dataset according to the present invention. Example 1: Method for constructing a real rainline layer dataset (corresponding to) Figure 1 Lower part 200-240) S1 (Acquiring Image Sequence): A black background or matte black curtain is placed inside the rain / fog simulation chamber 200 to make the image appear as a near-black field in the absence of rain; the camera position and imaging parameters are fixed, and rain line shooting 210 is performed to obtain the image sequence. Preferably, the sequence length is 10–200 frames, and the resolution is any one of 720p / 1080p / 4K; each sequence segment records the acquisition batch number. Related to camera parameters such as exposure time, gain, aperture, and white balance.

[0028] S2 (Background estimation and decoupling separation): (a) Black reference: Acquired black reference image with sprinkler off / no rain. Or preset constant black field (The constant matrix after all-zero or dark level compensation), let ; (b) Dark / flat field correction: When dark level drift or uneven illumination exists, the dark field is sampled. A draw ,implement ,in Gain correction diagram (by (Calculated with preset uniform grayscale), and Time dimension statistical results as background image ; (c) Time dimension statistics: In non-pure black background scenes, using or Obtain background image B; (d) Low-rank sparse decomposition: vectorizing the sequence frames into a matrix Solve ,Pick Corresponding to the background and reconstructing .

[0029] The basic form of decoupling: for each frame Calculate the initial image of the rainline layer. , This indicates that the pixel value is truncated to [0, 255].

[0030] Steady-state difference / quantile difference form: When the background is not strictly static, first perform steady-state segment filtering (e.g., remove frames where the average brightness change exceeds a threshold), then... The candidate components of the rain line are obtained, among which , These are the high and low quantile statistical images of the sequence in the time dimension, respectively; when slight camera shake is present, geometric transformation can be obtained by feature point / phase correlation registration. And on After registration, the quantiles are calculated.

[0031] S3 (Normalization and Gamma Correction): Upper quantile truncation and normalization: calculation upper quantile threshold in ,make ; Gamma mapping: Gamma coefficient Adaptive determination: for example, based on saturation pixel ratio Rain line highlight ratio The histogram skewness is used as input, and selection is based on segmentation rules. ∈[2.0,5.0]; take the larger value when local highlighting is obvious. To compress highlights while preserving details of rain streaks in shadows.

[0032] S3 (Denoising and Morphological Constraints).

[0033] right or After performing denoising (median / bilateral / guided filtering), morphological opening and closing operations are performed; and then the results are calculated based on the area of ​​the connected components. Aspect Ratio Filtering non-rain line patches, for example, retaining Connected regions with an area of ​​≥3 and falling within a preset range are used to enhance the slender rain line structure. S4 (Quality Control): (1) Background Residual Rate: The mean or variance of the low-threshold region is statistically analyzed in a black background scene, and it is required to be lower than the threshold; (2) Saturated Pixel Ratio: Statistically analyzed ≥254 pixel ratio; (3) Rain line edge density: for implement (3) Edge detection and statistical analysis of edge pixel proportion; (4) Fog occlusion proportion: estimated based on low-frequency energy proportion or contrast reduction index; (5) Near-duplication deduplication: for calculate If the similarity with the sample in the library is higher than the threshold, it will be removed; (6) Proportion of slender structures: statistically satisfying the aspect ratio. ≥ The proportion of connected pixels to the total number of rain line pixels, or the statistical directional consistency (peak percentage of the main direction / directional entropy).

[0034] S5 (Tag and Storage Organization): Generate for each qualified sample (For example ):{ , , , , , , , }; and according to "Intensity Map / / Store in one or a combination of these formats (lossless formats such as PNG / TIFF). Generate an index file (CSV / JSONL) for use by... Alternatively, batch retrieval sampling can be used; a partition file (train / val / test) can be generated, and samples from the same batch or the same steady-state segment can be constrained not to cross sets.

[0035] Example 2: Generating synthetic images of rainy days using real rainline layer datasets (corresponding to...) Figure 1 Upper part 100-300) Get a clean background image (100), perform depth estimation (110) to obtain the depth map. (120). Regarding Perform the inverse transformation and normalization (130) to obtain the depth weights. or transmittance-related characterization .

[0036] Samples were selected from the rainline database. (240) Generate rain line superposition terms with deep correlation modulation (or (Equivalent form).

[0037] In the rainy day image generation module 140, an atmospheric scattering model is used to generate images. and Coupling, for example: ,in Brightness of atmospheric light or fog background. Weights are added to the rain lines; the composite image of the rainy day is output. (300). It should be noted that Example 2 is used to illustrate the reusability and controllable synthesis capability of the real rainline dataset; the present invention does not limit the specific algorithm implementation of depth estimation, nor does it limit the specific form of the atmospheric scattering model, and the module can be implemented using existing methods or their equivalents.

[0038] Example 3: Devices, Electronic Devices, and Storage Media The construction apparatus includes: an acquisition module (executing S1), a separation module (executing background estimation and decoupling separation in S2), a calibration module (executing S3), a quality inspection module (executing S4), and a labeling and storage module (executing S5). The electronic device includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the method described in claims 1-10; the program instructions are stored on a computer-readable storage medium.

[0039] Feasibility description This invention focuses on image sequence processing and data engineering organization, and can be implemented without relying on specific hardware structures. Its key steps (background estimation, decoupling and separation, gamma correction, quality control, labeling and index generation) can all be implemented by general-purpose computing devices, making it suitable for engineering deployment and large-scale dataset production.

Claims

1. A method for constructing a real rainline layer dataset, characterized in that, Includes the following steps: S1. Acquire at least one set of image sequences containing real rain lines, wherein the image sequences correspond to the same or approximately the same background imaging conditions; S2. Perform background estimation on the image sequence to obtain a background image. And based on the background image The image sequence is decoupled and separated to obtain the initial image of the rainline layer. ; S3. Initial image of the rainline layer Intensity normalization was performed, and gamma correction was applied to obtain a gamma-corrected rainline layer image. Gamma correction is used to suppress high-brightness saturation and enhance the controllable expression of the rainline layer; S4. The gamma-corrected rainline layer image Quality control was carried out to remove samples with background residue, fog dominance, overexposure, low contrast, or texture abnormalities, so as to obtain a qualified rainline layer sample set. S5. Establish label information associated with the acquisition environment parameters and / or imaging parameters for each qualified rainline sample, and store it according to a preset data organization structure to form a real rainline dataset.

2. The method according to claim 1, characterized in that, The background estimation in step S2 includes any of the following methods: (1) Obtain a black field reference image under rainless conditions as background image B, or use a preset constant black field as background image B. (2) After performing dark field correction and / or flat field correction on the image sequence, the correction result is used as background image B; (3) In a non-pure black background scene, the image sequence is subjected to median statistics or quantile statistics in the time dimension to obtain background image B; (4) In non-pure black background scenes, low-rank sparse decomposition is used to separate the background component from the rain line component to obtain background image B.

3. The method according to claim 1, characterized in that, The decoupling separation in step S2 satisfies any of the following forms: (1) Where I is the rainline image, This is a truncation function; (2) Under fixed camera and fixed background conditions, steady-state segment screening is performed on the image sequence, and rain line candidate components are obtained based on time dimension difference or quantile difference, and then thresholding and connected component constraints are applied to obtain... In cases of camera shake or changes in viewing angle, image registration is performed on the image sequence before the differential extraction is performed.

4. The method according to claim 1, characterized in that, Step S3 includes: right After performing upper quantile truncation and normalization, gamma mapping is performed to obtain... ; where the gamma coefficient The gamma map is adaptively determined based on the saturated pixel ratio, rainline highlight ratio, and / or histogram distribution; the gamma map satisfies: 。 5. The method according to claim 1, characterized in that, Step S3 further includes: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] or Perform denoising and morphological constraint processing, wherein the morphological constraint processing includes any one or more of opening operations, closing operations, elongated structure preservation, or isolated noise point removal.

6. The method according to claim 1, characterized in that, The quality control in step S4 includes at least one of the following indicators: (1) The background residual rate is less than the preset threshold; (2) The proportion of saturated pixels is less than the preset threshold; (3) The density of the rain line edge is within the preset range; (4) The proportion of fog obstruction is less than the preset threshold; (5) Texture similarity deduplication: When the similarity with an existing sample is greater than the threshold, it is judged as a near duplicate and removed or downweighted; (6) The proportion of the elongated structure of the rain line is within a preset range. The proportion of the elongated structure is calculated by any one or more of the following: directional consistency, linear filter response, or connected domain aspect ratio statistics.

7. The method according to claim 1, characterized in that, The label information in step S5 includes at least any combination of the following: rain line intensity level, exposure time, gain, color temperature / illuminance, resolution, collection batch number, and sample quality score.

8. The method according to claim 1, characterized in that, The real rainline layer dataset is stored in a background-independent manner and includes at least one of the following data representations: (1) Single-channel rain line intensity map; (2) Rain line transparency map alpha; (3) Binary rain line mask; (4) Combined expression of "intensity map + mask" or "intensity map + alpha".

9. The method according to claim 1, characterized in that, Step S5 further includes: generating an index file and a data partitioning file, wherein the index file is used to support retrieval and sampling by rain line intensity level, exposure parameters, or acquisition batch; the data partitioning file is used to generate a training set, a validation set, and a test set, and restricts samples of the same acquisition segment from crossing sets.

10. The method according to claim 1, characterized in that, It also includes the step of generating a synthetic image of rainy weather using the real rainline layer dataset, including: A1. Obtain a clean background image ; A2. The clean background image Depth estimation is performed to obtain a depth map. ; A3. Regarding the depth map Perform inverse transformation and normalization to obtain the depth weight map. or transmittance-related characterization ; A4. Select gamma-corrected rainline images from the real rainline dataset. And based on the depth weight map The gamma-corrected rain line layer image Deep correlation modulation was performed to obtain the rain line superposition term. ; A5. Based on the atmospheric scattering model, the clean background image is... Overlay item with the rain line Coupled, outputting a composite image of rainy days. .

11. An apparatus for constructing a real rainline layer dataset, characterized in that, include: The acquisition module is used to acquire at least one image sequence containing real rain lines; The separation module is used to perform background estimation and decoupling separation to obtain the initial image of the rainline layer. ; The correction module is used to correct the initial image of the rainline layer. Normalization and gamma correction were performed to obtain gamma-corrected rainline images. ; The quality control module is used to process gamma-corrected rainline layer images. Perform quality control and screen qualified samples; The labeling and storage module is used to generate label information for qualified samples and store them according to a preset data organization structure to form a real rainline layer dataset. The functions of the above modules correspond to the implementation of the method described in any one of claims 1 to 10.

12. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 10.

Images