Lens contamination image generation method and apparatus
By constructing a generation path based on a diffusion model, high-fidelity images of dirty lenses are generated, solving the problem of decreased lens dirt perception capability in intelligent driving, providing high-confidence synthetic data, and improving the training effect of the perception model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BYD CO LTD
- Filing Date
- 2026-01-27
- Publication Date
- 2026-06-05
AI Technical Summary
In existing technologies, in the perception tasks of intelligent driving, lens dirt leads to a decrease in perception ability, but manually collected lens dirt data is scarce and has limited features, making collection difficult.
By constructing a generation path based on a diffusion model, high-fidelity dirty images of lenses are generated, including initializing a random grid image of dirt, generating a dirty mosaic image and a contour image, and combining a dirty template and a weighted image with a natural scene image to generate highly realistic and diverse dirty images.
It significantly improves the realism and diversity of dirty lens images, provides high-confidence synthetic data for training perception models, and overcomes the problems of harsh edges and insufficient material variation in traditional methods.
Smart Images
Figure CN122156358A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for generating images of dirty lenses. Background Technology
[0002] In related technologies, during the perception tasks of intelligent driving, image data from cameras is easily affected by dirt on the camera lenses, leading to a decrease in perception capabilities. However, manually collected lens dirt data is relatively scarce, and the dirt features collected are singular and difficult to obtain. Summary of the Invention
[0003] The lens dirt image generation method and apparatus provided in this application embodiments at least partially solve the above-mentioned problems. The first aspect of this application provides a method for generating a lens dirt image, the method comprising:
[0004] Based on the initial dirty random grid image, a dirty mosaic image is obtained;
[0005] Based on the dirty mosaic image, a dirty outline image is obtained;
[0006] Based on the dirt contour image, a dirt template weight image is obtained;
[0007] Based on the first natural scene image, a dirty template image is obtained;
[0008] Based on the dirty template image and the dirty template weight image, a dirty overlay image and a dirty weight overlay image are obtained;
[0009] The dirt fusion result image is obtained based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image.
[0010] A method for generating a dirty lens image, the method comprising:
[0011] Based on the initial dirty random grid image, a dirty weight smoothed image is obtained;
[0012] Based on the dirt weight smoothing image, a dirt template weight image is obtained;
[0013] Based on the first natural scene image, a dirty template image is obtained;
[0014] Based on the dirty template image and the dirty template weight image, a dirty overlay image and a dirty weight overlay image are obtained;
[0015] The dirt fusion result image is obtained based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image.
[0016] Optionally, obtaining the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image includes:
[0017] The dirt overlay image, the dirt weight overlay image, and the second natural scene image are overlaid to obtain the dirt fusion result image.
[0018] Optionally, after obtaining the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image, the process includes:
[0019] The dirt region truth map is obtained based on the dirt fusion result map, the dirt weight overlay map, and the second natural scene map.
[0020] Optionally, obtaining the dirt overlay image and the dirt weight overlay image based on the dirt template image and the dirt template weight image includes:
[0021] Randomly select the dirty template image and the dirty template weight image;
[0022] Randomly adjust the selected dirty template image and the dirty template weight image;
[0023] The randomly adjusted dirty template image and the dirty template weight image are fused to generate a dirty overlay image and a dirty weight overlay image.
[0024] Optionally, the random adjustment of the selected dirty template image and the dirty template weight image includes:
[0025] Randomly scale the dirty template image and the dirty template weight image; and / or,
[0026] The fusion positions of the dirty template image and the dirty template weight image are randomly adjusted.
[0027] Optionally, fusing the randomly adjusted dirty template image and the dirty template weight image to generate a dirty overlay image and a dirty weight overlay image includes:
[0028] The randomly adjusted dirty template image and the dirty template weight image are fused to generate a dirty overlay image and a dirty weight overlay image.
[0029] When the number of fusions exceeds a preset threshold, a dirt overlay map and a dirt weight overlay map are output.
[0030] Optionally, obtaining the dirty mosaic image based on the initialized dirty random grid image includes:
[0031] Peak regions and edges are extracted from the initialized dirty random grid image to obtain a dirty mosaic image;
[0032] And / or, obtaining the dirt outline image based on the dirt mosaic image includes:
[0033] The dirty mosaic image is input into a diffusion model to obtain a dirty outline image;
[0034] And / or, obtaining the dirt template weight image based on the dirt contour image includes:
[0035] The dirt contour image is smoothed to obtain a dirt template weight image.
[0036] Optionally, obtaining a dirt-weighted smoothed image based on an initialized dirty random grid image includes:
[0037] The initial dirty random grid image is smoothed to obtain a dirty weight smoothed image;
[0038] And / or, obtaining the dirt template weight image based on the dirt weight smoothing image includes:
[0039] The dirt weight smoothing image is subjected to contrast enhancement and edge smoothing processing to obtain a dirt template weight image.
[0040] An apparatus that operates and / or stores a method for generating a lens smudge image as described in any of the preceding claims.
[0041] This application achieves high-fidelity simulation of the physical characteristics of dirt in real lenses by constructing a generation path based on a diffusion model. It overcomes the problems of harsh edges and lack of material variation caused by traditional methods that rely on preset geometric shapes or simple blurring. It significantly improves the realism and diversity of the generated dirty images and provides high-confidence synthetic data for the training of perception models.
[0042] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0044] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.
[0045] Figure 1 This is a flowchart (a) of a method for generating a dirty lens image provided in an exemplary embodiment of this disclosure.
[0046] Figure 2 This is a flowchart of another method for generating a dirty lens image provided in an exemplary embodiment of this disclosure;
[0047] Figure 3 This is a flowchart (II) of a method for generating a dirty lens image provided in an exemplary embodiment of this disclosure.
[0048] Figure 4 This is a flowchart (III) of a method for generating a dirty lens image provided in an exemplary embodiment of this disclosure. Detailed Implementation
[0049] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.
[0050] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, features defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0051] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0052] This application provides a method for generating images of a dirty lens, such as... Figure 1 As shown, the method includes:
[0053] Step S100: Obtain a dirty mosaic image based on the initialized dirty random grid image;
[0054] In some implementations, a grayscale image can be initialized first, and the image can be divided into grids with the number of grids in the width or height direction ranging from (14, 18) and the grayscale range from (0, 255), with the grayscale of the image edge grids being 0. Then, peak regions and edges are extracted from the grid image: specifically, a random threshold is first set, ranging from (246, 249), and grid regions in the weighted image that are above the threshold are extracted to obtain a peak region binary image; then, a morphological dilation operation is performed on the peak region binary image, and the peak region binary image is subtracted to obtain a peak edge binary image; finally, a dirty mosaic image is generated based on the above binary image: specifically: mosaic image = peak region binary image * 255 + peak edge binary image * random grid image.
[0055] Step S200: Obtain the dirt outline image based on the dirt mosaic image;
[0056] In a specific implementation, the dirt contour image can be obtained by inputting a dirt mosaic image into a diffusion model and processing it. Furthermore, before using the diffusion model, it is trained using manually labeled ground truth segments and the processed mosaic images. The mosaic images serve as the model input, and the ground truth segments represent the true values for training the model.
[0057] Step S300: Obtain the dirt template weight image based on the dirt contour image;
[0058] In a specific implementation, the dirty template weight image can be obtained by processing the dirty contour image through Gaussian filtering.
[0059] Step S400: Obtain the dirty template image based on the first natural scene image;
[0060] Specifically, a natural scene image is randomly selected. This image can come from open-source datasets such as the Intelligent Driving Dataset or COCO (an open-source and universal visual dataset), without any limitations. Then, a dirty template image is obtained through random cropping, blurring, color enhancement, and brightness variation. Specifically, the image can be randomly cropped into square image blocks, with the cropping size formula being: original image shorter side * size factor (0.3~0.5). The image is then smoothed by randomly selecting mean filtering or Gaussian filtering, with the filter kernel size being: cropped image size * blur factor (0.05~0.2). The image is then color-enhanced by randomly transforming the RGB (red, green, and blue) channels of the image to simulate color diversity. Finally, the image brightness is varied by randomly adjusting the RGB values of the image: RGB value * brightness factor (0.2~1.4) to simulate dirt with different brightness levels (pixels with a grayscale value greater than 255 are set to 255).
[0061] Step S500: Based on the dirty template image and the dirty template weight image, obtain the dirty overlay image and the dirty weight overlay image;
[0062] In the specific implementation, the RGB zero-value map can be initialized first, with its size and original... Figure 1 To initialize, use as the initial dirty overlay image; then initialize the image with a single channel value of 1, the size and original Figure 1 First, a dirty template image and a dirty template weight image are generated as the initial dirty weight overlay image. Then, the scaling factor (0.8~1.7) of the image is randomly generated to scale the dirty template image and the dirty template weight image. A fusion position is randomly generated to obtain the rectangular region to be fused on the original image, i.e., the fusion region. Care must be taken to check for out-of-bounds errors during generation. Then, a dirty template weight image and a dirty template image are input to update the dirty overlay image within the fusion region. The dirty overlay image can be generated according to the following formula:
[0063] Dirt overlay image [blended region] = Dirt overlay image [blended region] * (1 - Dirt template weight image) + Dirt template image * Dirt template weight image;
[0064] Finally, the dirt weight overlay map within the fusion region can be updated. The dirt weight overlay map can be generated according to the following formula: Dirt weight overlay map [fusion region] = Dirt weight overlay map [fusion region] * (1 - Dirt template weight image);
[0065] Finally, return to the scaling step of randomly generating the image, and perform the next overlay until the specified number of times is completed, to obtain the final dirty weight overlay image and the final dirty overlay image.
[0066] Step S600: Obtain the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image.
[0067] In the specific implementation, the two images mentioned above are superimposed with the second natural scene image to obtain the dirt fusion result image, where the dirt fusion result image = second natural scene image * dirt weight superimposed image + dirt superimposed image.
[0068] In the above embodiments, by constructing a generation path based on a diffusion model, a high-fidelity simulation of the physical characteristics of dirt in real lenses is achieved. This overcomes the problems of harsh edges and lack of material variation caused by traditional methods that rely on preset geometric shapes or simple blurring. It significantly improves the realism and diversity of the generated dirty images and provides high-confidence synthetic data for the training of perception models.
[0069] In some embodiments, step S100 may include:
[0070] Step S110: Perform morphological opening operation on the initialized dirty random grid image, with a 3×3 cross kernel as the structuring element, to remove isolated noise points;
[0071] Step S120: Extract the gradient magnitude map using the Sobel operator, combine it with Otsu adaptive thresholding to preserve significant edge structures, and output a binarized dirty mosaic image.
[0072] In this step, the initial random grid image of dirt is generated by a 128×128 pixel binary noise matrix, with each pixel randomly set to 1 with a probability of 0.3~0.7, simulating the initial distribution of non-uniform contamination on the lens surface. Morphological opening operations effectively eliminate minor noise interference, and the Sobel operator and Otsu thresholding work together to preserve the local protrusions and discontinuous edge features of typical dirt such as mud spots and gravel scratches, forming a dirt mosaic image with topological realism, which serves as the conditional input for the diffusion model. In a specific implementation, the initial random grid image can also be initialized by dividing the image into grids according to the image size, with the number of grids in the width or height direction ranging from (14, 18), and randomly selecting gray values to fill each grid block, with a gray value range of (0, 255), and the gray value of the image edge grids being 0. The initial dirty random grid image is composed of multiple mosaic squares in both the horizontal and vertical directions. Each mosaic square can be composed of 150x150 pixels. The gray value within each mosaic square is consistent, resulting in a dirty mosaic image 101, which is used as the conditional input for the diffusion model.
[0073] In some embodiments, step S200 may include:
[0074] The dirty mosaic image is input into a pre-trained diffusion model based on the DDPM architecture. The training set contains 10,000 real dirty images from vehicle cameras. Through a 100-step forward noise and reverse noise reduction process, a dirty contour image 102 with continuous gray-scale transition and natural edge blurring is generated, with pixel values ranging from [0,1].
[0075] This diffusion model takes a dirty mosaic image as input and learns the light diffusion, capillary effect and material texture distribution of real dirt. The output image shows a soft gradient at the edges and the internal grayscale distribution conforms to the physical diffusion morphology of oil stains, rainwater film, etc., which is significantly better than the artifact edges generated by traditional Gaussian blur or morphological dilation.
[0076] In some embodiments, step S300 may include:
[0077] The dirt contour image was smoothed using Gaussian smoothing with a kernel size of 5×5 and a standard deviation of σ=1.2 to eliminate the small high-frequency noise that may be introduced by the diffusion model, resulting in a smooth dirt template weight image 103.
[0078] This weighted image serves as a transparency mask for subsequent fusion. Each pixel value represents the occlusion intensity at that location, ensuring that dirt and background exhibit a gradual attenuation characteristic that conforms to optical laws when fused, avoiding hard-edge artifacts and improving visual naturalness.
[0079] In some embodiments, step S400 may include:
[0080] Select the first natural scene image (this image can be from a sunny / rainy road scene in the Baidu Apollo or KITTI dataset), randomly crop a 256×256 region, and generate a dirty template image 104 by randomly offsetting the H channel by ±15°, scaling the S channel by ±20%, and setting a brightness and contrast enhancement factor of 0.2~1.4.
[0081] This step simulates the differences in optical reflection of different materials (mud, oil stains, water film), and through color and brightness perturbation, gives the template image a visual representation of various types of dirt, providing diverse "dirty content" for subsequent overlay.
[0082] In some embodiments, step S500 may include:
[0083] The dirty template image and the dirty template weight image are fused by pixel-level multiplication to form an initial dirty overlay image and a dirty weight overlay image. Then, multiple sets of template and weight images are randomly selected, and each set is independently subjected to random scaling (scaling factor follows a uniform distribution [0.8~1.7]) and random displacement (offset within ±1 / 4 width of the image boundary). The results are then superimposed on the cumulative image. This process is repeated 3~8 times to finally output the dirty overlay image and the dirty weight overlay image.
[0084] This multi-stage overlay mechanism simulates the overlay effect of multiple foreign objects (such as mud, rainwater, and insect remains) falling simultaneously on the lens surface in a real scene. The scaling and displacement before each overlay ensure that the size of the dirt covers 2×2 to 400×400 pixels, and the position distribution covers the entire image area, avoiding concentration in the central area. It effectively simulates the spatial distribution preferences of highway mud splashes and tunnel drips, generating dirt distributions with complex forms such as single points, multiple points, overlaps, and occlusions.
[0085] In some embodiments, step S600 may include:
[0086] The dirty overlay image, the dirty weight overlay image, and the second natural scene image are fused pixel-weightedly to ensure that the fusion result conforms to the optical transmittance attenuation law. This fusion method can simulate the semi-transparent occlusion of the image by water film in rainy scenes and the occlusion effect of local high-density dirt in mud splash scenes, avoiding the distortion of being too bright or too dark caused by simple superposition. The generated dirty fusion result, Figure 108, has real physical consistency.
[0087] This application also provides a method for generating a lens dirt image, such as... Figure 2 As shown, the method includes:
[0088] Step S100': Obtain a dirt weight smoothing image based on the initialized dirty random grid image;
[0089] In the specific implementation, the initial dirty random grid image can be smoothed first to obtain a dirty weighted smooth image. Specifically, the random grid image can be initialized as follows: initialize a grayscale image, divide the image into grids, with the number of grids in the width or height direction ranging from (14, 18), and the grayscale range from (0, 255), with the grayscale of the image edge grids being 0; then perform weighted image smoothing on the image: apply Gaussian filtering and mean filtering to the interior of the image (excluding the zero-value edge region), with a kernel size of: grid size * grid blur factor (0.4~0.7); then perform grayscale normalization (normalized grayscale = original grayscale image ÷ maximum grayscale value);
[0090] Step S200': Obtain the dirt template weight image based on the dirt weight smoothing image;
[0091] Specifically, the dirty image can be processed by contrast enhancement and edge smoothing to obtain a dirty image with weighted mode. For example, first, the contrast can be enhanced by calculating the nth power of the weight value (0.7~1.5), setting pixels with weight values less than the minimum weight threshold (0.5) to zero, and finally multiplying the weight value by a scaling factor of 1.2. Then, the edges can be smoothed by performing a Gaussian filter on the entire image with the filter kernel size equal to the grid size, and then normalizing it to obtain the dirty image with weighted mode.
[0092] Step S400: Obtain the dirty template image based on the first natural scene image;
[0093] Step S500: Based on the dirty template image and the dirty template weight image, obtain the dirty overlay image and the dirty weight overlay image;
[0094] Step S600: Obtain the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image. Steps S400, S500, and S600 can be the same as or similar to the methods in the above embodiments. Relevant details can be found in the above description and will not be repeated here.
[0095] In the above embodiments, this approach is suitable for edge computing platforms with limited computing resources. By using fast smoothing and contrast enhancement to replace the diffusion model, it achieves efficient dirt generation, complementing method 1 and improving system deployment flexibility.
[0096] In some embodiments, step S100' may include:
[0097] The initial dirty random mesh image 100 is Gaussian smoothed with a kernel size of 7×7 and a standard deviation σ=1.5 to obtain a preliminarily smoothed dirty weighted image 112. This step preserves the patch structure of the original mesh while blurring the edges to form a continuous gray-level distribution, providing a basis for subsequent enhancement.
[0098] In some embodiments, step S200' may include:
[0099] The dirt-weighted smoothed image 112 is contrast-enhanced using the CLAHE algorithm (block size 8×8, contrast limit 2.0) to enhance local grayscale differences. Then, Non-Local Mean Denoising (NLM) is performed with a 7×7 search window and a 3×3 similarity window to suppress jagged edges and preserve the overall structure, resulting in the dirt template weighted image 103. In some implementations, the dirt-weighted smoothed image can be calculated to the power of n (0.7~1.5), pixels with weight values less than the minimum weight threshold (0.5) are set to zero, and finally, the weight values are multiplied by a scaling factor of 1.2. Gaussian blurring is then applied with a kernel size of 31×31 and a standard deviation σ=6, followed by normalization.
[0100] In the above embodiments, see Figure 3 or Figure 4 After obtaining the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image, the method may further include step 700:
[0101] The true value map of the dirty area is obtained based on the dirty fusion result map, the dirty weight superposition map, and the second natural scene map.
[0102] In the specific implementation, the second natural scene image can be averaged along the channel direction and converted to grayscale to obtain the original grayscale image. Then, the difference image between the fused result image and the original image can be calculated, where the difference image = average(absolute value(fused result image - original image)) / (1.0 + original grayscale image). Then, the difference image after adding the weighted overlay image can be calculated: difference image = square root(difference image * (1 - weighted overlay image)). Then, the obtained difference image is truncated to a high threshold: pixel values in the difference image that are greater than the high threshold are truncated to be equal to the high threshold. Then, low threshold truncation is performed: pixel values in the difference image that are less than the low threshold are set to 0. Finally, the difference image is Gaussian smoothed and normalized to obtain the ground truth image of the dirty area.
[0103] In the above embodiments, by automatically generating pixel-level ground truth labels, the problems of high cost and poor consistency of manual annotation are avoided, providing accurate training basis for supervised learning.
[0104] In the above embodiments, see Figures 3-4 The step 500, which involves obtaining a dirt overlay image and a dirt weight overlay image based on the dirt template image and the dirt template weight image, may further include:
[0105] Step S510: Randomly select a dirty template image and a dirty template weight image;
[0106] Step S520: Randomly adjust the selected dirty template image and dirty template weight image; in some embodiments, randomly adjusting the selected dirty template image and dirty template weight image includes: randomly scaling the dirty template image and dirty template weight image; and / or, randomly adjusting the fusion position of the dirty template image and dirty template weight image.
[0107] In the specific implementation, the RGB zero-value map can be initialized first, with its size and original... Figure 1 To initialize, use as the initial dirty overlay image; then initialize the image with a single channel value of 1, the size and original Figure 1 The initial dirty template image and the dirty template weight image are then randomly generated. The scaling factor (0.8~1.7) of the image is then used to scale the dirty template image and the dirty template weight image. The fusion position is randomly generated to obtain the rectangular area to be fused on the original image, i.e. the fusion area. When generating the fusion area, attention should be paid to the boundary judgment.
[0108] In the above embodiments, random scaling is achieved through bilinear interpolation, with the scaling factor following U [0.8~1.7], ensuring that the generated dirt size covers the entire range from 2×2 pixels (tiny dust) to 400×400 pixels (large area of oil diffusion); the fusion position offset is uniformly sampled within ±1 / 4 of the image width and height to simulate high-frequency scenes of lens edge contamination, such as mud splashing from the rear wheels located at the lower edge of the image and water dripping from the tunnel located at the upper part.
[0109] Step S530: Fuse the randomly adjusted dirty template image and the dirty template weight image to generate a dirty overlay image and a dirty weight overlay image.
[0110] Specifically, after step S520, a dirty template weight image and a dirty template image can be input to update the dirty overlay image in the fusion region. The dirty overlay image can be generated according to the following formula:
[0111] Dirt overlay image [blended region] = Dirt overlay image [blended region] * (1 - Dirt template weight image) + Dirt template image * Dirt template weight image;
[0112] Finally, update the dirt weight overlay map within the fusion region. The dirt weight overlay map can be generated according to the following formula: Dirt weight overlay map [fusion region] = Dirt weight overlay map [fusion region] * (1 - Dirt template weight image);
[0113] Finally, return to the scaling step of randomly generating the image, and perform the next overlay until the specified number of times is completed, to obtain the final dirty weight overlay image and the final dirty overlay image.
[0114] In the above embodiments, this step enables dynamic combination and iterative enhancement of dirty samples, simulating the complex form of multiple pollution types coexisting in real-world scenarios.
[0115] In some embodiments, the randomly adjusted dirty template image and the dirty template weight image are fused to generate a dirty overlay image and a dirty weight overlay image, including: fusing the randomly adjusted dirty template image and the dirty template weight image to generate a dirty overlay image and a dirty weight overlay image; and outputting the dirty overlay image and the dirty weight overlay image when the number of fusions is greater than a preset threshold.
[0116] In the above embodiments, the preset threshold for the number of times can be set to 3 to 8 times. Each superposition is based on the previous result. Before fusion, the dirt superposition image and the weight superposition image are normalized to prevent pixel overflow. This mechanism simulates the process of real lens contaminants accumulating over time, such as continuous rain causing mud spots to repeatedly adhere and oil stains to spread, generating an evolutionary sequence of "single point → aggregation → fusion → large area occlusion", providing the perception model with progressive training samples from light to heavy pollution.
[0117] See Figures 1 to 4 Step 100: Obtain a dirty mosaic image based on the initialized dirty random grid image, which may specifically include:
[0118] Peak regions and edges are extracted from the initial dirty random grid image to obtain a dirty mosaic image;
[0119] And / or, step 200: Based on the dirt mosaic image, obtain a dirt outline image, including:
[0120] The dirty mosaic image is input into the diffusion model to obtain the dirty outline image;
[0121] And / or, step 300: Based on the dirt contour image, obtain the dirt template weight image, including:
[0122] The dirt contour image is smoothed to obtain the dirt template weight image.
[0123] In the above embodiments, the relevant steps and methods are the same as or similar to those in the above embodiments, and can be referred to the above description for details. No specific limitations are made here. The above three sub-steps constitute a complete path, forming a chain of "grid → mosaic → diffusion → smoothing" to generate a high-fidelity weighted image.
[0124] See also Figures 1 to 4 Step 100': Based on the initialized dirty random grid image, obtain a dirty weight smoothed image, including:
[0125] The initial dirty random grid image is smoothed to obtain a dirty weight smoothed image;
[0126] And / or, step 200': Obtain a dirt template weight image based on the dirt weight smoothing image, including:
[0127] Contrast enhancement and edge smoothing are performed on the dirt weight smoothed image to obtain the dirt template weight image.
[0128] In the above embodiments, the relevant steps and methods are the same as or similar to those in the above embodiments, and can be referred to the above description for details. No specific limitations are made here. This path forms a "mesh → smoothing → enhancement" chain, suitable for rapid deployment at edge devices.
[0129] This application also provides an apparatus capable of running and / or storing a method for generating lens dirt images as described in any of the above embodiments. This apparatus may be a computer, server, embedded device, or memory, and is not specifically limited thereto.
[0130] In the above embodiments, the device can be an in-vehicle intelligent driving domain controller or a cloud training server. The hardware includes at least one GPU and a solid-state storage unit, or the software is integrated into a module of the software stack. During the training phase, the device can generate 20-50 sets of dirty images with ground truth labels per second, generating over 10,000 high-quality samples daily for training the visual perception model. During the inference phase, the device can dynamically load a pre-generated template library and overlay it onto the original camera images in real time for system robustness testing under extreme contamination scenarios.
[0131] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0132] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
Claims
1. A method for generating an image of a dirty lens, characterized in that, The method includes: Based on the initial dirty random grid image, a dirty mosaic image is obtained; Based on the dirty mosaic image, a dirty outline image is obtained; Based on the dirt contour image, a dirt template weight image is obtained; Based on the first natural scene image, a dirty template image is obtained; Based on the dirty template image and the dirty template weight image, a dirty overlay image and a dirty weight overlay image are obtained; The dirt fusion result image is obtained based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image.
2. A method for generating an image of a dirty lens, characterized in that, The method includes: Based on the initial dirty random grid image, a dirty weight smoothed image is obtained; Based on the dirt weight smoothing image, a dirt template weight image is obtained; Based on the first natural scene image, a dirty template image is obtained; Based on the dirty template image and the dirty template weight image, a dirty overlay image and a dirty weight overlay image are obtained; The dirt fusion result image is obtained based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image.
3. The generation method according to any one of claims 1 or 2, characterized in that, The step of obtaining the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image includes: The dirt overlay image, the dirt weight overlay image, and the second natural scene image are overlaid to obtain the dirt fusion result image.
4. The generation method according to any one of claims 1 or 2, characterized in that, After obtaining the dirt fusion result image based on the dirt overlay image, the dirt weight overlay image, and the second natural scene image, the process includes: The dirt region truth map is obtained based on the dirt fusion result map, the dirt weight overlay map, and the second natural scene map.
5. The generation method according to any one of claims 1 or 2, characterized in that, The step of obtaining a dirt overlay image and a dirt weight overlay image based on the dirt template image and the dirt template weight image includes: Randomly select the dirty template image and the dirty template weight image; Randomly adjust the selected dirty template image and the dirty template weight image; The randomly adjusted dirty template image and the dirty template weight image are fused to generate a dirty overlay image and a dirty weight overlay image.
6. The generation method according to claim 5, characterized in that, The random adjustment of the selected dirty template image and the dirty template weight image includes: Randomly scale the dirty template image and the dirty template weight image; and / or, The fusion positions of the dirty template image and the dirty template weight image are randomly adjusted.
7. The generation method according to claim 5, characterized in that, The step of fusing the randomly adjusted dirty template image and the dirty template weight image to generate a dirty overlay image and a dirty weight overlay image includes: The randomly adjusted dirty template image and the dirty template weight image are fused together to generate the dirty template image and the dirty template weight image. When the number of fusions exceeds a preset threshold, a dirt overlay map and a dirt weight overlay map are output.
8. The generation method according to claim 1, characterized in that, The process of obtaining a dirty mosaic image based on an initialized dirty random grid image includes: Peak regions and edges are extracted from the initialized dirty random grid image to obtain a dirty mosaic image; And / or, obtaining the dirt outline image based on the dirt mosaic image includes: The dirty mosaic image is input into a diffusion model to obtain a dirty outline image; And / or, obtaining the dirt template weight image based on the dirt contour image includes: The dirt contour image is smoothed to obtain a dirt template weight image.
9. The generation method according to claim 2, characterized in that, The step of obtaining a dirt-weighted smoothed image based on an initialized dirty random grid image includes: The initial dirty random grid image is smoothed to obtain a dirty weight smoothed image; And / or, obtaining the dirt template weight image based on the dirt weight smoothing image includes: The dirt weight smoothing image is subjected to contrast enhancement and edge smoothing processing to obtain a dirt template weight image.
10. A device, characterized in that, The device operates and / or stores a method for generating a lens smudge image as described in any one of claims 1 to 9.