Multi-mode rainy day digital twinborn scene fusion construction method in automatic driving sample library

Abstract

The invention discloses a snowy day digital twinborn scene fusion construction method in an automatic driving sample library. The method comprises the following steps: acquiring an unprocessed image in an original picture data set in clear weather; obtaining an initial rain line model and an initial raindrop model; the obtained model is improved according to image characteristics, and a random multi-source rain line model and a heterogeneous mapping raindrop model are obtained; and according to the random multi-source rain line model and the heterogeneous mapping raindrop model, processing the acquired image in the data set to obtain a rain-added image. According to the invention, rain lines caused by falling of raindrops due to gravity and a local convex effect caused by falling of the raindrops to the front of a lens can be effectively simulated, scene reconstruction from a sunny condition to a rainy condition is realized, and a high-quality test sample library is constructed.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and specifically relates to a method for fusion construction of multi-modal rainy day digital twin scenes in an autonomous driving sample library. Background Art

[0002] Visual perception in rainy days has become one of the challenges facing autonomous driving technology. Effective sample data is the key to testing visual perception performance. Rainy day data has problems such as difficulty in collecting control samples and difficulty in controlling environmental variables. Rainy day scenes can be divided into rain line-based sample libraries, raindrop-based sample libraries, and raindrop-rain line combined sample libraries according to the type of rain. Existing rainy day sample library research focuses on rain line models. Based on the BSD dataset, Yang Wenhan et al. proposed the Rain100H and Rain100L sample libraries. Among them, Rain100H publicly disclosed 1,800 training sets containing 3-5 layers of rain lines and 100 test sets, and Rain100L publicly disclosed 200 training sets and 100 test sets of single-layer rain lines. Zhang He et al. used PhotoShop tools to randomly extract part of the BSD and UCID datasets and proposed the Rain800 sample library, which contains 700 pairs of rain line training sets and 100 test sets. FuXueyang et al. proposed the Rain1400 sample library, which synthesizes 14 images of different rain line directions and sizes from 1,000 images taken from the BSD and UCID datasets and Google images. QuanRuijie et al. proposed the RainDS sample library, introduced the rain line data acquisition method, collected 250 sets of real raindrop and rain line images and synthesized 1,200 sets of rainy day images. WangTianyu et al. proposed the SPA-Data sample library, which contains a large number of high-resolution rainy day and clear image pairs. LiSiyuan et al. proposed the MPID sample library, which covers various types of rainy day data such as raindrops, rain lines, rain fog, etc. according to the needs of multiple tasks. The existing rainy day data set synthesis method is relatively complicated to operate, and there is little research on the control of raindrops, so the raindrop effect is not good. Summary of the invention

[0003] The purpose of the present invention is to provide a method for fusion construction of multimodal rainy day digital twin scenes in an autonomous driving sample library to solve the problems existing in the prior art. The present invention can effectively simulate the rain lines caused by the falling of raindrops due to gravity and the local convex effect caused by raindrops falling in front of the lens, thereby realizing scene reconstruction from sunny conditions to rainy conditions and constructing a high-quality test sample library.

[0004] In order to achieve the above object, the present invention adopts the following technical scheme:

[0005] The method for fusion construction of multi-modal rainy day digital twin scenes in the autonomous driving sample library includes the following steps:

[0006] Step 1: Obtain the images in the original image dataset under unprocessed sunny weather;

[0007] Step 2: Obtain the initial rain line model and the initial raindrop model;

[0008] Step 3: Improve the model obtained in step 2 according to the image characteristics to obtain a random multi-source rain line model and a heterogeneous mapped raindrop model;

[0009] Step 4: According to the random multi-source rain line model and the heterogeneously mapped raindrop model obtained in step 3, the image in the data set obtained in step 1 is processed to obtain an image after raining.

[0010] Furthermore, in step 2, the initial rain line model is established as follows:

[0011]

[0012] Among them, I R_S represents the image after fusion of multiple rain line layers, S R_i represents the i-th rain line layer; J represents the image under the sunny scene; Represents the cumulative effect of n rain line layers;

[0013] In step 2, the initial raindrop model is established as follows:

[0014]

[0015] Among them, I R_d is the rainy scene image under the action of the raindrop model; M is the binary mask image, which represents the raindrop influence area. When M=1, it means that a certain pixel in the image to be processed is located in the raindrop influence area, and when M=0, it means that it is not in the raindrop influence area; D is the raindrop influence area image; J is the image under the sunny scene.

[0016] Furthermore, in step 3, the rain line model and raindrop model obtained in step 2 are improved to obtain a random multi-source rain line model and a raindrop model of heterogeneous mapping, including the following steps:

[0017] Step 3.1: Randomly generate a single rain line layer;

[0018] Step 3.2: Combine the depth information and fuse multiple rain line layers;

[0019] Step 3.3: Control the angle fluctuation of multiple layers of rain lines and perform multi-source rain line fusion;

[0020] Step 3.4: Design a heterogeneous mapping single raindrop convex vision algorithm for the image after multi-source rain line fusion;

[0021] Step 3.5: Design an anti-collision mechanism for a single raindrop;

[0022] Step 3.6: After the anti-collision design, a raindrop design is performed for the single raindrop taking into account the influence of the neighborhood.

[0023] Furthermore, a single rain line layer is randomly generated in step 3.1 as follows:

[0024] Map=Random(m,n,δ r )

[0025] Among them, m, n are the image length and width in the original image dataset; δ r is the rain line perception density; Map is the generated rain line distribution map, Random() is the random distribution mapping;

[0026] Based on the rain line distribution map, a rain line filter is designed to integrate the basic rain line with the distribution map:

[0027] S = RainFilter(R r ,Map)

[0028] Among them, S is the rain line layer; RainFilter is the rain line filter, which is composed of multiple layers of filtering; Map is the rain line distribution map; R r To filter the rain kernel, it is as follows:

[0029] R r =gaussfilt(R B (L,θ)

[0030] Among them, R B is the initial rain line shape; L is the size of the rain line; θ is the direction of the rain line; gaussfilt is the Gaussian filter.

[0031] Furthermore, in step 3.2, multiple rain line layers are fused by combining depth information:

[0032] The rain line perception density is modeled as:

[0033] δ r =ρd

[0034] Among them, ρ is the perception coefficient; d is the scene depth, and a single layer of rain line is expressed as:

[0035]

[0036] Among them, S is a single-layer rain line map, δ is the rain line density, θ is the rain line angle, k is the adjustment coefficient, Map() is the rain line distribution map, and R() is the rain line shape.

[0037] Furthermore, in step 3.3, the angle fluctuation of multiple layers of rain lines is controlled to perform multi-source rain line fusion, and the method is as follows:

[0038]

[0039] Among them, S R This is the rain line effect after multi-source fusion. n δ directions, n θ The cumulative effect of the perceived density; therefore, rain lines at different levels are affected by the scene depth, and the fused model is expressed as:

[0040]

[0041] In the formula, e -kd(i) is the influence of the depth of the i-th rain line scene on the rain line attenuation; k is the attenuation coefficient; δ ri is the perceived density of the rain line at the i-th layer; θ j is the jth tilt angle; S ij (δ ri ,θ j ) is δ ri and θ j Corresponding rain line layer effect.

[0042] Furthermore, in step 3.4, a heterogeneous mapping single raindrop convex vision algorithm is designed, and a circle is selected at any position as the raindrop area, denoted as M i , represents the i-th raindrop area, and constructs a mapping relationship, namely:

[0043]

[0044] in, is the image of the raindrop area after mapping; T(·) is the mapping function; D i is the original raindrop area image. T realizes geometric transformation by calculating the mapping relationship between the input image xy axis coordinates and the output coordinates, and converts the Cartesian coordinates of the input image into polar coordinates, that is:

[0045]

[0046] Among them, x and y are the horizontal and vertical coordinates of the image respectively; r and θ are the polar diameter and angle of the corresponding polar coordinates respectively; To map the transformed coordinates, the transformed area is mirrored and then mean filtered.

[0047] Furthermore, step 3.5 designs an anti-collision mechanism for a single raindrop, namely:

[0048] M c =M i &∑M i-1

[0049] Among them, M cM is the collision detection mask; i is the mask image of the current i-th raindrop; ∑M i-1 is the accumulation of the previous i-1 raindrop areas; & is the AND operation of the image. c If there is 1 in M, it means that the current raindrop collides with the previous raindrop and the current effect is abandoned. c A mask of all 0s indicates that there is no collision.

[0050] Furthermore, in step 3.6, a raindrop design considering the neighborhood influence is performed on the single raindrop after the anti-collision design, and the raindrop influence area after the anti-collision design is modified to:

[0051]

[0052] Among them, D i is the original raindrop area image; M Ω_i is the mask image M i neighborhood, due to D i The size is large, so shrink it as follows:

[0053]

[0054] Among them, r Ω is the shrinkage rate; τ i is the compensation coefficient, which compensates for the influence of light outside the image. Different empirical values ​​are set for different images. D' i For the shrunk raindrop mapping image, combined with steps 3.4-3.5, the raindrop model of heterogeneous mapping is described as:

[0055]

[0056] in, n d The accumulation of raindrop effects.

[0057] Furthermore, in step 4, the random multi-source rain line model and the heterogeneously mapped raindrop model obtained in step 3 are applied to the image in the data set obtained in step 1 to obtain an image after raining, which is described as follows:

[0058]

[0059] Among them, I R This is the image after rain.

[0060] Compared with the prior art, the present invention has the following beneficial technical effects:

[0061] The present invention can effectively achieve the effect of adding rain lines and raindrops of different concentrations to the picture, and complete the mapping process in the digital twin. The main idea of ​​the present invention is to obtain the rain line effect based on the construction of the rain line model. First, the single-layer rain line effect is obtained, and the improved rain line model is obtained by combining the image depth information to fuse the multi-source rain lines. Then a raindrop model is designed to simulate raindrops in real weather. The present invention can achieve the effect of adding different amounts of rain to the image. The image after raining gradually blurs as the rain line density and raindrop density increase, and the raining effect obtained is no different from the actual one. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings in the specification are used to provide further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0063] Figure 1 It is a schematic diagram of the process of the present invention;

[0064] Figure 2 is the 11th frame image in the data set, used as the original image;

[0065] Figure 3(a) to Figure 3(d) For Figure 2 Set the number of raindrops n d =10, the rain line density is δ r =The processing results of 5, 10, 15, 20;

[0066] Figure 4(a) to Figure 4(d) For Figure 2 Set the number of raindrops n d =20, the rain line density is δ r =The processing results of 5, 10, 15, 20;

[0067] Figure 5(a) to Figure 5(d) For Figure 2 Set the number of raindrops n d =30, the rain line density is δ r =The processing results of 5, 10, 15, 20;

[0068] Figure 6(a) to Figure 6(d) For Figure 2 Set the number of raindrops n d =50, the rain line density is δ r =The processing results of 5, 10, 15, 20;

[0069] Figure 7(a) to Figure 7(d) For Figure 2 Set the number of raindrops n d =50, the rain line density is δ r =The processing results of 5, 10, 15, 20;

[0070] Figure 8(a)-Figure 8(c) The corresponding non-parameter evaluation index information entropy, average gradient and FADE value curves, where different color curves correspond to different rain line perception densities δ r , corresponding to the green, red, blue, and pink curves respectively;

[0071] Figure 8(d)-Figure 8(f) The following are the corresponding evaluation indicators PSNR, SSIM and fuzzy value curves. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] See also Figure 1 , the present invention improves the initial rain line model and raindrop model, and proposes a random multi-source rain line model and heterogeneously mapped raindrops. For the rain line model, a single rain line layer is first randomly generated; secondly, the scene depth information is fused; finally, the scene depth and the random multiple rain line layers are combined. For the raindrop model, firstly, a single raindrop convex model for subsequent mapping is designed; secondly, a raindrop anti-collision mechanism is designed to prevent two raindrops from appearing in the same position; finally, a heterogeneously mapped raindrop model is constructed by combining the rain information around the raindrops. The improved rain line model and raindrop model are applied to sunny images, and it can be concluded that the image is close to the real situation in both subjective perception and objective evaluation, and the image quality is significantly reduced.

[0075] The specific steps are as follows:

[0076] Step 1: Obtain the images in the original image dataset under unprocessed sunny weather;

[0077] Step 2: Establish the initial rainy day model and initial raindrop model as follows:

[0078] Establish the initial rain line model:

[0079]

[0080] Among them, I R_S represents the image after fusion of multiple rain line layers, S R_i represents the i-th rain line layer; J represents the image under the sunny scene; Represents the cumulative effect of n rain line layers.

[0081] Create an initial raindrop model:

[0082] The raindrop model can be expressed as:

[0083]

[0084] Among them, I R_d is the rainy scene image under the raindrop model; M is the binary mask image, which indicates the raindrop influence area. When M=1, it means that a certain pixel in the image to be processed is in the raindrop influence area, and when M=0, it means that it is not in the raindrop influence area; D is the raindrop influence area image; J is the image under the sunny scene.

[0085] Step 3: Improve the model obtained in step 2 to obtain a random multi-source rain line model and a heterogeneous mapped raindrop model:

[0086] Step 3.1: Generate a random single rain line layer as follows:

[0087] Map=Random(m,n,δ r )

[0088] Where m, n are the length and width of the image in the original image dataset under clear weather obtained in step 1; δ r is the rain line perception density; Map is the generated rain line distribution map, and Random() is the random distribution mapping. Based on the rain line distribution map, the basic rain line and distribution map are fused by designing a rain line filter. Among them, the rain line filter is:

[0089] S = RainFilter(R r ,Map)

[0090] Among them, S is the rain line layer; RainFilter is the rain line filter, which is composed of multiple layers of filtering; Map is the rain line distribution map; R rTo filter the rain core, as the basic rain line shape, its size, direction, speed and other factors will affect the imaging effect, so it can be described as follows:

[0091] R r =gaussfilt(R B (L,θ)

[0092] Among them, R B is the initial rain line shape; L is the size of the rain line; θ is the direction of the rain line; gaussfilt is the Gaussian filter;

[0093] Step 3.2: Combine depth information and fuse multiple rain line layers:

[0094] The rain line perception density can be modeled as:

[0095] δ r =ρd

[0096] Where ρ is the perception coefficient and d is the scene depth. A single layer of rain line can be expressed as:

[0097]

[0098] S is a single-layer rain line map, δ is the rain line density, θ is the rain line angle, k is the adjustment coefficient, Map() is the rain line distribution map, and R() is the rain line shape.

[0099] Step 3.3: Control the angle fluctuation of multiple layers of rain lines and perform multi-source rain line fusion:

[0100] To fuse multiple rain line layers, the method is as follows:

[0101]

[0102] Among them, S R This is the rain line effect after multi-source fusion. n δ directions, n θ The cumulative effect of perceived density.

[0103] Therefore, rain lines at different levels are affected by the scene depth, and the fused model can be expressed as:

[0104]

[0105] In the formula, e -kd(i) is the influence of the depth of the i-th rain line scene on the rain line attenuation; k is the attenuation coefficient, k is 0.01; δ ri is the perceived density of the rain line at the i-th layer; θ j is the jth tilt angle; S ij (δ ri ,θ j ) is δri and θ j Corresponding rain line layer effect.

[0106] Step 3.4: Design a heterogeneous mapping single raindrop convex vision algorithm:

[0107] Select a circle at any position as the raindrop area, denoted as M i , represents the i-th raindrop area. Construct a mapping relationship, namely:

[0108]

[0109] in, is the image of the raindrop area after mapping; T(·) is the mapping function; D i is the original raindrop area image. T realizes geometric transformation by calculating the mapping relationship between the input image xy axis coordinates and the output coordinates. The Cartesian coordinates of the input image are converted to polar coordinates, that is:

[0110]

[0111] Among them, x and y are the horizontal and vertical coordinates of the image respectively; r and θ are the polar diameter and angle of the corresponding polar coordinates respectively; is the coordinate after mapping transformation. The transformed area is mirrored and then mean filtered.

[0112] Step 3.5: Design an anti-collision mechanism for a single raindrop:

[0113] Design anti-collision detection when generating raindrop positions, namely:

[0114] M c =M i &∑M i-1

[0115] Among them, M c M is the collision detection mask; i is the mask image of the current i-th raindrop; ∑M i-1 is the accumulation of the previous i-1 raindrop areas; & is the AND operation of the image. c If there is 1 in M, it means that the current raindrop collides with the previous raindrop and the current effect is abandoned. c A mask of all 0s indicates that there is no collision.

[0116] Step 3.6: After the anti-collision design, the single raindrop is designed to take the influence of the neighborhood into consideration:

[0117] Modify the raindrop influence area to:

[0118]

[0119] Among them, Di is the original raindrop area image; M Ω_i is the mask image M i Neighborhood. Since D i The size is larger and needs to be shrunk to a certain extent, as follows:

[0120]

[0121] Among them, r Ω is the shrinkage rate; τ i D' is the compensation coefficient, which compensates for the influence of light outside the image. Different empirical values ​​are set for different images. i is the shrunk raindrop mapping image. Combined with steps 3.4 to 3.6, the raindrop model can be described as:

[0122]

[0123] in, n d The accumulation of raindrop effects.

[0124] The obtained random multi-source rain line model and heterogeneously mapped raindrop model are applied to the images in the unprocessed clear weather dataset obtained in step 1 to obtain the rained image, which is described as follows:

[0125]

[0126] Among them I R This is the image after rain.

[0127] Figure 2 It is the 11th frame image in the data set, which is used as the original image. Figure 3(a) to Figure 3(d) For Figure 2 Set the number of raindrops n d =10, the rain line density is δ r =The processing results of 5, 10, 15, 20; Figure 4(a) to Figure 4(d) For Figure 2 Set the number of raindrops n d =20, the rain line density is δ r =The processing results of 5, 10, 15, 20; Figure 5(a) to Figure 5(d) For Figure 2 Set the number of raindrops n d =30, the rain line density is δ r =The processing results of 5, 10, 15, 20; Figure 6(a) to Figure 6(d) For Figure 2 Set the number of raindrops n d =50, the rain line density is δ r =The processing results of 5, 10, 15, 20; Figure 7(a) to Figure 7(d) For Figure 2 Set the number of raindrops n d =50, the rain line density is δ r =5, 10, 15, 20. Figure 8(a)-(c) shows the information entropy, average gradient and FADE value curves of the non-parametric evaluation index. The different color curves in the figure correspond to different rain line perception densities δ r , corresponding to the green, red, blue, and pink curves. Figure 8(d)-(f) shows the corresponding evaluation indicators PSNR, SSIM, and fuzzy value curves. Table 1 shows the corresponding evaluation indicators PSNR, SSIM, and fuzzy value curves with n d Table 2 shows the objective index evaluation of each picture with the change of δ r Objective indicator evaluation of each picture with changes.

[0128] For Figures 3 to 7, it can be seen that the information entropy, average gradient and FADE in the figures change relatively obviously with the rain line density. The information entropy and average gradient first increase and then decrease with the rain line density, and FADE first decreases and then increases. The number of raindrops has little effect on the non-parametric evaluation, and the information entropy, average gradient and FADE only fluctuate slightly. As the density increases, PSNR, SSIM and fuzzy coefficient continue to decrease, and the image quality decreases significantly. After the number of raindrops increases, the parameter-containing evaluation changes little, but it also shows a downward trend, indicating that the image quality is also gradually decreasing. From Tables 1-2 and Figure 8(a)-Figure 8(f) It can be seen that as δ r As the value of n increases, the corresponding fuzzy coefficient, PSNR, and SSIM gradually decrease, the information entropy and average gradient increase first and then decrease, and the FADE decreases first and then increases. d The information entropy, FADE and fuzzy coefficient increase slightly, the average gradient, PSNR and SSIM decrease continuously, and the image quality decreases continuously, which is consistent with the actual situation. Therefore, after the processing of the present invention, the information loss in the rainy scene image is controlled by the rainfall perception coefficient and the number of raindrops, and the rainy day effect with different rainfall can be effectively achieved.

[0129] Table 1n d Objective indicators of change

[0130]

[0131] Table 2δ r Objective indicators of change

[0132]

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the pending claims of the invention.

Claims

1. A method for fusion construction of multi-modal rainy day digital twin scenes in an autonomous driving sample library, characterized in that: The following steps are involved: Step 1: Obtain the images in the original image dataset under unprocessed sunny weather; Step 2: Obtain the initial rain line model and the initial raindrop model; Step 3: Improve the model obtained in step 2 according to the image characteristics to obtain a random multi-source rain line model and a heterogeneous mapped raindrop model; Step 4: According to the random multi-source rain line model and the heterogeneously mapped raindrop model obtained in step 3, the image in the data set obtained in step 1 is processed to obtain an image after raining.

2. The method for fusion construction of multimodal rainy day digital twin scenes in the autonomous driving sample library according to claim 1 is characterized in that: In step 2, the initial rain line model is established as follows: Among them, I R_S represents the image after fusion of multiple rain line layers, S R_i represents the i-th rain line layer; J represents the image under the sunny scene; Represents the cumulative effect of n rain line layers; In step 2, the initial raindrop model is established as follows: Among them, I R_d is the rainy scene image under the action of the raindrop model; M is the binary mask image, which represents the raindrop influence area. When M=1, it means that a certain pixel in the image to be processed is located in the raindrop influence area, and when M=0, it means that it is not in the raindrop influence area; D is the raindrop influence area image; J is the image under the sunny scene.

3. The method for fusion construction of multimodal rainy day digital twin scenes in the autonomous driving sample library according to claim 1 is characterized in that: In step 3, the rain line model and raindrop model obtained in step 2 are improved to obtain a random multi-source rain line model and a raindrop model of heterogeneous mapping, including the following steps: Step 3.1: Randomly generate a single rain line layer; Step 3.2: Combine the depth information and fuse multiple rain line layers; Step 3.3: Control the angle fluctuation of multiple layers of rain lines and perform multi-source rain line fusion; Step 3.4: Design a heterogeneous mapping single raindrop convex vision algorithm for the image after multi-source rain line fusion; Step 3.5: Design an anti-collision mechanism for a single raindrop; Step 3.6: After the anti-collision design, a raindrop design is performed for the single raindrop taking into account the influence of the neighborhood.

4. The method for constructing multimodal rainy day digital twin scene fusion in the autonomous driving sample library according to claim 3 is characterized in that: In step 3.1, a single rain line layer is randomly generated as follows: Map=Random(m,n,δ r ) Among them, m, n are the image length and width in the original image dataset; δ r is the rain line perception density; Map is the generated rain line distribution map, Random() is the random distribution mapping; Based on the rain line distribution map, a rain line filter is designed to integrate the basic rain line with the distribution map: S=RainFilter(R r ,Map) Among them, S is the rain line layer; RainFilter is the rain line filter, which is composed of multiple layers of filtering; Map is the rain line distribution map; R r To filter the rain kernel, it is as follows: R r =gaussian filter(R B (L,θ)) Among them, R B is the initial rain line shape; L is the size of the rain line; θ is the direction of the rain line; gaussfilt is the Gaussian filter.

5. The method for constructing multimodal rainy day digital twin scene fusion in the autonomous driving sample library according to claim 4 is characterized in that: In step 3.2, multiple rain line layers are fused together by combining depth information: The rain line perception density is modeled as: d r =ρd Among them, ρ is the perception coefficient; d is the scene depth, and a single layer of rain line is expressed as: Among them, S is a single-layer rain line map, δ is the rain line density, θ is the rain line angle, k is the adjustment coefficient, Map() is the rain line distribution map, and R() is the rain line shape.

6. The method for fusion construction of multimodal rainy day digital twin scenes in the autonomous driving sample library according to claim 5 is characterized in that: In step 3.3, the angle fluctuation of multiple layers of rain lines is controlled to fuse multiple sources of rain lines. The method is as follows: Among them, S R This is the rain line effect after multi-source fusion. n δ directions, n θ The cumulative effect of the perceived density; therefore, rain lines at different levels are affected by the scene depth, and the fused model is expressed as: In the formula, e -kd(i) is the influence of the depth of the i-th rain line scene on the rain line attenuation; k is the attenuation coefficient; δ ri is the perceived density of the rain line at the i-th layer; θ j is the jth tilt angle; S ij (δ ri ,θ j ) is δ ri and θ j Corresponding rain line layer effect.

7. The method for fusion construction of multimodal rainy day digital twin scenes in the autonomous driving sample library according to claim 6 is characterized in that: In step 3.4, a heterogeneous mapping single raindrop convex vision algorithm is designed, and a circle is selected at any position as the raindrop area, denoted as M i , represents the i-th raindrop area, and constructs a mapping relationship, namely: in, is the image of the raindrop area after mapping; T(·) is the mapping function; D i is the original raindrop area image. T realizes geometric transformation by calculating the mapping relationship between the input image xy axis coordinates and the output coordinates, and converts the Cartesian coordinates of the input image into polar coordinates, that is: Among them, x and y are the horizontal and vertical coordinates of the image respectively; r and θ are the polar diameter and angle of the corresponding polar coordinates respectively; To map the transformed coordinates, the transformed area is mirrored and then mean filtered.

8. The method for constructing multimodal rainy day digital twin scene fusion in the autonomous driving sample library according to claim 7, characterized in that: Step 3.5 designs an anti-collision mechanism for a single raindrop, namely: M c =M i &∑M i-1 Among them, M c is the collision detection mask; M i is the mask image of the current i-th raindrop; ∑M i-1 is the accumulation of the previous i-1 raindrop areas; & is the AND operation of the image. c If there is 1 in M, it means that the current raindrop collides with the previous raindrop and the current effect is abandoned. c A mask of all 0s indicates that there is no collision.

9. The method for constructing multimodal rainy day digital twin scene fusion in the autonomous driving sample library according to claim 8, characterized in that: In step 3.6, the raindrop design considering the neighborhood influence is performed on the single raindrop after the anti-collision design, and the raindrop influence area after the anti-collision design is modified to: Among them, D i is the original raindrop area image; M Ω_i is the mask image M i neighborhood, due to D i The size is large, so shrink it as follows: Among them, r Ω is the shrinkage rate; τ i is the compensation coefficient, which compensates for the influence of light outside the image. Different empirical values ​​are set for different images. D' i For the shrunk raindrop mapping image, combined with steps 3.4-3.5, the raindrop model of heterogeneous mapping is described as: in, n d The accumulation of raindrop effects.

10. The method for fusion construction of multi-modal rainy day digital twin scenes in the autonomous driving sample library according to claim 9, characterized in that: In step 4, the random multi-source rain line model and the heterogeneous mapped raindrop model obtained in step 3 are applied to the image in the data set obtained in step 1 to obtain the rained image, which is described as follows: Among them, I R This is the image after rain.

Images