A camera model construction method for intelligent vehicle testing in a complex rainfall environment

Abstract

The application provides a camera model construction method for intelligent automobile testing in a complex rainfall environment, which comprises camera geometric model construction and camera physical model construction; the camera geometric model is used to determine the pixel position of a target object in the camera imaging plane in a test scene; the camera physical model determines the pixel intensity of the corresponding pixel position based on light tracking through light source information, rainfall information and camera parameter information, and determines the corresponding image information features through the pixel intensity. The application can provide prior knowledge before virtual simulation or closed field testing, predict the test results, guide the search of the virtual simulation edge test scene or the actual arrangement of the target object in the closed field, reduce the test cost, improve the test efficiency and shorten the test cycle.

Description

Technical Field

[0001] This invention relates to a camera model construction method, and more particularly to a camera model construction method for testing intelligent vehicles in complex rainy environments. Background Technology

[0002] Scenario-based testing is an effective way to ensure the practical deployment and safe application of intelligent vehicles. However, due to the complex and variable environment in which intelligent vehicles actually drive, traditional real-vehicle testing methods suffer from drawbacks such as uncontrollable test scenarios, poor scenario repeatability, and high scenario setup costs. Therefore, simulation-based test scenario construction methods have become a hot research topic in recent years. Meanwhile, the evaluation object of intelligent vehicle testing has shifted from the traditional binary system of humans and vehicles to a strongly coupled system of humans, vehicles, environment, and tasks. The impact of meteorological environment on intelligent vehicles is not only reflected in the road adhesion ability of traditional vehicles but also has a more significant impact on the perception system. Cameras are one of the most commonly used sensors in perception systems. As passive sensors, they are more susceptible to the influence of meteorological environmental factors in the scenario compared to active sensors such as lidar and millimeter-wave radar. Rain, as one of the most common complex weather conditions in the real world, frequently occurs in driving environments. The function of camera sensors is greatly affected in rainy environments, which can amplify the impact and ultimately lead to the failure of intelligent vehicle planning and control modules, causing serious traffic accidents. Therefore, in the process of designing test scenarios for intelligent vehicle camera sensors, it is necessary to consider how to design corresponding rainfall information to test the functions of intelligent vehicles under different camera parameters, target object types, and other factors. This requires a highly realistic camera model and rainfall blur model as the basis for constructing test scenarios. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a camera model construction method for intelligent vehicle testing in complex rainy environments, comprising the following steps:

[0004] (1) Camera geometry model construction:

[0005] Camera geometry models are used to determine the pixel positions of target objects in the camera's imaging plane during intelligent vehicle testing scenarios.

[0006] First, the geometric features of the target object are simplified using an axis-aligned bounding box. Then, based on the camera's mounting parameters and built-in parameters, the pixel positions of the target object within the camera's imaging plane are determined, serving as input to the camera's physical model. The specific steps are as follows:

[0007] (1.1) Simplify the features of the target object using AABB:

[0008] This invention uses axis-aligned bounding boxes (AABBs) to initially screen the pixel space and identify pixel groups that may provide key features of the target object: starting from each pixel point in the image plane, the intersection points of the ray and the x and y planes of the target object's AABB are calculated, and the expression for ray r(t) is:

[0009]

[0010] In the formula, r(t) is the ray formed by the line connecting the pixel in the image plane to the lens; o is the pixel coordinate. t is the unit vector along the line connecting the pixel and the center of the lens; t is the distance the light travels.

[0011] The intersection point t with the target object AABB is obtained by using the intersection point relationship. i =max[t min ], t o =min[t max ];t i t represents the distance the ray travels when it enters the AABB. o t represents the distance the ray travels after leaving the AABB; min t is the distance between the ray and the intersection point of the plane parallel to AABB; max The distance between the ray and the AABB parallel plane from the intersection point; if and only if t i <t o When the condition is met, the pixel is determined to be a valid pixel; otherwise, it is considered an invalid pixel. A valid pixel is one whose extended line connecting the pixel and the optical center intersects the AABB of the target object in the scene. An invalid pixel is one whose extended line connecting the pixel and the optical center does not intersect the AABB of the target object in the scene.

[0012] (1.2) Determine pixel positions based on geometric relationships:

[0013] Using H o and W o Describe a point on the target plane, where H O W refers to the distance between the point and the horizontal plane. O This refers to the distance of the point from the central axis of the target object. The x and y coordinates of the corresponding pixel plane point are (ima). v ,ima u ),ima v Expressed as:

[0014]

[0015] In the formula, the line connecting the center of the camera lens and the target point is denoted as ι, and δ hon The angle between the camera's optical axis and ι is expressed as:

[0016]

[0017] For IMA u The calculation is expressed as:

[0018]

[0019]

[0020] In the formula, RS x RS y For the horizontal and vertical resolution of CMOS, θ I Let x be the angle between the camera's optical axis and the vertical line. c ,y c ,z c (x) represents the camera's position relative to the world coordinate system. T ,y T This refers to the position information of the target object relative to the world coordinate system. Let f be the length, width, and height of the target object, and f be the camera focal length.

[0021] (ima) is obtained by testing the geometric positional relationships of the scene. v ,ima u This value, as the output of the camera's geometric model, will be used as the input of the camera's physical model, which will then output the pixel at that location.

[0022] (2) Camera physical model construction:

[0023] The camera physics model construction includes:

[0024] (2.1) Calculate pixel intensity under ideal conditions based on ray tracing;

[0025] Ray tracing calculates pixel values ​​at pixel locations output by the camera's geometric model. It breaks down the rendering task into several light rays originating from the camera and intersecting with the scene. The pixel intensity is then calculated based on information such as surface material and normal vectors at the intersection points, combined with light source information. Specific steps include:

[0026] (2.1.1) Light source analysis:

[0027] The light source is a major factor affecting pixel calculation. The radiant intensity I (unit: watts / solid angle) of the light source refers to the luminous flux Q (unit: watts) per unit solid angle per unit time, expressed as:

[0028]

[0029] In the formula, h represents Planck's constant; c represents the speed of light; λ(t) represents the wavelength of the light source as a function of time; Ω represents the solid angle; and φ represents the power of the light source.

[0030] Intelligent vehicles often operate in sunlight. Since the sun is far from the target object sensed by the camera, sunlight is abstracted as a point light source. The radiation range of the point light source is a sphere with radius r. The power of the light source received by a finite area A on the surface of this sphere is called irradiance E, and its expression is:

[0031]

[0032] In the formula, θ represents the incident ray and the normal vector to the object surface. The angle between them; r represents the distance between the point light source and the target object;

[0033] (2.1.2) Construction of the physical model of the target object:

[0034] When a target object in a scene receives irradiance E from a light source, the properties of the incident light will change depending on the surface normal and material characteristics of the target object. Generally, the light intensity will weaken, the wavelength will change according to the object's material, and the propagation direction will change. The bidirectional reflection distribution function (BRDF) is used to describe the reflectivity of a material surface at different incident and exit angles, and its expression is:

[0035]

[0036] In the formula, L(α) r ,β r E(α) represents the surface radiometry of the target object, expressed in watts per square meter * azimuth angle; i ,β i ) represents the surface irradiance of the target object, in watts per square meter; α and β describe the angular relationship between the incident and reflected rays;

[0037] (2.1.3) Camera model construction:

[0038] First, establish the surface emissivity L(α) of the target object. r ,β r ) and image irradiance E I The relationship between the two areas, for an area of ​​dA i A pixel plane with area dA s The target object, from the solid angle dω corresponding to the pixel plane at the center of the lens. i The solid angle dω of the target object corresponding to the center sSince they are the same size, according to the definition of a solid angle:

[0039]

[0040] dA i with dA s Proportional relationship:

[0041]

[0042] In the formula, α is the angle between the line connecting the corresponding pixel to the target point and the optical axis; θ is the line connecting the lens center to the corresponding pixel and the normal vector of that point on the target object. The angle between them; f is the camera focal length; z is the horizontal distance from the feature point of the target object to the center of the lens; the solid angle dω of this feature point with respect to the camera lens plane. L Expressed as:

[0043]

[0044] In the formula, D is the diameter of the camera lens;

[0045] According to the law of conservation of energy, the lens moves from dA... s The received luminous flux and the projection onto dA i If the luminous flux of a pixel plane is the same, then the image irradiance E I Expressed as:

[0046] E I ·dA i =L·(dA) s ·cosθ)·dω L (12)

[0047] Substituting equations (10-11) into equation (12) yields the image irradiance E. I The final expression is:

[0048]

[0049] Digital cameras convert this light signal into a charge signal. For each photosensitive unit on the image sensor, the number I of electrons released by the photoelectric effect after receiving image irradiance is... e Represented as:

[0050] I e =T∫E I ·S·q(λ)dλ (14)

[0051] In the formula, T is the integration time; S represents the spatial variation of sensitivity within the photosensitive unit; q(λ) represents the photoelectric conversion function of the camera; after this, the charge signal generated within the photosensitive unit is amplified by an amplifier circuit and output as a digital signal via an analog-to-digital converter, which is then buffered in the storage unit. The output quantity is DN, which is the camera's raw response (RR), expressed as:

[0052]

[0053] In the formula, g′ represents the gain factor of the analog circuit; V os η represents the bias voltage; η represents the quantization step size of the analog-to-digital conversion; however, RR cannot be directly used in image processing algorithms and requires a series of post-processing algorithms to output the final RGB three-channel image information. The specific post-processing steps include linear processing, white balance transformation, color interpolation, color space conversion, brightness correction, and gamma correction to output the final color image.

[0054] The first step is linear processing. Since different camera manufacturers have different upper and lower limits for RR storage, the range of the analog camera should be considered when selecting the corresponding values. This invention defines the lower limit as black and the upper limit as saturation. The image RR obtained through linear processing... Lin Represented as:

[0055]

[0056] The second step is white balance processing. This process mainly involves multiplying the three RGB channels by different gain coefficients to compensate for the effects of the different spectral sensitivities of the three color filters. If image brightness processing is completely disregarded in this step, and the multiplication coefficients for each channel are exactly proportional, then the impact of color changes on the image is equivalent. Generally, the gain of the G channel is set to 1, and white balance processing of the image is performed by changing the gains of the R and B channels. Let's assume the gain settings for these two channels are... R and gain B For a color filter arranged in [RGGB], the white balance processing matrix is:

[0057]

[0058] The third step is color interpolation, also known as de-mosaicing. After white balance processing, a one-dimensional image matrix is ​​obtained. To acquire image information, this step requires determining the RGB three-channel values ​​for each pixel position. The expression is:

[0059]

[0060] This step yielded three-dimensional image data;

[0061] The fourth step is color space conversion. The color space will output different values ​​depending on the display device. This value is mainly determined by A... c2s The matrix determines that A c2s Composed of two parts A c←x and A x←s Composition, the expression is:

[0062] A c2s =(A c←x ·A x←s ) -1 (19)

[0063] In the formula A x←s The following information was found in the standards published by the International Commission on Illumination (CIE):

[0064]

[0065] As for A c←x This depends on the specific camera model;

[0066] The fifth step is brightness correction and gamma correction. Before this step, all operations were performed to ensure that the image information and scene present a linear distribution. However, for actual intelligent vehicle applications, it is necessary to ensure a certain level of brightness information. This step is mainly achieved by adjusting the global brightness of the image after color space conversion.

[0067] (2.2) Determine pixel intensity abrupt changes based on rainfall fuzzy model;

[0068] The aforementioned fuzzy rainfall model construction:

[0069] Rainfall is a meteorological condition characterized by rapid movement and random distribution of varying shapes and sizes. The impact of rainfall on images is primarily manifested in abrupt changes in pixel intensity. Furthermore, raindrops exhibit strong randomness, failing to appear within the camera's field of view according to a fixed pattern. This invention aims to establish a mathematical model of the impact of rainfall on vehicle-mounted cameras, modeling four real-world characteristics of raindrops: particle-scale distribution, effective rainfall distribution, spatial random interference, and abrupt changes in pixel intensity. This raindrop fuzzy mathematical model is then used to process the camera's digital density (DN) information, establishing a camera model for testing intelligent vehicles in rainfall environments.

[0070] Raindrop particle-scale distribution refers to the different quantity distributions of raindrops of different sizes in rainfall environments with varying rainfall amounts, describing the distribution of raindrops of different sizes in different rainfall amounts. Effective rainfall distribution means that, for camera sensors, not all raindrop interference affects the functions of intelligent vehicles; the range of raindrop influence needs to be determined based on the geometric properties of the target object in the test scene. Spatial random interference refers to the randomness of the impact of raindrops on pixel intensity; based on the effective distribution, this randomness needs to be modeled to accurately describe the location and time randomness of the raindrop's impact on image pixel intensity. Intensity abrupt change characteristics refer to the abrupt change in pixel intensity at the location of raindrops compared to locations without raindrops, specifically, the pixel intensity at that location is greater than the pixel intensity in an environment without rainfall. This invention uses rejection sampling to model the raindrop particle-scale distribution; utilizes effective regions to model the effective distribution of raindrops; utilizes chaotic mapping to model the random distribution of rainfall; and utilizes Gaussian impulses to model the intensity abrupt changes caused by raindrops on pixel intensity, simulating real rainfall environments for intelligent vehicle testing and verification. Specific steps include:

[0071] (2.2.1) Effective distribution simulation:

[0072] This invention defines the area that effectively affects camera function as the effective distribution area, and its volume is denoted as V. in When the target is a vehicle, the region is mapped from the center of the camera lens to the target object's AABB (aligned as the origin). The volumes of the two triangular pyramids formed by the lines connecting the first intersecting surfaces to the origin are the corresponding volumes (V). in Using this value and the raindrop particle distribution curve N(a) ra Multiplying these two numbers yields the distribution of different raindrop sizes within that range.

[0073] (2.2.2) Simulation of random disturbances:

[0074] This invention selects a chaotic mapping method to model positional randomness. The specific chaotic mapping expression is as follows:

[0075] y = δsin(πx) i ) (twenty one)

[0076] In the formula, δ is the system parameter, δ∈[0,1]. Chaos occurs when δ∈(0.87,0.93)∪(0.95,1); V in The x, y, and z coordinates of the midpoint are simultaneously subjected to chaotic mapping to obtain the corresponding random distribution results;

[0077] (2.2.3) Raindrop particle-scale simulation:

[0078] This invention assumes that raindrops are spherical for analysis, and considers the vertical downward velocity v of the raindrops. ra The following description is provided using empirical formulas:

[0079]

[0080] In this empirical formula, a ra For the raindrop radius, the corresponding unit relationship is: a ra The unit is millimeters; v ra The unit is meters per second;

[0081] The size of raindrops will exhibit a random distribution N(a) within a certain space. ra The input for this distribution is a raindrop diameter of 2a. ra This value describes the distribution of the number of raindrops of different diameters within a certain space, and this distribution is described using the log-normal distribution expression:

[0082]

[0083] In the formula, σ represents the standard deviation of the raindrop size; N T D represents the total number of simulated raindrops per unit volume; D is the diameter of the camera lens; D g This represents the average diameter of the raindrops.

[0084] Raindrop particle-scale simulations employ rejection sampling and use a normal distribution for auxiliary screening, selecting N(a) as the sample size. ra Let p(x) be an auxiliary function and q(x) satisfy a standard normal distribution. We choose a value for M such that p(x) is always below Mq(x); We randomly sample x... (i) ~q(x), u~U(0,1), accept samples that satisfy the following formula, otherwise reject them:

[0085]

[0086] (2.2.4) Simulation of intensity mutation:

[0087] Because the movement of raindrops affects the intensity of a segment of pixels during the camera's sampling time, a "rainband" effect is produced. The specific number of pixels affected is described by the following expression:

[0088]

[0089] In the formula, I eff τ represents the number of pixels whose intensity is affected by a single raindrop; τ represents the frame rate of the camera being tested; dis represents the horizontal distance between the raindrop and the center of the camera lens.

[0090] For a single pixel, the number of raindrops affecting that pixel is determined by effective distribution simulation to be I. e The effect ΔI caused by rainfall on this pixel is expressed as:

[0091]

[0092] In the formula, ΔI max I represents the maximum effect of raindrops on pixel intensity. e The number of raindrops that affect the intensity of a pixel, G c It is a Gaussian impulse model;

[0093] The first three models describe the effectiveness, particle size distribution, and randomness of raindrops, respectively. However, the actual impact on pixel intensity also needs to be described by a model. Due to their internal reflective properties, raindrops collect light from a large field of view, leading to abrupt changes in pixel intensity. These changes are less affected by ambient background light. Based on these properties, this invention models the effect of raindrops on pixel intensity using a Gaussian pulse model, expressed as:

[0094] G c (x)=ε*exp[(x) 2 / (2σ g 2 (27)

[0095] In the formula, ε is the peak value of the Gaussian pulse, and σ g ε represents the standard deviation of the Gaussian pulse; ε is an important parameter describing the intensity of the abrupt change, and the value of this parameter is related to the value of the raindrop (x). drop ,y drop ,z drop ) and camera (x c ,y c ,z c The distance Dis is related to the distance of ), and the expression is:

[0096]

[0097]

[0098]

[0099] In the formula, T c It is the camera exposure time; τ ra N is the time it takes for a raindrop to travel the distance corresponding to one pixel; c It is the camera's F-number; L r and L b These represent scene irradiance under conditions of rainfall and no rainfall, respectively; calculate each I... eThe corresponding ε is then substituted into equation (26) to calculate the ΔI corresponding to that pixel; simultaneously, the maximum value ΔI of the raindrop's influence on the pixel intensity is calculated. max At that time, a ra The average value in the simulated raindrops is selected; finally, the influence ΔI of the rainfall environment on each pixel is determined, and the final image information is output.

[0100] The beneficial effects of this invention are:

[0101] The camera model proposed in this invention consists of a camera geometric model and a camera physical model. The camera geometric model is used to determine the pixel position of the target object in the test scene within the camera's imaging plane. The camera physical model determines the pixel intensity of the corresponding pixel position based on ray tracing using light source information, rainfall information, and camera parameter information. The pixel intensity is then used to determine the corresponding image information features. For rainfall features, rejection sampling is used to model the distribution of raindrop particles at the particle scale; effective regions are used to model the effective distribution of raindrops; chaotic mapping is used to model the random distribution of rainfall; and Gaussian pulses are used to model the intensity abrupt changes caused by raindrops to pixel intensity. This method can provide prior knowledge before virtual simulation or closed-site testing, predicting test results. It can be used to guide the search of virtual simulation edge testing scenes or the actual arrangement of target objects in closed sites, reducing testing costs, improving testing efficiency, and shortening the testing cycle. Attached Figure Description

[0102] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0103] Figure 2 This is a schematic diagram illustrating the simplification of target object features using AABB in this invention;

[0104] Figure 3 This is a schematic diagram of the AABB of the present invention;

[0105] Figure 4 This is a schematic diagram illustrating how the pixel position is determined based on geometric relationships according to the present invention;

[0106] Figure 5 This is a schematic diagram of the imaging model of the ideal single-lens optical system of the present invention;

[0107] Figure 6 This is a schematic diagram of the image post-processing workflow of the present invention;

[0108] Figure 7 This is a schematic diagram of the white balance output of the present invention;

[0109] Figure 8 This is a schematic diagram of the global image brightness adjustment of the present invention;

[0110] Figure 9This is a schematic diagram of the raindrop simulation method of the present invention;

[0111] Figure 10 This is a schematic diagram of the effective distribution simulation of the present invention;

[0112] Figure 11 This is a schematic diagram illustrating the generation of raindrops at random locations using chaotic mapping, as described in this invention.

[0113] Figure 12 This is a schematic diagram of the raindrop particle distribution curve of the present invention;

[0114] Figure 13 This is a schematic diagram showing the results of raindrop particle-scale simulation in this invention;

[0115] Figure 14 This is a schematic diagram illustrating the effect of a single raindrop on a pixel according to the present invention. Detailed Implementation

[0116] like Figure 1 As shown: This invention provides a camera model construction method for intelligent vehicle testing in complex rainy environments, comprising the following steps:

[0117] 1. Camera geometry model construction

[0118] Camera geometry models are used to determine the pixel positions of target objects in the camera's imaging plane during intelligent vehicle testing scenarios.

[0119] First, the geometric features of the target object are simplified using an axis-aligned bounding box. Then, the pixel position of the target object within the camera's imaging plane is determined based on the camera's mounting parameters and built-in parameters, serving as input to the camera's physical model.

[0120] 1.1 Using AABB to simplify the features of the target object

[0121] like Figure 2 As shown: In the process of constructing a test scenario for intelligent vehicles, it is necessary to consider the spatial position design of specific target objects. If the target objects are directly modeled, it will consume a lot of computing resources. Therefore, this invention uses axis-aligned bounding boxes (AABB) to perform initial screening of the pixel space and determine the pixel group that may provide key features of the target objects.

[0122] like Figure 3 As shown, specifically, starting from each pixel point on the image plane, the intersection points of the ray and the x and y planes of the target object's AABB are obtained. The expression for ray r(t) is:

[0123]

[0124] In the formula, o represents the pixel coordinates; t is the unit vector along the line connecting the pixel and the center of the lens; t is the distance the light travels.

[0125] The intersection point t with the target object AABB is obtained by using the intersection point relationship. i =max[t min ], t o =min[t max ];t i t represents the distance the ray travels when it enters the AABB. o t represents the distance the ray travels after leaving the AABB; min t is the distance between the ray and the intersection point of the plane parallel to AABB; max The distance between the ray and the AABB parallel plane from the intersection point; if and only if t i <t o When a pixel is deemed valid, it is considered a valid pixel; otherwise, it is considered an invalid pixel. A valid pixel is defined as one whose extended line connecting the pixel to the optical center intersects the AABB (alignment, angle, and block) of the target object in the scene. An invalid pixel is one whose extended line connecting the pixel to the optical center does not intersect the AABB of the target object in the scene. The purpose of proposing the valid pixel filtering method is to determine whether a pixel is valid as a preliminary screening, which can speed up the target object screening process and improve operational efficiency.

[0126] 1.2 Determine pixel position based on geometric relationships

[0127] like Figure 4 As shown: This invention uses the back plane of AABB as an example to describe pixel positions, utilizing H O and W O Describe a point on a plane, where W O W refers to the distance between the point and the horizontal plane. O This refers to the distance of the point from the central axis of the target object. The x and y coordinates of the corresponding pixel plane point are (ima). v ,ima u ),ima v Expressed as:

[0128]

[0129] In the formula, the line connecting the center of the camera lens and the target point is denoted as ι, and δ hon The angle between the camera's optical axis and l is expressed as:

[0130]

[0131] Similarly, for IMA u The calculation is expressed as:

[0132]

[0133]

[0134] In the formula, RS x RS y For the horizontal and vertical resolution of CMOS, θ I Let x be the angle between the camera's optical axis and the vertical line. c ,y c ,z c (x) represents the camera's position relative to the world coordinate system. T ,y T This refers to the position information of the target object relative to the world coordinate system. Let f be the length, width, and height of the target object, and f be the camera focal length.

[0135] (ima) is obtained by testing the geometric positional relationships of the scene. v ,ima u This value, as the output of the camera's geometric model, will be used as the input of the camera's physical model, and the physical model will output the pixel at that location.

[0136] 2. Camera Physical Model Construction

[0137] The camera physics model construction includes:

[0138] 2.1. Calculate pixel intensity under ideal conditions based on ray tracing;

[0139] The camera physics model is based on ray tracing to calculate the pixel values ​​of the pixel positions output by the camera's geometric model. Ray tracing breaks down the rendering task in the scene into several rays originating from the camera and intersecting with the scene. It calculates the pixel intensity based on information such as the surface material and normal vectors at the intersection points, combined with the light source information.

[0140] 2.1.1 Light Source Analysis

[0141] The light source is a major factor affecting pixel calculation. The radiant intensity I (unit: watts / solid angle) of the light source refers to the luminous flux Q (unit: watts) per unit solid angle per unit time, expressed as:

[0142]

[0143] In the formula, h represents Planck's constant; c represents the speed of light; λ(t) represents the wavelength of the light source as a function of time; Ω represents the solid angle; and φ represents the power of the light source.

[0144] Intelligent vehicles often operate in sunlight. Since the sun is far from the target object sensed by the camera, sunlight can be abstracted as a point light source. The radiation range of the point light source is a sphere with radius r. The power of the light source received by a finite area A on the surface of this sphere is called irradiance E, and its expression is:

[0145]

[0146] In the formula, θ represents the incident ray and the normal vector to the object surface. The angle between them; r represents the distance between the point light source and the target object.

[0147] 2.1.2 Physical Model of the Target Object

[0148] For intelligent vehicle camera sensors, the physical model of the target object in the scene needs to consider the optical reflection characteristics of the target object, which also determines the size of the corresponding pixels. After the target object in the scene receives irradiance E from the light source, the properties of the incident light will change according to the surface normal vector and material properties of the target object. Generally speaking, the light intensity will weaken, the wavelength will change according to the material of the object, and the propagation direction will change. The bidirectional reflection distribution function (BRDF) is used to describe the reflectivity of the material surface at different incident and exit angles, and its expression is:

[0149]

[0150] In the formula, L(α) r ,β r E(α) represents the surface radiometry of the target object, expressed in watts per square meter * azimuth angle; i ,β i ) represents the surface irradiance of the target object, in watts per square meter; α and β describe the angular relationship between incident and reflected light.

[0151] 2.1.3 Camera Model

[0152] like Figure 5 As shown: First, establish the surface radioactivity L(α) of the target object. r ,β r ) and image irradiance E I The relationship between the two areas, for an area of ​​dA i The pixel plane, the solid angle dω corresponding to this plane from the center of the lens. i Area dA of the corresponding target object s solid angle dω s Since they are the same size, according to the definition of a solid angle:

[0153]

[0154] dA i with dA s The proportional relationships are as follows:

[0155]

[0156] In the formula, α is the angle between the line connecting the corresponding pixel to the target point and the optical axis; θ is the line connecting the lens center to the corresponding pixel and the normal vector of that point on the target object. The angle between them; f is the camera focal length; z is the horizontal distance from the feature point of the target object to the center of the lens; the solid angle dω of this feature point with respect to the camera lens plane. L Expressed as:

[0157]

[0158] In the formula, D is the diameter of the camera lens;

[0159] According to the law of conservation of energy, the lens moves from dA... s The received luminous flux and the projection onto dA i If the luminous flux of a pixel plane is the same, then the image irradiance E I Expressed as:

[0160] E I ·dA i =L·(dA) s ·cosθ)·dω L (12)

[0161] Substituting equations (10-11) into equation (12) yields the image irradiance E. I The final expression is:

[0162]

[0163] Digital cameras convert this light signal into a charge signal. For each photosensitive unit on the image sensor, the number I of electrons released by the photoelectric effect after receiving image irradiance is... e Represented as:

[0164] I e =T∫E I ·S·q(λ)dλ (14)

[0165] In the formula, T is the integration time; S represents the spatial variation of sensitivity within the photosensitive unit; q(λ) represents the photoelectric conversion function of the camera; after this, the charge signal generated within the photosensitive unit is amplified by an amplifier circuit and output as a digital signal via an analog-to-digital converter, which is then buffered in the storage unit. The output quantity is DN, which is the camera's raw response (RR), expressed as:

[0166]

[0167] In the formula, g′ represents the gain factor of the analog circuit; V os η represents the bias voltage; η represents the quantization step size of the analog-to-digital conversion; however, RR cannot be directly used in image processing algorithms and requires a series of post-processing algorithms to output the final RGB three-channel image information. The specific post-processing steps include linear processing, white balance transformation, color interpolation, color space conversion, brightness correction and gamma correction to output the final color image.

[0168] like Figure 6 As shown: The first step is linear processing. Since different camera manufacturers have different values ​​for the upper and lower limits of RR storage, the range of the analog camera should be considered when selecting the corresponding values. This invention defines the lower limit as black and the upper limit as saturation. The image RR obtained by the linear processing is then... Lin Represented as:

[0169]

[0170] The second step is white balance processing. This process mainly involves multiplying the three RGB channels by different gain coefficients to compensate for the effects of the different spectral sensitivities of the three color filters. If image brightness processing is completely disregarded in this step, and the multiplication coefficients for each channel are exactly proportional, then the impact of color changes on the image is equivalent. Generally, the gain of the G channel is set to 1, and white balance processing of the image is performed by changing the gains of the R and B channels. Let's assume the gain settings for these two channels are... R and gain B For a color filter arranged in [RGGB], the white balance processing matrix is:

[0171]

[0172] The third step is color interpolation, also known as de-mosaicing. Through white balance processing, a one-dimensional image matrix is ​​obtained. In order to obtain image information, this step requires determining the RGB three-channel values ​​corresponding to each pixel position.

[0173] like Figure 7As shown, with Figure 7 Taking a B-pixel as an example, if the color filter is arranged in [RGGB] and the pixel's position is (i,j), then the B-channel value at that position is the B-pixel value, and the R-channel value at that position is the average of the values ​​of the surrounding four R-channels. The G-channel value can be obtained similarly. The expression is:

[0174]

[0175] The RGB values ​​for other locations are obtained in the same way, and the three-dimensional image data is obtained through this step.

[0176] The fourth step is color space conversion. The color space will output different values ​​depending on the display device. This value is mainly determined by A... c2s The matrix determines that A c2s Composed of two parts A c←x and A x←s The expression is:

[0177] A c2s =(A c←x ·A x←s ) -1 (19)

[0178] In the formula A x←s The following information was found in the standards published by the International Commission on Illumination (CIE):

[0179]

[0180] As for A c←x This depends on the specific camera model.

[0181] The fifth step is brightness and gamma correction. Prior to this step, all operations ensured a linear distribution of image information and scene. However, for practical intelligent vehicle applications, a certain level of brightness information needs to be maintained. This step is mainly achieved by performing global brightness adjustment on the image after color space conversion. The specific implementation effect is as follows: Figure 8 As shown.

[0182] 2.2. Determining pixel intensity abrupt changes based on a rainfall fuzzy model;

[0183] The rainfall model construction described above:

[0184] Rainfall is a meteorological condition characterized by rapid movement and random distribution of various shapes and sizes. Numerous researchers have provided empirical formulas for shape, size, and speed. Since common raindrops typically have a diameter of less than 1 mm, approximating raindrops as spherical, transparent objects is a premise of this invention. Furthermore, previous research indicates that the impact of rainfall on images primarily manifests as abrupt changes in pixel intensity. Additionally, raindrops exhibit strong randomness, failing to appear within the camera's field of view according to a fixed pattern. This invention aims to establish a mathematical model of the impact of rainfall on vehicle-mounted cameras, modeling four real-world characteristics of raindrops: particle-scale distribution, effective rainfall distribution, spatial random interference, and abrupt changes in pixel intensity. This raindrop fuzzy mathematical model is then used to process the camera's digital density (DN) information, establishing a camera model for testing intelligent vehicles in rainfall environments.

[0185] Raindrop particle-scale distribution refers to the different quantity distributions of raindrops of different sizes in rainfall environments with varying rainfall amounts, describing the distribution of raindrops of different sizes in different rainfall amounts. Effective rainfall distribution means that, for camera sensors, not all raindrop interference affects the functions of intelligent vehicles; the range of raindrop influence needs to be determined based on the geometric properties of the target objects in the test scene. Spatial random interference refers to the randomness of the impact of raindrops on pixel intensity; based on the effective distribution, this randomness needs to be modeled to accurately describe the location and time-based randomness of raindrops affecting image pixel intensity. Intensity abrupt change characteristics refer to the abrupt change in pixel intensity at the location of raindrops compared to locations without raindrops, specifically, the pixel intensity at that location is greater than the pixel intensity in an environment without rainfall. This invention uses rejection sampling to model the raindrop particle-scale distribution; utilizes effective regions to model the effective distribution of raindrops; utilizes chaotic mapping to model the random distribution of rainfall; and utilizes Gaussian impulses to model the intensity abrupt changes caused by raindrops on pixel intensity, simulating real rainfall environments for application in intelligent vehicle testing and verification, such as... Figure 9 As shown.

[0186] 2.2.1 Effective Distribution Simulation

[0187] For intelligent vehicle testing, not all raindrops within a unit volume of space will affect the test results. It is necessary to identify the area that effectively affects camera functionality to quantify the degree of impact on the test results. This invention defines this area as the effective distribution area, and its volume is denoted as V. in When the target is a vehicle, the region is mapped from the center of the camera lens to the target object's AABB (aligned as the origin). The volumes of the two triangular pyramids formed by the lines connecting the first intersecting surfaces to the origin are the corresponding volumes (V). in ,like Figure 10 As shown in the figure, the sum of the volumes of o-ABCD and o-DEFC constitutes V.in Using this value and the raindrop particle distribution curve N(a) ra Multiplying these two numbers yields the distribution of different raindrop sizes within that range.

[0188] 2.2.2 Simulation of Random Disturbance

[0189] The effective distribution defines the area where raindrops affect camera perception, but raindrops of different sizes are not uniformly distributed on V. in Therefore, it is necessary to model spatial randomness. This invention chooses the chaotic mapping method to model positional randomness. The specific chaotic mapping expression is as follows:

[0190] y = δsin(πx) i ) (twenty one)

[0191] In the formula, δ is the system parameter, δ∈[0,1]. Chaos occurs when δ∈(0.87,0.93)∪(0.95,1); V in The x, y, and z coordinates of the midpoint are simultaneously subjected to chaotic mapping to obtain the corresponding random distribution results, such as... Figure 11 As shown in the figure, the random location distribution of raindrops with different rainfall amounts within a unit volume is simulated.

[0192] 2.2.3 Raindrop Particle Scale Simulation

[0193] like Figure 12 As shown, the raindrop radius size a ra The distribution typically varies between 0.1 mm and 3.5 mm. Smaller raindrops exhibit a spherical distribution, while as the raindrop radius increases, the shape gradually approaches an ellipsoid. Based on Beard's empirical model of raindrop shape, when the raindrop radius is less than 2 mm, most raindrops are transparent spheres. Therefore, this invention assumes a spherical shape for analysis. Regarding the vertical downward velocity v of the raindrop... ra The following description is provided using empirical formulas:

[0194]

[0195] In this empirical formula, the unit correspondence is: a ra The unit is meters; v ra The unit is meters per second.

[0196] The size of raindrops will exhibit a random distribution N(a) within a certain space. ra The input for this distribution is a raindrop diameter of 2a. ra This value describes the distribution of the number of raindrops of different diameters within a certain space, and this distribution is described using the log-normal distribution expression:

[0197]

[0198] In the formula, σ represents the standard deviation of the raindrop size; N T D represents the total number of simulated raindrops per unit volume; g This represents the average diameter of the raindrops.

[0199] Raindrop particle-scale simulations employ rejection sampling and use a normal distribution for auxiliary screening, selecting N(a) as the sample size. ra Let p(x) be an auxiliary function and q(x) satisfy a standard normal distribution. We choose a value M such that p(x) is always below Mq(x). Experiments show that for small rain, a value of M of 9.2 * 10^- ... 3 The M value for moderate rain is taken as 1.13 * 10. 4 The M value for heavy rain is taken as 2.13 * 10. 5 Random sampling x (i) ~q(x), u~U(0,1), accept samples that satisfy the following formula, otherwise reject them:

[0200]

[0201] The results of raindrop particle-scale simulations are as follows: Figure 13 As shown.

[0202] 2.2.4 Simulation of Intensity Mutation

[0203] Because the movement of raindrops affects the intensity of a segment of pixels during the camera's sampling time, a "rainband" effect is produced, such as... Figure 14 As shown, the specific number of pixels affected, I eff Described by the following expression:

[0204]

[0205] In the formula, τ represents the frame rate of the camera being measured; dis represents the horizontal distance between the raindrop and the center of the camera lens.

[0206] For a single pixel, the number of raindrops affecting that pixel can be determined by effective distribution simulation as I. e The impact of rainfall on this pixel is expressed as:

[0207]

[0208] In the formula, ΔI represents the influence of raindrops on pixel intensity. max I represents the maximum effect of raindrops on pixel intensity. e The number of raindrops that affect the intensity of a pixel, G c It is a Gaussian impulse model;

[0209] The first three models described the distribution effectiveness, randomness, and particle-scale distribution characteristics of raindrops. This section describes the pixel intensity abrupt changes actually caused by raindrops. Due to the reflective properties of the internal liquid, raindrops absorb light from a large field of view, resulting in pixel intensity abrupt changes at the raindrop locations in the image. This abrupt change value is independent of the ambient background lighting, and its upper limit is approximately the same. Based on these properties, this paper defines the influence of raindrops on pixel intensity G... c It is modeled using a Gaussian impulse model. A Gaussian impulse can be expressed as:

[0210] G c (x)=ε*exp[(x) 2 / (2σ g 2 (27)

[0211] In the formula, ε is the peak value of the Gaussian pulse, and σ g ε represents the standard deviation of the Gaussian pulse; ε is an important parameter describing the intensity of the abrupt change, and the value of this parameter is related to the value of the raindrop (x). drop ,y drop ,z drop ) and camera (x c ,y c ,z c The distance Dis is related to the distance of ), and the expression is:

[0212]

[0213]

[0214]

[0215] In the formula, T c It is the camera exposure time; τ ra N is the time it takes for a raindrop to travel the distance corresponding to one pixel; c It is the camera's F-number; L r and L b These represent scene irradiance under conditions of rainfall and no rainfall, respectively; calculate each I... e The corresponding A is substituted into equation (26) to calculate the ΔI corresponding to the pixel; at the same time, the maximum value ΔI of the raindrop's influence on the pixel intensity is calculated. max At that time, a ra The average value in the simulated raindrops is selected; finally, the influence ΔI of the rainfall environment on each pixel is determined, and the final image information is output.

[0216] This embodiment applies a camera model to rain-affected environment testing of intelligent vehicles:

[0217] The model proposed in this invention is applied to the testing of intelligent vehicles in rainy environments. The purpose of the test is to search for edge scenes of the test object, which refer to test scenes where the expected functions cannot be met due to functional limitations of the test object. For rainy test environments, given the light source information, rainfall information, location information, and information of the camera under test, the method proposed in this invention can predict in advance whether the corresponding scene belongs to an edge scene. The judgment result can be used as a priori scene parameter setting for virtual simulation testing, or directly as a scene layout scheme for closed-site real vehicle testing, reducing testing costs, shortening the testing cycle, and improving testing efficiency. The simplified expression of the camera model proposed in this invention is as follows:

[0218] Score = CM(Light source information, rainfall information, location information, camera information under test) (31)

[0219] In the formula, CM() represents the camera model; Score represents the test value of the scene; Score is mainly mapped through ΔI output by the model. The method of this invention can determine the ΔI corresponding to the position of each pixel in the image, and use the three-channel original image information output after camera post-processing to test the algorithm under test.

Claims

1. A method for constructing a camera model for testing intelligent vehicles in complex rainy environments, characterized in that: Includes the following steps: (1) Camera geometry model construction: Camera geometry models are used to determine the pixel positions of target objects in the camera's imaging plane during intelligent vehicle testing scenarios. First, the geometric features of the target object are simplified using an axis-aligned bounding box. Then, the pixel position of the target object within the camera's imaging plane is determined based on the camera's mounting parameters and built-in parameters, serving as input to the camera's physical model. (2) Camera physical model construction: The camera physics model construction includes: (2.1) Calculate pixel intensity under ideal conditions based on ray tracing; Ray tracing calculates pixel values ​​at pixel locations output by the camera's geometric model. Ray tracing breaks down the rendering task in a scene into several intersecting rays originating from the camera and interacting with the scene. It then calculates pixel intensity based on surface material and normal vector information at the intersection points, combined with light source information. The steps include: (2.1.1) Light source analysis; (2.1.2) Construction of the physical model of the target object; (2.1.3) Camera model construction; (2.2) Determine pixel intensity abrupt changes based on rainfall fuzzy model; The aforementioned fuzzy rainfall model construction: To establish a mathematical model of the impact of rainfall on vehicle-mounted cameras, four real-world characteristics of raindrops—particle-scale distribution, effective rainfall distribution, spatial random interference, and pixel intensity abrupt changes—were modeled. This rainfall fuzzy mathematical model was then used to process the camera's digital density (DN) information, establishing a camera model for intelligent vehicle testing in rainfall environments. The steps include: (2.2.1) Effective distribution simulation; (2.2.2) Simulation of random disturbances; (2.2.3) Raindrop particle-scale simulation; (2.2.4) Intensity mutation simulation.

2. The camera model construction method for intelligent vehicle testing in complex rainy environments according to claim 1, characterized in that: The camera geometry model construction steps in step (1) include: (1.1) Simplify the features of the target object using AABB: The pixel space is initially screened using axis-aligned bounding boxes (AABBs) to identify pixel groups that may provide key features of the target object: starting from each pixel in the image plane, the intersection points of the ray and the x and y planes of the target object's AABB are calculated. The expression for ray r(t) is as follows: In the formula, r(t) is the ray formed by the line connecting the pixel in the image plane to the lens; o is the pixel coordinate. t is the unit vector along the line connecting the pixel and the center of the lens; t is the distance the light travels. The intersection point t with the target object AABB is obtained by using the intersection point relationship. i =max[t min ], t o =min[t max ];t i t represents the distance the ray travels when it enters the AABB. o t represents the distance the ray travels after leaving the AABB; min t is the distance between the ray and the intersection point of the plane parallel to AABB; max The distance between the ray and the AABB parallel plane from the intersection point; if and only if t i <t o When the condition is met, the pixel is determined to be a valid pixel; otherwise, it is considered an invalid pixel. A valid pixel is one whose extended line connecting the pixel and the optical center intersects the AABB of the target object in the scene. An invalid pixel is one whose extended line connecting the pixel and the optical center does not intersect the AABB of the target object in the scene. (1.2) Determine pixel positions based on geometric relationships: Using H O and W O Describe a point on the target plane, where H O W refers to the distance between the point and the horizontal plane. O This refers to the distance of the point from the central axis of the target object. The x and y coordinates of the corresponding pixel plane point are (ima). v ,ima u ),ima v Expressed as: In the formula, the line connecting the center of the camera lens and the target point is denoted as ι, and δ hon The angle between the camera's optical axis and ι is expressed as: For IMA u The calculation is expressed as: In the formula, RS x RS y For the horizontal and vertical resolution of CMOS, θ I Let x be the angle between the camera's optical axis and the vertical line. c ,y c ,z c (x) represents the camera's position relative to the world coordinate system. T ,y T This refers to the position information of the target object relative to the world coordinate system. Let f be the length, width, and height of the target object, and f be the camera focal length. (ima) is obtained by testing the geometric positional relationships of the scene. v ,ima u This value, as the output of the camera geometry model, will be used as the input of the camera physical model, and the camera physical model will output the pixel at that location.

3. A camera model construction method for intelligent vehicle testing in complex rainy environments according to claim 1 or 2, characterized in that: Step (2.1) of calculating the ideal pixel intensity based on ray tracing includes: (2.1.1) Light source analysis: The radiant intensity I of a light source refers to the luminous flux Q per unit solid angle per unit time, expressed as: In the formula, h represents Planck's constant; c represents the speed of light; λ(t) represents the wavelength of the light source as a function of time; Ω represents the solid angle; and φ represents the power of the light source. Intelligent vehicles often operate in sunlight. Since the sun is far from the target object sensed by the camera, sunlight is abstracted as a point light source. The radiation range of the point light source is a sphere with radius r. The power of the light source received by a finite area A on the surface of this sphere is called irradiance E, and its expression is: In the formula, θ represents the incident ray and the normal vector to the object surface. The angle between them; r represents the distance between the point light source and the target object; (2.1.2) Construction of the physical model of the target object: The bidirectional reflectance distribution function is used to describe the reflectance of a material surface at different incident and exit angles, and its expression is: In the formula, L(α) r ,β r E(α) represents the surface radiometry of the target object, expressed in watts per square meter * azimuth angle; i ,β i ) represents the surface irradiance of the target object, in watts per square meter; α and β describe the angular relationship between the incident and reflected rays; (2.1.3) Camera model construction: First, establish the surface emissivity L(α) of the target object. r ,β r ) and image irradiance E I The relationship between the two areas, for an area of ​​dA i A pixel plane with area dA s The target object, from the solid angle dω corresponding to the pixel plane at the center of the lens. i The solid angle dω of the target object corresponding to the center s They are the same size, according to the definition of solid angle: dA i with dA s Proportional relationship: In the formula, α is the angle between the line connecting the corresponding pixel to the target point and the optical axis; θ is the line connecting the lens center to the corresponding pixel and the normal vector of that point on the target object. The angle between them; f is the camera focal length; z is the horizontal distance from the feature point of the target object to the center of the lens; the solid angle dω of this feature point with respect to the camera lens plane. L Expressed as: In the formula, D is the diameter of the camera lens; According to the law of conservation of energy, the lens moves from dA... s The received luminous flux and the projection onto dA i If the luminous flux of a pixel plane is the same, then the image irradiance E I Expressed as: AND I ·from i =L·(dA s ·cosθ)·dω L dA i with dA s The proportional relationship and the solid angle dω of the feature point with respect to the camera lens plane L Substituting the relational expression into the above formula yields the image irradiance E. I The final expression is: Digital cameras convert this light signal into a charge signal. For each photosensitive unit on the image sensor, the number I of electrons released by the photoelectric effect after receiving image irradiance is... e Represented as: I e =T∫E I ·S·q(λ)dλ In the formula, T is the integration time; S represents the spatial variation of sensitivity within the photosensitive unit; q(λ) represents the photoelectric conversion function of the camera; after this, the charge signal generated within the photosensitive unit is amplified by an amplifier circuit and output as a digital signal via an analog-to-digital converter, which is then buffered in the storage unit. The output quantity is DN, which is the camera's raw response value, expressed as: In the formula, g′ represents the gain factor of the analog circuit; V os η represents the bias voltage; η represents the quantization step size of the analog-to-digital conversion; however, the camera's raw response value needs to be processed by a post-processing algorithm to output the final RGB three-channel image information. The post-processing steps include linear processing, white balance transformation, color interpolation, color space conversion, brightness correction and gamma correction, to output the final color image.

4. The camera model construction method for intelligent vehicle testing in complex rainy environments according to claim 3, characterized in that: The post-processing steps include: The first step is linear processing. The lower limit is defined as black, and the upper limit as saturation. The resulting image from this linear processing will have an RR value. Lin Represented as: The second step is white balance processing. The gain of the G channel is set to 1. White balance processing of the image is performed by changing the gains of the R and B channels, with the corresponding gain settings for these two channels being [gain value missing]. R and gain B For a color filter arranged in [RGGB], the white balance processing matrix is: The third step is color interpolation, also known as de-mosaicing. Through white balance processing, a one-dimensional image matrix is ​​obtained, determining the RGB three-channel values ​​for each pixel position. The expression is: B(i,j)=B(i,j) This step yielded three-dimensional image data; Step 4: Color space conversion, color space values ​​are changed from A... c2s The matrix determines that A c2s Composed of two parts A c←x and A x←s Composition, the expression is: A c2s =(A c←x ·A x←s ) -1 In the formula A x←s The following information was found in the standards published by the International Commission on Illumination (CIE): For A c←x This depends on the specific camera model; Step 5: Brightness and Gamma Correction. This step involves adjusting the global brightness of the image after color space conversion.

5. The camera model construction method for intelligent vehicle testing in complex rainy environments according to claim 1, characterized in that: Step (2.2) of determining pixel intensity abrupt changes based on the rainfall fuzzy model includes: (2.2.1) Effective distribution simulation: The effective distribution area is defined as the region that effectively affects camera function, and its volume is denoted as V. in Using this value and the raindrop particle distribution curve N(a) ra Multiplying these two numbers yields the distribution of different raindrop sizes within that range. (2.2.2) Simulation of random disturbances: The chaotic mapping method is chosen to model the randomness of location, and the expression for the chaotic mapping is as follows: y=δsin(πx i ) In the formula, δ is the system parameter, δ∈[0,1]. Chaos occurs when δ∈(0.87,0.93)∪(0.95,1); V in The x, y, and z coordinates of the midpoint are simultaneously subjected to chaotic mapping to obtain the corresponding random distribution results; (2.2.3) Raindrop particle-scale simulation: If we assume the raindrop is spherical, then for the raindrop's vertical downward velocity v... ra Described using empirical formulas: In this empirical formula, a ra For the raindrop radius, the corresponding unit relationship is: a ra The unit is millimeters, v ra The unit is meters per second; The size of raindrops will exhibit a random distribution N(a) within a certain space. ra The input for this distribution is a raindrop diameter of 2a. ra This value describes the distribution of the number of raindrops of different diameters within a certain space, and this distribution is described using the log-normal distribution expression: In the formula, σ represents the standard deviation of the raindrop size; N T D represents the total number of simulated raindrops per unit volume; D is the diameter of the camera lens; D g This represents the average diameter of the raindrops. Raindrop particle-scale simulations employ rejection sampling and use a normal distribution for auxiliary screening, selecting N(a) as the sample size. ra Let p(x) be an auxiliary function and q(x) satisfy a standard normal distribution. We choose a value for M such that p(x) is always below Mq(x); We randomly sample x... (i) ~q(x), u~U(0,1), accept samples that satisfy the following formula, otherwise reject them: (2.2.4) Simulation of intensity mutation: Because the movement of raindrops affects the intensity of a segment of pixels during the camera's sampling time, creating a "rainband" effect, the number of affected pixels is described by the following expression: In the formula, I eff τ represents the number of pixels whose intensity is affected by a single raindrop; τ represents the frame rate of the camera being tested; dis represents the horizontal distance between the raindrop and the center of the camera lens; and f is the camera's focal length. For a single pixel, the number of raindrops affecting that pixel is determined by effective distribution simulation to be I. e The effect ΔI caused by rainfall on this pixel is expressed as: In the formula, ΔI max I represents the maximum effect of raindrops on pixel intensity. e The number of raindrops that affect the intensity of a pixel, G c It is a Gaussian impulse model; The effect of raindrops on pixel intensity is modeled using a Gaussian pulse model, which is expressed as: G c (x)=ε*exp[(x) 2 / (2σ g 2 )] In the formula, ε is the peak value of the Gaussian pulse, and σ g ε represents the standard deviation of the Gaussian pulse; ε is an important parameter describing the intensity of the abrupt change, and the value of this parameter is related to the value of the raindrop (x). drop ,y drop ,z drop ) and camera (x c ,y c ,z c The distance Dis is related to the distance of ), and the expression is: In the formula, T c It is the camera exposure time; τ ra N is the time it takes for a raindrop to travel the distance corresponding to one pixel; c It is the camera's F-number; L r and L b These represent scene irradiance under conditions of rainfall and no rainfall, respectively; calculate each I... e The corresponding ε, and substitute it into the equation The calculation yields the ΔI corresponding to that pixel; simultaneously, the maximum value ΔI of the raindrop's influence on pixel intensity is calculated. max At that time, a ra The average value in the simulated raindrops is selected; finally, the influence ΔI of the rainfall environment on each pixel is determined, and the final image information is output.

Images