A Functional Simulation Test Method for Smart Car Cameras in High-Light Environments

Abstract

This invention relates to a camera function testing method, and more particularly to a simulation testing method for intelligent vehicle camera functions in strong light environments. First, based on camera parameters, the correspondence between the position of the strong light source in the world coordinate system and the pixel position in the image coordinate system is determined. Second, the influence radius of the light source is determined based on the geometric characteristics of strong light. Finally, the pixel positions affected by strong light are output as the strong light influence area. A camera imaging physical model is established according to the "light-electricity-digital" signal propagation process. The specific numerical values ​​of the strong light environment's influence on pixel intensity are determined based on the physical characteristics of the strong light source. Based on the geometric model, a more realistic strong light influence area is simulated by combining a strong light pixel threshold. Based on the strong light influence area and the corresponding pixel values, this is superimposed onto a non-strong light image captured by the actual vehicle to synthesize a strong light simulation image, which is then used to test the camera function. This method has the advantages of low testing cost, high testing efficiency, and precise and controllable simulation parameters.

Description

Technical Field

[0001] This invention relates to a camera function testing method, and more particularly to a method for simulating the function testing of a smart car camera in a strong light environment. Background Technology

[0002] Camera-based environmental perception boasts advantages such as high recognition rate, low cost, and long lifespan, making it widely used in intelligent vehicles. However, cameras are passive sensors, and external lighting conditions directly affect their imaging, thus impacting the normal operation of their environmental perception functions. When light from strong light sources reaches the camera, reflection and scattering between the camera's optical elements create noise. This can degrade image quality and produce artifacts. In particular, the strong light environments commonly encountered in intelligent vehicle driving environments, such as those generated by vehicle headlights or sunlight, can easily cause camera malfunctions and even serious traffic accidents. There is an urgent need to establish functional testing methods for intelligent vehicle cameras in strong light environments to improve their robustness and environmental adaptability. However, real-world testing in strong light conditions on actual roads or closed environments suffers from high testing costs, long testing cycles, and difficulties in controlling and reproducing the scenarios. Simulation testing methods offer advantages such as low testing costs, high controllability, high repeatability, and short testing cycles. However, when using simulation methods to test camera functions in strong light environments, the confidence level of the test results is insufficient, making it difficult to accurately test the adaptability of camera functions in strong light environments. In addition, in-depth research is urgently needed on establishing accurate and controllable strong light simulation models. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method for simulating the functionality of a smart car camera in strong light environments, comprising the following steps:

[0004] (1) The types of strong light sources to be simulated include vehicle high beams and sunlight;

[0005] (2) Construction of the geometric model for imaging in strong light environment:

[0006] First, the correspondence between the position of the strong light source in the world coordinate system and the pixel position in the image coordinate system is determined based on the camera parameters. Second, the influence radius of the light source is determined based on the geometric characteristics of the strong light. Finally, the pixel position affected by the strong light is output as the strong light influence area, and the information of this area is used as the input of the physical model of strong light environment imaging.

[0007] (2.1) Construction of the geometric model of the strong light source location:

[0008] At night, vehicle high beams are mostly generated in the lower half of the image, while sunlight is mostly generated in the upper half of the image.

[0009]

[0010] A geometric model of the location of a strong light source is used to determine the coordinates of the light source in the world coordinate system. plane with pixel coordinate system The correspondence is expressed as follows:

[0011] In the formula, These are the pixel coordinates corresponding to the light source in the image coordinate system. This represents the z-axis coordinate of the light source in the camera coordinate system. Representing pixel coordinates respectively The number of pixels per unit distance in both axes, expressed in pixels per meter (pixel / m). The offset coordinates of the optical center projected onto the pixel plane in the pixel coordinate system; This is the rotation and translation transformation matrix performed when transforming from the world coordinate system to the camera coordinate system; For the light source in the world coordinate system Axis coordinate values.

[0012] (2.2) Construction of the geometric model of the region affected by strong light source:

[0013] Modeling the geometric pixel influence range of two different strong light sources in an image:

[0014] Among them, the strong light generated by the vehicle's high beams affects the radius of the circle. The expression is:

[0015]

[0016] In the formula, This is the camera focal length, expressed in pixels. The distance between the light source and the camera; The effective distance between the illuminated plane produced by the light source and the strong light source. For vehicle high beams, this means the distance between the illuminated plane produced by the light source and the strong light source exceeds the effective distance. Subsequently, the strong glare from the vehicle's high beams on the pixels becomes negligible. Determined based on the characteristics of different high beams; The light source scattering angle represents the angle between the light rays produced by the high beam, expressed in degrees.

[0017] The strong sunlight affects the radius of the circle. Regardless of the distance between the light source and the camera The changes occur and are related to the parameters of the camera being measured. The characteristics of the Gaussian beam are used to model it, and its expression is:

[0018]

[0019] In the formula, is the maximum light intensity of the beam, i.e., the light intensity of the beam at the waist position; r is the radial distance between the camera and the center of the sunlight source; It is the waist radius, the value of which determines scope;

[0020] Waist radius The value of is related to the wavelength of light. and camera focal length The relevant expression is:

[0021]

[0022] in, These are constants related to the camera system.

[0023] (3) Construction of physical model for imaging in strong light environment:

[0024] In the physical model, in order to determine the correspondence between strong light environment and pixel intensity, a camera imaging physical model is first established according to the "light-electricity-digital" signal propagation process. Then, the specific values ​​of the influence of strong light environment on pixel intensity are determined according to the physical characteristics of strong light source. Based on the geometric model, combined with the strong light pixel threshold in the physical model, a more realistic strong light influence area is simulated.

[0025] (3.1) Construction of the camera imaging physical model:

[0026] The camera imaging physics model establishes a physical propagation model between the brightness of a specific input light source and the output pixel value. First, the scene radiometry is established. (unit: ) and the image irradiance received by the camera image (unit: The relationship between )

[0027]

[0028] In the formula, The angle of incidence of the light ray; The diameter of the camera lens; digital cameras will... When light signals are converted into electrical charge signals, the number of electrons released by the photoelectric effect after receiving image irradiance for each photosensitive unit on the image sensor is... Represented as:

[0029]

[0030] In the formula, The integration time is in seconds. This indicates the spatial variation in sensitivity within the photosensitive unit; The photoelectric conversion function of a camera refers to the number of electrons released for every joule of energy absorbed at a specified wavelength. The wavelength of light within the camera's light-sensing range.

[0031] The charge signal generated within the photosensitive unit is amplified by an amplifier circuit and then output as a digital signal via an analog-to-digital converter. The (Raw Response) is buffered in the storage unit, and the output quantity is DN, which is a digital signal. This is the camera's raw response value, expressed as:

[0032]

[0033] In the formula, Represents the gain factor of analog circuits; Represents the bias voltage; This represents the quantization step size for analog-to-digital conversion.

[0034] For digital signals A series of camera post-processing steps are performed to output the final RGB three-channel image information. The post-processing steps include linear processing, white balance transformation, color interpolation, color space conversion, brightness correction and gamma correction, and finally output a color image.

[0035] The first step is linear processing for digital signals. The storage of upper and lower bound values, with the lower bound defined as... The upper limit is defined as The image obtained by the linear processing procedure Represented as:

[0036]

[0037] The second step is white balance processing. The RGB channels are multiplied by different gain coefficients to simulate the effects of three different color filters. The gain of the G channel is set to 1. White balance processing of the image is performed by changing the gains of the R and B channels, with the corresponding gain settings being... For a color filter arranged in [RGGB], the white balance processing matrix is:

[0038]

[0039] The third step is color interpolation, also known as de-mosaicing. After color interpolation, the one-dimensional image matrix output by white balance is a three-channel color image. In order to obtain image information, this step requires determining the RGB three-channel position information corresponding to each pixel.

[0040] Step 4: Color space conversion. The color space will output different values ​​depending on the display device. This value is mainly determined by... The matrix determines, Depend on It consists of two parts, and the expression is:

[0041]

[0042] In the formula, This represents the transformation matrix from the camera color space to the standard color space, used to realize the mapping relationship between the camera-captured image and the standard color space; This represents the transformation matrix from the camera color space to the intermediate reference color space. This matrix is ​​determined based on the specific imaging parameters of different cameras. This represents the transformation matrix from the standard color space to the intermediate reference color space, which can be determined based on standard data published by the International Commission on Illumination (CIE). It was found in the standards published by the International Commission on Illumination, and for This needs to be confirmed based on the specific parameters of the camera in different simulations;

[0043]

[0044] Step 5: Brightness and gamma correction, which involves global brightness adjustment of the image after color space conversion.

[0045] (3.2) Physical characteristics of strong light sources:

[0046] To determine the scene radiance L produced by a strong light source, we analyze the physical characteristics of the strong light source itself, and the expression is:

[0047]

[0048] In the formula, Power of a high-intensity light source, measured in watts (W). The area that receives the light source, ; The angle formed by the line connecting the strong light source and the center of the camera lens and the perpendicular line to the camera plane; The solid angle produced by a strong light source. The unit is .

[0049] According to step (3.2), the corresponding strong light pixel thresholds are determined by using the scene radiance of vehicle high beams and sunlight respectively. Then, in combination with step (3.1), pixels that simultaneously meet the threshold conditions in the RGB channels of the actually acquired strong light images are screened based on the pixel thresholds.

[0050] (4) The strong light influence area output by the geometric model constructed in step 2 and the pixel values ​​corresponding to the strong light area output by the physical model in step 3 are superimposed on the non-strong light image collected by the actual vehicle to synthesize a strong light simulation image. The camera function is tested using this image.

[0051] The beneficial effects of this invention are:

[0052] This invention proposes a method for simulating and testing the functionality of intelligent vehicle cameras in strong light environments. This method simulates the impact of strong light environments on images using a geometric-physical fusion model. This impact is then synthesized onto a real vehicle image captured without strong light to obtain a simulated image of the strong light environment. This simulated image is then used to test the functionality of the intelligent vehicle camera. By establishing a geometric model for strong light environment imaging, the influence area of ​​strong light in the image is determined. Based on the camera's physical model and the physical characteristics of different strong light sources, the strong light pixel values ​​are determined. The test results for the same camera function under strong light conditions are compared with those from real-world environments, large-scale model methods, simulation software, and the method of this invention. This verifies that the proposed method can improve the confidence level of simulation testing methods. Compared to traditional simulation software and large-scale model methods, this invention can more accurately and effectively test the functionality of intelligent vehicle cameras in strong light environments. Furthermore, compared to physical scene testing, it has advantages such as lower testing costs, higher testing efficiency, and more precise and controllable simulation parameters. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall architecture of the method of the present invention;

[0054] Figure 2 The types of light sources that produce strong light; (a) high beams of vehicles at night, (b) sunlight;

[0055] Figure 3 A schematic diagram illustrating how the radius of the circle affected by the strong light generated by a vehicle's high beams changes with the distance between the light source and the camera.

[0056] Figure 4 For the purposes of this invention Different values ​​correspond to Schematic diagram; (a) (b) (c) ;

[0057] Figure 5 This is the image post-processing procedure of the present invention;

[0058] Figure 6 This is a schematic diagram of the white balance output of the present invention;

[0059] Figure 7 This is a schematic diagram of the image post-processing workflow of the present invention;

[0060] Figure 8 This is a schematic diagram of the pixels after being filtered using the strong light pixel threshold according to the present invention;

[0061] Figure 9 This is a schematic diagram of a real vehicle data acquisition platform in a specific embodiment of the present invention;

[0062] Figure 10 This is a schematic diagram of a strong light environment constructed by virtual simulation software in a specific embodiment of the present invention;

[0063] Figure 11 This is a schematic diagram of the simulation results of generating a strong light environment using a large model in a specific embodiment of the present invention;

[0064] Figure 12 This is a schematic diagram comparing test results using only the geometric model and the geometric-physical model in a specific embodiment of the present invention;

[0065] Figure 13 This is a schematic diagram of camera function test results obtained in a real strong light environment in a specific embodiment of the present invention.

[0066] Figure 14 This is a schematic diagram of camera function test results obtained using pure digital simulation software in a specific embodiment of the present invention;

[0067] Figure 15 This is a schematic diagram of camera function test results obtained using the large model image generation method in a specific embodiment of the present invention;

[0068] Figure 16 This is a schematic diagram of the camera function test results obtained using the method of the present invention in a specific embodiment of the present invention. Detailed Implementation

[0069] like Figure 1 As shown, the present invention provides a method for simulating the function of an intelligent car camera in a strong light environment, comprising the following steps:

[0070] (1) The types of strong light sources to be simulated include vehicle high beams and sunlight;

[0071] (2) Construction of the geometric model for imaging in strong light environment:

[0072] (2.1) Construction of the geometric model of the strong light source location:

[0073] like Figure 2 As shown, the actual images reveal that the two light sources producing strong light are located in different positions, and the radii of the resulting circles are also different. At night, the high beams of vehicles are more frequently observed in the lower half of the image, such as... Figure 2 As shown in (a); sunlight is mostly generated in the upper half of the image, such as... Figure 2As shown in (b), both types of strong light environments can severely affect camera functionality. This invention discusses the geometric models of the positions of these two strong light sources respectively.

[0074] A geometric model of the location of a strong light source is used to determine the coordinates of the light source in the world coordinate system. plane with pixel coordinate system The correspondence is expressed as follows:

[0075]

[0076] In the formula, These are the pixel coordinates corresponding to the light source in the image coordinate system. This represents the z-axis coordinate of the light source in the camera coordinate system. Representing pixel coordinates respectively The number of pixels per unit distance in both axes, expressed in pixels per meter (pixel / m). The offset coordinates of the optical center projected onto the pixel plane in the pixel coordinate system; This is the rotation and translation transformation matrix performed when transforming from the world coordinate system to the camera coordinate system; For the light source in the world coordinate system Axis coordinate values.

[0077] (2.2) Construction of the geometric model of the region affected by strong light source:

[0078] The strong light from a vehicle's high beams affects the radius of a circle. The radius of the circle affected by strong sunlight changes with the distance D between the light source and the camera; Since the distance D between the target and the camera is relatively large, this embodiment assumes... It will not follow The changes occur and are related to the parameters of the camera being tested. These are determined separately. and The geometric model is then used to model the range of geometric pixels in an image affected by two different strong light sources.

[0079] For vehicle high beams, when the distance between the illuminated plane and the high beam source exceeds the effective distance... Subsequently, the strong glare from the vehicle's high beams on the pixels becomes negligible. The determination is based on the characteristics of different high beams. Figure 3 Demonstrated certainty The specific process, including the light source scattering angle The angle between the light rays produced by the high beam, in degrees; obtained based on the camera pinhole imaging model. The specific expression:

[0080]

[0081] In the formula, This is the camera focal length, expressed in pixels.

[0082] For sunlight, this embodiment uses the characteristics of a Gaussian beam to model it, and its specific expression is as follows:

[0083]

[0084] In the formula, It is the maximum light intensity of the beam, that is, the light intensity of the beam at the waist position; It is the radial distance between the camera and the center of the sunlight source; It is the waist radius, the value of which determines Scope of impact, such as Figure 4 As shown.

[0085] In actual camera function testing applications, the waist radius The value of is related to the wavelength of light. and camera focal length The relevant expression is:

[0086]

[0087] in, These are constants related to the camera system.

[0088] After collecting a large number of real strong light images, it was found that the distribution of the real strong light influence area is not strictly in a regular circle. Therefore, this embodiment, based on the geometric model, combined with the strong light pixel threshold conditions in the physical model, made a selection and supplement to simulate a more realistic strong light influence area than a single circle.

[0089] (3) Construction of physical model for imaging in strong light environment:

[0090] In the physical model, in order to determine the correspondence between strong light environment and pixel intensity, the camera imaging physical model is first established according to the "light-electricity-digital" signal propagation process, and then the specific value of the influence of strong light environment on pixel intensity is determined according to the physical characteristics of strong light source.

[0091] (3.1) Construction of the camera imaging physical model:

[0092] The camera imaging physics model establishes a physical propagation model between the brightness of a specific input light source and the output pixel value. First, the scene radiometry is established. (unit: ) and the image irradiance received by the camera image (unit: The relationship between )

[0093]

[0094] In the formula, The angle of incidence of the light ray; The diameter of the camera lens; digital cameras will... When light signals are converted into electrical charge signals, the number of electrons released by the photoelectric effect after receiving image irradiance for each photosensitive unit on the image sensor is... Represented as:

[0095]

[0096] In the formula, The integration time is in seconds. This indicates the spatial variation in sensitivity within the photosensitive unit; The photoelectric conversion function of a camera refers to the number of electrons released for every joule of energy absorbed at a specified wavelength.

[0097] The charge signal generated within the photosensitive unit is amplified by an amplifier circuit and then output as a digital signal via an analog-to-digital converter. The (Raw Response) is buffered in the storage unit, and the output quantity is DN, which is a digital signal. This is the camera's raw response value, expressed as:

[0098]

[0099] In the formula, Represents the gain factor of analog circuits; Represents the bias voltage; This represents the quantization step size for analog-to-digital conversion.

[0100] However, for digital signals In this case, it cannot be directly used for image processing algorithms. A series of camera post-processing steps are required to output the final RGB three-channel image information. Specific post-processing steps include linear processing, white balance transformation, color interpolation, color space conversion, brightness correction, and gamma correction, ultimately outputting a color image, such as... Figure 5 As shown.

[0101] The first step is linear processing, because different camera manufacturers handle digital signals differently. The upper and lower limits of storage have different values. When selecting the corresponding values, attention should be paid to the range of the simulated camera. In this embodiment, the lower limit is defined as... The upper limit is defined as The image obtained by the linear processing procedure Represented as:

[0102]

[0103] The second step is white balance processing. This process multiplies the RGB channels by different gain coefficients to simulate the effects of different color filters having varying spectral sensitivities. If image brightness processing is completely disregarded in this step, the multiplication coefficients for each channel are perfectly proportional, and the effects of color changes are equivalent. The gain of the G channel is set to 1, and white balance processing is performed by changing the gains of the R and B channels. Let's assume the corresponding gain settings for these two channels are... For a color filter arranged in [RGGB], the white balance processing matrix is:

[0104]

[0105] The third step is color interpolation, also known as de-mosaicing. After color interpolation, the one-dimensional image matrix output by white balance becomes a three-channel color image. In order to obtain image information, this step needs to determine the RGB three-channel position information corresponding to each pixel.

[0106] by Figure 6 Taking pixel B as an example, the color filter is arranged in [RGGB], and the position of this pixel is... The B channel value at that location is the B pixel value, the R channel value is the average of the values ​​of the four surrounding R channels, and the G channel value is obtained similarly. The expression is:

[0107]

[0108] The RGB values ​​for other locations are obtained in the same way, and the three-channel image data is obtained through this step.

[0109] The fourth step is color space conversion. The color space outputs different values ​​depending on the display device. This value is mainly determined by… The matrix determines, Depend on and It consists of two parts, and the expression is:

[0110]

[0111] In the formula, This represents the transformation matrix from the camera color space to the standard color space, used to realize the mapping relationship between the camera-captured image and the standard color space; This represents the transformation matrix from the camera color space to the intermediate reference color space. This matrix is ​​determined based on the specific imaging parameters of different cameras. This represents the transformation matrix from the standard color space to the intermediate reference color space, which can be determined based on standard data published by the International Commission on Illumination (CIE). It was found in the standards published by the International Commission on Illumination, and for This needs to be confirmed based on the specific parameters of the camera in different simulations;

[0112]

[0113] The fifth step is brightness and gamma correction. Prior to this step, all operations ensured a linear distribution of image information and scene. However, for practical applications in intelligent vehicles, a certain level of brightness information needs to be maintained. This step performs global brightness adjustment on the image after color space conversion. The specific effect is as follows: Figure 7 As shown.

[0114] (3.2) Physical characteristics of strong light sources:

[0115] To determine the scene radiance produced by a strong light source It is necessary to analyze this based on the physical characteristics of the strong light source itself. The specific expression is as follows:

[0116]

[0117] In the formula, Power of a high-intensity light source, in units of ; The area that receives the light source, ; The angle formed by the line connecting the strong light source and the center of the camera lens and the perpendicular line to the camera plane; The solid angle produced by a strong light source. The unit is .

[0118] This embodiment analyzes the physical characteristics of high beam headlights and sunlight at night to determine the high-intensity light pixel threshold, and obtains the corresponding simulated high-intensity light pixel value based on the threshold and the high-intensity light image. The physical characteristics of the two high-intensity light sources in this embodiment are shown in Table 1;

[0119] Table 1 Physical characteristics of vehicle high beams and sunlight

[0120]

[0121] The corresponding pixel thresholds are determined using the scene radiance of vehicle high beams and sunlight, respectively. The camera physical parameters used in this embodiment are shown in Table 2. Based on the camera physical parameters in Table 2, the strong light pixel thresholds for vehicle high beams and sunlight are obtained. Then, pixels in the strong light image whose RGB channels simultaneously meet the threshold conditions are selected based on these pixel thresholds, such as... Figure 8 As shown, it can be concluded that the range of influence of strong light determined solely by the geometric model cannot accurately simulate strong light; it is necessary to supplement it with pixels that meet the physical conditions.

[0122] Table 2. Simulated camera physical parameters and values

[0123]

[0124] (4) The strong light influence area output by the geometric model constructed in step 2 and the pixel values ​​corresponding to the strong light area output by the physical model in step 3 are superimposed on the non-strong light image collected by the actual vehicle to synthesize a strong light simulation image. The camera function is tested using this image.

[0125] Verification experiments and result analysis:

[0126] First, the platform used for the verification experiment is introduced. Then, the camera function is tested using traditional simulation software, large-scale model methods, the method of this invention, and real-world high-light environments. Finally, the test results in the real environment are used as the ground truth, and the proposed method is verified by comparing the test results obtained from the large-scale model, simulation software, and the method of this invention. Under the same high-light scene, the closer the test results output by the testing method are to the test results in the real environment, the more effective the method is.

[0127] The experimental platform is a real-vehicle data acquisition platform to verify the real-world environmental conditions of vehicle high beams and sunlight. Figure 9 As shown, the system includes a real high beam headlight, a vehicle target, and an image data acquisition platform. The image data acquisition platform consists of a white-box camera and an image data storage device. Using this real-vehicle platform, image information of the real high beam headlight and the vehicle target under sunlight at different distances can be acquired, and the corresponding camera function test results serve as the true values ​​for the verification experiment.

[0128] In a real-world sunlight scenario, the vehicle data acquisition platform collects data in a closed area facing the sunlight, during which the target object moves slowly forward. In a real-world high-beam scenario, the target object is stationary in front of the data acquisition platform, while the platform slowly moves towards the real high beams. Due to the risk of collision in this experiment, a balloon-like vehicle was chosen as the target object. Based on the above two strong light scenarios, images of the target object in clear weather without sunlight and without high beams were acquired, respectively, and used as inputs to the method of this invention.

[0129] Setting up a pure simulation software environment:

[0130] This invention uses virtual simulation software to construct a strong light environment for sunlight and vehicle high beams, such as... Figure 10 As shown, the camera simulation parameters are selected to be consistent with the parameters of the real environment.

[0131] Image generation of strong light effects based on the Sora large model:

[0132] Sora is OpenAI's latest large-scale text-to-video generation model, representative of the field of large-scale models, achieving a leapfrog transformation from text to video. This invention utilizes textual descriptions of strong light test scenes to obtain corresponding images; the simulation results are as follows... Figure 11 As shown.

[0133] Comparison experiment between geometric model and geometric-physical model:

[0134] To verify the necessity of the geometry-physics model, this invention compares the test results obtained using only the geometry model with the test results obtained using the geometry-physics model, such as... Figure 12 As shown. Using the geometric model alone, the effect on the image is not realistic enough, making it difficult for the test result (0.93) to be sufficiently similar to the real environment (0.79). The geometric-physical model proposed in this invention is obtained based on the geometric model. The physical model makes the simulated strong light image more closely resemble the real conditions by filtering pixel thresholds, so that the YOLO algorithm test result (0.77) is closer to the test result of the real strong light environment (0.79).

[0135] Results and analysis of camera functionality tests in high-light environments:

[0136] Test scenarios were constructed in real-world environments (Scene R), using PreScan simulation software (Scene S), the large model method (Scene M), and the method of this invention (Scene C), targeting two strong light sources: sunlight (numbered 1, 2, 3, 4) and vehicle high beams (numbered 5, 6, 7, 8). These scenarios included ultra-long-range (numbered 1, 5), long-range (numbered 2, 6), medium-range (numbered 3, 7), and short-range (numbered 4, 8) vehicles as targets. The four different actual distances were determined by the pixel distance between the bottom edge of the target's ground truth bounding box and the bottom edge of the image.

[0137] The camera function selected for the verification experiment is the YOLO series target recognition function commonly used in intelligent vehicles. The probability that the target object output by the recognition function is a vehicle is used as the test result. This invention uses the same algorithm under test in the verification experiment, and its network framework, parameter weights, etc., remain unchanged during the experiment. The test results of testing the camera function output using a real environment, simulation software, a large model, and the method of this invention are as follows: Figure 13 , Figure 14 , Figure 15 and Figure 16As shown. This invention primarily focuses on the impact of simulated strong light on target areas in images and tests the target recognition function. Although other background areas of the scene have some impact on the target detection algorithm, this impact is smaller compared to the target image area, and therefore has little effect on judging the similarity between the simulated method and the real environment.

[0138] The specific experimental results are shown in Table 3, which presents the test results for the real environment, traditional simulation software, large model method and the method of the present invention, as well as the absolute value of the error of the simulation method compared with the real environment. The present invention uses this absolute value of error to evaluate the performance of the simulation method. The smaller the absolute value of error, the better the performance of the simulation method.

[0139] Table 3. Results of Camera Function Test

[0140]

[0141] Note: "-" indicates that the algorithm cannot identify the target; the value in parentheses represents the absolute value of the error, × indicates that the result does not match the true trend, and √ indicates that the result matches the true trend.

[0142] The experimental results in Table 3 show that, in terms of accuracy, the absolute values ​​of the errors obtained by the method of this invention for the two strong light environments of daylight and high beam (0.12 and 0.03) are both smaller than those obtained by the large model method (0.42 and 0.30) and the results obtained by the traditional simulation software (0.39 and 0.45).

[0143] Regarding effectiveness, traditional simulation software exhibits test results that contradict real-world conditions in scenarios S1, S5, and S6. Specifically, when the camera cannot identify the target in the real environment (R5, R6), the traditional simulation software still outputs test results with high confidence (S5, S6). Conversely, when the camera can identify the target in the real environment (R1), the traditional simulation software fails to output a recognition result (S1). The large model method shows the same trend as the real environment (R5) in scenario M5, but exhibits test results contradicting the real environment (R6) in scenario M6. However, the large model method cannot quantitatively determine the longitudinal distance between the target vehicle and the vehicle itself, thus failing to achieve quantitative simulation. The method proposed in this invention outputs test results with the same trend as the real environment (R5, R6) in both scenarios C5 and C6, without exhibiting a trend contradicting the real environment.

Claims

1. A method for simulating and testing the functionality of an intelligent car camera in a strong light environment, characterized in that: Includes the following steps: (1) Types of strong light sources include vehicle high beams and sunlight; (2) Construction of the geometric model for imaging in strong light environment: First, construct the geometric model of the position of the strong light source, and determine the correspondence between the position of the strong light source in the world coordinate system and the pixel position in the image coordinate system according to the camera parameters; Secondly, a geometric model of the area affected by strong light source is constructed, and the influence radius of the light source is determined according to the geometric characteristics of strong light. Finally, the pixel positions affected by strong light are output as the strong light influence area, and the information of this area is used as the input of the physical model of strong light environment imaging. (3) Construction of physical model for imaging in strong light environment: First, a physical model of camera imaging is established according to the "optical-electronic-digital" signal propagation process. The steps include establishing the scene radiance and the image irradiance received by the camera. The relationship between the two is calculated by determining the number of electrons released by the photoelectric effect after the digital camera receives the image irradiance. The charge signal generated in the photosensitive unit is amplified by the amplifier circuit and output as a digital signal RR by the analog-to-digital converter. A series of camera post-processing steps are performed on the digital signal RR to output the final RGB three-channel image information. Secondly, the scene radiance generated by the strong light source is determined based on its own physical characteristics. The scene radiance of vehicle high beams and sunlight is used to determine the strong light pixel threshold that affects the pixel intensity. Based on this pixel threshold, pixels in the strong light image whose RGB channels simultaneously meet the threshold conditions are selected. By supplementing the strong light influence area determined by the geometric model with pixels that meet the physical conditions, a more realistic strong light influence area can be obtained. (4) The strong light influence area output by the geometric model constructed in step 2 and the pixel values ​​corresponding to the strong light area output by the physical model in step 3 are superimposed on the non-strong light image collected by the actual vehicle to synthesize a strong light simulation image. The camera function is tested using this strong light simulation image.

2. The method for simulating and testing the functionality of an intelligent car camera in a strong light environment according to claim 1, characterized in that: The method for constructing the geometric model of the location of the strong light source is as follows: A geometric model of the location of a strong light source is used to determine the coordinates of the light source in the world coordinate system. plane with pixel coordinate system The correspondence is expressed as follows: In the formula, These are the pixel coordinates corresponding to the light source in the image coordinate system. This represents the Z-axis coordinate of the light source in the camera coordinate system. and Representing pixel coordinates respectively The number of pixels per unit distance in both axes, expressed in pixels per meter (pixel / m). and R represents the offset coordinates of the optical center projected onto the pixel plane in the pixel coordinate system; R and T are the rotation and translation transformation matrices performed when transforming from the world coordinate system to the camera coordinate system. For the light source in the world coordinate system Axis coordinate values.

3. The method for simulating and testing the functionality of an intelligent car camera in a strong light environment according to claim 1, characterized in that: The method for constructing the geometric model of the region affected by the strong light source is as follows: Modeling the geometric pixel influence range of vehicle high beams and daylight sources in an image: Among them, the strong light generated by the vehicle's high beams affects the radius of the circle. The expression is: In the formula, is the camera focal length, in pixels; D is the distance between the light source and the camera. The effective distance between the illuminated plane produced by the light source and the strong light source. For vehicle high beams, this means the distance between the illuminated plane produced by the light source and the strong light source exceeds the effective distance. Subsequently, the strong glare from the vehicle's high beams on the pixels becomes negligible. Determined based on the characteristics of different high beams; The light source scattering angle represents the angle between the light rays produced by the high beam, expressed in degrees. The strong sunlight affects the radius of the circle. The Gaussian beam is modeled using its characteristics, and its expression is: In the formula, is the maximum light intensity of the beam, i.e., the light intensity of the beam at the waist position; r is the radial distance between the camera and the center of the sunlight source; It is the waist radius, the value of which determines scope; Waist radius The value of is related to the wavelength of light. and camera focal length The relevant expression is: Where k is a constant related to the camera system.

4. The method for simulating and testing the functionality of an intelligent car camera in a strong light environment according to claim 1, characterized in that: The method for constructing the camera imaging physical model is as follows: First, establish the scene radiance L and the image irradiance received by the camera. The relationship between them: In the formula, The angle of incidence of the light ray; The diameter of the camera lens; Digital cameras will The light signal is converted into a charge signal, and the number of electrons released by the photoelectric effect in each photosensitive unit on the image sensor after receiving the image irradiance is... Represented as: In the formula, T is the integration time in seconds; S represents the spatial fluctuation of sensitivity within the photosensitive unit. Represents the photoelectric conversion function of the camera; The wavelength of light within the camera's light-sensing range; The charge signal generated within the photosensitive unit is amplified by an amplifier circuit and output as a digital signal RR via an analog-to-digital converter. This digital signal is buffered in a storage unit, and the output quantity is DN. The expression for this digital signal RR is: In the formula, Represents the gain factor of analog circuits; Represents the bias voltage; This represents the quantization step size for analog-to-digital conversion.

5. A method for simulating the function of an intelligent car camera in a strong light environment according to claim 1 or 4, characterized in that: The digital signal RR is subjected to a series of camera post-processing steps to output the final RGB three-channel image information. The steps include: linear processing, white balance transformation, color interpolation, color space conversion, brightness correction and gamma correction, and finally output color image. Step 1: Linear processing. For the upper and lower limits of the digital signal RR storage, the lower limit is defined as... The upper limit is defined as The image obtained by the linear processing procedure Represented as: The second step is white balance processing. The RGB channels are multiplied by different gain coefficients to simulate the effects of three different color filters. The gain of the G channel is set to 1. White balance processing of the image is performed by changing the gains of the R and B channels, with the corresponding gain settings being... For a color filter arranged in [RGGB], the white balance processing matrix is: The third step is color interpolation, also known as demosaicing. After color interpolation, the one-dimensional image matrix output by white balance is a three-channel color image. In order to obtain image information, this step needs to determine the RGB three-channel position information corresponding to each pixel. Step 4: Color space conversion. The color space will output different values ​​depending on the display device. This value is mainly determined by... The matrix determines, Depend on and It consists of two parts, and the expression is: In the formula, This represents the transformation matrix from the camera color space to the standard color space, used to realize the mapping relationship between the camera-captured image and the standard color space; This represents the transformation matrix from the camera color space to the intermediate reference color space. This matrix is ​​determined based on the specific imaging parameters of different cameras. This represents the transformation matrix from the standard color space to the intermediate reference color space. It was found in the standards published by the International Commission on Illumination, and for This needs to be confirmed based on the specific parameters of the camera in different simulations; Step 5: Brightness and gamma correction, which involves global brightness adjustment of the image after color space conversion.

6. The method for simulating and testing the functionality of an intelligent car camera in a strong light environment according to claim 1, characterized in that: The scene radiance L is calculated based on the physical characteristics of the strong light source itself, and the expression is: In the formula, Power of a high-intensity light source, measured in watts (W). The area that receives the light source, ; The diameter of the camera lens; The angle formed by the line connecting the strong light source and the center of the camera lens and the perpendicular line to the camera plane; The solid angle produced by a strong light source. The unit is D is the distance between the light source and the camera.

Images