Adaptive light field adjustment and image enhancement method for driving record camera

Abstract

The application relates to a driving record camera shooting method for adaptive light field adjustment and image enhancement, and relates to the technical field of image data processing, which comprises the following steps: receiving an adaptive shooting instruction through an intelligent vehicle accessory; acquiring real-time light field information through a light field sensing component in response to the adaptive shooting instruction; generating real-time photography parameters based on the real-time light field information; collecting real-time images based on the real-time photography parameters through a vehicle-mounted driving recorder; determining real-time enhancement parameters based on the real-time light field information; and performing image enhancement on the real-time images based on the real-time enhancement parameters, so that the driving record image quality is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to a driving record camera shooting method for adaptive light field adjustment and image enhancement. BACKGROUND

[0002] Driving record is a technical means for recording the real-time image of the front (or surrounding) environment of the vehicle during driving by installing a camera device on the vehicle, and the core purpose is to provide reliable video evidence for driving safety, accident responsibility identification, and legal evidence.

[0003] When the vehicle-mounted driving recorder records the low-light driving road condition, part of the details in the obtained video image is not clear due to the dark environment brightness, and therefore, in order to ensure driving safety, image enhancement needs to be performed on the video image of the road in front of the vehicle. Histogram equalization is one of the commonly used image enhancement methods, which includes obtaining a gray level histogram according to the image, then performing enhancement on the image according to the gray level histogram and the gray level value of each pixel point in the image to obtain an enhanced image. However, the existing gray level histogram acquisition method in the image enhancement technology based on histogram equalization is as follows: the clipping amount is obtained through a gray level histogram clipping rule, and then the clipping amount is supplemented into other related gray level histogram according to the inherent supplement method. However, due to the existence of strong light regions such as passing car lights, street lamps, and street shop lights in the video image of the road in front of the vehicle, the existing gray level histogram acquisition method in the image enhancement technology based on histogram equalization and the subsequent image enhancement technology will have the following situation: in addition to enhancing the dark regions in the video image of the road in front of the vehicle, the remaining interference regions such as passing car lights, street lamps, and street shop lights are also enhanced, so that the enhanced effect cannot effectively highlight the weak light regions of the road in front of the vehicle, and the image enhancement effect is poor.

[0004] Therefore, a driving record camera shooting method for adaptive light field adjustment and image enhancement is needed to improve the image quality of the driving record. SUMMARY

[0005] The present application provides a driving record camera shooting method for adaptive light field adjustment and image enhancement, which comprises: receiving an adaptive camera shooting instruction through an intelligent vehicle-mounted accessory; acquiring real-time light field information through a light field sensing component in response to the adaptive camera shooting instruction; generating real-time photography parameters based on the real-time light field information; collecting real-time images based on the real-time photography parameters through a vehicle-mounted driving recorder; determining real-time enhancement parameters based on the real-time light field information; and performing image enhancement on the real-time images based on the real-time enhancement parameters.

[0006] Further, the light field sensing component includes a plurality of light field sensors arranged at different positions of the vehicle; and installing the light field sensing component includes: obtaining experimental data; determining a plurality of key global light features based on the experimental data; determining a plurality of key sensing positions based on the experimental data and the plurality of key global light features; and installing the plurality of light field sensors based on the plurality of key sensing positions.

[0007] Further, obtaining the experimental data includes: determining a plurality of simulation scenes and a plurality of experimental sensing positions; arranging a light field sensor at each experimental sensing position; determining a plurality of sets of experimental photography parameters; and obtaining experimental sensing data of the plurality of experimental sensing positions and experimental images of the plurality of sets of experimental photography parameters for each simulation scene, wherein the experimental data includes the experimental sensing data of the plurality of experimental sensing positions and the experimental images of the plurality of sets of experimental photography parameters corresponding to each simulation scene.

[0008] Further, determining the plurality of key global light features based on the experimental data includes: determining an optimal experimental photography parameter of each simulation scene based on the experimental images of the plurality of sets of experimental photography parameters; determining a plurality of global light features; extracting a feature value of each global light feature based on the experimental sensing data of the plurality of experimental sensing positions corresponding to each simulation scene; determining a photography parameter influence coefficient of each global light feature based on the feature value of the global light feature corresponding to each simulation scene and the optimal experimental photography parameter of each simulation scene; and determining the plurality of key global light features based on the photography parameter influence coefficient of each global light feature.

[0009] Further, determining the plurality of key sensing positions based on the experimental data and the plurality of key global light features includes: calculating a light similarity between any two experimental sensing positions based on the experimental sensing data of the two experimental sensing positions in each simulation scene; screening the plurality of experimental sensing positions based on the light similarity between any two experimental sensing positions to determine a plurality of candidate sensing positions; determining a plurality of position combinations, wherein the position combination includes at least two candidate sensing positions; calculating an optimization value of each position combination based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photography parameter of each simulation scene corresponding to each position combination; and determining the plurality of key sensing positions based on the optimization value of each position combination.

[0010] Further, the calculating the optimization value of the position combination based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photography parameter of each simulation scene comprises: calculating a correlation coefficient of each key global light feature and the optimal experimental photography parameter based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photography parameter of each simulation scene; and calculating the optimization value of the position combination based on the correlation coefficient of each key global light feature and the optimal experimental photography parameter.

[0011] Further, the real-time light field information comprises real-time sensing data collected by the light field sensor of each key sensing position; and the generating the real-time photography parameter based on the real-time light field information comprises: determining feature values of a plurality of key global light features corresponding to each simulation scene based on the experimental sensing data of each key sensing position in the simulation scene; determining real-time feature values of the plurality of key global light features based on the real-time sensing data collected by the light field sensor of each key sensing position; determining a matched simulation scene based on the feature values of the plurality of key global light features corresponding to each simulation scene and the real-time feature values of the plurality of key global light features; and generating the real-time photography parameter based on the optimal experimental photography parameter of the matched simulation scene.

[0012] Further, the determining the real-time enhancement parameter based on the real-time light field information comprises: determining a mapping relationship between the plurality of key sensing positions and the image region; and determining a regional enhancement parameter of each image region based on real-time sensing data collected by the mapped key sensing position, wherein the regional enhancement parameter of the image region at least comprises a brightness gain coefficient, a color saturation parameter and a white balance parameter, and the real-time enhancement parameter at least comprises the regional enhancement parameter of each image region.

[0013] Further, the determining the mapping relationship between the plurality of key sensing positions and the image region comprises: determining a plurality of test scenes; for each test scene, acquiring test light field information, generating test photography parameters based on the test light field information, and collecting test images based on the test photography parameters by the vehicle-mounted event data recorder; determining a plurality of image units; for each test image, segmenting the test image into unit images based on the plurality of image units, and extracting a gray scale feature of the unit image; for each image unit, calculating a correlation coefficient of each key sensing position and the image unit based on test sensing data collected by each key sensing position in each test scene and the gray scale feature of the unit image of the image unit corresponding to each test scene of the image unit, and determining a key sensing position mapped by the image unit based on the correlation coefficient of each key sensing position and the image unit; and merging the plurality of image units based on the key sensing position mapped by each image unit to determine the mapping relationship between the plurality of key sensing positions and the image region.

[0014] Further, based on the real-time sensing data collected by the mapping key sensing position, the region enhancement parameter of the image region is determined, including: for each image region, based on the test images of the plurality of test scenes, determining the test sensing data of the mapping key sensing position collected by the image region corresponding to each test scene, and obtaining the optimal region enhancement parameter of the region image corresponding to each test scene of the image region; based on the test sensing data of the mapping key sensing position collected by the image region corresponding to each test scene and the real-time sensing data of the mapping key sensing position collected by the image region, the matching test scene is determined; based on the optimal region enhancement parameter of the region image corresponding to the matching test scene of the image region, the region enhancement parameter of the image region is determined.

[0015] Compared with the prior art, the driving record camera method provided by the present specification has at least the following beneficial effects:

[0016] 1. By real-time acquisition of light field information and generation of targeted photography parameters, dynamic optimization of exposure, focusing and other settings can be achieved, reducing overexposure, underexposure or blur problems, making the collected real-time images clearer and more detailed. Based on real-time light field information to determine enhancement parameters, it can quickly adapt to different lighting conditions (such as day and night alternation, tunnel entry and exit, strong light backlight, etc.), automatically adjust brightness, color, etc., to ensure that the image remains good visibility in complex scenes, and real-time adjustment of photography and enhancement parameters can avoid image quality degradation due to environmental changes, ensuring complete and accurate recording of key information (such as license plates, road signs, accident details, etc.), providing a reliable basis for subsequent evidence or analysis;

[0017] 2. Through the layout of multi-position light field sensors and the analysis of key global light features, complex light field changes around the vehicle (such as day and night alternation, tunnel entry and exit, backlight, etc.) can be accurately captured, providing a reliable basis for real-time photography parameter generation, significantly improving image acquisition quality, and reducing overexposure, underexposure or color distortion problems. Based on simulated scene experimental data, through light similarity screening and position combination optimization, the optimal key sensing position is determined to avoid redundant arrangement, ensure comprehensive and efficient light field sensing coverage, reduce hardware cost, and improve environmental adaptability;

[0018] 3. By real-time acquisition of key sensing position data, combined with the global light features of the pre-set simulated scene, the matching simulated scene of the current environment is quickly determined, and its optimal experimental photography parameters are directly called to generate real-time parameters, ensuring the accuracy and efficiency of parameter adaptation, and significantly improving image acquisition quality;

[0019] 4. Based on the mapping relationship between the key sensing position and the image area, the brightness, color, white balance and other parameters are independently adjusted for the real-time sensing data of different areas (such as the strong light area outside the vehicle and the shadow area inside the vehicle), avoiding the loss of local details or overexposure caused by global adjustment, so that each area of the image achieves the best visual effect. Through the test scene to verify the mapping relationship and the area enhancement parameters, it is ensured that the method can still accurately identify the key light features in complex real scenes (such as rainy days, foggy days, backlight), dynamically output adaptive parameters, improve the environmental adaptability of the driving recorder, rely on a large amount of experimental data and test scene analysis, establish a quantitative correlation model of light features, sensing positions and image parameters, reduce the randomness of manual parameter adjustment, make the parameter generation and enhancement process more scientific and stable, and ensure the clear recording of key information (such as license plates and road signs). BRIEF DESCRIPTION OF DRAWINGS

[0020] The present specification will be further illustrated in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0021] Figure 1 is a flowchart of the adaptive light field adjustment and image enhancement driving recording camera method shown in an embodiment of the present application;

[0022] Figure 2 is a flowchart of determining a plurality of key sensing positions in an embodiment of the present application;

[0023] Figure 3 is a flowchart of determining the mapping relationship between the plurality of key sensing positions and the image area in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description.

[0025] Figure 1 is a flowchart of the adaptive light field adjustment and image enhancement driving recording camera method shown in an embodiment of the present application, as shown in Figure 1 The adaptive light field adjustment and image enhancement driving recording camera method can include the following steps.

[0026] Step 110, receiving an adaptive camera instruction through an intelligent vehicle accessory.

[0027] The smart car accessory integrates voice recognition (supports natural language instructions such as "turn on adaptive camera"), gesture control (recognizes hand waving, fist clenching, etc. through the built-in IMU sensor), and tactile feedback (confirms instruction reception through a vibration motor), allowing users to trigger adaptive camera instructions through voice, gestures, or physical contact.

[0028] The smart car accessory uses Bluetooth technology to establish a high-speed, low-latency (<10ms) bidirectional communication link with the car camera and electronic control unit, ensuring real-time transmission of instructions.

[0029] In step 120, in response to the adaptive camera instruction, real-time light field information is obtained through the light field sensing component.

[0030] The light field sensing component includes multiple light field sensors arranged at different positions of the vehicle. The light field sensor is a core sensor component for capturing light field information by recording the intensity, direction, color, phase, and other attributes of light in space. The light field sensor can include light intensity sensors, color temperature sensors, and spectral sensors, etc.

[0031] In some embodiments, the light field sensing component is installed, including:

[0032] Obtain experimental data;

[0033] Based on the experimental data, determine a plurality of key global light features;

[0034] Based on the experimental data and the plurality of key global light features, determine a plurality of key sensing positions;

[0035] Based on the plurality of key sensing positions, install a plurality of light field sensors.

[0036] In some embodiments, obtaining experimental data includes:

[0037] Determine a plurality of simulated scenes and a plurality of experimental sensing positions;

[0038] Set up a light field sensor at each experimental sensing position;

[0039] Determine a plurality of sets of experimental photography parameters;

[0040] For each simulated scene, obtain experimental sensing data of the plurality of experimental sensing positions and experimental images of the plurality of sets of experimental photography parameters, wherein the experimental data includes experimental sensing data of the plurality of experimental sensing positions corresponding to each simulated scene and experimental images of the plurality of sets of experimental photography parameters.

[0041] Specifically, the plurality of simulation scenarios can simulate the light field conditions of real driving environments. For example, noon strong light (100,000 lux), night without street lamps (<10 lux), tunnel entry and exit (light intensity mutation 1000 times), heavy rain (visibility 50 meters), thick fog (visibility 10 meters), snowy day (ground reflectivity 80%) and the like.

[0042] The target scenario (such as "night heavy rain, visibility 20 meters") is reproduced by an environmental simulation device (such as an LED light box, a rain and fog generator). The scene parameters (such as the actual light intensity = 8 lux, with an error <5% from the set value) are verified using high-precision sensors (such as a laser range finder).

[0043] The plurality of experimental sensing positions can be different positions of the vehicle, for example, the inside of the front windshield, the base of the rearview mirror, the shark fin / luggage rack on the roof, the tail / rear windshield and the like.

[0044] At least one of the photography parameters (such as frame rate, exposure time, ISO, aperture and the like) in any two groups of experimental photography parameters is different.

[0045] In some embodiments, based on the experimental data, a plurality of key global light features are determined, including:

[0046] For each simulation scenario, based on the experimental images of the plurality of groups of experimental photography parameters, the optimal experimental photography parameters of the simulation scenario are determined;

[0047] A plurality of global light features are determined;

[0048] For each simulation scenario, based on the experimental sensing data of the plurality of experimental sensing positions corresponding to the simulation scenario, the feature value of each global light feature is extracted;

[0049] For each global light feature, based on the feature value of the global light feature corresponding to each simulation scenario and the optimal experimental photography parameters, the photography parameter influence coefficient of the global light feature is determined;

[0050] Based on the photography parameter influence coefficient of each global light feature, a plurality of key global light features are determined.

[0051] Specifically, a plurality of image evaluation indicators (such as definition, color restoration, peak signal-to-noise ratio and the like) can be set, the scores of the experimental images of the experimental photography parameters in the plurality of image evaluation indicators are determined, and the experimental photography parameters with the highest comprehensive score are selected as the optimal experimental photography parameters of the simulation scenario.

[0052] The global light feature is used to reflect the comprehensive situation of light at a plurality of positions. For example, the plurality of global light features can include average light intensity, light intensity standard deviation, maximum light intensity difference, average color temperature, color temperature standard deviation, maximum color temperature difference and the like.

[0053] For any one global light feature and any one photographic parameter, the global light feature is substituted into the correlation coefficient (e.g., Pearson correlation coefficient, etc.) calculation formula in the feature value of each simulation scene and the value of the photographic parameter in the optimal experimental photographic parameter corresponding to each simulation scene, to calculate the correlation coefficient of the global light feature and the photographic parameter, and take the absolute value of the correlation coefficient as the influence coefficient of the global light feature on the photographic parameter.

[0054] For any one global light feature, the influence coefficients of the global light feature on each photographic parameter are weighted and summed to obtain the photographic parameter influence coefficient of the global light feature.

[0055] The global light feature with a photographic parameter influence coefficient greater than a photographic parameter influence coefficient threshold (e.g., 0.6) is taken as a key global light feature.

[0056] The above process, through the photographic parameter influence coefficient threshold screening, eliminates light features with weak influence on the parameter, and only retains key features with strong guidance on parameter adjustment. This reduces the feature dimension in subsequent key sensing position determination and photographic parameter determination, and improves the calculation efficiency.

[0057] Figure 2 is a flowchart for determining a plurality of key sensing positions in an embodiment of the present application, as shown in Figure 2 In some embodiments, based on experimental data and a plurality of key global light features, a plurality of key sensing positions are determined, including:

[0058] For any two experimental sensing positions, based on the experimental sensing data of the two experimental sensing positions in each simulation scene, the light similarity of the two experimental sensing positions is calculated;

[0059] Based on the light similarity of any two experimental sensing positions, a plurality of experimental sensing positions are screened to determine a plurality of candidate sensing positions;

[0060] A plurality of position combinations are determined, wherein a position combination includes at least two candidate sensing positions;

[0061] For each position combination, based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photographic parameter of each simulation scene, the optimization value of the position combination is calculated;

[0062] Based on the optimization value of each position combination, a plurality of key sensing positions are determined.

[0063] Specifically, the cosine similarity of the experimental sensing data of the two experimental sensing positions in each simulated scene is calculated as the light similarity of the two experimental sensing positions, ensuring that the candidate sensing positions have different responses to the light features.

[0064] For the two experimental sensing positions with a light similarity greater than a light similarity threshold (e.g., 0.7), one of the experimental sensing positions is retained as a candidate sensing position.

[0065] The plurality of position combinations can be determined by an exhaustive method.

[0066] In some embodiments, the optimization value of the position combination is calculated based on the experimental sensing data of each candidate sensing position included in the position combination in each simulated scene and the optimal experimental photography parameter of each simulated scene, including:

[0067] The correlation coefficient of each key global light feature and the optimal experimental photography parameter is calculated based on the experimental sensing data of each candidate sensing position included in the position combination in each simulated scene and the optimal experimental photography parameter of each simulated scene.

[0068] The optimization value of the position combination is calculated based on the correlation coefficient of each key global light feature and the optimal experimental photography parameter.

[0069] Specifically, for each position combination and each simulated scene, the feature value of each key global light feature of the position combination corresponding to the simulated scene is calculated based on the experimental sensing data of each candidate sensing position included in the position combination in each simulated scene. For example, if the key global light feature is the average light intensity, the mean value of the light intensity of each candidate sensing position included in the position combination is calculated as the feature value of the average light intensity of the position combination corresponding to the simulated scene.

[0070] For any kind of key global light feature and any kind of photography parameter, the feature value of the key global light feature of the position combination corresponding to each simulated scene and the value of the photography parameter in the optimal experimental photography parameter corresponding to each simulated scene are substituted into the correlation coefficient (e.g., Pearson correlation coefficient, etc.) calculation formula to calculate the correlation coefficient of the key global light feature and the photography parameter corresponding to the position combination, and the absolute value of the correlation coefficient is taken as the influence coefficient of the key global light feature corresponding to the position combination on the photography parameter.

[0071] For any kind of key global light feature, the influence coefficients of the key global light feature corresponding to the position combination on each kind of photography parameter are averaged to calculate the parameter optimization value of the key global light feature corresponding to the position combination.

[0072] The parameter optimization values of each kind of key global light feature corresponding to the position combination are averaged to obtain the optimization value of the position combination.

[0073] For example, the optimization value of the position combination can be calculated according to the following formula:

[0074]

[0075] wherein, is the optimization value of the ith position combination, is the influence coefficient of the nth key global light feature corresponding to the ith position combination on the mth photographic parameter, is the total number of photographic parameters, is the total number of key global light features.

[0076] The above formula converts the optimization contribution of the position combination to multiple parameters and multiple features into a single comparable numerical value through double summation and normalization, avoids subjective evaluation or single index deviation, and improves the objectivity of the screening decision. The complex correlation among the position combination, the light feature, and the photographic parameter is simplified into a calculable average correlation coefficient, reducing the dimension of the optimization problem and improving the calculation efficiency. If the absolute correlation coefficient sum of a certain position combination is generally large, it indicates that it can more accurately guide the parameter adjustment through the light feature (for example, in a high dynamic scene, the strong correlation between the standard deviation of the light intensity and the exposure compensation can avoid overexposure / underexposure). Therefore, the larger the absolute correlation coefficient sum, the higher the optimization value, indicating that the contribution of the position combination to the parameter optimization is more significant.

[0077] The position combination with the largest optimization value is selected as the optimal position combination, and the candidate sensing position included in the optimal position combination is selected as the key sensing position.

[0078] An optical field sensor is installed at each key sensing position.

[0079] The real-time optical field information includes real-time sensing data collected by the optical field sensor at each key sensing position.

[0080] Step 120 screens the key position based on the optimization value, directly associates the effective guidance of parameter adjustment, avoids the waste of computing resources caused by redundant positions, and reduces the sensor deployment density and data acquisition amount. The position combination with a high optimization value can more accurately map the relationship between the light feature and the optimal parameter (for example, in a high dynamic scene, the strong correlation between the light intensity standard deviation of the key position and the exposure compensation), improving the accuracy of parameter generation. By screening the candidate positions covering diversified light patterns, it is ensured that the key position combination can provide effective feature values in various simulation scenes, supporting cross-scene parameter optimization.

[0081] Step 130, based on the real-time optical field information, generates real-time photographic parameters.

[0082] In some embodiments, step 130 specifically includes:​​

[0083] For each simulation scenario, based on the experimental sensing data of each key sensing position in the simulation scenario, determine the feature values of the plurality of key global light features corresponding to the simulation scenario;

[0084] Based on the real-time sensing data collected by the light field sensor of each key sensing position, determine the real-time feature values of the plurality of key global light features;

[0085] Based on the feature values of the plurality of key global light features corresponding to each simulation scenario and the real-time feature values of the plurality of key global light features, determine the matched simulation scenario;

[0086] Based on the optimal experimental photography parameters of the matched simulation scenario, generate real-time photography parameters.

[0087] Specifically, for each simulation scenario, calculate the cosine similarity of the feature values of the plurality of key global light features corresponding to the simulation scenario and the real-time feature values of the plurality of key global light features.

[0088] For example only, the simulation scenario with the largest cosine similarity is taken as the matched simulation scenario. The optimal experimental photography parameters of the matched simulation scenario can be taken as the real-time photography parameters.

[0089] For another example, the simulation scenario with a cosine similarity greater than a cosine similarity threshold (for example, 0.6) is taken as the matched simulation scenario, and the optimal experimental photography parameters of the matched simulation scenario are averaged to obtain the real-time photography parameters.

[0090] Step 140, collecting real-time images based on the real-time photography parameters by the vehicle-mounted driving recorder.

[0091] Step 150, determining real-time enhancement parameters based on real-time light field information.

[0092] In some embodiments, step 150 specifically includes:

[0093] determining a mapping relationship between the plurality of key sensing positions and the image regions;

[0094] For each image region, based on the real-time sensing data collected by the mapped key sensing position, determine the regional enhancement parameters of the image region, wherein the regional enhancement parameters of the image region at least include a brightness gain coefficient, a color saturation parameter and a white balance parameter, and the real-time enhancement parameters at least include the regional enhancement parameters of each image region.

[0095] Figure 3 is a flowchart for determining the mapping relationship between the plurality of key sensing positions and the image regions in an embodiment of the present application, as shown in Figure 3As shown, in some embodiments, determining the mapping relationship between the plurality of key sensing positions and the image region comprises:

[0096] Determining a plurality of test scenes, wherein the plurality of test scenes are selected from a scene set covering typical driving environments (such as midday backlight, night street lamp, tunnel entrance and exit, cloudy weather, etc.), and the test scenes cover the extremity and diversity of light distribution (such as high dynamic range, low light, mixed light source), to ensure that the mapping relationship has generalization ability.

[0097] For each test scene, test light field information is obtained, and based on the test light field information, test photography parameters are generated, and based on the test photography parameters, test images are collected by the vehicle-mounted event data recorder, wherein the determination manner of the test photography parameters is similar to that of the real-time photography parameters, which will not be described herein.

[0098] Determining a plurality of image units, specifically, dividing the image into a plurality of fixed-size image units (such as 32x32 pixel blocks);

[0099] For each test image, the test image is segmented into unit images based on the plurality of image units, and the gray scale features of the unit images are extracted, specifically, each test image is divided into a plurality of fixed-size image units (such as 32x32 pixel blocks), and the gray scale features (such as mean, standard deviation, entropy) of each unit are extracted.

[0100] For each image unit, based on the test sensing data collected by each key sensing position in each test scene and the gray scale features of the unit images corresponding to each test scene of the image unit, the correlation coefficient of each key sensing position and the image unit is calculated, and based on the correlation coefficient of each key sensing position and the image unit, the key sensing position mapped by the image unit is determined.

[0101] Based on the key sensing position mapped by each image unit, the plurality of image units are merged to determine the mapping relationship between the plurality of key sensing positions and the image region.

[0102] Specifically, for each key sensing position, based on the test sensing data collected by the key sensing position in the test scene, the light feature (such as light intensity, color temperature, etc.) of the key sensing position is determined. The light feature of the key sensing position in each test scene and the gray scale feature of the unit image corresponding to each test scene of the image unit can be substituted into the correlation coefficient (such as Pearson correlation coefficient, Spearman rank correlation coefficient, etc.) calculation formula to calculate the correlation coefficient of the key sensing position and the image unit. The key sensing position with a correlation coefficient greater than a correlation coefficient threshold (such as 0.6) can be taken as the key sensing position mapped by the image unit.

[0103] If the coincidence degree of the key sensing positions mapped by any two image units is greater than a coincidence degree threshold (e.g., 80%), the two image units are merged. After all the image unit merging is completed, a plurality of image regions are determined, wherein each image region includes at least one image unit.

[0104] The key sensing positions mapped by the two image units are de-coincided and merged as the key sensing positions of the merged region.

[0105] In some embodiments, based on the real-time sensing data collected by the mapped key sensing positions, a region enhancement parameter of the image region is determined, including:

[0106] For each image region, based on test images of a plurality of test scenes, test sensing data of the image region corresponding to each test scene collected by the mapped key sensing positions is determined, and an optimal region enhancement parameter of a region image of the image region corresponding to each test scene is obtained;

[0107] Based on the test sensing data of the image region corresponding to each test scene collected by the mapped key sensing positions and the real-time sensing data collected by the key sensing positions mapped by the image region, a matched test scene is determined.

[0108] Based on the optimal region enhancement parameter of the region image of the image region corresponding to the matched test scene, a region enhancement parameter of the image region is determined.

[0109] Specifically, for each image region, a plurality of sets of region enhancement parameters are determined, based on each set of region enhancement parameters, the region image of the image region corresponding to the test scene is image-enhanced to generate an enhanced region image, the scores of the enhanced region image in a plurality of image quality evaluation indicators (such as peak signal-to-noise ratio, structural similarity, and local contrast) are determined, the difference between the scores of the region image before enhancement in the plurality of image quality evaluation indicators and the scores of the enhanced region image in the plurality of image quality evaluation indicators is calculated, and the enhancement effect score corresponding to the region enhancement parameter is obtained. The region enhancement parameter with the largest enhancement effect score is taken as the optimal region enhancement parameter of the region image of the image region corresponding to the test scene.

[0110] The cosine similarity of the test sensing data of the image region corresponding to each test scene collected by the mapped key sensing positions and the real-time sensing data collected by the key sensing positions mapped by the image region is calculated.

[0111] Only as an example, the simulation scene with the largest cosine similarity is taken as the matched test scene. The optimal region enhancement parameter of the matched test scene can be taken as the region enhancement parameter of the image region.

[0112] Further, as an example, a test scene with a cosine similarity greater than a cosine similarity threshold (e.g., 0.6) is regarded as a matched test scene, and an average of optimal region enhancement parameters of the matched test scenes is taken as the region enhancement parameter of the image region.

[0113] Step 150 ensures that real-time sensing data directly corresponds to the region in the image actually affected by it, avoiding local overexposure / underexposure or color deviation caused by global parameter adjustment, by establishing a mapping relationship between key sensing positions and image regions. According to dynamic adjustment of light intensity data of the mapping region, it ensures that dark details are visible (such as night license plate recognition rate increased by more than 20%), while avoiding overexposure of highlight areas leading to texture loss. Combined with the color reference after white balance correction, the color performance of natural scenes (such as green plants and the sky) is enhanced, and the image clarity perceived by the human eye is improved by 15%-30%. Through real-time correction of the color temperature data of the key sensing position, the color deviation of the light source (such as the blue deviation of indoor LED lights and the warm deviation during the twilight period) is eliminated, ensuring that the color restoration error of white objects (such as lane lines and traffic signs) is ΔE<5 (within the acceptable range of the human eye). By calculating the correlation coefficient of the test sensing data of the key sensing position and the image unit gray scale feature, the causal relationship between the sensor data and the image region is accurately identified.

[0114] Step 160 performs image enhancement on the real-time image based on the real-time enhancement parameters.

[0115] Specifically, the brightness gain coefficient is a multiplicative factor for linear scaling of the pixel value of the image region, used to adjust the overall brightness. The color saturation parameter is an adjustment factor for controlling the purity of image colors, used to enhance or suppress the degree of color brightness. The white balance parameter is a parameter group for independent gain adjustment of RGB channels, used to correct color temperature deviation.

[0116] First, based on the white balance parameter, ensure that the color reference is correct, avoid subsequent brightness / saturation adjustment to amplify color deviation, then adjust the brightness of the image region according to the brightness gain coefficient, and finally correct the color based on the color saturation parameter.

[0117] The images of all enhanced image regions are spliced to generate an enhanced real-time image.

[0118] Finally, it should be understood that the embodiments described in the specification are only used to illustrate the principles of the embodiments of the specification. Other variations can also be within the scope of the specification. Therefore, as an example rather than limitation, alternative configurations of the embodiments of the specification can be considered consistent with the teachings of the specification. Accordingly, the embodiments of the specification are not limited to the embodiments explicitly introduced and described in the specification.

Claims

1. A method for recording a driving scene with adaptive light field adjustment and image enhancement, characterized in that, The method comprises the following steps: receiving adaptive shooting instructions through an intelligent vehicle accessory; obtaining real-time light field information through a light field sensing component in response to the adaptive shooting instructions; generating real-time shooting parameters based on the real-time light field information; collecting real-time images based on the real-time shooting parameters through a vehicle-mounted dashcam; determining real-time enhancement parameters based on the real-time light field information; performing image enhancement on the real-time images based on the real-time enhancement parameters; wherein the light field sensing component comprises a plurality of light field sensors arranged at different positions of the vehicle; installing the light field sensing component comprises the following steps: obtaining experimental data; determining a plurality of key global light features based on the experimental data, wherein the global light features are used to reflect the comprehensive situation of light at multiple positions, and the plurality of global light features include average illumination intensity, illumination intensity standard deviation, maximum illumination intensity difference, average color temperature, color temperature standard deviation, and maximum color temperature difference; determining a plurality of key sensing positions based on the experimental data and the plurality of key global light features; installing a plurality of light field sensors based on the plurality of key sensing positions; obtaining experimental data comprises the following steps: determining a plurality of simulated scenes and a plurality of experimental sensing positions; arranging light field sensors at each experimental sensing position; determining a plurality of sets of experimental shooting parameters; for each simulated scene, obtaining experimental sensing data of the plurality of experimental sensing positions and experimental images of the plurality of sets of experimental shooting parameters, wherein the experimental data includes experimental sensing data of the plurality of experimental sensing positions corresponding to each simulated scene and experimental images of the plurality of sets of experimental shooting parameters. 2.The adaptive light field adjustment and image enhancement method for a driving recording camera according to claim 1, wherein, determining a plurality of key global light features based on the experimental data comprises the following steps: for each simulated scene, determining optimal experimental shooting parameters of the simulated scene based on experimental images of the plurality of sets of experimental shooting parameters; determining a plurality of global light features; for each simulated scene, extracting a feature value of each global light feature based on experimental sensing data of the plurality of experimental sensing positions corresponding to the simulated scene; for each global light feature, determining a shooting parameter influence coefficient of the global light feature based on the feature value of the global light feature corresponding to each simulated scene and the optimal experimental shooting parameters of each simulated scene; determining a plurality of key global light features based on the shooting parameter influence coefficients of each global light feature. 3.The adaptive light field adjustment and image enhancement method for a driving recording camera according to claim 2, wherein, determining a plurality of key sensing positions based on the experimental data and the plurality of key global light features comprises the following steps: for any two experimental sensing positions, calculating a light similarity of the two experimental sensing positions based on experimental sensing data of the two experimental sensing positions in each simulated scene; based on the light similarity of any two experimental sensing positions, screening the plurality of experimental sensing positions to determine a plurality of candidate sensing positions; determining a plurality of position combinations, wherein the position combination includes at least two candidate sensing positions; for each position combination, calculating an optimization value of the position combination based on experimental sensing data of each candidate sensing position included in the position combination in each simulated scene and optimal experimental shooting parameters of each simulated scene; determining a plurality of key sensing positions based on the optimization value of each position combination.

4. The method of claim 3, wherein, calculating an optimization value of the position combination based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photography parameter of each simulation scene, comprising: calculating a correlation coefficient of each key global light feature and the optimal experimental photography parameter based on the experimental sensing data of each candidate sensing position included in the position combination in each simulation scene and the optimal experimental photography parameter of each simulation scene; calculating an optimization value of the position combination based on the correlation coefficient of each key global light feature and the optimal experimental photography parameter.

5. The self-adaptive light field adjustment and image enhancement method for driving record camera according to claim 4, characterized in that, The real-time light field information includes real-time sensing data collected by the light field sensor of each key sensing position; The real-time photography parameter is generated based on the real-time light field information, comprising: For each simulation scene, determining the feature values of the multiple key global light features corresponding to the simulation scene based on the experimental sensing data of each key sensing position in the simulation scene; determining the real-time feature values of the multiple key global light features based on the real-time sensing data collected by the light field sensor of each key sensing position; determining the matching simulation scene based on the feature values of the multiple key global light features corresponding to each simulation scene and the real-time feature values of the multiple key global light features; generating the real-time photography parameter based on the optimal experimental photography parameter of the matching simulation scene. 6.The adaptive light field adjustment and image enhancement method for a driving recording camera according to claim 4, wherein, The real-time enhancement parameter is determined based on the real-time light field information, comprising: determining the mapping relationship between the multiple key sensing positions and the image region; For each image region, determining the regional enhancement parameter of the image region based on the real-time sensing data collected by the mapped key sensing position, wherein the regional enhancement parameter of the image region at least includes a brightness gain coefficient, a color saturation parameter and a white balance parameter, and the real-time enhancement parameter at least includes the regional enhancement parameter of each image region.

7. The self-adapting light field adjustment and image enhancement method for a driving recording camera according to claim 6, wherein, determining the mapping relationship between the multiple key sensing positions and the image region, comprising: determining multiple test scenes; For each test scene, acquiring test light field information, generating test photography parameters based on the test light field information, and collecting test images based on the test photography parameters through the vehicle-mounted event data recorder; determining multiple image units; For each test image, segmenting the test image into unit images based on the multiple image units, and extracting the gray scale features of the unit images; For each image unit, calculating the correlation coefficient of each key sensing position and the image unit based on the test sensing data collected by each key sensing position in each test scene and the gray scale features of the unit images of each test scene corresponding to the image unit, and determining the key sensing position mapped by the image unit based on the correlation coefficient of each key sensing position and the image unit; merging the multiple image units based on the key sensing position mapped by each image unit to determine the mapping relationship between the multiple key sensing positions and the image region.

8. The self-adaptive light field adjustment and image enhancement method for driving record camera according to claim 7, characterized in that, determining the regional enhancement parameter of the image region based on the real-time sensing data collected by the mapped key sensing position, comprising: For each image region, based on the test images of the multiple test scenes, determine the test sensing data of the mapping of the key sensing positions of the image region corresponding to each test scene, obtain the optimal region enhancement parameter of the region image of the image region corresponding to each test scene; Based on the test sensing data of the mapping of the key sensing positions of the image region corresponding to each test scene and the real-time sensing data of the mapping of the key sensing positions of the image region, determine the matched test scene; Based on the optimal region enhancement parameter of the region image of the image region corresponding to the matched test scene, determine the region enhancement parameter of the image region.

Images