An image shooting parameter adaptive adjustment method, system and vehicle

Abstract

The application discloses an adaptive adjustment method and system of image shooting parameters and a vehicle. The method performs image recognition on images in the automatic driving process of the vehicle by constructing a target image recognition model, determines a scene state in which the vehicle is currently located, and selects the most adaptive target image shooting parameter according to the current scene state, so as to realize the corresponding switching of the target image shooting parameter according to the real-time scene state of the vehicle. Since the target image shooting parameter matched based on the real-time scene state can effectively guarantee the imaging effect of an ISP, the problem that the imaging effect of the ISP is poor due to the sudden change of an external scene can be effectively overcome.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an adaptive adjustment method, system, and vehicle for image capture parameters. Background Technology

[0002] New energy vehicles equipped with autonomous driving technology are gaining increasing attention and development. In autonomous driving technology, considering the reliability and ease of implementation, vision-based solutions are increasingly used to achieve autonomous driving control. This involves capturing image data through cameras, converting it into detectable images via an ISP, using deep learning to identify targets such as vehicles and pedestrians, and then making judgments and decision-making commands for driving behavior (such as braking, deceleration, following the vehicle in front, steering, etc.) to control the vehicle to perform corresponding actions.

[0003] Currently, in vision-based autonomous driving technology, the imaging quality of the Information Signal Processor (ISP) directly impacts the performance of autonomous driving. Clearer imaging allows for greater distance, higher precision, and greater accuracy in recognizing vehicles and pedestrians, leading to more precise judgments and decisions regarding driving behavior, thus making autonomous driving safer and more intelligent. However, outdoor driving conditions vary greatly (e.g., highways, main roads, mixed pedestrian and vehicle roads) and driving scenarios are diverse (e.g., daytime, nighttime, sunny days, rainy days, dusty / foggy days), making it difficult to guarantee the imaging quality of the ISP.

[0004] Therefore, existing autonomous vehicles suffer from poor image quality in the ISP due to sudden changes in the external scene during real-time image acquisition. Summary of the Invention

[0005] In view of this, it is necessary to provide an adaptive adjustment method, system and vehicle for image acquisition parameters to solve the problem of poor image quality of ISP caused by sudden changes in external scene during real-time image acquisition in existing autonomous vehicles.

[0006] To address the above problems, this invention provides an adaptive adjustment method for image capturing parameters, comprising:

[0007] Acquire image samples, as well as target image capture parameter samples;

[0008] An initial image recognition model is established by inputting image samples into the initial image recognition model and using the corresponding target image capture parameter samples as sample labels to train the initial image recognition model, thereby obtaining a fully trained target image recognition model.

[0009] Acquire real-time images, input the real-time images into the target image recognition model to obtain the corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters;

[0010] The initial image recognition model includes at least one of the following: initial camera preprocessing parameter recognition sub-model, initial image brightness recognition sub-model, initial image color recognition sub-model, initial image noise recognition sub-model, and initial image distortion recognition sub-model.

[0011] Furthermore, image samples and target image capture parameter samples are obtained, including:

[0012] Obtain initial image samples and their corresponding initial image capture parameters;

[0013] Based on the initial image samples, the initial image shooting parameters are adjusted and the images are reshot to obtain multiple transition image samples and their corresponding transition image shooting parameters.

[0014] Select the optimal transition image sample from multiple transition image samples, and determine the optimal transition image shooting parameters corresponding to the optimal transition image sample as the target image shooting parameter sample;

[0015] The initial image samples include multiple image samples taken in the same scene under different external environmental conditions.

[0016] Furthermore, an initial image recognition model is established. Image samples are input into the initial image recognition model, and the corresponding target image capture parameter samples are used as sample labels to train the initial image recognition model, resulting in a fully trained target image recognition model, including:

[0017] Establish an initial image recognition model;

[0018] The image samples are divided into blocks to obtain multiple key feature blocks;

[0019] Multiple key feature blocks are input into the initial image recognition model to obtain target image capture parameter samples corresponding to the image samples. The corresponding target image capture parameter samples are used as sample labels to continuously iterate and train the initial image recognition model to obtain a fully trained target image recognition model.

[0020] Furthermore, multiple key feature blocks are input into the initial image recognition model to obtain target image capture parameter samples corresponding to the image samples, including:

[0021] The initial camera preprocessing parameter recognition sub-model is used to identify defect samples of key feature blocks and determine the corrected defect shooting parameter samples of key feature blocks based on the defect samples.

[0022] The initial image brightness recognition sub-model is used to identify brightness samples of key feature blocks and determine the corrected brightness shooting parameter samples of key feature blocks based on the brightness samples.

[0023] The initial image color recognition sub-model is used to identify the color samples of key feature blocks and determine the corrected color shooting parameter samples of key feature blocks based on the color samples.

[0024] The initial image noise recognition sub-model is used to identify noise samples of key feature blocks and determine the corrected noise shooting parameter samples of key feature blocks based on the noise samples.

[0025] The initial image distortion recognition sub-model is used to identify distortion samples of key feature blocks and determine the correction distortion shooting parameter samples of key feature blocks based on the distortion samples.

[0026] The target image capture parameter sample includes at least one of the following: defect correction capture parameter sample, brightness correction capture parameter sample, color correction capture parameter sample, noise correction capture parameter sample, and distortion correction capture parameter sample.

[0027] Furthermore, real-time images are acquired and input into the target image recognition model to obtain the corresponding target image capture parameters. The image capture parameters are then adjusted based on these target image capture parameters, including:

[0028] Acquire real-time images;

[0029] Using AI technology to analyze real-time images and determine if there are any sudden changes in the scene;

[0030] When a sudden change in scene is detected, the real-time image is input into the target image recognition model to obtain the corresponding target image shooting parameters, and the image shooting parameters are adjusted based on the target image shooting parameters.

[0031] Furthermore, adjusting the image capture parameters based on the target image capture parameters also includes:

[0032] An updated image is obtained based on the target image capture parameters;

[0033] AI technology is used to determine whether the updated image achieves the target effect;

[0034] When it is determined that the updated image does not achieve the target effect, the updated image is input into the target image recognition model to obtain the updated target image shooting parameters until an image that achieves the target effect is obtained.

[0035] Furthermore, the initial camera preprocessing parameter recognition sub-model, and / or the initial image brightness recognition sub-model, and / or the initial image color recognition sub-model, and / or the initial image noise recognition sub-model, and / or the initial image distortion recognition sub-model run simultaneously.

[0036] Furthermore, image capture parameters include ISP parameters and camera configuration parameters.

[0037] To address the above problems, the present invention also provides an adaptive adjustment system for image capturing parameters, comprising:

[0038] The sample acquisition module is used to acquire image samples and target image capture parameter samples;

[0039] The target image recognition model acquisition module is used to establish an initial image recognition model. It inputs image samples into the initial image recognition model and uses the corresponding target image capture parameter samples as sample labels to train the initial image recognition model, thereby obtaining a fully trained target image recognition model.

[0040] The image capture parameter adjustment module is used to acquire real-time images, input the real-time images into the target image recognition model to obtain the corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters;

[0041] The initial image recognition model includes at least one of the following: initial camera preprocessing parameter recognition sub-model, initial image brightness recognition sub-model, initial image color recognition sub-model, initial image noise recognition sub-model, and initial image distortion recognition sub-model.

[0042] To address the aforementioned problems, the present invention also provides a vehicle, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the adaptive adjustment method for image capture parameters as described above.

[0043] The beneficial effects of adopting the above technical solution are as follows: The present invention provides an adaptive adjustment method, system and vehicle for image shooting parameters. The method constructs a target image recognition model to perform image recognition on images during the autonomous driving process of the vehicle, determines the current scene state of the vehicle, and selects the most suitable target image shooting parameters according to the current scene state. This realizes the switching of target image shooting parameters according to the real-time scene state of the vehicle. Since the target image shooting parameters matched based on the real-time scene state can effectively guarantee the imaging effect of the ISP, it can effectively overcome the problem of poor imaging effect of the ISP caused by sudden changes in the external scene. Attached Figure Description

[0044] Figure 1A flowchart illustrating an embodiment of the adaptive adjustment method for image capture parameters provided by the present invention;

[0045] Figure 2 This is a schematic flowchart of an embodiment of obtaining a sample provided by the present invention;

[0046] Figure 3 This is a flowchart illustrating an embodiment of the target image recognition model obtained by the present invention.

[0047] Figure 4 This is a schematic diagram showing the result of an embodiment of image sample segmentation provided by the present invention;

[0048] Figure 5 A flowchart illustrating an embodiment of the present invention for obtaining corresponding target image capture parameters;

[0049] Figure 6 This is a schematic flowchart illustrating an embodiment of the present invention for obtaining an image that achieves the target effect;

[0050] Figure 7 A schematic diagram of an embodiment of the adaptive adjustment system for image capture parameters provided by the present invention;

[0051] Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0052] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0053] Before describing the embodiments, let's first explain the ISP:

[0054] ISP (Image Signal Processing) is a unit mainly used to process the output signals of front-end image sensors to match image sensors from different manufacturers.

[0055] New energy vehicles equipped with autonomous driving technology are gaining increasing attention and development. Currently, my country's autonomous driving technology has gradually transitioned from the L1 or L2 intelligent assisted driving stage to the L3 or L4 advanced autonomous driving stage. In autonomous driving technology, considering reliability and ease of implementation, vision-based solutions are increasingly used for autonomous driving control. This involves capturing image data through cameras, converting it into detectable images via an ISP, using deep learning to identify targets such as vehicles and pedestrians, and then making driving behavior judgments and decision-making commands (such as braking, deceleration, following the vehicle in front, and steering) to control the vehicle to perform corresponding actions.

[0056] Currently, in vision-based autonomous driving technology, the imaging quality of the Information Signal Processor (ISP) directly impacts the performance of autonomous driving. Clearer imaging allows for greater distance, higher precision, and greater accuracy in recognizing vehicles and pedestrians, leading to more precise judgments and decisions regarding driving behavior, thus making autonomous driving safer and more intelligent. However, outdoor driving conditions vary greatly (e.g., highways, main roads, mixed pedestrian and vehicle roads) and driving scenarios are diverse (e.g., daytime, nighttime, sunny days, rainy days, dusty / foggy days), making it difficult to guarantee the imaging quality of the ISP.

[0057] Furthermore, in existing ISP technologies applied to autonomous driving, the ISP parameters are often configured after system startup, ensuring the ISP remains well-adapted to a specific driving scenario during operation. However, a single set of ISP parameters cannot adapt to all scenarios. For example, a set of parameters that produces good images in bright daylight often fails to produce the same good results in low-light conditions at night (potentially resulting in a dark image but overexposed lights making vehicles unclear). This is due to the inherent implementation mechanism of existing ISP algorithms; a single ISP calibration parameter can only be well-adapted to a certain type of scenario. Parameters for sunny days can only be applied to sunny scenarios, and parameters for nighttime scenarios can only be applied to nighttime scenarios.

[0058] Undoubtedly, there are many application scenarios for autonomous driving, and some of these scenarios have contradictory parameters. For example, when entering or exiting tunnels or underground parking garages, the change in lighting from bright to dark is extremely drastic within a very short time. This results in either an image displayed as completely dark in a dark environment when ISP processing technology is suitable for daytime conditions, or an image displayed as completely white in a bright environment when ISP processing technology is suitable for dark conditions.

[0059] Therefore, existing autonomous vehicles suffer from poor image quality in the ISP due to sudden changes in the external scene during real-time image acquisition.

[0060] To address the aforementioned problems, this invention provides an adaptive adjustment method, system, and vehicle for image capture parameters, which will be described in detail below.

[0061] like Figure 1 As shown, Figure 1 A flowchart illustrating an embodiment of the adaptive adjustment method for image capture parameters provided by the present invention includes:

[0062] Step S101: Obtain image samples and target image capture parameter samples;

[0063] Step S102: Establish an initial image recognition model. Input image samples into the initial image recognition model and use the corresponding target image capture parameter samples as sample labels to train the initial image recognition model and obtain a fully trained target image recognition model.

[0064] Step S103: Acquire real-time images, input the real-time images into the target image recognition model to obtain the corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters;

[0065] The initial image recognition model includes at least one of the following: initial camera preprocessing parameter recognition sub-model, initial image brightness recognition sub-model, initial image color recognition sub-model, initial image noise recognition sub-model, and initial image distortion recognition sub-model.

[0066] In this embodiment, firstly, image samples and target image capture parameter samples are acquired; then, an initial image recognition model is established by inputting the image samples into the initial image recognition model and using the corresponding target image capture parameter samples as sample labels to train the initial image recognition model, thereby obtaining a fully trained target image recognition model; finally, a real-time image is acquired by inputting the real-time image into the target image recognition model to obtain the corresponding target image capture parameters, and the image capture parameters are adjusted based on the target image capture parameters; wherein, the initial image recognition model includes at least one of the following: an initial camera preprocessing parameter recognition sub-model, an initial image brightness recognition sub-model, an initial image color recognition sub-model, an initial image noise recognition sub-model, and an initial image distortion recognition sub-model.

[0067] In this embodiment, a target image recognition model is constructed to perform image recognition on images during the autonomous driving process of the vehicle, determine the current scene state of the vehicle, and select the most suitable target image shooting parameters according to the current scene state. This enables the target image shooting parameters to be switched according to the real-time scene state of the vehicle. Since the target image shooting parameters matched based on the real-time scene state can effectively guarantee the imaging effect of the ISP, it can effectively overcome the problem of poor imaging effect of the ISP caused by sudden changes in the external scene.

[0068] In a preferred embodiment, the image capture parameters include ISP parameters and camera configuration parameters.

[0069] In a preferred embodiment, in step S101, in order to obtain image samples and target image acquisition parameter samples, such as... Figure 2 As shown, Figure 2 A schematic flowchart of an embodiment of obtaining a sample provided by the present invention includes:

[0070] Step S111: Obtain the initial image samples and their corresponding initial image capture parameters;

[0071] Step S112: Based on the initial image samples, adjust the initial image shooting parameters and reshoot to obtain multiple transition image samples and their corresponding transition image shooting parameters;

[0072] Step S113: Select the optimal transition image sample from multiple transition image samples, and determine the optimal transition image shooting parameters corresponding to the optimal transition image sample as the target image shooting parameter sample;

[0073] The initial image samples include multiple image samples taken in the same scene under different external environmental conditions.

[0074] In this embodiment, firstly, initial image samples and their corresponding initial image shooting parameters are obtained; then, based on the initial image samples, the initial image shooting parameters are adjusted and the images are reshot to obtain multiple transition image samples and their corresponding transition image shooting parameters; finally, the optimal transition image sample among the multiple transition image samples is selected, and the optimal transition image shooting parameters corresponding to the optimal transition image sample are determined as the target image shooting parameter sample; wherein, the initial image samples include multiple image samples taken in the same scene under different external environmental conditions.

[0075] In this embodiment, based on the scene corresponding to the initial image sample, the image shooting parameters are adjusted to take multiple shots, so that multiple transition image samples can be obtained for a certain scene. Then, by comparison, the best transition image sample with the best effect is selected, and the best transition image shooting parameters corresponding to the best transition image sample are used as the target image shooting parameter sample. Thus, when any image is obtained, the matching image shooting parameters can be obtained according to the current scene state of the image.

[0076] In a preferred embodiment, in step S102, in order to obtain a fully trained target image recognition model, such as... Figure 3 As shown, Figure 3 A flowchart illustrating an embodiment of the target image recognition model obtained by the present invention includes:

[0077] Step S121: Establish an initial image recognition model;

[0078] Step S122: Divide the image samples into blocks to obtain multiple key feature blocks;

[0079] Step S123: Input multiple key feature blocks into the initial image recognition model respectively, and use the corresponding target image shooting parameter samples as sample labels to continuously iterate and train the initial image recognition model to obtain a fully trained target image recognition model.

[0080] In this embodiment, by dividing the image sample into blocks, multiple key feature blocks with the characteristics of the image sample are obtained. Then, the initial image recognition model performs data analysis and processing on the key feature blocks, which effectively reduces the amount of data processing, thereby improving data processing efficiency and reducing the time to obtain target image shooting parameters, thereby improving the reaction speed of autonomous vehicles during operation.

[0081] In step S132, for each image sample, on the one hand, ISP processing is performed on it; on the other hand, AI is used to intelligently identify its corresponding scene effect. When sampling the effect image of each type, in order to improve real-time performance, the image sample is divided into blocks, so that only the key feature blocks need to be sampled to obtain the classification features of the entire image sample.

[0082] Because driving scene images have an overall consistent characteristic—that is, if the scene is bright at midday and sunny, the entire image is bright, and if it is dark at night, the entire image is dark—only a portion of the image feature blocks need to be sampled. Furthermore, since different ISP classification algorithms use different processing methods, the position and size of the sampled feature blocks also differ, and this is automatically switched by the system according to the ISP processing flow.

[0083] In one specific embodiment, such as Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the result of an embodiment of image sample segmentation provided by the present invention. In a driving scenario on a sunny midday day, the intense sunlight exceeds the linear range of the camera, resulting in an overexposed image that appears entirely white, making it difficult to see driving details. When a frame of camera data is processed by the ISP (Image Signal Processor) for brightness and contrast, ISP effect detection is performed. Since the ISP features of the entire image are consistent, only key feature blocks within a small local area around the driving road need to be sampled, thereby reducing the amount of data processed and improving real-time performance.

[0084] In a preferred embodiment, in step S133, the initial image recognition model includes at least one of the following: an initial camera preprocessing parameter recognition sub-model, an initial image brightness recognition sub-model, an initial image color recognition sub-model, an initial image noise recognition sub-model, and an initial image distortion recognition sub-model. The function of each sub-model is explained below.

[0085] First, based on the initial camera preprocessing parameters, the sub-model identifies defect samples of key feature blocks, and determines the corrected defect shooting parameter samples of key feature blocks based on the defect samples.

[0086] In other words, the initial camera preprocessing parameter recognition sub-model can identify inherent defects in key feature blocks, such as bad pixels, abnormal black levels, noise, and mosaic. Based on the recognition results, the current shooting parameters are adjusted to determine the corrected defect shooting parameter samples for key feature blocks, thereby obtaining a better image in the current scene state based on the corrected defect shooting parameter samples to meet the needs of autonomous vehicles.

[0087] Based on the sample of shooting parameters with corrected defects, a better image under the current scene condition can be obtained. At least the following optimization effects can be achieved in the ISP process, such as: bad pixel correction, black level correction, raw noise reduction, and de-mosaic.

[0088] It should be noted that inherent defects also include other identifiable defects.

[0089] Second, based on the initial image brightness recognition sub-model, the brightness samples of key feature blocks are identified, and the corrected brightness shooting parameter samples of key feature blocks are determined according to the brightness samples.

[0090] In other words, the initial image brightness recognition sub-model can identify brightness issues in key feature blocks, such as resolution, image wide dynamic range, brightness, contrast, white balance, and automatic exposure. Based on the recognition results, the current shooting parameters are adjusted to determine the corrected brightness shooting parameter samples for key feature blocks. This enables the generation of a better image for the current scene state based on the corrected brightness shooting parameter samples, thus meeting the needs of autonomous vehicles.

[0091] Based on the sample of shooting parameters with corrected brightness, a better image under the current scene condition can be obtained. At least the following optimization effects can be achieved in the ISP process, such as: Gamma setting, image wide dynamic range setting, brightness setting, contrast setting, automatic white balance, automatic exposure, etc.

[0092] It should be noted that brightness issues also include other factors that affect image brightness.

[0093] Third, based on the initial image color recognition sub-model, color samples of key feature blocks are identified, and corrected color shooting parameter samples of key feature blocks are determined according to the color samples.

[0094] In other words, the initial image color recognition sub-model can identify color problems in key feature blocks, such as color matrix, high color resolution, black level adjustment, color gamut transformation, hue saturation, and image curve restoration. Based on the recognition results, the current shooting parameters are adjusted to determine the corrected color shooting parameter samples for key feature blocks, thereby obtaining a better image for the current scene state based on the corrected color shooting parameter samples to meet the needs of autonomous vehicles.

[0095] Based on the corrected color shooting parameter samples, a better image under the current scene condition can be obtained. At least the following optimization effects can be achieved in the ISP process, such as: color matrix setting, high color resolution, black level adjustment, color gamut transformation, hue and saturation setting, and image curve restoration.

[0096] It should be noted that color issues also include other factors that affect the color of an image.

[0097] Fourth, based on the initial image noise recognition sub-model, noise samples of key feature blocks are identified, and the corrected noise shooting parameter samples of key feature blocks are determined according to the noise samples.

[0098] In other words, the initial image noise recognition sub-model can identify noise problems in key feature blocks, such as 2D noise, 3D noise, and edge noise. Based on the recognition results, the current shooting parameters are adjusted to determine the corrected noise shooting parameter samples for key feature blocks, thereby obtaining a better image under the current scene state based on the corrected noise shooting parameter samples to meet the needs of autonomous vehicles.

[0099] By obtaining a better image under the current scene state based on the sample of shooting parameters with corrected noise, at least the following optimization effects can be achieved in the ISP process, such as: 2D image denoising, 3D image denoising, edge denoising, etc.

[0100] It should be noted that noise issues also include other noise problems that can be identified and affect the image.

[0101] Fifth, based on the initial image distortion recognition sub-model, distortion samples of key feature blocks are identified, and the distortion correction shooting parameter samples of key feature blocks are determined according to the distortion samples.

[0102] In other words, the initial image distortion recognition sub-model can identify image distortion problems in key feature blocks, such as camera lens tilt, image distortion, edge weakening, and image sharpening. Based on the recognition results, the current shooting parameters are adjusted to determine the corrected distortion shooting parameter samples for key feature blocks, thereby obtaining a better image under the current scene state based on the corrected distortion shooting parameter samples to meet the needs of autonomous vehicles.

[0103] By obtaining a better image under the current scene state based on the sample of shooting parameters with corrected noise, at least the following optimization effects can be achieved in the ISP process, such as: camera lens correction, image distortion correction, edge enhancement, image sharpening, etc.

[0104] It should be noted that distortion problems also include other issues that cause image distortion.

[0105] It should be noted that although the initial camera preprocessing parameter recognition sub-model, initial image brightness recognition sub-model, initial image color recognition sub-model, initial image noise recognition sub-model, and initial image distortion recognition sub-model each have their own functions, in actual applications, different vehicles face different scenarios during autonomous driving. Therefore, the initial image recognition model can select some or all of these sub-models to process data according to actual needs, thereby obtaining the target image shooting parameter sample corresponding to the image sample based on at least one of the corrected defect shooting parameter sample, corrected brightness shooting parameter sample, corrected color shooting parameter sample, corrected noise shooting parameter sample, and corrected distortion shooting parameter sample.

[0106] In other embodiments, the number of any sub-model can be adjusted as needed to improve the reliability of the final target image capture parameters.

[0107] Furthermore, for an initial image recognition model that includes multiple sub-models, in order to improve the efficiency of data processing, all sub-models run simultaneously; that is, the initial camera preprocessing parameter recognition sub-model, and / or the initial image brightness recognition sub-model, and / or the initial image color recognition sub-model, and / or the initial image noise recognition sub-model, and / or the initial image distortion recognition sub-model, which are in the same initial image recognition model, run simultaneously.

[0108] In a preferred embodiment, in step S103, based on the obtained fully trained target image recognition model, in order to adjust the image capturing parameters of the autonomous vehicle in real time to obtain the corresponding target image capturing parameters, such as... Figure 5 As shown, Figure 5 A flowchart illustrating an embodiment of the present invention for obtaining corresponding target image capture parameters includes:

[0109] Step S131: Acquire real-time images;

[0110] Step S132: Use AI technology to evaluate the scene effects of the real-time image and determine if there are any sudden scene changes;

[0111] Step S133: When a sudden change in scene is determined, the real-time image is input into the target image recognition model to obtain the corresponding target image capture parameters.

[0112] In this embodiment, firstly, a real-time image is acquired; then, the real-time image is processed using AI technology to determine if there are any sudden scene changes; finally, when a sudden scene change is determined, the real-time image is input into the target image recognition model to obtain the corresponding target image capture parameters.

[0113] In this embodiment, by performing scene effects on real-time images, it is possible to effectively determine whether there are sudden changes in the current scene state of the vehicle, thereby reducing the workload of the target image recognition model. That is, when the autonomous vehicle is in a stable scene state and running stably, the target image recognition model is not activated, and the target image shooting parameters are also avoided.

[0114] In one specific embodiment, when the scene changes, if the initial image recognition model includes multiple sub-models such as the initial camera preprocessing parameter recognition sub-model, the initial image brightness recognition sub-model, the initial image color recognition sub-model, the initial image noise recognition sub-model, and the initial image distortion recognition sub-model, after obtaining the corrected shooting parameter samples corresponding to each sub-model, contradictions may occur. To ensure the final imaging effect, it is also necessary to perform an effect verification on the image obtained based on the target image shooting parameters to obtain an image that achieves the target effect, such as... Figure 6 As shown, Figure 6 This is a schematic flowchart illustrating an embodiment of the present invention for obtaining an image that achieves the target effect.

[0115] First, an updated image is obtained based on the target image capture parameters. Then, AI technology is used to detect the image processing effect of the ISP to determine whether the updated image achieves the target effect. Finally, if it is determined that the updated image does not achieve the target effect, the updated image is input into the target image recognition model to obtain updated target image capture parameters until an image that achieves the target effect is obtained.

[0116] In this embodiment, AI technology is used to detect the effect of the image and adjust and update the target image shooting parameters in real time. By systematically correcting the image shooting parameters, the overall optimization can be achieved, resulting in an image with the best overall effect.

[0117] By constructing a target image recognition model, images during the autonomous driving process of the vehicle are recognized to determine the current scene state of the vehicle. Based on the current scene state, the most suitable target image shooting parameters are selected, and the target image shooting parameters are switched according to the real-time scene state of the vehicle. Since the target image shooting parameters matched based on the real-time scene state can effectively guarantee the imaging effect of the ISP, it can effectively overcome the problem of poor imaging effect of the ISP caused by sudden changes in the external scene.

[0118] To address the aforementioned problems, the present invention also provides an adaptive adjustment system for image capturing parameters, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of an embodiment of the adaptive adjustment system for image capturing parameters provided by the present invention. The adaptive adjustment system 700 for image capturing parameters includes:

[0119] The sample acquisition module 701 is used to acquire image samples and target image capture parameter samples;

[0120] The target image recognition model acquisition module 702 is used to establish an initial image recognition model, input image samples into the initial image recognition model, and use the corresponding target image capture parameter samples as sample labels to train the initial image recognition model and obtain a fully trained target image recognition model.

[0121] The image capture parameter adjustment module 703 is used to acquire real-time images, input the real-time images into the target image recognition model to obtain the corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters;

[0122] The initial image recognition model includes at least one of the following: initial camera preprocessing parameter recognition sub-model, initial image brightness recognition sub-model, initial image color recognition sub-model, initial image noise recognition sub-model, and initial image distortion recognition sub-model.

[0123] The present invention also provides a vehicle, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the adaptive adjustment method for image capturing parameters as described above. The memory stores the adaptive adjustment program for image capturing parameters.

[0124] In some embodiments, the memory may be an internal storage unit of a computer device, such as a hard disk or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store application software and various types of data installed on the computer device, such as program code for installing the computer device. The memory can also be used to temporarily store data that has been output or will be output. In one embodiment, an adaptive adjustment program for image capture parameters can be executed by a processor to implement the adaptive adjustment method for image capture parameters according to various embodiments of the present invention.

[0125] In some embodiments, the processor may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in memory or process data, such as executing an adaptive adjustment program for image capture parameters.

[0126] Furthermore, the present invention also provides an electronic device for real-time ISP processing of data from multiple cameras, such as... Figure 8 As shown, Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention.

[0127] The electronic device includes: an MCU control module 100, a camera control module 101, a Sensor data preprocessing related ISP function module 102, an image brightness / contrast related ISP function module 103, an image color related ISP function module 104, an image denoising related ISP function module 105, an image distortion correction / enhancement related ISP function module 106, a data caching module 107, a data acquisition module for key driving areas in the ISP image 108, a multi-core AI processor module 109, and a real-time ISP / camera parameter adjustment (lookup table) module 110. The operation instructions for each unit are as follows:

[0128] Before using this device, it is necessary to obtain the optimal imaging effect under various driving scenarios through actual road tests, and determine the calibration parameters of various ISP processing functions required for the optimal effect under these driving scenarios, as well as the corresponding camera configuration parameters. Furthermore, this device uses AI to detect the ISP imaging effect. Therefore, it is necessary to train the AI ​​neural network for ISP effect detection based on the calibration parameters of various driving scenarios and the optimal effect as a reference, and obtain the parameters and weights of the trained AI neural network.

[0129] Before using this device, the ISP calibration parameters, camera configuration parameters, and AI neural network parameter weights for various scenarios need to be saved to an external SD card or EMMC.

[0130] When the device is powered on, the MCU control module 100 first operates normally and reads the parameters of each functional module of the ISP and the camera configuration parameters under the default driving scenario (generally a normal daytime lighting scenario) from the SD card or EMMC, and then configures them to the relevant functional modules of the ISP (modules 102-106). The camera parameters are configured to the external cameras through the camera control module 101 to enable them to work normally. The MCU control module 100 also reads the parameters and weights of the AI ​​neural network from the SD card or EMMC and sends them to the multi-core AI processor module 109, which then saves them to the external DDR. After that, the MCU control module 100 starts the multi-core AI processor module 109, which then reads the relevant weight parameters from the external DDR to perform AI neural network detection.

[0131] After power-on, the MCU control module 100 reads the ISP calibration parameters of other scenarios and the camera collaborative adaptation parameters of the current scenario from the SD card EMMC and sends them to the ISP / camera parameter real-time calibration (lookup table) module 110 and stores them in module 111 respectively, storing them in the form of tables according to the scenario.

[0132] After the camera captures raw data and sends it to the Sensor Data Preprocessing (ISP) module 102 for preprocessing, the processing effect of the current frame is sent to subsequent modules and also to the data cache module 107 for caching. To achieve real-time performance, the data cache module 107 uses a ping-pong operation, alternately saving the nth and n+1th frames to RAMA and RAMB. When the Sensor Data Preprocessing (ISP) module 102 writes data to RAMA or RAMB, the data acquisition module 108 for the key driving areas in the ISP image reads the ISP effect data from B or A accordingly. This enables sampling and detection of the current frame, real-time adjustment after detection, and the generation of a new ISP effect in the next frame, achieving uninterrupted real-time adjustment and control, thus making autonomous driving safer and improving target recognition accuracy.

[0133] Using the above method, the data acquisition module 108 of the driving-critical area in the ISP image only samples and reads key feature blocks in the driving-related range from RAMA or B of the data cache module 107, instead of the entire ISP effect image, thereby reducing the amount of data used, improving processing speed, and achieving higher real-time performance.

[0134] Since the various functional modules of the IPS operate continuously in a pipeline, meaning that each function processes each image frame simultaneously, the data cache module 107 simultaneously reads the effect data written to RAM by modules 102-106 and sends it to the multi-core AI processor module 109 for AI-based ISP effect detection. The multi-core AI processor module 109 is a multi-core AI processor with multiple AI processing cores that can work independently and in parallel. Each core can simultaneously detect the effect data of each function of the ISP.

[0135] When an AI core in the multi-core AI processor module 109 detects that a certain ISP effect needs adjustment, it sends the information requiring adjustment (e.g., reducing brightness and overexposure due to an overly bright scene) to the ISP / camera parameter real-time adjustment (lookup table) module 110. The ISP / camera parameter real-time adjustment (lookup table) module 110 converts this information into a corresponding parameter table format and reads the ISP and camera parameters from the table in module 111 according to the corresponding table format. Each RAM in module 111 stores parameters from one parameter table. Module 111 retrieves the corresponding parameters from the ISP / camera parameter real-time adjustment (lookup table) module 110 and sends them to the ISP / camera parameter real-time adjustment (lookup table) module 110, which then adjusts the corresponding ISP module and camera adaptation within the current frame. The reason for using dedicated hardware circuitry to look up the adjustment parameters is that calculating or searching through the MCU is very slow and makes it difficult to meet the real-time requirement of adjusting in the current frame and implementing in the next frame. Using dedicated hardware circuits to perform table lookup operations can achieve the fastest speed and improve real-time performance.

[0136] When the Sensor data preprocessing related ISP function modules 102-106 produce images with good ISP effects after adjustment, they are sent to the deep learning target recognition module through buffering. Due to the good image quality, the deep learning module can easily achieve a longer detection distance and more accurate detection precision, thereby improving the overall performance of autonomous driving.

[0137] It should be noted that AI technology, by simulating human thinking, enables machines to flexibly solve various complex problems. Its characteristics include the following:

[0138] First, the goal of AI technology is to enable machines to simulate human thinking, thereby facilitating the handling of complex real-world problems. AI technology allows for the design of a wide variety of algorithms that can help machines understand complex problems and continuously improve their performance based on ever-increasing amounts of data.

[0139] Secondly, AI technology can greatly improve work efficiency. AI can automatically process large amounts of information, quickly solve problems, execute tasks more accurately, and effectively reduce the possibility of machine errors.

[0140] Therefore, AI technology can achieve high-precision monitoring of the vehicle's external scene and quickly obtain the corresponding reverse calibration parameters based on the monitoring results, so as to adjust the vehicle's shooting parameters in real time and effectively overcome the problem of poor ISP imaging effect caused by sudden changes in the external scene.

[0141] In one specific embodiment, the electronic device can be a circuit structure or a chip that sends the image to the AI ​​module in real time for detection (too bright or too dark), thereby generating corresponding adjustment parameters for the ISP to achieve a normal image output effect in the next frame.

[0142] It should be noted that while adjusting the ISP, it is also necessary to make corresponding readjustments to the camera (such as adjusting exposure time, HDR compression curve, etc.). Due to the limitations of the ISP algorithm and the processing performance of the chip, adjusting the ISP alone may not achieve the best results for very wide dynamic range changes. Adjusting the ISP and the camera simultaneously can result in better image quality.

[0143] The following is an example of the tuning method in a specific driving scenario:

[0144] When vehicles enter or exit tunnels, the image changes abruptly from very bright to very dark, and vice versa. Under these conditions of large dynamic range light changes, the camera's raw data will experience noise and altered current. Therefore, AI is used to detect and adjust the performance data of the ISP (In-Screen Display) module 102 related to sensor data preprocessing.

[0145] In a tunnel environment, if the image is too bright or too dark, causing the camera data to be too white or too black, the ISP function module 103, which detects the image brightness / contrast, will adjust it.

[0146] Due to the narrow space inside the tunnel, the lighting is not standard white, but a mixture of various lights such as vehicle headlights, tunnel ceiling lights, and emergency lights, which causes the colors of the image to become abnormal and make it difficult to distinguish the colors of vehicle shapes and traffic light signs. Therefore, it is necessary to detect and adjust the image color-related ISP function module 104 accordingly.

[0147] In driving scenarios involving rain, smog, or dust, the images captured by the camera exhibit a dense, grainy noise effect. Because distant vehicles are small, they can be obscured by these grains and become difficult to identify. Therefore, the image denoising-related ISP module 105 needs to be detected and adjusted to make the image balanced and clear, allowing distant vehicles to be more clearly visible and easier to identify.

[0148] In driving scenarios involving rain, smog, or dust, the camera's image is relatively dark, causing the edge color of light-colored vehicles or small bicycles to be close to the color of the cement road, making it difficult to identify light-colored vehicles. It is necessary to detect and adjust the image distortion / enhancement related ISP function module 106 to make the edges of light-colored vehicles and bicycles more obvious, thus making them easier to identify.

[0149] In one specific embodiment, the electronic device may be a circuit structure or a chip to implement the adaptive adjustment method for image capture parameters described above.

[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0151] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An adaptive adjustment method for image acquisition parameters, applied to autonomous driving of vehicles, characterized in that, include: Acquire image samples, as well as target image capture parameter samples; An initial image recognition model is established by inputting the image samples into the initial image recognition model and using the corresponding target image capture parameter samples as sample labels to train the initial image recognition model, thereby obtaining a fully trained target image recognition model. Acquire real-time images, input the real-time images into the target image recognition model to obtain the corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters; The initial image recognition model includes at least one of the following: an initial camera preprocessing parameter recognition sub-model, an initial image brightness recognition sub-model, an initial image color recognition sub-model, an initial image noise recognition sub-model, and an initial image distortion recognition sub-model. The process of establishing an initial image recognition model involves inputting the image samples into the initial image recognition model, using the corresponding target image capture parameter samples as sample labels, and training the initial image recognition model to obtain a fully trained target image recognition model. Establish an initial image recognition model; The image sample is divided into blocks to obtain multiple key feature blocks; The multiple key feature blocks are respectively input into the initial image recognition model to obtain the target image shooting parameter samples corresponding to the image samples. The target image shooting parameter samples are used as sample labels to continuously iterate and train the initial image recognition model to obtain a fully trained target image recognition model. The step of inputting the multiple key feature blocks into the initial image recognition model to obtain the target image capture parameter samples corresponding to the image samples includes: The initial camera preprocessing parameter recognition sub-model is used to identify the defect samples of the key feature blocks and determine the correction defect shooting parameter samples of the key feature blocks based on the defect samples. The initial image brightness recognition sub-model is used to identify the brightness samples of the key feature blocks and determine the corrected brightness shooting parameter samples of the key feature blocks based on the brightness samples. The initial image color recognition sub-model is used to identify the color samples of the key feature blocks and determine the corrected color shooting parameter samples of the key feature blocks based on the color samples. The initial image noise recognition sub-model is used to identify noise samples of the key feature blocks and determine the corrected noise shooting parameter samples of the key feature blocks based on the noise samples. The initial image distortion recognition sub-model is used to identify distortion samples of the key feature blocks and determine the correction distortion shooting parameter samples of the key feature blocks based on the distortion samples. The target image capture parameter sample includes at least one of the defect correction capture parameter sample, the brightness correction capture parameter sample, the color correction capture parameter sample, the noise correction capture parameter sample, and the distortion correction capture parameter sample.

2. The adaptive adjustment method for image capturing parameters according to claim 1, characterized in that, The acquisition of image samples and target image capture parameter samples includes: Obtain initial image samples and their corresponding initial image capture parameters; Based on the initial image sample, the initial image shooting parameters are adjusted and the image is reshot to obtain multiple transition image samples and their corresponding transition image shooting parameters; Select the optimal transition image sample from the plurality of transition image samples, and determine the optimal transition image shooting parameters corresponding to the optimal transition image sample as the target image shooting parameter sample; The initial image samples include multiple image samples taken in the same scene under different external environmental conditions.

3. The adaptive adjustment method for image capturing parameters according to claim 1, characterized in that, The process of acquiring a real-time image, inputting the real-time image into the target image recognition model to obtain corresponding target image capture parameters, and adjusting the image capture parameters based on the target image capture parameters includes: Acquire real-time images; The AI ​​technology is used to analyze the scene effects of the real-time images to determine whether there are any sudden scene changes. When a sudden change in scene is detected, the real-time image is input into the target image recognition model to obtain the corresponding target image shooting parameters, and the image shooting parameters are adjusted based on the target image shooting parameters.

4. The adaptive adjustment method for image capturing parameters according to claim 3, characterized in that, The step of adjusting the image capture parameters based on the target image capture parameters further includes: An updated image is obtained based on the target image capture parameters; The AI ​​technology is used to determine whether the updated image achieves the target effect. When it is determined that the updated image does not achieve the target effect, the updated image is input into the target image recognition model to obtain updated target image shooting parameters until an image that achieves the target effect is obtained.

5. The adaptive adjustment method for image capturing parameters according to claim 1, characterized in that, The initial camera preprocessing parameter recognition sub-model, and / or the initial image brightness recognition sub-model, and / or the initial image color recognition sub-model, and / or the initial image noise recognition sub-model, and / or the initial image distortion recognition sub-model operate simultaneously.

6. The adaptive adjustment method for image capturing parameters according to claim 1, characterized in that, The image capture parameters include ISP parameters and camera configuration parameters.

7. An adaptive adjustment system for image capturing parameters, characterized in that, include: The sample acquisition module is used to acquire image samples and target image capture parameter samples; The target image recognition model acquisition module is used to establish an initial image recognition model, input the image sample into the initial image recognition model, and use the corresponding target image shooting parameter sample as the sample label to train the initial image recognition model to obtain a fully trained target image recognition model. An image capture parameter adjustment module is used to acquire real-time images, input the real-time images into the target image recognition model to obtain corresponding target image capture parameters, and adjust the image capture parameters based on the target image capture parameters; The initial image recognition model includes at least one of the following: an initial camera preprocessing parameter recognition sub-model, an initial image brightness recognition sub-model, an initial image color recognition sub-model, an initial image noise recognition sub-model, and an initial image distortion recognition sub-model. The target image recognition model acquisition module is also used to establish an initial image recognition model; The image sample is divided into blocks to obtain multiple key feature blocks; The multiple key feature blocks are respectively input into the initial image recognition model to obtain the target image shooting parameter samples corresponding to the image samples. The target image shooting parameter samples are used as sample labels to continuously iterate and train the initial image recognition model to obtain a fully trained target image recognition model. The step of inputting the multiple key feature blocks into the initial image recognition model to obtain the target image capture parameter samples corresponding to the image samples includes: The initial camera preprocessing parameter recognition sub-model is used to identify the defect samples of the key feature blocks and determine the correction defect shooting parameter samples of the key feature blocks based on the defect samples. The initial image brightness recognition sub-model is used to identify the brightness samples of the key feature blocks and determine the corrected brightness shooting parameter samples of the key feature blocks based on the brightness samples. The initial image color recognition sub-model is used to identify the color samples of the key feature blocks and determine the corrected color shooting parameter samples of the key feature blocks based on the color samples. The initial image noise recognition sub-model is used to identify noise samples of the key feature blocks and determine the corrected noise shooting parameter samples of the key feature blocks based on the noise samples. The initial image distortion recognition sub-model is used to identify distortion samples of the key feature blocks and determine the correction distortion shooting parameter samples of the key feature blocks based on the distortion samples. The target image capture parameter sample includes at least one of the defect correction capture parameter sample, the brightness correction capture parameter sample, the color correction capture parameter sample, the noise correction capture parameter sample, and the distortion correction capture parameter sample.

8. A vehicle, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the adaptive adjustment method for image capture parameters as described in any one of claims 1-6.

Images