Image processing method and system and vehicle

Abstract

The embodiment of the invention provides an image processing method and system and a vehicle. The method comprises the following steps: acquiring a current image; and inputting the current image into a trained exposure label model, so that the exposure label model outputs an exposure prediction label of the current image, and the exposure prediction label is used for image processing. According to the embodiment of the invention, the exposure prediction label of the current image is output based on the exposure label model to process the image, for example, the exposure parameter of the image is adjusted, so that the rationality of the adjusted exposure parameter is ensured, and the display quality of the image is improved.

Description

Technical Field

[0001] The present invention relates to the field of video technology, and in particular to an image processing method, an in-vehicle imaging system, an electronic device, a vehicle, and a computer-readable storage medium. Background Technology

[0002] In practice, exposure parameters are one of the key factors affecting image quality.

[0003] Traditional image exposure control methods typically rely on engineers' subjective adjustments and accumulated experience. When faced with complex and ever-changing image shooting scenarios, this method often fails to ensure the rationality of exposure parameters, resulting in poor image display quality. Summary of the Invention

[0004] The purpose of this invention is to provide an image processing method to solve the problem of poor image display quality caused by unreasonable exposure parameters. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, an image processing method is provided, the method comprising:

[0006] Get the current image;

[0007] The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label of the current image, and the exposure prediction label is used for image processing.

[0008] Optionally, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. The method further includes:

[0009] Acquire exposure images with different exposure parameters for various shooting scenarios;

[0010] Extract objective indicator data corresponding to multiple objective indicators from the exposed image;

[0011] Generate sample images and corresponding exposure prediction labels for the sample images based on the objective indicator data;

[0012] The exposure label model to be trained is obtained by using the sample image and the exposure prediction label corresponding to the sample image.

[0013] Optionally, generating sample images and corresponding exposure prediction labels based on the objective indicator data includes:

[0014] Search for target objective indicator data that conforms to the preset exposure value search strategy from the objective indicator data;

[0015] The exposure range is determined based on the objective indicator data of the target.

[0016] Multiple sample exposure parameters are extracted from the exposure parameters in each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images; each sample image has a corresponding exposure prediction label.

[0017] Optionally, the objective indicators are image color-related information; the objective indicators include at least one of the following: average image brightness, contrast, saturation, effective Gaussian exposure, overexposure percentage, and black percentage.

[0018] The preset exposure optimal value search strategy is a search strategy composed of the objective indicators; the preset exposure optimal value search strategy includes at least a search strategy composed of a single objective indicator, and / or, multiple or all of the objective indicators.

[0019] Optionally, the exposure range includes multiple ranges describing different exposure states; the exposure range includes at least an overexposure range, a normal exposure range, and an underexposure range, and the exposure prediction label includes at least an overexposure label, a normal exposure label, and an underexposure label.

[0020] Optionally, determining the exposure range based on the target objective indicator data includes:

[0021] Obtain the exposure parameters of the exposure image corresponding to the target objective index data;

[0022] The exposure range is determined based on the exposure parameters of the exposure image corresponding to the objective indicator data of the target.

[0023] Optionally, determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data includes:

[0024] Obtain the maximum and minimum values ​​of the exposure parameters of the exposure image corresponding to the target objective index data, and set the normal exposure range based on the maximum and minimum values ​​of the exposure parameters.

[0025] Optionally, determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data further includes:

[0026] The overexposure range is determined based on the maximum value of the exposure parameters of the exposed image corresponding to the target objective index data;

[0027] Alternatively, the underexposure range can be determined based on the minimum value of the exposure parameters of the exposed image corresponding to the objective index data of the target.

[0028] Alternatively, set the overexposure range and / or underexposure range based on the normal exposure range.

[0029] Optionally, objective indicator data corresponding to multiple objective indicators are extracted from the exposed image, including:

[0030] The exposed image is processed according to the first step length parameter to obtain a densely exposed image of any exposure level;

[0031] Extract objective indicator data corresponding to multiple objective indicators from the densely exposed image.

[0032] Optionally, before processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels, the method further includes:

[0033] The exposed image is converted using an inverse tone mapping algorithm to obtain a high dynamic range image;

[0034] The step of processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels includes: processing the high dynamic range image according to the first step length parameter to obtain densely exposed images of arbitrary exposure levels.

[0035] Optionally, searching for target objective indicator data that conforms to a preset exposure suitability search strategy from the objective indicator data includes:

[0036] According to the second step length parameter, the target objective indicator data that meets the preset exposure value search strategy is searched step by step from the objective indicator data.

[0037] Optionally, searching for target objective indicator data that conforms to a preset exposure suitability search strategy from the objective indicator data includes:

[0038] According to the second step length parameter, the target objective indicator data that conforms to the preset exposure value search strategy and is within the preset exposure compensation range is searched step by step from the objective indicator data.

[0039] Optionally, the preset exposure value search strategy is a preset range interval set according to the objective indicators respectively, and the objective indicator data falls within the preset range interval if the preset exposure value search strategy is met.

[0040] Optionally, multiple sample exposure parameters are extracted from the exposure parameters in each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images, including:

[0041] According to the third step length parameter, multiple sample exposure parameters of different exposure levels are extracted from the exposure parameters of each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters of different exposure levels to obtain sample images of different exposure levels in each exposure range and the exposure prediction labels corresponding to the sample images.

[0042] Optionally, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image, including:

[0043] The current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features;

[0044] The hierarchical features are input into the logistic regression layer of the exposure label model to obtain the exposure prediction label of the current image.

[0045] Optionally, the current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features, including:

[0046] The current image is input into the shallow feature extraction layer of the exposure label model to obtain high-dimensional features;

[0047] The high-dimensional features are input into the deep feature extraction layer of the exposure label model to obtain deep features.

[0048] Optionally, the high-dimensional features are features related to overexposure percentage and black percentage, contrast, and saturation; the deep features are features related to average image brightness and effective Gaussian exposure.

[0049] Optionally, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. The exposure prediction label is used for image processing, including:

[0050] The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label and exposure level of the current image, and the exposure prediction label and exposure level are used for image processing.

[0051] Optionally, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. The exposure prediction label is used for image processing. The method further includes:

[0052] The exposure parameters are adjusted based on the exposure prediction label and the exposure level of the current image, which are used for image processing.

[0053] Optionally, adjusting the exposure parameters based on the exposure prediction label of the current image and the exposure level includes:

[0054] When the exposure prediction label of the current image is an underexposure label, obtain the underexposure parameters corresponding to the underexposure label;

[0055] The adjusted exposure parameters are obtained by increasing the exposure parameters of the current image based on the underexposure parameters and / or the exposure level.

[0056] Optionally, adjusting the exposure parameters based on the exposure prediction label of the current image and the exposure level includes:

[0057] When the exposure prediction label of the current image is an overexposure label, obtain the overexposure parameters corresponding to the overexposure label;

[0058] The exposure parameters of the current image are reduced based on the overexposure parameters and the exposure level to obtain the adjusted exposure parameters.

[0059] Optionally, adjusting the exposure parameters based on the exposure prediction label of the current image and the exposure level includes:

[0060] When the exposure prediction label of the current image is the normal exposure label, the exposure parameters of the current image are used as the adjusted exposure parameters.

[0061] Optionally, the current image includes at least the current image of the video stream, and the method further includes:

[0062] The current image and / or the next frame of the video stream are displayed according to the adjusted exposure parameters.

[0063] Optionally, before adjusting the exposure parameters based on the exposure prediction label of the current image, the method further includes:

[0064] When the video stream is being monitored to be playing, the current image of the video stream is acquired.

[0065] Optionally, the method further includes:

[0066] Preprocess the current image;

[0067] The preprocessing includes at least image format conversion, normalization, and image resolution adjustment.

[0068] In another aspect of the present invention, a computer-readable storage medium is also provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform any of the image processing methods described above.

[0069] In another aspect of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image processing methods described above.

[0070] In another aspect of the present invention, an in-vehicle imaging system is also provided, including the image processing device described above.

[0071] In another aspect of the present invention, a vehicle is also provided, which implements any of the image processing methods described above.

[0072] Compared with related technologies, the embodiments of the present invention have at least the following advantages:

[0073] In this embodiment of the invention, a current image is acquired and input into a trained exposure labeling model, causing the model to output an exposure prediction label for the current image. This exposure prediction label is used for image processing. This embodiment of the invention processes the image based on the exposure prediction label output by the exposure labeling model, for example, adjusting the image's exposure parameters. This ensures the rationality of the adjusted exposure parameters and improves the image's display quality. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0075] Figure 1 This is a schematic diagram of a photometer-based automatic exposure method.

[0076] Figure 2 Here is an example image of an exposure table;

[0077] Figure 3 This is a flowchart of the steps of an image processing method provided in an embodiment of the present invention;

[0078] Figure 4 This is a general framework diagram of a video automatic exposure control imaging system provided in an embodiment of the present invention;

[0079] Figure 5 This is a schematic diagram of a three-stage deep learning model provided in an embodiment of the present invention;

[0080] Figure 6 This is a basic flowchart of an AI-based automatic video exposure control algorithm provided in an embodiment of the present invention;

[0081] Figure 7 This is a flowchart of an exposure determination method provided in an embodiment of the present invention;

[0082] Figure 8This is a flowchart of an embodiment of the present invention for increasing exposure;

[0083] Figure 9 This is a flowchart of an embodiment of the present invention for reducing exposure;

[0084] Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0085] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0086] To enable those skilled in the art to better understand the embodiments of the present invention, some terms and names involved in the embodiments of the present invention will be explained below.

[0087] HDR: High dynamic range.

[0088] LDR: Low dynamic range.

[0089] Android: Android operating system.

[0090] Camera: A camera.

[0091] RGB: A color encoding system that includes three channels: R (red), G (green), and B (blue).

[0092] RGBA: A color encoding system that includes four channels: R (red), G (green), B (blue), and Alpha.

[0093] YUV: A color encoding system that includes three channels: Y (luminance), U (chrominance), and V (chrominance).

[0094] AE stands for Automatic Exposure. Specifically, automatic exposure refers to a camera (such as an in-vehicle social security system) automatically adjusting the exposure based on the intensity of light to prevent overexposure or underexposure.

[0095] TMO: Tone Mapping Operator, is a technique that transforms high dynamic range images into densely exposed images, making the visual perception of a high dynamic range image approximate in media with limited dynamic range.

[0096] iTMO: inverseToneMappingOperator, is a technique for converting standard dynamic range images into high dynamic range images.

[0097] Exposure bracketing refers to creating 3 or 5 images with different exposure levels in a single shot, using intermediate, reduced, and increased exposure values. Among these images, the one with the most suitable exposure can be obtained, and these images can also be used to create an HDR image.

[0098] EV: Exposure Value. Exposure value is a metric used to represent camera exposure settings. It combines aperture size, shutter speed, and ISO sensitivity to describe the lighting conditions of a video stream (image) shooting scene.

[0099] In recent years, with the rapid advancement of automotive intelligence, traditional car functions can no longer meet people's needs. More intelligent car accessories have brought people a modern experience of being connected to everything. The development of intelligent vehicles is inseparable from the support of in-vehicle imaging systems. With the rapid development of intelligent vehicles, in-vehicle imaging systems have been further improved and popularized in terms of functionality, resulting in higher quality in-vehicle imaging systems. Higher quality in-vehicle imaging systems greatly enhance the user experience of using intelligent vehicles, and are therefore favored by more car owners.

[0100] In-vehicle cameras typically face more complex shooting scenarios and harsher shooting conditions, such as lighting limitations due to the camera's mounting location. Considering both hardware limitations and harsher operating environments, controlling exposure is more difficult for in-vehicle cameras in complex lighting conditions, often resulting in poorer image quality. Specifically, poor image quality manifests in overexposed areas, lack of contrast in dark (underexposed) areas, and relatively higher noise levels. These issues severely impact the user experience of in-vehicle imaging systems.

[0101] To address the aforementioned issues, in-vehicle imaging systems typically incorporate automatic exposure processing capabilities. (Refer to...) Figure 1 This is a flowchart illustrating an automatic exposure method based on photometry. Traditional vehicle imaging systems can control exposure parameters based on photometry. The specific process includes: acquiring an input image; measuring illumination; searching a pre-made exposure table based on the measured illumination to adjust exposure parameters; acquiring the exposure parameters; and acquiring an output image. The output image is the input image converted based on the adjusted exposure parameters. For example, refer to... Figure 2 The image shown is an example of an exposure table, where the horizontal axis represents the Exposure index and the vertical axis represents the Exposure time (exposure time / shutter speed).

[0102] However, illumination measurement cannot accurately identify the scene content corresponding to the image being captured, thus easily leading to misjudgment of the environment. For example, in-vehicle imaging systems cannot accurately determine the lighting conditions in the shooting scene, and consequently cannot automatically expose the image appropriately based on these conditions. This can result in overexposure or underexposure, leading to images that are too bright or too dark, affecting image quality and information accuracy. Furthermore, due to the limited dynamic range of in-vehicle cameras, in extreme lighting conditions, the system may not be able to effectively handle large-scale brightness differences, resulting in overexposed or underexposed areas within the same image, thereby reducing the image's information richness.

[0103] Furthermore, traditional in-vehicle imaging systems do not consider the interrelationships between images in the video stream when deciding on the exposure parameters of the image (images in the video stream). They often only smooth the obtained exposure parameters through a smoothing range. Therefore, they cannot adapt to the problem of exposure parameter jumps caused by rapid changes in the environment in in-vehicle application scenarios, which leads to flickering of video brightness, affects the image quality of the in-vehicle imaging system, and may also have a negative impact on the decision-making of the driver assistance system, resulting in a negative user experience.

[0104] To address the aforementioned problems, this invention provides an image processing method. This method designs an objective exposure evaluation standard and sets multiple exposure prediction labels for different shooting scenarios. By obtaining the exposure prediction label corresponding to the current image (e.g., the current image (image frame) in a video stream), the exposure parameters can be adjusted based on the exposure prediction label. The adjusted exposure parameters are then used to process the image. For example, the adjusted exposure parameters can be used to display the next frame of the video stream, thus constraining the image based on the exposure prediction label and solving the problem of unreasonable image exposure parameters. For instance, it solves the previous problem of poor continuity and consistency of exposure parameters between adjacent images in a video stream, improving the display quality of images such as those in a video stream.

[0105] Reference Figure 3 This is a flowchart of the steps of an image processing method provided in an embodiment of the present invention, as follows: Figure 3 As shown, the method may specifically include the following steps:

[0106] Step 301: Obtain the current image.

[0107] In this embodiment of the invention, the current image can be an image that requires exposure parameter adjustment, or it can be an image used to adjust other images as a reference.

[0108] Step 302: Input the current image into the trained exposure label model so that the exposure label model outputs the exposure prediction label of the current image, and the exposure prediction label is used for image processing.

[0109] Image processing may include adjusting the exposure parameters of an image.

[0110] In this embodiment of the invention, the exposure label model is a pre-trained machine learning model that can output information such as the exposure prediction label corresponding to the current image based on the input current image.

[0111] For example, the exposure label model in this embodiment of the invention can be a lightweight network structure. Since the lightweight network structure cannot fit complex data, the design principle of relative labels is adopted in the label design. The exposure prediction label can include multiple labels such as a first label, a second label, and a third label. For example, the first label can be an underexposure label, the second label can be a normal exposure label, and the third label can be an overexposure label.

[0112] When entering image processing, such as the exposure adjustment stage, the exposure prediction label for the current image can be output by the AI-AE algorithm (exposure label model). Then, image processing can be performed on the current image or other images based on the exposure prediction label. For example, the exposure parameters of the vehicle camera can be adjusted based on the exposure prediction label corresponding to the current image. In this way, the vehicle camera will take and display the next frame image according to the adjusted exposure parameters.

[0113] For example, if the exposure prediction label of the current frame image is underexposed, it means that the overall area or part of the current image is dark and details are lost due to insufficient exposure. In this case, the exposure enhancement step is entered. The exposure can be increased by adjusting the exposure parameters, such as increasing the exposure time (shutter speed), increasing the aperture size (decreasing the aperture value), increasing the ISO sensitivity, and supplementing light, etc.

[0114] For example, if the exposure prediction label of the current frame image is the normal exposure label, it means that the entire area or part of the current image is within the ideal exposure range, with rich details, and there is no need to adjust the exposure parameters to increase or decrease the exposure.

[0115] For example, if the exposure prediction label of the current frame image is overexposed, it means that the overall area or part of the current image is too bright and details are lost due to overexposure. In this case, the exposure reduction step is entered. The exposure amount can be reduced by adjusting the exposure parameters, such as reducing the exposure time (shutter speed), reducing the aperture size (increasing the aperture value), reducing the ISO sensitivity, and reducing the light, etc.

[0116] Of course, the above-mentioned exposure prediction labels are merely examples. In practical applications, more exposure prediction labels can be defined according to actual needs. This embodiment of the invention does not impose any limitations on this.

[0117] In this embodiment of the invention, a current image is acquired and input into a trained exposure labeling model, causing the model to output an exposure prediction label for the current image. This exposure prediction label is used for image processing. This embodiment of the invention processes the image based on the exposure prediction label output by the exposure labeling model, for example, adjusting the image's exposure parameters. This ensures the rationality of the adjusted exposure parameters and improves the image's display quality.

[0118] In one embodiment of the present invention, the current image may at least include the current image of the video stream, and the method further includes:

[0119] The current image and / or the next frame of the video stream are displayed according to the adjusted exposure parameters.

[0120] In this embodiment of the invention, the current image can be any image, or it can be an image from a series of consecutively captured images. Other images can be the next image or all images in the series. Furthermore, the current image can also be a frame from a video stream, and other images can be the next frame after the current image in the video stream. For ease of explanation, the current image will primarily be an image from a video stream, but this should not be construed as a limitation of the invention.

[0121] In some specific examples, the image processing method of this invention can be applied to vehicle-mounted imaging systems, and of course, it can also be applied to other automatic exposure systems that require automatic exposure. This invention does not impose any limitations on this. For ease of explanation, the following primarily uses vehicle-mounted imaging systems as examples, but this should not be construed as a limitation of the invention.

[0122] Reference Figure 4This is a general framework diagram of a video automatic exposure control imaging system provided in an embodiment of the present invention. The automatic exposure algorithm module (AE control module) relies on the camera image data (video stream / camera data) provided by the Camera API. It calculates and obtains the appropriate exposure parameters (optimal exposure parameters) for the current shooting scene using the AI-AE (Artificial Intelligence-Auto-ExposureAlgorithm) algorithm, and sends this information to the AndroidCameraHAL layer. After obtaining the imaging effect under these exposure parameters, it returns the result to the Camera application service module for image display. Specifically, the various modules of the video automatic exposure control imaging system are shown below:

[0123] AI-AE algorithm: An AI-based automatic exposure algorithm that allows users to control the exposure parameters of the video captured in the current scene;

[0124] Android APP: refers to a specific business application (Application) in the Android system.

[0125] Camera application business module: refers to the various specific camera services in the APP, such as camera preview, video recording, taking pictures, and the data module for the camera screen required by the APP.

[0126] AndroidFramework: Refers to the interface layer provided by the Android system for apps to use system-related functions.

[0127] CameraAPI: refers to the collection of interface classes that provide apps with camera functionality.

[0128] cameraimagedata: Image data captured by the camera (e.g., video stream).

[0129] AndroidCameraHAL: The interaction layer in the Android system that connects the Android Framework layer and the driver layer for camera function operations.

[0130] Solid line with arrow: Direction of function interface call; the arrow points to the location of the called interface.

[0131] Dashed lines with arrows: the direction of video stream data flow.

[0132] Exposure parameters refer to the settings used to control the exposure of images (such as images from video streams) during shooting. Exposure parameters typically include parameters such as aperture, shutter speed, and ISO sensitivity.

[0133] In some embodiments of the present invention, the current image may be an image from a video stream. The video stream may be a video stream captured by an in-vehicle camera (camera / vehicle-mounted camera) based on an in-vehicle imaging system. The video stream may be displayed on a display screen of an in-vehicle device. For example, the in-vehicle camera may capture the environment around the vehicle, including other vehicles, lanes, traffic signs, and obstacles, etc. Specifically, the video stream typically includes multiple frames of images. The image currently acquired in the video stream is the current image, and the image of the next frame after the current image in the video stream can be used as other images that need to be processed.

[0134] In the above embodiments, the current image of the video stream is acquired and input into a trained exposure labeling model. The exposure labeling model then outputs an exposure prediction label for the current image. The exposure parameters of the current image can then be adjusted based on this exposure prediction label, and subsequently, the next image in the video stream can be displayed based on the adjusted exposure parameters. This embodiment of the invention outputs an exposure prediction label for the current image based on the exposure labeling model and adjusts the exposure parameters of the next image based on this label, thereby ensuring the continuity and consistency of the exposure parameters of adjacent images in the video stream and improving the display quality of the video stream.

[0135] In one embodiment of the present invention, the method may further include:

[0136] When the video stream is being monitored to be playing, the current image of the video stream is acquired.

[0137] In this embodiment of the invention, it is possible to monitor in real time whether the video stream of the vehicle imaging system is on (whether it is playing). If the video stream is being played, the current image of the video stream can be obtained, and the exposure parameters of the current image or other images can be adjusted according to the current image.

[0138] In this embodiment of the invention, after adjusting the exposure parameters based on the current image of the video stream, the next image of the video stream can be displayed based on the adjusted exposure parameters. Since this embodiment can obtain the exposure prediction label corresponding to the current image of the video stream through an exposure label model, and the exposure prediction label reflects the exposure state of the current image, such as underexposure, normal exposure, or overexposure, the exposure parameters can be quickly and adaptively adjusted based on the exposure prediction label. Because there is no need to identify the complex shooting scene of the vehicle-mounted camera, the complexity of the data is reduced. Furthermore, after adjusting the exposure parameters according to the exposure prediction label of the current image of the video stream, the next image of the video stream can be displayed immediately based on the adjusted exposure parameters, fully considering the continuity of the shooting scene of the video stream, avoiding obvious inter-frame jumps, and ensuring the video quality of the video stream.

[0139] In one embodiment of the present invention, after acquiring the current image, the method further includes:

[0140] Preprocess the current image;

[0141] The preprocessing includes at least image format conversion, normalization, and image resolution adjustment.

[0142] In this embodiment of the invention, the acquired current image, such as the current image of a video stream acquired through an in-vehicle imaging system, is a color image. Specifically, the color image may include, but is not limited to, image formats such as RGBA, RGB, and YUV. For example, the image format typically used to acquire a color image is a 4-channel RGBA image. After acquiring the current image as a color image, preprocessing can be performed on the current image to improve the efficiency and effectiveness of subsequent analysis, thereby better enabling the adjustment of exposure parameters.

[0143] In some examples, preprocessing may include, but is limited to, image format conversion, normalization, image resolution adjustment, etc. Taking an image in RGBA format as an example, the preprocessing process could be as follows: First, convert the 4-channel I... RGBA Convert to a 3-channel RGB image The conversion process involves extracting the first three channels of the RGBA format image, i.e., I RGB =I RGBA [0:3]; Secondly, I RGB Normalization is performed by dividing by 255 (255 represents the maximum brightness value of each color channel (Red R, Green G, Blue B) in an RGB image) to make I RGB The data range remains between 0 and 1, i.e., I RGB =I RGB / 255.0.

[0144] In the above embodiments, after acquiring the current image, preprocessing it before inputting it into the exposure label model to predict the exposure prediction label can improve the analysis efficiency and effect of the exposure label model on the current image, thereby better completing the adjustment of the image's exposure parameters.

[0145] In one embodiment of the present invention, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. The method further includes:

[0146] Acquire exposure images with different exposure parameters for various shooting scenarios;

[0147] Extract objective indicator data corresponding to multiple objective indicators from the exposed image;

[0148] Generate sample images and corresponding exposure prediction labels for the sample images based on the objective indicator data;

[0149] The exposure label model to be trained is obtained by using the sample image and the exposure prediction label corresponding to the sample image.

[0150] In this embodiment of the invention, exposure images with different exposure parameters for various shooting scenes can be acquired, i.e., bracketed exposure images. Then, objective indicator data corresponding to objective indicators can be extracted from the bracketed exposure images. This embodiment of the invention designs an objective evaluation method that comprehensively considers objective indicators closely related to the content of the image shooting scene. The objective indicators are image color-related information. For example, the objective indicators may include at least image average brightness, contrast, saturation, effective Gaussian exposure, overexposure percentage, and black percentage.

[0151] In a specific example, the calculation formulas for each objective indicator are as follows:

[0152] The formula for calculating the average brightness is:

[0153]

[0154] Dead black, or statistical dark area, is calculated using the following formula:

[0155]

[0156] Overexposure, also known as statistical bright area measurement, is calculated using the following formula:

[0157]

[0158] Effective exposure based on threshold range statistics is calculated using the following formula:

[0159]

[0160] Effective exposure based on Gaussian mapping statistics is calculated using the following formula:

[0161]

[0162] in,

[0163]

[0164] The formula for calculating contrast is:

[0165]

[0166] The formula for calculating saturation is:

[0167]

[0168] Of course, the above objective indicators and their corresponding calculation formulas are merely examples. In practical applications, other objective indicators and their corresponding calculation formulas that are closely related to the image scene content can be comprehensively considered. This embodiment of the invention does not impose any limitations on this.

[0169] After obtaining objective indicator data, multiple sample images and corresponding exposure prediction labels can be generated based on the objective indicator data. Then, the sample images and their corresponding exposure prediction labels can be used to train the model, resulting in a trained exposure label model, which is used to predict the exposure prediction label of the current image for image processing.

[0170] In the above embodiments, exposure images with different exposure parameters under various shooting scenarios are obtained, and objective index data corresponding to multiple objective indicators are extracted from them. A large number of sample images and corresponding exposure prediction labels can be generated as training data to train the exposure label model, which can improve the accuracy and generalization ability of the exposure label model.

[0171] In one embodiment of the present invention, generating a sample image and an exposure prediction label corresponding to the sample image based on the objective indicator data includes:

[0172] Search for target objective indicator data that conforms to the preset exposure value search strategy from the objective indicator data;

[0173] The exposure range is determined based on the objective indicator data of the target.

[0174] Multiple sample exposure parameters are extracted from the exposure parameters in each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images; each sample image has a corresponding exposure prediction label.

[0175] This invention provides an exposure suitability search strategy based on objective indicators. The preset exposure suitability search strategy is a search strategy composed of the objective indicators. For example, the preset exposure suitability search strategy can be a search strategy composed of a single objective indicator (a single-indicator exposure suitability search strategy), or it can be a search strategy composed of multiple objective indicators (a composite indicator exposure suitability search strategy).

[0176] In one specific example, a search strategy for optimal exposure values ​​is set for a single metric:

[0177] The shortest exposure with a black area smaller than a preset threshold is used as the exposure value search strategy.

[0178] I correct_exp =arg min[E dead_black <Threshold_1]

[0179] The longest exposure with an overexposed area smaller than a preset threshold is used as the optimal exposure value search strategy; correct_exp =arg max[E over_exp <Threshold_2]

[0180] The exposure value closest to the average brightness value and the preset threshold is used as the exposure value search strategy;

[0181] I correct_exp =arg min[E gray_mean -Threshold_3]

[0182] The exposure with the largest effective Gaussian exposure area is used as the exposure value search strategy;

[0183] I crrect_exp =argmax[E well_exp_gauss ]

[0184] Use the exposure at the contrast peak as the exposure optimal value search strategy;

[0185] I correct_exp =arg max[E contrast ]

[0186] The exposure at the saturation peak is used as the optimal exposure value search strategy;

[0187] I correct_exp =argmax[E saturation ]

[0188] Set an appropriate exposure search strategy for composite metrics:

[0189] Combine overexposure and average brightness indices to set an exposure search strategy:

[0190] I correct_exp =(argmax[E over_exp <Threshold_2]+argmin[E gray_mean -Threshold_3]) / 2

[0191] Combine Gaussian exposure, contrast, and saturation metrics to set an optimal exposure search strategy:

[0192] I correct_exp =(arg max[E well_exp_gauss ]+arg max[E contrast ]+arg max[E saturation ]) / 3

[0193] A search strategy for optimal exposure value is set by combining Gaussian exposure, contrast, saturation, brightness average, and overexposure metrics:

[0194] I correct_exp =(arg max[E well_exp_gauss ]+arg max[E contrast ]+arg max[E saturation ]+

[0195] arg min[E gray_mean -Threshold_3]+arg max[E over_exp <Threshold_2]) / 5

[0196] The preset exposure suitability search strategy is a preset range interval set according to objective indicators. The objective indicator data must fall within the preset range interval to meet the preset exposure suitability search strategy. In this embodiment, after obtaining the objective indicator data corresponding to the objective indicator, a search can be performed based on the set exposure suitability search strategy to find target objective indicator data that meets the preset exposure suitability search strategy. The target objective indicator data is the objective indicator data of a normally exposed image. The exposed images corresponding to the objective indicator data each have corresponding exposure parameters (exposure amount), and the exposed images can be overexposed, normally exposed, or underexposed images. Corresponding exposure prediction labels can be set for these exposed images, such as overexposed, normally exposed, and underexposed labels.

[0197] This invention allows for the determination of exposure intervals based on objective indicator data. Different exposure intervals describe different exposure states. For example, exposure intervals may include at least overexposure, normal exposure, and underexposure intervals. Objective indicator data can be assigned to an exposure interval based on corresponding exposure prediction labels, thereby determining the exposure interval of the corresponding exposure image. Subsequently, multiple sample exposure parameters can be extracted from the exposure parameters of the exposure images in each exposure interval. The corresponding exposure images are then adjusted according to these sample exposure parameters to obtain sample images. Each sample image has a corresponding exposure prediction label, and the sample images and their corresponding exposure prediction labels can be used as training data for model training.

[0198] In one embodiment of the present invention, determining the exposure range based on the target objective indicator data includes:

[0199] Obtain the exposure parameters of the exposure image corresponding to the target objective index data;

[0200] The exposure range is determined based on the exposure parameters of the exposure image corresponding to the objective indicator data of the target.

[0201] In this embodiment of the invention, the exposed images may each have corresponding objective index data and exposure parameters. After obtaining the target objective index data, since the target objective index data is the objective index data corresponding to the normally exposed image, the exposure parameters of the exposed image corresponding to the target objective index data can be obtained. Then, based on the exposure parameters of the exposed image corresponding to the target objective index data, one or more exposure ranges can be determined. For example, the normal exposure range can be determined based on the exposure parameters of the exposed image corresponding to the target objective index data. Furthermore, the underexposed range, overexposed range, or underexposed range and overexposed range can also be determined based on the target objective index data.

[0202] In one embodiment of the present invention, determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data includes:

[0203] Obtain the maximum and minimum values ​​of the exposure parameters of the exposure image corresponding to the target objective index data, and set the normal exposure range based on the maximum and minimum values ​​of the exposure parameters.

[0204] In this embodiment of the invention, after determining the target objective index data, the maximum and minimum values ​​of the exposure parameters of the exposure image corresponding to the target objective index data are obtained. Then, the maximum value is taken as TH_EV_MAX and the minimum value is taken as TH_EV_MIN, thereby determining the optimal exposure range (normal exposure range).

[0205] Pbest ∈[TH_EV_MIN, TH_EV_MAX]

[0206] Where TH EV MAX is the maximum EV threshold (maximum exposure value threshold) of the searched optimal exposure range, and THEV MIN is the minimum EV threshold (minimum exposure value threshold) of the searched optimal exposure range.

[0207] In one embodiment of the present invention, determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data further includes:

[0208] The overexposure range is determined based on the maximum value of the exposure parameters of the exposed image corresponding to the target objective index data;

[0209] Alternatively, the underexposure range can be determined based on the minimum value of the exposure parameters of the exposed image corresponding to the objective index data of the target.

[0210] Alternatively, set the overexposure range and / or underexposure range based on the normal exposure range.

[0211] In this embodiment of the invention, after determining the normal exposure range, overexposure range and / or underexposure range can also be determined simultaneously. For example, the normal exposure range can first be determined based on the maximum and minimum values ​​of the exposure parameters of the exposed image corresponding to the target objective index data.

[0212] P best ∈[TH_EV_MIN, TH_EV_MAX]

[0213] Furthermore, the overexposure range is determined based on the maximum value of the exposure parameters of the exposed image corresponding to the target objective index data.

[0214] P over_exp ∈[TH-EV_MAX, +∞], or, determine the underexposure range based on the minimum value of the exposure parameters of the exposed image corresponding to the target objective index data.

[0215] P under_exp ∈[-∞, TH_EV_MIN], or, determine the overexposure range based on the maximum value of the exposure parameters of the exposed image corresponding to the objective index data of the target.

[0216] P over_exp The underexposure interval P is determined by the minimum value of the exposure parameters of the exposed image corresponding to the target objective index data, ∈[TH_EV_MAX, +∞]. under_exp ∈[-∞, TH_EV_MIN].

[0217] In this embodiment of the invention, the exposure range can be determined based on the exposure parameters of the exposure image corresponding to the target objective index data, namely, overexposure range, normal exposure range, and underexposure range. Then, multiple sample exposure parameters are further extracted from the exposure parameters of the exposure image in each exposure range, and the corresponding exposure images are adjusted according to these sample exposure references to obtain sample images. Finally, the sample images and their corresponding exposure prediction labels can be used to form a training sample set (training data) to train the exposure label model (machine learning model). When a preset convergence condition is reached (e.g., the loss value of the exposure label model reaches a preset loss value, or the number of iterations of the model reaches a preset number of iterations, etc.), the trained exposure label model is obtained. Subsequently, the exposure label model can output the corresponding exposure prediction label based on the input image (e.g., the current image) to quickly and adaptively adjust the exposure parameters of the image based on the exposure prediction label, thereby improving the image quality of the displayed image.

[0218] In one embodiment of the present invention, extracting objective indicator data corresponding to multiple objective indicators from the exposed image includes:

[0219] The exposed image is processed according to the first step length parameter to obtain a densely exposed image of any exposure level;

[0220] Extract objective indicator data corresponding to multiple objective indicators from the densely exposed image.

[0221] The step size parameter (first step step size parameter) is the amount of data that changes each time during the increment or decrement process, and it can be a fixed value.

[0222] In this embodiment of the invention, the exposed image can be processed according to the first step length parameter. For example, the weighting value of the first step length parameter can be increased or decreased based on the exposure parameters corresponding to the exposed image, thereby obtaining a densely exposed image of any exposure level. For instance, assuming the original exposure parameter of an exposed image is 10 and the first step length parameter is 1, a densely exposed image of any exposure level can be obtained. The exposure parameters corresponding to the densely exposed image can be 10+1, 10+2, 10+3, etc. After obtaining the densely exposed image, objective index data corresponding to multiple objective indicators are extracted from the densely exposed image for subsequent determination of the exposure range.

[0223] In one embodiment of the present invention, before processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels, the method further includes:

[0224] The exposed image is converted using an inverse tone mapping algorithm to obtain a high dynamic range image;

[0225] The step of processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels includes: processing the high dynamic range image according to the first step length parameter to obtain densely exposed images of arbitrary exposure levels.

[0226] In the field of image processing, dynamic range refers to the range of light exposure values ​​in a shooting scene that an image can capture. The dynamic range of natural scenes that the human eye can observe can reach 10,000:1. Ordinary cameras can often only shoot images with a limited low dynamic range (such as 100:1 to 300:1). High dynamic range images have a wider dynamic range and can more realistically reproduce the light and shadow effects of the real scene. Therefore, they can produce images or videos with richer layers, more realistic images, and higher quality.

[0227] In this embodiment of the invention, after acquiring bracketed exposure image sequences of various shooting scenes...

[0228] imgs = {img1, img2, ...}

[0229] That is, after exposing the image, the Inverse Tone Mapping (iTMO) algorithm can be used to perform bracketing exposure fusion on the exposed image to obtain a high-quality high dynamic range image (HDR data):

[0230] hdr=iTMO(img1, img2,...)

[0231] Then, a sequence of densely exposed images at any exposure level, i.e., densely exposed images (LDR data), can be obtained using HDR data and automatic exposure compensation:

[0232] ldr i =TMO(hdr, ev) i )

[0233] One method for automatically setting exposure compensation is to use the TMO algorithm to process HDR data step by step according to a preset first-step length parameter. For example, the first-step length parameter can be increased or decreased for each HDR data step to obtain a densely exposed image at any exposure level.

[0234] Finally, objective indicator data corresponding to objective indicators can be obtained from densely exposed images.

[0235] In one embodiment of the present invention, searching for target objective indicator data that conforms to a preset exposure suitability search strategy from the objective indicator data includes:

[0236] According to the second step length parameter, the target objective indicator data that meets the preset exposure value search strategy is searched step by step from the objective indicator data.

[0237] In this embodiment of the invention, the preset exposure appropriate value search strategy can be a search strategy composed of a single objective indicator (single indicator exposure appropriate value search strategy), or the preset exposure appropriate value search strategy can be a search strategy composed of multiple objective indicators (composite indicator exposure appropriate value search strategy). If the preset exposure appropriate value search strategy is a search strategy composed of a single objective indicator, the target objective indicator data that conforms to the preset exposure appropriate value search strategy can be directly searched from the objective indicator data.

[0238] If the preset exposure fit search strategy is a search strategy composed of multiple objective indicators, then it is necessary to find the best objective indicator data that simultaneously satisfies multiple objective indicators. Therefore, based on the preset second step size parameter (e.g., 0.01EV), the target objective indicator data that conforms to the preset exposure fit search strategy can be accurately searched from the objective indicator data step by step.

[0239] In one embodiment of the present invention, searching for target objective indicator data that conforms to a preset exposure suitability search strategy from the objective indicator data includes:

[0240] According to the second step length parameter, the target objective indicator data that conforms to the preset exposure value search strategy and is within the preset exposure compensation range is searched step by step from the objective indicator data.

[0241] In this embodiment of the invention, exposure suitability search can be performed within a preset exposure compensation range (e.g., -5EV to 5EV) with a predetermined second step size parameter (e.g., 0.01EV) to accurately search for target objective index data that conforms to the preset exposure suitability search strategy and is within the preset exposure compensation range from the objective index data.

[0242] In one embodiment of the present invention, multiple sample exposure parameters are extracted from the exposure parameters of each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images, including:

[0243] According to the third step length parameter, multiple sample exposure parameters of different exposure levels are extracted from the exposure parameters of each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters of different exposure levels to obtain sample images of different exposure levels in each exposure range and the exposure prediction labels corresponding to the sample images.

[0244] To avoid the bias of photometering in evaluating image quality and the inconsistency of subjective exposure assessments, this invention employs a more comprehensive exposure evaluation system and objective exposure evaluation methods to label exposure data, thereby obtaining a labeled exposure dataset. The comprehensive indicator system covers multiple aspects of exposure, including image performance in terms of brightness distribution (overall image brightness, local brightness such as overexposure and black areas, effective exposure, etc.), contrast, and color accuracy (color saturation). By designing objective calculation methods for each image quality indicator, objective exposure evaluation is achieved.

[0245] In this embodiment of the invention, after determining the exposure range based on the target objective index data, multiple sample images of different exposure levels under each exposure range can be generated according to a preset third step size parameter (e.g., 0.33EV). Then, an exposure dataset can be obtained based on the sample images corresponding to different exposure levels under each exposure range and the exposure prediction labels corresponding to the sample images, which can be used to train the label prediction model to be trained. For example, assuming there are 3 exposure levels, the underexposure range can include 3 exposure levels: level 1, level 2, and level 3.

[0246] In one embodiment of the present invention, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image, including:

[0247] The current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features;

[0248] The hierarchical features are input into the logistic regression layer of the exposure label model to obtain the exposure prediction label of the current image.

[0249] In this embodiment of the invention, the exposure labeling model may include multiple feature extraction layers and a logistic regression layer. The feature extraction layers can extract multi-level features, thereby improving the expressive power of the features. The logistic regression layer, acting as a classifier, classifies the image using the features extracted by the feature layers and provides guidance for exposure adjustment. The number of feature extraction layers can be designed according to actual needs. After acquiring the current image, the current image can be input into the feature extraction layers of the exposure labeling model to obtain hierarchical features. Then, the hierarchical features are input into the logistic regression layer of the exposure labeling model to obtain the exposure prediction label for the current image, and the image is then processed based on the exposure prediction label.

[0250] In one embodiment of the present invention, the current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features, including:

[0251] The current image is input into the shallow feature extraction layer of the exposure label model to obtain high-dimensional features;

[0252] The high-dimensional features are input into the deep feature extraction layer of the exposure label model to obtain deep features.

[0253] In this embodiment of the invention, to meet the real-time requirements of video streaming shooting scenarios and to mitigate limitations imposed by in-vehicle computing resources, a three-stage lightweight deep learning model is designed. By training the deep learning model with a labeled exposure dataset, the model can learn appropriate exposure levels for various shooting scenarios through its scene awareness capabilities, thereby guiding exposure control.

[0254] Reference Figure 5 This diagram illustrates a three-stage deep learning model provided in this embodiment of the invention. The network structure of the three-stage deep learning model mainly consists of a shallow feature extraction layer, a deep feature extraction layer, and a logistic regression layer. The shallow feature extraction layer can learn local features, such as overexposure, black levels, contrast, and color saturation. The deep feature extraction layer can learn global features, such as the overall brightness distribution of the image and effective exposure, by increasing its receptive field. The logistic regression layer acts as a classifier, classifying the image based on the features extracted by the feature layers and providing guidance for exposure adjustment. The receptive field of the model refers to the size of the input data region that a specific neuron in the network can "see."

[0255] Meanwhile, since lightweight network structures cannot fit complex data, a relative label design principle was adopted in the label design. For example, exposure prediction labels (labels) can include underexposure labels, normal exposure labels, and overexposure labels. During the exposure adjustment stage, the exposure parameters are adjusted based on the exposure prediction labels given by the AI-AE algorithm of the deep learning model. This reduces the complexity of data processing and also takes into account the continuity of the shooting scene of the video stream, avoiding obvious inter-frame jumps in the video stream.

[0256] In this embodiment of the invention, I is obtained by preprocessing the current image of the video stream. RGB Then, first I RGB The image is fed into a shallow feature extraction module. This shallow feature extraction layer can consist of convolutional layers, ReLU activation function layers, and batch normalization (BN) layers. The shallow feature extraction layer can extract the 3-channel image I... RGB Mapped to high-dimensional features F of the C channel Shallow At the same time, it extracts some low-level features of the current image (such as color and texture), providing a higher-level abstract feature foundation for the deep feature extraction layer, which is beneficial to improving the performance of machine learning models in tasks.

[0257] Next, the high-dimensional feature FShallow The data is fed into a deep feature extraction layer to extract deep features F. Deep The deep feature extraction layer consists of multiple feature extraction modules. It can improve the model's receptive field and learn more abstract and high-level feature representations.

[0258] Finally, the deep features F Deep The image is fed into a logistic regression layer, which outputs the exposure prediction label I corresponding to the current image. Result According to exposure prediction tag I Result The corresponding preset exposure parameters (hyperparameters, i.e. step parameters) are obtained. After analyzing the exposure parameters of the current image, the exposure parameters are adjusted to preview and display the next image based on the adjusted exposure parameters.

[0259] In the above embodiments, the current image is input into the feature extraction layer and logistic regression layer of the exposure label model to output the exposure prediction label of the current image, which improves the accuracy and computational efficiency of the exposure prediction label prediction, thereby improving the display quality of the image.

[0260] Reference Figure 6 This is a basic flowchart of an AI-based automatic video exposure control algorithm provided in an embodiment of the present invention. The specific exposure control process includes:

[0261] First, the AE control module determines whether the video stream is on. If it is on, it means the in-vehicle device is playing a video stream. The current image of the video stream is then preprocessed. After processing by the AI ​​module (exposure prediction model), an exposure prediction label is output. Based on the output exposure prediction label, corresponding exposure parameters are adjusted, such as increasing, decreasing, or maintaining the exposure. After adjustment, the exposure parameters are sent out so that the next image of the video stream is displayed based on these parameters. After sending out the parameters, the system continues to check whether the video stream is on. If it is on, the above steps are repeated; if the video stream is not on, the system continues to monitor its on / off status.

[0262] For example, the exposure prediction label may include a first label, a second label, and a third label. The first label is an underexposure label, the second label is a normal exposure label, and the third label is an overexposure label. If the AI ​​module outputs the first label, the automatic exposure system enters the exposure enhancement stage; if the AI ​​module outputs the second label, the automatic exposure system determines that it is normal exposure and does not adjust the current exposure parameters; if the AI ​​module outputs the third label, the automatic exposure system enters the exposure reduction stage.

[0263] After the AE exposure control of the current image in the video stream ends, the next image of the video stream is obtained using the adjusted exposure parameters and sent back to the AE control module.

[0264] To adjust the image exposure level to a suitable range, it is necessary to first determine the current image exposure value. (Refer to...) Figure 7 Here is a flowchart of an exposure determination process provided in an embodiment of the present invention. The specific exposure determination process is as follows:

[0265] In this embodiment of the invention, after obtaining the exposure prediction label of the current image output by the label prediction model, a judgment is made. Specifically, if the label is the first label (underexposed label), it means that the current image is underexposed, and the exposure is increased; if the label is the second label (normal exposure label), it means that the current image is normally exposed, and the current exposure and exposure parameters are maintained; if the label is the third label (overexposed label), it means that the current image is overexposed, and the exposure is reduced.

[0266] In one embodiment of the present invention, the current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. The exposure prediction label is used for image processing, including:

[0267] The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label and exposure level of the current image, and the exposure prediction label and exposure level are used for image processing;

[0268] The current image is then input into the trained exposure label model, so that the exposure label model outputs an exposure prediction label for the current image. This exposure prediction label is used for image processing. The method further includes:

[0269] The exposure parameters are adjusted based on the exposure prediction label and the exposure level of the current image, which are used for image processing.

[0270] In practical implementation, when an image is underexposed or overexposed, its display quality is poor. However, in reality, achieving normal exposure may require adjusting the exposure parameters by a small or large margin. Therefore, adjusting the exposure parameters too much or too little may result in exceeding the normal exposure parameters. Thus, in this embodiment of the invention, the exposure label model outputs not only an exposure prediction label but also an exposure level. Based on the exposure level, it can be specifically determined how much the exposure parameters need to be reduced or increased. In this way, image processing can be performed based on the exposure prediction label and the exposure level, enabling the adjusted exposure parameters to achieve better image display quality when used to display the image.

[0271] In the above embodiments, the exposure label model can output an exposure prediction label and an exposure level. Thus, the exposure parameters can be adjusted together based on the exposure prediction label and the exposure level, resulting in a higher image quality for the image displayed based on the adjusted exposure parameters.

[0272] In one embodiment of the present invention, adjusting exposure parameters based on the exposure prediction label of the current image and / or the exposure level includes:

[0273] When the exposure prediction label of the current image is an underexposure label, obtain the underexposure parameters corresponding to the underexposure label;

[0274] The exposure parameters of the current image are increased based on the underexposure parameters and the exposure level to obtain the adjusted exposure parameters.

[0275] Reference Figure 8 This invention provides a flowchart for increasing exposure. The process begins by obtaining the current exposure level of the current image, i.e., the exposure parameters of the current image. Next, it reads the preset hyperparameters, i.e., the step size parameter (underexposure parameter) for increasing exposure. Using the current exposure level, combined with the step size parameter and the exposure level, the required exposure parameters are calculated. After calculation, the exposure parameters are output to adjust the next image in the video stream. For example, assuming the exposure label model outputs "-2", the exposure prediction label output by the exposure label model is determined to be an underexposure label, and the exposure level corresponding to the underexposure label is 2. Therefore, it is determined that two step size parameters need to be added to the current exposure level to calculate the required exposure parameters.

[0276] The underexposure parameter can be the same as the first step length parameter used in training the exposure label model. It can be understood that during the training of the exposure label model, the first step length parameter determines the step size of the exposure label model when adjusting the exposure parameters. If the underexposure parameter is the same as the first step length parameter, that is, the step size of the adjustment during underexposure will be consistent with the step size during model training, thereby ensuring the stability and predictability of the exposure parameter adjustment, and thus ensuring the display quality of the image.

[0277] It should be noted that if the required precision for adjusting the exposure parameters is low, and it is only necessary to ensure that the image is not underexposed, then the exposure parameters of the current image can be increased based on the underexposure parameters to obtain the adjusted exposure parameters. This embodiment of the invention does not impose any restrictions on this.

[0278] In the above embodiments, if the exposure prediction label of the current image is an underexposure label, it means that the image display quality is poor and it is in an underexposure state. In this case, the exposure parameter can be increased. Displaying the image based on the increased exposure parameter can prevent the image from being underexposure, thereby ensuring the display quality of the image.

[0279] In one embodiment of the present invention, adjusting exposure parameters based on the exposure prediction label of the current image and / or the exposure level includes:

[0280] When the exposure prediction label of the current image is an overexposure label, obtain the overexposure parameters corresponding to the overexposure label;

[0281] The exposure parameters of the current image are reduced based on the overexposure parameters and the exposure level to obtain the adjusted exposure parameters.

[0282] Reference Figure 9 This invention provides a flowchart for reducing exposure. The process begins by obtaining the current exposure level of the current image, i.e., the exposure parameters of the current image. Next, pre-set hyperparameters are read, i.e., the step size parameter (overexposure parameter) for reducing exposure is obtained. Finally, the current exposure level, combined with the step size parameter and the exposure level, is used to calculate the required exposure parameters. After calculation, the exposure parameters are output to adjust the next image in the video stream. For example, if the exposure label model outputs "+1", it can be determined that the exposure prediction label output by the exposure label model is an overexposure label, and the exposure level corresponding to the overexposure label is 1. Therefore, it can be determined that the step size parameter needs to be reduced by 1 from the current exposure level to calculate the required exposure parameters.

[0283] The overexposure parameter can be the same as the first step length parameter used in training the exposure label model. This means that during the training of the exposure label model, the first step length parameter determines the step size of the exposure label model when adjusting the exposure parameters. If the overexposure parameter is the same as the first step length parameter, that is, the step size of the adjustment when overexposure occurs will be consistent with the step size during model training, thereby ensuring the stability and predictability of the exposure parameter adjustment, and thus guaranteeing the display quality of the image.

[0284] It should be noted that if the required precision for adjusting the exposure parameters is low, and it is only necessary to ensure that the image is not overexposed, then the exposure parameters of the current image can be reduced based on the overexposure parameters to obtain the adjusted exposure parameters. This embodiment of the invention does not impose any restrictions on this.

[0285] In the above embodiments, if the exposure prediction label of the current image is an overexposure label, it means that the image display quality is poor and it is in an overexposed state. The exposure parameter can be reduced, and the image can be displayed based on the reduced exposure parameter so that the image will not be overexposed, thereby ensuring the display quality of the image.

[0286] In one embodiment of the present invention, adjusting exposure parameters based on the exposure prediction label of the current image and / or the exposure level includes:

[0287] When the exposure prediction label of the current image is the normal exposure label, the exposure parameters of the current image are used as the adjusted exposure parameters.

[0288] In this embodiment of the invention, when the exposure prediction label of the current image is the normal exposure label, that is, the automatic exposure system determines that the exposure is normal, the exposure parameters of the current image do not need to be adjusted, that is, the exposure parameters of the current image can continue to be used.

[0289] In the above embodiments, if the current image's exposure prediction label is the normal exposure label, it means that a good image display effect has been achieved at this time, and there is no need to adjust the exposure parameters, thus ensuring the image display quality.

[0290] By applying the embodiments of this invention, a real-time, deep learning-based automatic exposure method for video streams can be implemented, improving the exposure control process of video streams: 1) A deep learning-based automatic exposure system: Thanks to the scene perception capability of deep learning algorithms, the automatic exposure system has stronger anti-interference capabilities in complex situations. It can stably output appropriate exposure values ​​for various complex shooting scenarios, such as different weather conditions, scenes with high contrast lighting intensity, and scenes with high-frequency changes. 2) In current traditional algorithms, exposure levels rely on subjective adjustments by engineers, resulting in poor continuity and consistency of exposure values. The automatic exposure method for video streams proposed in this invention designs an objective evaluation standard. By setting multiple exposure prediction labels, the exposure prediction labels output by the machine learning model (exposure label model) constrain the video stream, thereby solving the problem of poor consistency in exposure levels between images in the video stream and improving the display quality of the video stream images.

[0291] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0292] It should also be noted that the embodiments of the present invention may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0293] This invention also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0294] Memory 1003 is used to store computer programs;

[0295] When the processor 1001 executes the program stored in the memory 1003, it implements any of the image processing methods described in the above embodiments:

[0296] Get the current image;

[0297] The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label of the current image, and the exposure prediction label is used for image processing.

[0298] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0299] The communication interface is used for communication between the aforementioned terminal and other devices.

[0300] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0301] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0302] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the image processing methods described in the above embodiments.

[0303] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image processing methods described in the above embodiments.

[0304] In another aspect of the present invention, an in-vehicle imaging system is also provided, including the image processing method described above.

[0305] In another aspect of the present invention, a vehicle is also provided, which implements any of the image processing methods described above.

[0306] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0307] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0308] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0309] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, The method includes: Get the current image; The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label of the current image, and the exposure prediction label is used for image processing.

2. The method according to claim 1, characterized in that, The method further includes inputting the current image into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. Acquire exposure images with different exposure parameters for various shooting scenarios; Extract objective indicator data corresponding to multiple objective indicators from the exposed image; Generate sample images and corresponding exposure prediction labels for the sample images based on the objective indicator data; The exposure label model to be trained is obtained by using the sample image and the exposure prediction label corresponding to the sample image.

3. The method according to claim 2, characterized in that, Generate sample images and corresponding exposure prediction labels for the sample images based on the objective indicator data, including: Search for target objective indicator data that conforms to the preset exposure value search strategy from the objective indicator data; The exposure range is determined based on the objective indicator data of the target. Multiple sample exposure parameters are extracted from the exposure parameters in each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images; each sample image has a corresponding exposure prediction label.

4. The method according to claim 3, characterized in that, The objective indicators are image color-related information; the objective indicators include at least one of the following: average image brightness, contrast, saturation, effective Gaussian exposure, overexposure percentage, and black percentage. The preset exposure optimal value search strategy is a search strategy composed of the objective indicators; the preset exposure optimal value search strategy includes at least a search strategy composed of a single objective indicator, and / or, multiple or all of the objective indicators.

5. The method according to claim 3, characterized in that, The exposure range includes multiple ranges describing different exposure states; the exposure range includes at least an overexposure range, a normal exposure range, and an underexposure range, and the exposure prediction label includes at least an overexposure label, a normal exposure label, and an underexposure label.

6. The method according to claim 3, characterized in that, Determining the exposure range based on the target objective indicator data includes: Obtain the exposure parameters of the exposure image corresponding to the target objective index data; The exposure range is determined based on the exposure parameters of the exposure image corresponding to the objective indicator data of the target.

7. The method according to claim 6, characterized in that, Determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data includes: Obtain the maximum and minimum values ​​of the exposure parameters of the exposed image corresponding to the target objective index data, and set the normal exposure range based on the maximum and minimum values ​​of the exposure parameters.

8. The method according to claim 7, characterized in that, The step of determining the exposure range based on the exposure parameters of the exposure image corresponding to the target objective index data further includes: The overexposure range is determined based on the maximum value of the exposure parameters of the exposed image corresponding to the target objective index data; Alternatively, the underexposure range can be determined based on the minimum value of the exposure parameters of the exposed image corresponding to the objective index data of the target. Alternatively, set the overexposure range and / or underexposure range based on the normal exposure range.

9. The method according to claim 2, characterized in that, Extract objective indicator data corresponding to multiple objective indicators from the exposed image, including: The exposed image is processed according to the first step length parameter to obtain a densely exposed image of any exposure level; Extract objective indicator data corresponding to multiple objective indicators from the densely exposed image.

10. The method according to claim 9, characterized in that, Before processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels, the method further includes: The exposed image is converted using an inverse tone mapping algorithm to obtain a high dynamic range image; The step of processing the exposed image according to the first step length parameter to obtain densely exposed images of different exposure levels includes: processing the high dynamic range image according to the first step length parameter to obtain densely exposed images of arbitrary exposure levels.

11. The method according to claim 3, characterized in that, Searching for target objective indicator data that conforms to the preset exposure suitability search strategy from the objective indicator data includes: According to the second step length parameter, the target objective indicator data that meets the preset exposure value search strategy is searched step by step from the objective indicator data.

12. The method according to claim 3, characterized in that, Searching for target objective indicator data that conforms to the preset exposure suitability search strategy from the objective indicator data includes: According to the second step length parameter, the target objective indicator data that conforms to the preset exposure value search strategy and is within the preset exposure compensation range is searched step by step from the objective indicator data.

13. The method according to claim 3, characterized in that, The preset exposure optimal value search strategy is a preset range interval set according to the objective indicators respectively, and the objective indicator data falls within the preset range interval if the preset exposure optimal value search strategy is met.

14. The method according to claim 3, characterized in that, Multiple sample exposure parameters are extracted from the exposure parameters of each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters to obtain sample images, including: According to the third step length parameter, multiple sample exposure parameters of different exposure levels are extracted from the exposure parameters of each exposure range, and the corresponding exposure images are adjusted according to the sample exposure parameters of different exposure levels to obtain sample images of different exposure levels in each exposure range and the exposure prediction labels corresponding to the sample images.

15. The method according to claim 1, characterized in that, Inputting the current image into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image, includes: The current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features; The hierarchical features are input into the logistic regression layer of the exposure label model to obtain the exposure prediction label of the current image.

16. The method according to claim 15, characterized in that, The current image is input into the feature extraction layer of the exposure label model to obtain hierarchical features, including: The current image is input into the shallow feature extraction layer of the exposure label model to obtain high-dimensional features; The high-dimensional features are input into the deep feature extraction layer of the exposure label model to obtain deep features.

17. The method according to claim 16, characterized in that, The high-dimensional features are those related to overexposure percentage, black percentage, contrast, and saturation; the deep features are those related to average image brightness and effective Gaussian exposure.

18. The method according to claim 1, characterized in that, The current image is input into a trained exposure labeling model, which outputs an exposure prediction label for the current image. This exposure prediction label is used for image processing, including: The current image is input into the trained exposure label model, so that the exposure label model outputs the exposure prediction label and exposure level of the current image, and the exposure prediction label and exposure level are used for image processing.

19. The method according to claim 18, characterized in that, The current image is input into a trained exposure labeling model, so that the exposure labeling model outputs an exposure prediction label for the current image. This exposure prediction label is used for image processing. The method further includes: The exposure parameters are adjusted based on the exposure prediction label and the exposure level of the current image, which are used for image processing.

20. The method according to claim 19, characterized in that, Adjusting exposure parameters based on the current image's exposure prediction label and the exposure level includes: When the exposure prediction label of the current image is an underexposure label, obtain the underexposure parameters corresponding to the underexposure label; The adjusted exposure parameters are obtained by increasing the exposure parameters of the current image based on the underexposure parameters and / or the exposure level.

21. The method according to claim 19, characterized in that, Adjusting exposure parameters based on the current image's exposure prediction label and the exposure level includes: When the exposure prediction label of the current image is an overexposure label, obtain the overexposure parameters corresponding to the overexposure label; The exposure parameters of the current image are reduced based on the overexposure parameters and the exposure level to obtain the adjusted exposure parameters.

22. The method according to claim 19, characterized in that, Adjusting exposure parameters based on the current image's exposure prediction label and the exposure level includes: When the exposure prediction label of the current image is the normal exposure label, the exposure parameters of the current image are used as the adjusted exposure parameters.

23. The method according to any one of claims 18-22, characterized in that, The current image includes at least the current image of the video stream, and the method further includes: The current image and / or the next frame of the video stream are displayed according to the adjusted exposure parameters.

24. The method according to claim 23, characterized in that, Before adjusting the exposure parameters based on the exposure prediction label of the current image, the method further includes: When the video stream is being monitored to be playing, the current image of the video stream is acquired.

25. The method according to claim 1, characterized in that, The method further includes: Preprocess the current image; The preprocessing includes at least image format conversion, normalization, and image resolution adjustment.

26. A vehicle-mounted imaging system, characterized in that, Used to implement the steps of the method as described in any one of claims 1-25.

27. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-25.

28. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-25.

29. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 27.

30. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1-25.

Images