Image enhancement method and device

Abstract

An image enhancement method comprises: generating an input image by concatenating an original input image with a depth map. The method comprises: encoding the input image using an encoder to generate bottleneck features; and injecting a perturbation vector into the bottleneck features. Feeding the bottleneck features injected with the perturbation vector to an image generator. The method further comprises: generating an enhanced image at the image generator based on the bottleneck features and the perturbation vector. The method further comprises: receiving the enhanced image and a clear image randomly selected from a clear image dataset at a discriminator, and determining an image enhancement score based on a comparison between the enhanced image and the randomly selected clear image. The method can provide multiple output images, reduce processing complexity, and improve the quality of the output images.

Description

Technical Field

[0001] The present invention relates generally to the fields of computer vision and machine learning, and more particularly to a method and apparatus for performing multimodal image enhancement in an unsupervised manner. Background Art

[0002] Currently, processing images captured under different weather and lighting conditions (e.g., clear, rainy, foggy, dark, blurry, or snowy conditions) for image enhancement is a prominent technical challenge. For example, in autonomous driving, capturing images under different weather and lighting conditions and performing image enhancement on such images in real-time or near-real time to facilitate safe autonomous driving is a significant challenge. This is because many features of such images that contribute to perception are typically not apparent due to different weather and lighting conditions.

[0003] Currently, certain methods have been proposed to process such images. For example, conventional methods use paired data of an input image and its corresponding output clear image to train a conventional model, which is used to map the input image to its corresponding output clear image. However, using labeled datasets (e.g., input images and their corresponding output clear images) to perform such supervised learning on conventional models requires considerable effort. Another conventional image dehazing method exists, which is based on a traditional atmospheric scattering model (i.e., a physics-based model) rather than the paired data used in supervised learning. However, conventional image dehazing methods are task-specific and difficult to generalize to other image enhancement tasks, such as image deblurring and low-light enhancement. Furthermore, conventional image enhancement methods only obtain a single output image from the input image, resulting in a small amount of information and low efficiency, thus limiting their use. Because existing methods are complex, inefficient, and task-specific image enhancement methods, they are of limited use and unsuitable for holistic image enhancement. Therefore, there is a technical problem regarding how to fully and holistically enhance images captured under different weather and lighting conditions for practical applications, such as promoting safe autonomous driving.

[0004] Therefore, based on the above discussion, there is a need to overcome the above-mentioned shortcomings associated with traditional image enhancement methods. Summary of the Invention

[0005] The present invention provides a multimodal image enhancement method and apparatus to facilitate safe autonomous driving. Because existing methods are complex, inefficient, and task-specific image enhancement methods with limited use and unsuitable for overall image enhancement, the present invention provides a solution to the existing problem of how to fully and holistically enhance images captured under different weather and lighting conditions. The present invention aims to provide a solution that at least partially addresses the problems encountered in the prior art and to provide a multimodal image enhancement method and apparatus that can fully and holistically enhance images captured under different weather and lighting conditions for various practical applications, such as facilitating safe autonomous driving.

[0006] The objects of the invention are achieved by the measures presented in the appended independent claims. Advantageous implementations of the invention are further defined in the dependent claims.

[0007] In one aspect, the present invention provides an image enhancement method, wherein the method includes: generating an input image by concatenating an original input image with a sharpened attention map or a depth map. The method also includes: generating a bottleneck feature by encoding the input image using an encoder. The method also includes: injecting a perturbation vector into the bottleneck feature. The method also includes: feeding the bottleneck feature injected with the perturbation vector to an image generator. The method also includes: generating an enhanced image at the image generator based on the bottleneck feature and the perturbation vector. The method also includes: receiving the enhanced image and a clear image randomly selected from a clear image dataset at a discriminator, and determining an image enhancement score based on a comparison between the enhanced image and the randomly selected clear image.

[0008] The disclosed method generates a controllable solution space (or a set of multiple output images) for image enhancement, rather than generating only one non-optimal output image. By accounting for the uncertainty in the original input image, the controllable solution space can be created more rationally and allows the search for the optimal solution (e.g., an improved and enhanced output image). Therefore, the disclosed method provides multiple output images and enables processing of the optimal output image, thereby exhibiting higher reliability and efficiency. The disclosed method uses the sharpened attention map or the depth map and thus provides better guidance for generating clearer and brighter output images. In addition, the disclosed method provides a unified approach for multiple image enhancement tasks (e.g., dehazing, deblurring, low-light enhancement, etc.), processing images captured under different weather and lighting conditions (e.g., foggy, clear, rainy, dark, or snowy environments) to perform unsupervised, controllable overall image enhancement on such images. For example, in an exemplary practical application, the method makes various image features of such enhanced images very prominent, which aid in perception, even in different weather and lighting conditions, to promote safe autonomous driving.

[0009] In one implementation, the method further includes: feeding back the discriminant score to the encoder and the image generator.

[0010] The image quality is gradually improved by feeding the discriminant scores back to the encoder and the image generator. In addition, a perceptual loss is calculated between the original input image and the enhanced image based on a pre-trained convolutional neural network (CNN) such as the VGG neural network to preserve structure at the feature level.

[0011] In another implementation, the disturbance vector is sampled according to a Gaussian distribution.

[0012] The sampling of the perturbation vector according to a Gaussian distribution results in generating a multimodal output (or multiple output images), thereby creating a solution space of output images for improved image enhancement.

[0013] In another implementation, the perturbation vector is updated according to pre-trained network weights.

[0014] The perturbation vector is used to determine the optimal output image (i.e., improved output) from the controllable solution space of the output image. During training, the perturbation vector has no predetermined network weights and acts as a random vector sampled from a Gaussian distribution at each training step. However, after the training is completed, the perturbation vector is updated according to the pre-trained (i.e., fixed) network weights and then adjusted (or fine-tuned) to search for the optimal solution (i.e., improved output image).

[0015] In another implementation, gradient descent is used to adjust the perturbation vector by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score.

[0016] The perturbation vector is randomly initialized and then adjusted using gradient descent by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score, wherein the adjusted (or fine-tuned) perturbation vector results in generation of an improved or most visually pleasing image.

[0017] In another implementation, the discriminator is a gradient-based multi-patch discriminator.

[0018] Using the described gradient-based multi-patch discriminator will further improve the quality of the output image.

[0019] In another implementation, the discriminator includes at least the following three network branches: a first network branch for collecting the output of a Gaussian blur generator; a second network branch for collecting an identity image; and a third network branch for collecting the output of a Gaussian Laplacian operator blur generator; wherein the result of the discriminator is obtained after summing the outputs generated by the three branches after each convolutional layer.

[0020] Using the three network branches of the discriminator can provide improved lighting control and sharp edge information in the enhanced image.

[0021] On the other hand, the present invention provides an image enhancement device, wherein the device is used to generate an input image by concatenating an original input image with a sharpened attention map or a depth map. The device is also used to generate a bottleneck feature by encoding the input image using an encoder. The device is also used to inject a perturbation vector into the bottleneck feature. The device is also used to feed the bottleneck feature injected with the perturbation vector to an image generator. The device is also used to generate an enhanced image at the image generator based on the bottleneck feature and the perturbation vector. The device is also used to receive the enhanced image and a clear image randomly selected from a clear image dataset at a discriminator, and determine an image enhancement score based on the difference between the enhanced image and the randomly selected clear image.

[0022] The device of the present invention achieves all the advantages and effects of the method described above.

[0023] In yet another aspect, the present invention provides a computer program comprising program code, which, when executed by a computer, causes the computer to perform the method.

[0024] After executing the method described in the present invention, the computer realizes all the advantages and effects of the method.

[0025] In yet another aspect, the present invention provides an electronic component mounted on a vehicle, the electronic component being operable to perform the method.

[0026] After executing the method of the present invention, the electronic component achieves all the advantages and effects of the method.

[0027] It should be understood that all of the above implementations may be combined.

[0028] It should be noted that all devices, elements, circuits, units and components described in this application can be implemented by software or hardware elements or any type of combination thereof. All steps performed by the various entities described in this application and the described functions to be performed by the various entities are intended to indicate that the corresponding entities are used to perform the corresponding steps and functions. Although in the description of the following specific embodiments, the specific functions or steps performed by the external entity are not reflected in the description of the specific detailed elements of the entity performing the specific steps or functions, it should be clear to the technician that these methods and functions can be implemented by corresponding hardware or software elements or any combination thereof. It should be understood that, without departing from the scope of the present invention as defined in the appended claims, various combinations of features of the present invention can be carried out.

[0029] Additional aspects, advantages, features and objects of the present invention will become apparent from the accompanying drawings and detailed description of illustrative implementations when read in conjunction with the following appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above summary and the following detailed description of illustrative embodiments may be better understood when read in conjunction with the accompanying drawings. For purposes of illustrating the present invention, exemplary configurations of the present invention are shown in the accompanying drawings. However, the present invention is not limited to the specific methods and instruments disclosed herein. Furthermore, those skilled in the art will appreciate that the drawings are not drawn to scale. Where possible, identical elements are represented by identical numerals.

[0031] Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which:

[0032] Figure 1 A flowchart of an image enhancement method provided by an embodiment of the present invention is shown;

[0033] Figure 2 A block diagram showing various exemplary components of an apparatus provided by an embodiment of the present invention;

[0034] Figure 3A A graphical representation of a learning (or training) image dehazing model provided by an embodiment of the present invention is shown;

[0035] Figure 3B shows a diagrammatic representation of an encoder provided by an embodiment of the present invention;

[0036] Figure 3C shows a graphical representation of a densely connected block provided by an embodiment of the present invention;

[0037] Figure 3D A graphical representation of an encoder-decoder structure with a Gaussian perturbation vector provided by an embodiment of the present invention is shown;

[0038] Figure 3E shows a graphical representation of a discriminator provided by an embodiment of the present invention;

[0039] Figure 3F A graphical representation of fine-tuning provided by an embodiment of the present invention to obtain an optimal output image is shown;

[0040] Figure 4 A diagram showing an exemplary implementation scenario of the image enhancement method and apparatus provided by an embodiment of the present invention is shown.

[0041] In the accompanying drawings, underlined numbers are used to indicate the item in which the underlined number appears or the item adjacent to the underlined number. Non-underlined numbers are used to indicate the item identified by the line connecting the non-underlined number to the item. When a number is not underlined and has an associated arrow, the non-underlined number is used to identify the general item to which the arrow points. DETAILED DESCRIPTION

[0042] The following detailed description describes embodiments of the present invention and ways in which these embodiments may be implemented. Although some modes of implementing the present invention have been disclosed, those skilled in the art will recognize that there may be other embodiments for implementing or practicing the present invention.

[0043] Figure 1 FIG2 shows a flow chart of an image enhancement method provided by an embodiment of the present invention. Figure 1 , an image enhancement method 100 is shown. The method 100 includes steps 102 to 112. The method 100 is composed of steps such as Figure 2 The device described in detail is executed.

[0044] The present invention provides the image enhancement method 100, wherein the method 100 includes:

[0045] (i) Generate the input image by concatenating the original input image with the sharpened attention map or depth map;

[0046] (ii) encoding the input image using an encoder to generate a bottleneck feature;

[0047] (iii) injecting a perturbation vector into the bottleneck feature;

[0048] (iv) feeding the bottleneck feature injected into the perturbation vector to an image generator;

[0049] (v) generating, at the image generator, an enhanced image based on the bottleneck feature and the perturbation vector;

[0050] (vi) At a discriminator, the enhanced image and a clean image randomly selected from a clean image dataset are received, and an image enhancement score is determined according to a difference between the enhanced image and the randomly selected clean image.

[0051] The present invention provides an image enhancement method 100. The method 100 is based on a generative adversarial network (GAN). The GAN can be used to generate multiple output images relative to an input image using an image generator. The generated multiple output images exhibit higher visual quality and provide useful image features required for perception in real time or near real time. The GAN can also include a discriminator that supports the image generator in generating realistic outputs.

[0052] In step 102, the method 100 includes generating an input image by concatenating an original input image with a sharpened attention map or depth map. The original input image corresponds to an image captured under one of the following conditions: foggy, misty, clear, rainy, overcast, snowy, or other adverse weather or lighting conditions. The original input image may also be referred to as a degraded image because it does not reveal useful image features suitable for practical applications or uses. For example, the original input image may be captured by a camera mounted on an autonomous vehicle, which may not reveal features required for safe autonomous driving under different weather and lighting conditions. In another example, the original input image includes one or more objects and may be captured by a handheld device (such as a smartphone) in a low-light, rainy environment. In such an environment, the one or more objects may not be clear. Therefore, in order to obtain useful features (such as object shapes and edges) from the original input image, the original input image is processed using the sharpened attention map or the depth map to generate the input image, which serves as input to the encoder in the next operation. Compared to the original input image, the input image exhibits improved features, such as improved visual quality, object shapes, and edge details. The sharpened attention map can be defined as a scalar matrix that represents the relative importance of multiple layers of activation at different two-dimensional (2D) spatial locations with respect to the target task (e.g., outputting a sharp image). The depth map can be defined as an image or image channel that provides information about the distance between the surface of scene objects and the viewpoint. In one implementation, the sharpened attention map or the depth map can be obtained from a pre-trained model or can be jointly trained in the network in an unsupervised manner.

[0053] In step 104, the method 100 further includes: encoding the input image using an encoder to generate a bottleneck feature. The input image is sent to the encoder to generate the bottleneck feature. The bottleneck feature refers to an encoded feature map extracted from the input image, and the encoded feature map has a smaller spatial size but more channels than the input image. For example, Figure 2 The encoder is described in detail.

[0054] At step 106, the method 100 further includes injecting a perturbation vector into the bottleneck feature. By injecting the perturbation vector into the bottleneck feature, a multimodal image output can be generated. The multimodal image output is generated by changing the appearance of the output image.

[0055] According to one embodiment, the perturbation vector is sampled according to a Gaussian distribution. In one implementation, the perturbation vector corresponds to a six-dimensional perturbation vector sampled according to the Gaussian distribution. In general, the Gaussian distribution can be defined as a bell-shaped curve following a normal distribution with an equal number of measurements above and below the mean. The perturbation vector is upsampled using a multi-layer perceptron (MLP) network. For example, Figure 3A The MLP network is described in detail. The upsampled perturbation vector is injected into the bottleneck feature by using the adaptive instance normalization (AdaIn) method. The adaptive instance normalization (AdaIn) method affects the image generation stage and generates a multimodal output image. For example, Figure 3A The adaptive instance normalization (AdaIn) method is described in detail.

[0056] According to one embodiment, the perturbation vector is updated according to the pre-trained network weights. During the training process, the perturbation vector has no predetermined network weights and acts as a random vector sampled according to a Gaussian distribution in each training step. However, after the training is completed, the perturbation vector is updated according to the pre-trained (i.e., fixed) network weights and then adjusted (or fine-tuned) to search for the optimal solution (i.e., an improved output image). The pre-trained network weights are based on the pre-trained encoder, the image generator, and the discriminator network weights. After the training is completed, such network weights are fixed and then the perturbation vector is fine-tuned.

[0057] According to one embodiment, gradient descent is used to adjust the perturbation vector by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score. The perturbation vector is randomly initialized and then gradient descent is used to adjust the perturbation vector by minimizing various losses such as the adversarial loss, the Frechet Inception distance (FID) score, or the structural similarity index (SSI) score. Generally, the adversarial loss can be defined as the difference between ground truth data (or original data or source data) and generated data calculated using a generative adversarial network (GAN). The FID score can be defined as a metric that calculates the distance between feature vectors calculated for a real image and a generated image using a pre-trained Inception network. The SSI score can also be called a structural similarity index measure (SSIM). The SSIM can be defined as a method for predicting the perceptual quality of digital television and movie pictures and other types of digital images and videos. The SSIM can be used to measure the similarity between two images. The updated perturbation vector (or the resulting perturbation vector) eventually converges to a point where the updated perturbation vector can cooperate with the decoder (or image generator) to generate a visually pleasing output image. The updated perturbation vector (or the resulting perturbation vector) can be applied to one or more images (e.g., test images).

[0058] In step 108, the method 100 further includes feeding the bottleneck feature injected with the perturbation vector to an image generator. The bottleneck feature injected with the perturbation vector is further fed to the image generator. The image generator can be configured to generate multiple output images that can improve visual quality and provide useful image features required for real-time or near-real-time perception. In step 110, the method 100 further includes generating, at the image generator, an enhanced image based on the bottleneck feature and the perturbation vector. In one implementation, the image generator can be configured to generate the enhanced image (i.e., the output image) using the bottleneck feature injected with the perturbation vector in each iteration. Using a different perturbation vector in each iteration can result in variations in various aspects, such as illumination or brightness control of the enhanced image. In this way, a Gaussian solution space is created for varying the appearance of the enhanced image, enabling control of the image enhancement output during testing. Additionally, the enhanced image can be sent back to the encoder to force the encoder to generate the same encoding features as the original input image (e.g., another similar bottleneck feature). Therefore, there is an L1-norm feature reconstruction loss between these two encoded features (e.g., two bottleneck features).

[0059] In step 112, the method 100 further includes receiving, at a discriminator, the enhanced image and a randomly selected clear image from the clear image dataset, and determining an image enhancement score based on a comparison between the enhanced image and the randomly selected clear image. The comparison is performed from the perspective of the authenticity of the enhanced image relative to the randomly selected clear image. Based on the received enhanced image and the randomly selected clear image from the clear image dataset, the discriminator is configured to determine whether the enhanced image is a fake image or a real clear image.

[0060] According to one embodiment, the discriminator is a gradient-based multi-patch discriminator. The gradient-based multi-patch discriminator includes multiple network branches and thus improves the output image quality.

[0061] According to one embodiment, the method 100 further includes feeding the discriminant score back to the encoder and the image generator. By feeding the discriminant score back to the encoder and the image generator, image quality is gradually improved. In addition, a perceptual loss is calculated between the original input image and the enhanced image based on a pre-trained convolutional neural network (CNN) (e.g., a VGG neural network) to preserve structure at the feature level.

[0062] According to one embodiment, the discriminator includes at least the following three network branches:

[0063] (a) The first network branch is used to collect the output of the Gaussian blur generator;

[0064] (b) a second network branch for collecting identity images;

[0065] (c) The third network branch is used to collect the output of the Laplacian of Gaussian fuzzy generator,

[0066] After summing the outputs generated by the three branches after each convolutional layer, the result of the discriminator is obtained.

[0067] The Gaussian blur generator output is responsible for introducing an illumination distribution in the output image that is closer to the target data (i.e., the enhanced image). The identity image collected by the second network branch is similar to the standard discriminator (e.g., The image generated by the discriminator). The Laplacian of Gaussian (LoG) blur generator output is responsible for generating sharper edges in the output image that are closer to the distribution of the target data (i.e., the enhanced image). The LoG can be defined as a two-dimensional isotropic measure of the second-order spatial derivative of an image. Or, in other words, the LoG highlights areas of rapid intensity changes in an image and is commonly used for edge detection. The sum of the outputs generated by the three branches after each convolutional layer is obtained from the discriminator to make true or false predictions for different patches in the output image. In this way, the discriminator can improve the quality of the output image.

[0068] Thus, the method 100 generates a controllable solution space (or a set of multiple output images) for image enhancement, rather than generating only a single non-optimal output image. By accounting for the uncertainty in the original input image, the controllable solution space can be more rationally created, allowing for the search for an optimal solution (e.g., an improved and enhanced output image). Thus, the method 100 provides multiple output images and enables processing of the optimal output image, thereby exhibiting greater reliability and efficiency. The method 100 utilizes the sharpened attention map or the depth map and, therefore, provides better guidance for generating clearer, brighter output images. Furthermore, the method 100 performs multiple image enhancement tasks (e.g., image dehazing, image deblurring, and / or low-light enhancement) to comprehensively enhance images captured in various weather and lighting conditions (e.g., foggy, clear, rainy, dark, or snowy environments). For example, in one exemplary practical application, the method 100 makes various features of such enhanced images highly distinct, which aid in perception, even in varying weather and lighting conditions, to facilitate safe autonomous driving. Furthermore, the method 100 can reduce processing complexity and improve the quality of the output images.

[0069] Steps 102 to 112 are merely illustrative, and other alternatives may be provided, such as adding one or more steps, deleting one or more steps, or providing one or more steps in a different order without departing from the scope of the claims herein.

[0070] Figure 2 FIG2 is a block diagram showing various exemplary components of an apparatus according to an embodiment of the present invention. Figure 2 , a block diagram 200 of an apparatus 202 is shown. The apparatus 202 includes a sharpened attention map or depth map 204, an encoder 206, an image generator 208, a memory 210, a discriminator 212, and a processor 214.

[0071] The apparatus 202 may comprise suitable logic, circuitry, interfaces and / or code for enhancing an image. The apparatus 202 may be configured to perform ( Figure 1 Examples of the apparatus 202 include, but are not limited to, a handheld device or electronic device or component that can be installed on a vehicle (e.g., an autonomous vehicle or a semi-autonomous vehicle), a mobile device, a portable device, etc., which is operable to perform the method 100.

[0072] The sharpened attention map or the depth map 204 is used to provide better guidance for generating an output image with higher visual quality compared to a traditional scattering model. The sharpened attention map or the depth map 204 can be a software program or mathematical expression or application that can be installed in the device 202.

[0073] The encoder 206 (also denoted as Enc) includes appropriate logic, circuitry, interfaces, and / or code, which may be defined as a network (e.g., a convolutional neural network (CNN) or a recurrent neural network (RNN), etc.) that takes input data (e.g., an image) and provides output data (e.g., an output image) based on a feature map or vector or tensor representing the underlying information of the input data. Examples of the encoder 206 include, but are not limited to, recursive neural networks, feedforward neural networks, deep belief networks and convolutional deep belief networks, self-organizing maps, deep Boltzmann machines, and stacked denoising autoencoders.

[0074] The image generator 208 (also denoted as G) includes suitable logic, circuitry, interfaces, and / or code for generating one or more enhanced, sharp output images that approximate the true data distribution. In one implementation, the image generator 208 can also be defined as a network that reconstructs the input data (i.e., the input image) based on the feature map or changes the feature map to a different but related representation. The image generator 208 can also be referred to as a decoder.

[0075] The memory 210 includes appropriate logic, circuitry, or interfaces for storing instructions executable by the processor 214. The memory 210 may also be used to store a clear image dataset. Implementation examples of the memory 210 may include, but are not limited to, an electrically erasable programmable read-only memory (EEPROM), a random access memory (RAM), a read-only memory (ROM), a hard disk drive (HDD), a flash memory, a solid-state drive (SSD), and / or a CPU cache. The memory 210 may store an operating system or other program product (including one or more operating algorithms) to operate the device 202.

[0076] The discriminator 212 (also denoted as D) comprises suitable logic, circuitry, interfaces, and / or code that may be operable to determine an image enhancement score based on a comparison of the one or more enhanced clear output images received from the image generator 208 with a randomly selected clear image from the clear image dataset stored in the memory 210. In other words, the authenticity of the one or more enhanced clear output images may be checked or compared against the randomly selected clear image.

[0077] The processor 214 includes appropriate logic, circuitry, interfaces, and / or code for executing instructions stored in the memory 210. In one example, the processor 214 can be a general-purpose processor. Other examples of the processor 214 can include, but are not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) processor, an application-specific integrated circuit (ASIC) processor, a reduced instruction set (RISC) processor, a very long instruction word (VLIW) processor, a central processing unit (CPU), a state machine, a data processing unit, and other processors or control circuits. In addition, the processor 214 can refer to one or more separate processors, processing devices, or processing units as part of a machine, such as the apparatus 202.

[0078] In operation, the apparatus 202 for image enhancement is provided, wherein the apparatus 202 is configured to generate an input image by concatenating an original input image with the sharpened attention map or the depth map 204. The apparatus 202 is further configured to generate a bottleneck feature by encoding the input image using the encoder 206. The apparatus 202 is further configured to inject a perturbation vector into the bottleneck feature. The apparatus 202 is further configured to feed the bottleneck feature injected with the perturbation vector to the image generator 208. The apparatus 202 is further configured to generate an enhanced image at the image generator 208 based on the bottleneck feature and the perturbation vector. The apparatus 202 is further configured to receive the enhanced image and a clean image randomly selected from the clean image dataset at the discriminator 212, and determine an image enhancement score based on the difference between the enhanced image and the randomly selected clean image.

[0079] According to one embodiment, the perturbation vector is sampled according to a Gaussian distribution. In one implementation, the perturbation vector is a six-dimensional perturbation vector sampled according to the Gaussian distribution.

[0080] According to one embodiment, the perturbation vector is updated based on pre-trained network weights. In another implementation, the perturbation vector is used in conjunction with the pre-trained network weights of the encoder 206 or the image generator 208. In this implementation, the device 202 is configured to start processing from a random perturbation vector.

[0081] According to one embodiment, gradient descent is used to adjust the perturbation vector by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score. In the case of the random perturbation vector, gradient descent is used to update the value of the random perturbation vector by minimizing various losses such as the adversarial loss, the Frechet Inception distance (FID) score, or the structural similarity index (SSI) score.

[0082] According to one embodiment, the discriminator 212 is a gradient-based multi-patch discriminator. The discriminator 212 (or gradient-based multi-patch discriminator) includes multiple network branches and thus improves the output image quality.

[0083] According to one embodiment, the discriminator 212 includes three network branches:

[0084] (a) The first network branch is used to collect the output of the Gaussian blur generator;

[0085] (b) a second network branch for collecting identity images;

[0086] (c) A third network branch configured to collect a Laplacian of Gaussian operator of the generated output, wherein the outputs generated by the three branches after each convolutional layer are summed to obtain the result of the discriminator 212. By using the first network branch, the second network branch, and the third network branch, the discriminator 212 is configured to predict true or false for different patches in the output image.

[0087] According to one embodiment, a computer program is provided, comprising program code, which, when executed by a computer, causes the computer to perform the method 100. Examples of the computer include, but are not limited to, a laptop computer, an electronic control unit (ECU) in a vehicle or an onboard computer of a vehicle, a desktop computer, a mainframe computer, a handheld computer, the processor 214, and other computing devices.

[0088] Thus, the apparatus 202 generates a controllable solution space (or a set of multiple output images) for image enhancement, rather than generating only a single non-optimal output image. By accounting for the uncertainty in the original input image, the controllable solution space can be more rationally created, allowing for the search for an optimal solution (e.g., an improved and enhanced output image). Thus, the apparatus 202 provides multiple output images and enables processing of the optimal output image, thereby exhibiting greater reliability and efficiency. The apparatus 202 utilizes the sharpened attention map or the depth map and, therefore, provides better guidance for generating clearer and brighter output images. Furthermore, the apparatus 202 performs multiple image enhancement tasks (e.g., image dehazing, image deblurring, and / or low-light enhancement) to comprehensively enhance images captured in different weather and lighting conditions (e.g., foggy, clear, rainy, dark, or snowy environments). For example, in one exemplary practical application, the apparatus 202 makes various features of such enhanced images highly apparent, which facilitate perception, even in varying weather and lighting conditions, to promote safe autonomous driving.

[0089] Figure 3A A graphical representation of a learning (or training) image dehazing model provided by an embodiment of the present invention is shown. Figure 3A It is a combination Figure 1 and Figure 2 The elements described in Reference Figure 3A , the figure shows a pictorial representation 300A including an original input image 302, a densely connected block 304, a bottleneck feature 306, a perturbation vector 308, a multi-layer perceptron (MLP) network 310, a residual block 312, an output image (i.e., an enhanced sharp output image 314), another bottleneck feature 316, and a convolutional network 318.

[0090] The original input image 302 (also denoted as X input ) corresponds to an image captured in a foggy environment during autonomous driving. input) can be referred to as a degraded image, which does not reveal the various parameters or features used to guide safe autonomous driving. In the illustrated representation 300A, the original input image 302 (e.g., a blurred input image) is considered for enhancement. However, the illustrated representation 300A is also applicable to blurred images or images captured in foggy, clear, rainy, dark, or snowy environments.

[0091] The sharpened attention map or the depth map 204 corresponds to a sharpened attention or depth estimation model, which is used to process the original input image 302 (ie, X input ) and provides an input image (e.g., a sharpened image). The input image (i.e., the sharpened image) exhibits high visual quality and thus provides better object shape and edge information present in the image. Thereafter, the input image (i.e., the sharpened image) is encoded using the encoder 206 (i.e., Enc).

[0092] The densely connected block 304 (also denoted as DenseBlk) can be referred to as one or more convolutional layers in a convolutional neural network (CNN), which is used to perform image classification training using the bottleneck features received from the encoder 206 (i.e., Enc). The one or more convolutional layers are connected (or convolved) to each other in a feedforward manner through multiplication or dot product. The densely connected block 304 (i.e., DenseBlk) is a fully connected layer, typically used for image classification tasks.

[0093] The bottleneck feature 306 (also denoted as C) refers to an encoded feature map of the input image (encoded by the encoder 206). Compared to the input image, the encoded feature map has a smaller spatial size but more channels. The bottleneck feature 306 (i.e., C) can also be referred to as a bottleneck content feature, which represents the content characteristics of the input image in an encoded form.

[0094] The perturbation vector 308 refers to a six-dimensional perturbation vector. In the graphical representation 300A, the perturbation vector 308 is sampled according to a Gaussian distribution. The perturbation vector 308 is upsampled by using a multi-layer perceptron (MLP) network 310. The MLP network 310 is defined as a type of feedforward artificial neural network (ANN). The MLP network 310 includes multiple neural network layers (e.g., an input layer, an output layer, and one or more hidden layers) of nonlinearly activated nodes. The MLP network 310 is a fully connected network, so each node in one layer is connected to every other node in another layer with a certain weight. The perturbation vector 308 (i.e., the upsampled perturbation vector) is injected into the bottleneck feature by using an adaptive instance normalization (AdaIn) method. The AdaIn method affects the image generation stage and generates a multimodal output image. The AdaIn method is typically used in image style transfer and image generation tasks to change the original input image 302 (i.e., X input )'s appearance.

[0095] The residual block 312 (also denoted as ResBlk) includes two convolutional layers (e.g., an input layer and an output layer) with a skip connection between the two convolutional layers to allow identity mapping. The residual block 312 (i.e., ResBlk) is used to prevent gradient vanishing and provide better feature representation of the bottleneck feature.

[0096] Thereafter, the bottleneck features injected into the perturbation vector 308 (i.e., the upsampled perturbation vector) are fed to the image generator 208 (i.e., G). The image generator 208 (i.e., G) is configured to generate an enhanced sharp output image 314 (also denoted as X) in each training iteration based on the bottleneck features injected into the upsampled perturbation vector 308. clear ), and the enhanced clear output image 314 (ie, X clear ) is added to the clear image dataset stored in the memory 210.

[0097] By using the encoder 206 (ie, Enc) to encode the enhanced clear output image 314 (ie, X clear ) is encoded, and then the enhanced clear output image 314 (ie, X clear), obtain the other bottleneck feature 316 (also represented as C'). The other bottleneck feature 316 (ie, C') is used to keep consistent with the bottleneck feature 306 (ie, C) through L1 norm loss.

[0098] In addition, the discriminator 212 (ie, D) is configured to receive the enhanced sharp output image 314 (ie, X) from the image generator 208 (ie, G). clear ) and a clear image randomly selected from the clear image dataset stored in the memory 210. The discriminator 212 (ie, D) is further configured to generate a clear output image 314 (ie, X) according to the enhanced clear output image 314. clear ) is compared with the randomly selected clear image to determine the image enhancement score.

[0099] The convolutional network 318 corresponds to a VGG network (e.g., VGG 16 or VGG 19), which is used for image classification and image feature detection. In addition, the original input image 302 (i.e., X input ) and the enhanced clear output image 314 (ie, X clear ) is configured relative to the perceptual loss (i.e., L percep ) remain consistent, thereby maintaining the image structure at the feature level. As shown in the figure, the convolutional network 318 is independent of the entire framework and is only used to increase the additional loss of the entire framework.

[0100] In operation, the original input image 302 (ie, X input ) is concatenated with the sharpened attention map or the depth map 204 and transmitted to the encoder 206 (i.e., Enc) and the densely connected block 304 (i.e., DenseBlk) to obtain the bottleneck feature 306 (i.e., C). The perturbation vector 308 is upsampled by the MLP network 310 through the AdaIn method used in the MLP network 310 and integrated with the bottleneck feature 306 (i.e., C). After the perturbation vector 308 is integrated with the bottleneck feature 306 (i.e., C), the bottleneck feature 306 (i.e., C) is fed to the image generator 208 (i.e., G) through the residual block 312 (i.e., ResBlk). The image generator 208 (i.e., G) is used to generate the enhanced sharp output image 314 (i.e., X) in each iteration. clear In each training (or learning) iteration of the image dehazing model, some parameters such as the enhanced clear output image 314 (i.e., X clear) lighting or brightness contrast. The different perturbation vectors 308 result in the creation of a Gaussian solution space of multiple output images for image enhancement, rather than creating only one output image, and enable processing of an optimal output image (such as the enhanced clear output image 314). In each training iteration, the enhanced clear output image 314 (i.e., X clear ) is fed to the discriminator 212 (i.e., D) together with a clean image randomly selected from the clean image dataset to determine whether the enhanced clean output image 314 is a fake image or a real clean image. Thereafter, the image generator 208 (i.e., G) is used to improve itself by minimizing the adversarial loss of the discriminator 212 (i.e., D). In addition, the enhanced clean output image 314 (i.e., X) is fed to the discriminator 212 (i.e., D) to determine whether the enhanced clean output image 314 is a fake image or a real clean image. clear ) is further concatenated with the sharpened attention map or the depth map 204 and encoded by using the encoder 206 (i.e., Enc) and the densely connected block 304 (i.e., DenseBlk) to obtain the other bottleneck feature 316 (i.e., C'). The other bottleneck feature 316 (i.e., C') is used to obtain the bottleneck feature 316 (i.e., C') through the L1 norm loss (i.e., C recon ) is consistent with the bottleneck feature 306 (ie, C). input ) and the enhanced clear output image 314 (ie, X clear ) relative to the perceptual loss (e.g., L percep ) is kept consistent so that in the enhanced clear output image 314 (ie, X clear ) in the original input image 302 (ie, X input ) of the aforementioned features.

[0101] After training the image dehazing model through multiple iterations, a controllable solution space of the output image (or enhanced image) is generated, and the optimal output image is searched from the controllable solution space of the output image (or enhanced image). In order to fine-tune the optimal output image, two methods are used, for example, Figure 3F The two methods are described in detail.

[0102] In this embodiment, the graphical representation 300A is used to learn (or train) the image dehazing model. In another embodiment, the graphical representation 300A can also be used for image deblurring or low-light enhancement, or for enhancing images captured in foggy, sunny, rainy, dark, or snowy environments.

[0103] Figure 3B A diagrammatic representation of an encoder provided by an embodiment of the present invention is shown. Figure 3B It is a combination Figure 1、 Figure 2 and Figure 3A The elements described in Reference Figure 3B , the figure shows ( Figure 2 3 . The encoder 206 includes an Inception residual block 320. The Inception residual block 320 includes a plurality of 1×1 convolution blocks 320A and a plurality of 3×3 convolution blocks 320B.

[0104] The Inception residual block 320 can be defined as a convolution block that combines multiple convolution branches that can capture various features of different patch sizes of the image (such as the bottleneck features used in the illustrative representation 300A).

[0105] In contrast to conventional residual blocks, the encoder 206 includes the Inception residual block 320 and thus provides an improved encoded feature map that combines local image content and global image content. The improved encoded feature map is obtained because the Inception residual block 320 of the encoder 206 includes the multiple 1×1 convolution blocks 320A and the multiple 3×3 convolution blocks 320B.

[0106] Figure 3C A diagrammatic representation of densely connected blocks provided by an embodiment of the present invention is shown. Figure 3C It is a combination Figure 1 、 Figure 2 、 Figure 3A and Figure 3B The elements described in Reference Figure 3C , the figure shows ( Figure 3A 300C of the densely connected block 304. The densely connected block 304 includes the plurality of 3×3 convolutional blocks 320B.

[0107] In order to obtain the bottleneck feature 306 (i.e., C) having a more meaningful and robust feature representation of the bottleneck feature received from the encoder 206 (i.e., Enc), the traditional residual block is replaced with the densely connected block 304 (i.e., DenseBlk). In the densely connected block 304 (i.e., DenseBlk), each of the multiple 3×3 convolution blocks 320B is connected to each other, thereby providing an improved feature representation of the bottleneck feature.

[0108] Figure 3D A graphical representation of an encoder-decoder structure with a Gaussian perturbation vector provided by an embodiment of the present invention is shown. Figure 3D It is a combination Figure 1 、 Figure 2 、 Figure 3A 、 Figure 3B and Figure 3C The elements described in Reference Figure 3D , a graphical representation 300D of an encoder-decoder structure with a Gaussian perturbation vector 322 is shown. The encoder-decoder structure with the Gaussian perturbation vector 322 includes the encoder 206 and the image generator 208 (also referred to as a decoder). The encoder-decoder structure with the Gaussian perturbation vector 322 also includes the densely connected block 304, the bottleneck feature 306, the perturbation vector 308, the MLP network 310, and the residual block 312.

[0109] The conventional encoder-decoder structure includes the conventional residual block and only provides a non-optimal output image, and therefore, in comparison, the conventional encoder-decoder structure is not a preferred structure. However, the encoder-decoder structure with the Gaussian perturbation vector 322 includes the densely connected block 304 (i.e., DenseBlk), the perturbation vector 308, and the MLP network 310, and provides a plurality of output images (or multimodal output images) Gaussian solution spaces for image enhancement instead of providing only one output image, and enables processing of an optimal output image (such as the enhanced clear output image 314).

[0110] Figure 3E FIG. 4 shows a diagrammatic representation of a discriminator provided by an embodiment of the present invention. Figure 3E It is a combination Figure 1 、 Figure 2 、 Figure 3A 、 Figure 3B 、 Figure 3C and Figure 3D The elements described in Reference Figure 3E , the figure shows ( Figure 2 ) is a diagrammatic representation of the discriminator 212. The discriminator 212 (i.e., D) includes a first network branch 212A, a second network branch 212B, and a third network branch 212C. The discriminator 212 (i.e., D) also includes a first convolutional layer 324A, a second convolutional layer 324B, a third convolutional layer 324C, and an output image 328.

[0111] Each of the first convolutional layer 324A, the second convolutional layer 324B, and the third convolutional layer 324C may also be referred to as a convolutional neural network (CNN). Generally, a convolutional neural network (CNN) can be defined as a network of highly interconnected processing elements. Optionally, each element is associated with a local memory (i.e., the memory 210) and is used for image recognition and processing (e.g., for processing the enhanced sharp output image 314).

[0112] The first network branch 212A is used to collect a Gaussian blur generator output 326A, which is responsible for introducing an illumination distribution in the enhanced sharp output image 314 that is closer to the target data distribution.

[0113] The second network branch 212B is used to collect an identity image 326B, which is similar to a standard discriminator (e.g., The images generated by the discriminator.

[0114] The third network branch 212C is used to collect a Laplacian of Gaussian (LoG) blur generator output 326C, which is responsible for generating sharper edges in the enhanced sharp output image 314 that are closer to the target data distribution.

[0115] After summing the outputs (i.e., 326A, 326B, 326C) generated by the three network branches (i.e., the first network branch 212A, the second network branch 212B, and the third network branch 212C) after each convolutional layer (i.e., the first convolutional layer 324A, the second convolutional layer 324B, and the third convolutional layer 324C), the discriminator 212 (i.e., D) obtains the output image 328. Based on the different patches formed in the enhanced clean output image 314 and the output image 328, the discriminator perceives whether the output image 328 is a real image or a fake image. A real image refers to an image that is similar to an input image, such as the enhanced clean output image 314 (also referred to as an augmented image), in terms of all image features. A fake image can be described as an image that can be artificially generated using software tools. In this way, the discriminator 212 can improve the output image quality compared to a conventional discriminator that generates an output image using only one network branch, thereby exhibiting lower image quality.

[0116] Figure 3FA graphical representation of fine-tuning provided by an embodiment of the present invention to obtain an optimal output image is shown. Figure 3F It is a combination Figure 1 、 Figure 2 、 Figure 3A 、 Figure 3B 、 Figure 3C 、 Figure 3D and Figure 3E The elements described in Reference Figure 3F , a graphical representation 300F of fine-tuning to obtain an optimal output image from a set of multiple sharp output images is shown.

[0117] In the graphical representation 300F, two methods may be used to obtain the optimal output image from the set of multiple clear output images that may be generated using the graphical representation 300A.

[0118] In the first method, the perturbation vector 308 is sampled according to the Gaussian distribution. In this method, a grid search is performed by interpolating the values of each two dimensions in the Gaussian perturbation and checking the image with the best visual quality. The corresponding perturbation vector is applied to all test images.

[0119] In a second approach, the perturbation vector 308 is updated based on pre-trained network weights. In this approach, the weights associated with the encoder 206, the image generator 208, the densely connected block 304, the bottleneck feature 306, the residual block 312, and the discriminator 212 have fixed values. Thereafter, processing begins with a random perturbation vector and uses gradient descent to update the value of the random perturbation vector by minimizing various losses such as the adversarial loss, the Frechet Inception Distance (FID) score, or the Structural Similarity Index (SSI) score. The updated perturbation vector (or the resulting perturbation vector) eventually converges to a point where the updated perturbation vector can be used with the image generator 208 to generate a visually pleasing output image. The updated perturbation vector (or the resulting perturbation vector) can also be applied to one or more test images.

[0120] Figure 4 A diagram showing an exemplary implementation scenario of the image enhancement method and apparatus provided by an embodiment of the present invention is shown. Figure 4 It is a combination Figure 1 、 Figure 2 、 Figures 3A to 3F The elements described in Reference Figure 4 , which illustrates the disclosed method and apparatus ( Figure 1 and Figure 2 ) in the field of autonomous driving. In the exemplary scenario 400, a vehicle 402 is shown moving along a portion of a road. Also shown are electronic components 404 and one or more image capture devices, such as image capture device 406, mounted on the vehicle 402. It should be understood that the vehicle 402 may include many other known components typically used in autonomous vehicles, which are omitted for brevity. For example, the vehicle 402 may include a battery for powering the image capture device 406 and the electronic components 404.

[0121] In the exemplary scenario 400, the vehicle 402 may be an autonomous vehicle or a semi-autonomous vehicle. The electronic components 404 may include appropriate logic, circuitry, interfaces, and / or code for performing multimodal image enhancement that comprehensively and holistically enhances images captured under varying weather and lighting conditions. For example, the electronic components 404 are configured to process and enhance images captured by the image capture device 406 in varying weather and lighting conditions (e.g., foggy, clear, rainy, overcast, or snowy environments) in real time or near real time by holistically performing various image enhancement tasks (e.g., image dehazing, image deblurring, and overall low-light enhancement) to facilitate safe autonomous driving. Alternatively, the electronic components 404 may make many features of such enhanced images highly apparent, which facilitates perception of the real environment surrounding the vehicle 402, even under varying weather and lighting conditions. This, in turn, enables safe autonomous driving of the vehicle 402. Examples of the electronic components 404 include, but are not limited to, an electronic control unit (ECU), onboard equipment, onboard computer, or other electronic components for the vehicle 402. The electronic component 404 may correspond to ( Figure 2 ) the device 202, wherein the electronic component 404 is used to perform ( Figure 1 ) method 100.

[0122] The electronic component 404 can be used to perform various image enhancement tasks while the vehicle 402 is driving. The electronic component 404 can be used to use the perturbation vector 308 sampled according to the Gaussian distribution, which further results in generating multiple clear output images relative to an original input image (i.e., a degraded image), and thus can select the optimal output image (i.e., an enhanced and improved image) from the multiple clear output images. Therefore, the electronic component 404 provides the optimal output image (i.e., the enhanced and improved image) for perception and enables the vehicle 402 to be driven reliably and safely. In addition, the electronic component 404 can be used to use the discriminator 212 (i.e., the gradient-based multi-patch discriminator), which can further improve the visual quality of the multiple clear output images.

[0123] In another implementation scenario, the apparatus 202 may be implemented as a handheld device, which is operable to perform ( Figure 1 In one example, the handheld device may be a smartphone that can fully process one or more images captured under different weather and lighting conditions using the method 100 to generate an enhanced image, such as a high-quality, realistic, photo-like, and clear output image perceived by the human eye. The method 100 enables the handheld device to predict the "realness" or "fakeness" of different patches in the output image, thereby improving the output image quality.

[0124] Without departing from the scope of the invention as defined by the appended claims, the embodiments of the invention described above may be modified. Expressions such as "including", "in combination with", "having", "is" etc. used to describe and claim the present invention are intended to be interpreted in a non-exclusive manner, i.e., allow items, parts or elements that are not explicitly described to also exist. References to the singular should also be interpreted as being related to the plural. The word "exemplary" used herein means "as an example, instance or illustration". Any embodiment described as "exemplary" is not necessarily interpreted as being more preferred or more advantageous than other embodiments, and / or excludes the combination of the features of other embodiments. The word "optionally" used herein means "provided in some embodiments and not provided in other embodiments". It should be understood that some features of the present invention described in the context of a separate embodiment for the sake of clarity may also be provided in a single embodiment by combination. On the contrary, the various features of the present invention described in the context of a single embodiment for the sake of brevity may also be provided individually, in any suitable combination, or as any other described embodiment of the present invention.

Claims

1. An image enhancement method (100), characterized in that: The method comprises (100): (i) generating an input image by concatenating the original input image (302) with a sharpened attention map or depth map (204), wherein the sharpened attention map is a scalar matrix used to represent the relative importance of multiple layers of activation at each two-dimensional spatial position for the target task; (ii) encoding the input image using an encoder (206) to generate a bottleneck feature; (iii) injecting a perturbation vector (308) into the bottleneck feature; (iv) feeding the bottleneck feature injected into the perturbation vector (308) to an image generator (208); (v) at the image generator (208), a controllable solution space of the output image is generated based on the bottleneck feature and the perturbation vector (308), and the perturbation vector (308) is used to determine an output enhanced image from the controllable solution space of the output image; (vi) at the discriminator (212), the enhanced image and a clean image randomly selected from a clean image dataset are received, and an image enhancement score is determined based on the difference between the enhanced image and the randomly selected clean image, and the discriminator (212) is a gradient-based multi-patch discriminator.

2. The method (100) according to claim 1, characterized in that The method (100) further includes feeding back the discriminant score to the encoder (206) and the image generator.

3. The method (100) according to claim 1, characterized in that The disturbance vector (308) is sampled according to a Gaussian distribution.

4. The method (100) according to claim 1, characterized in that The perturbation vector is updated according to the pre-trained network weights (308).

5. The method (100) according to claim 4, characterized in that The perturbation vector is adjusted using gradient descent by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score (308).

6. The method (100) according to claim 3, characterized in that The discriminator (212) includes at least the following three network branches: (a) a first network branch (212A) for collecting Gaussian blur generator output (326A); (b) a second network branch (212B), for collecting an identity image (326B); (c) The third network branch (212C) is used to collect the output of the Laplacian of Gaussian fuzzy generator (326C), After summing the outputs generated by the three branches after each convolutional layer, the result of the discriminator (212) is obtained.

7. An image enhancement device (202), characterized in that The device (202) is used to: (i) generating an input image by concatenating the original input image (302) with a sharpened attention map or depth map (204), wherein the sharpened attention map is a scalar matrix used to represent the relative importance of multiple layers of activation at each two-dimensional spatial position for the target task; (ii) encoding the input image using an encoder (206) to generate a bottleneck feature; (iii) injecting a perturbation vector (308) into the bottleneck feature; (iv) feeding the bottleneck feature injected into the perturbation vector (308) to an image generator (208); (v) at the image generator (208), a controllable solution space of the output image is generated based on the bottleneck feature and the perturbation vector (308), and the perturbation vector (308) is used to determine an output enhanced image from the controllable solution space of the output image; (vi) at the discriminator (212), the enhanced image and a clean image randomly selected from a clean image dataset are received, and an image enhancement score is determined based on a comparison between the enhanced image and the randomly selected clean image, wherein the discriminator (212) is a gradient-based multi-patch discriminator.

8. The device (202) according to claim 7, characterized in that The disturbance vector (308) is sampled according to a Gaussian distribution.

9. The device (202) according to claim 7, characterized in that The perturbation vector is updated according to the pre-trained network weights (308).

10. The device (202) according to claim 9, characterized in that The perturbation vector is adjusted using gradient descent by reducing the adversarial loss, the Frechet Inception distance score, or the structural similarity index score (308).

11. The device (202) according to claim 7, characterized in that The discriminator (212) includes three network branches: (a) a first network branch (212A) for collecting Gaussian blur generator output (326A); (b) a second network branch (212B), for collecting an identity image (326B); (c) a third network branch (212C) for collecting the Laplacian of Gaussian operator of the generated output (326C), After summing the outputs generated by the three branches after each convolutional layer, the result of the discriminator (212) is obtained.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method (100) of claim 1 are implemented.

13. An electronic component (404) mounted on a vehicle (402), characterized in that: The electronic component (404) is operable to perform the method (100) according to claim 1.

Images