Automated image evaluation, automated setting of processing parameters, image processing, image processing system, and image analysis unit

Abstract

Relates to a computer-implemented method for automated image evaluation (28), to a computer-implemented method for automated setting (42) of processing parameters (18) of an image processing processor (16), and to a computer-implemented method for image processing (10) of an input image (12) of an image sensor (14). The invention further relates to an image processing system (22) and to an image analysis unit (26).

Description

Technical Field

[0001] The invention relates to a computer-implemented method according to the invention for automated image evaluation, a computer-implemented method according to the invention for automated setting of processing parameters of an image processing processor, and a computer-implemented method according to the invention for image processing of input images of an image sensor. Furthermore, the invention relates to an image processing system and an image analysis unit. Background Art

[0002] Image processing processors are implemented as highly specialized microprocessors and play a decisive role in digital image processing. These image processing processors are also often referred to as image signal processors (ImageSignal Prozessoren: ISPs) and have the ability to process raw images with a high dynamic range provided by image sensors and convert them into a format that can be used for image analysis. This process enables the creation of digital images in a form that can be efficiently interpreted and analyzed by computer applications and processing algorithms.

[0003] Image processing algorithms are embedded in the image processing processor, which can be configured through processing parameters. These image processing algorithms implement a series of functions to improve the input image and optimize it for use through image analysis. For example, these image processing algorithms can sharpen the input image, correct color, reduce noise, and perform other forms of image optimization.

[0004] One challenge in the development of image processing systems is to find suitable processing parameters that are optimally coordinated with the special requirements of image analysis. This means that the image processing system must be able to extract the maximum information from the input image in order to maximize the performance of the image analysis.

[0005] The known method for setting processing parameters comprises several steps. First, a record is created manually and different assumptions are made about the best image representation for image analysis. The next step consists in evaluating the individual parameter sets of the processing parameters. This is done by an image quality expert, often referred to as a "Golden Eye", who evaluates the output images of the image processing by visual inspection on a computer monitor. In addition, the image quality expert evaluates different characteristic curves, such as noise characteristics, in order to obtain, in particular, an overall estimate of the image quality and the effectiveness of the processing parameters.

[0006] This process requires a high degree of expertise and experience in order to ensure that the processing parameters are optimally coordinated with the requirements of the image analysis and provide the best possible image quality. Summary of the invention

[0007] According to the present invention, a computer-implemented method for automated image evaluation of an input image processed by an image processing processor is provided. The method comprises the steps of providing an input image, performing a first image processing on the input image by the image processing processor according to processing parameters of the image processing processor, outputting the processed input image as an output image, performing a second image processing on the output image by a trained image evaluation algorithm, and outputting an image evaluation result according to the second image processing, wherein the image evaluation result describes at least one image feature of the output image, and a subsequent image analysis of the output image is related to the at least one image feature. This allows an objective and traceable image evaluation. The image quality of the processed input image is improved. Subsequent image analysis can be performed more reliably, more accurately and more quickly. The image evaluation can be performed independently of the display device that is read out by an image quality expert.

[0008] The computer-implemented method can be applied in a semi-autonomous or autonomous system. For example, the system can be a vehicle or a robot.

[0009] Image evaluation can identify image features of the output image. This image feature can be the image sharpness of the input image. Image resolution, color depth, exposure, contrast and / or noise.

[0010] An image processing processor is understood to be a microprocessor which is used specifically for processing digital images. The image processing processor can be dedicated or integrated in a higher-level microprocessor.

[0011] The input image may be generated by an image sensor. The input image may be a raw image of the image sensor. The input image may exist as a single image or as an image sequence.

[0012] The input image can be a recording of the surroundings, in particular the vehicle surroundings of the vehicle. The input image can be present as a digital image.

[0013] The processing parameters may be parameters regarding noise suppression, sharpening, white balance, exposure, color correction, lens correction, demosaicing and / or compression. The compression may be bit depth reduction and / or the use of the compressed formats JPEG or HEIF.

[0014] The image evaluation result may be output for detection by a user. The image evaluation result may have at least one evaluation information. The image evaluation result may be output via a Rest-API.

[0015] The image evaluation result of the output image can be calculated independently of the comparison with the corresponding input image.

[0016] The image characteristics may be image quality and / or image suitability for subsequent image analysis.

[0017] The image assessment algorithm may be implemented separately from the image analysis. The image assessment algorithm may be a Blind / No-Reference Image Spatial Quality Evaluator (BRISQUE), in which the image quality is assessed by measuring the spatial deviations (Aberrationen) in the image. The image assessment algorithm may be a Naturalness Image Quality Evaluator (NIQE), which applies statistical features to the assessment of image quality. The image assessment algorithm may be a Perceptual Image Quality Assessment (PIQA), in which machine learning is used to simulate a human perception model for the image quality. The image assessment algorithm may be a Blind / No Reference Image Sharpness Metric (BRISM), in which the sharpness of the image is measured. The image assessment algorithm may be a No-Reference Quality Metric (NRQM), in which the image quality is calculated based on features such as brightness, contrast, and color saturation.

[0018] The image evaluation algorithm can be implemented during the image analysis, so that the image evaluation result can be calculated directly from the influence on the image analysis.

[0019] Autonomous systems may be controllable based on image analysis.

[0020] The computer-implemented program may be implemented in a software container.

[0021] In a preferred embodiment of the invention, the image evaluation algorithm advantageously processes the output image by machine learning. Machine learning can be performed by applying a neural network, a decision tree and / or a support vector machine (SVM). The neural network can be a convolutional neural network. The neural network can be trained by deep learning.

[0022] According to the present invention, a computer-implemented method for automatically setting processing parameters of an image processing processor according to the present invention is further provided. The method comprises the steps of providing an input image, performing a first image processing on the input image by an image processing processor according to the processing parameters of the image processing processor, outputting the processed input image as an output image, performing an automated image evaluation according to the above, and setting the processing parameters according to the image evaluation results. The setting of the processing parameters can be performed according to a trained correlation between the processing parameters and the image evaluation results. This correlation can be trained by machine learning.

[0023] The computer-implemented method may be implemented in a software container. The software container may be the same as or different from the software container on which the computer-implemented method for automated image assessment is run.

[0024] The automatically set processing parameters may be some or all processing parameters of the image processing processor.

[0025] In a special embodiment of the invention, it is advantageous that the numerical optimizer outputs suggested processing parameters based on the image evaluation results. The numerical optimizer can apply gradient descent methods, evolutionary algorithms ( Algorithms) and / or swarm optimization (Schwarmoptimierung).

[0026] The numerical optimizer can be assigned to the image evaluation algorithm. The numerical optimizer can be implemented separately from the image evaluation algorithm, in particular downstream of the image evaluation algorithm. The processing parameters can be transferred, in particular set, via the REST API.

[0027] The image evaluation algorithm can be assigned to the image analysis and can form an image evaluation result based on the result of the image analysis. This allows application-related image evaluation. The image evaluation algorithm can also be performed separately from the image analysis.

[0028] In a special embodiment of the invention, it is advantageous if the suggested processing parameters are applied to the image processing processor. The suggested processing parameters can at least partially replace previously preset processing parameters.

[0029] In a special embodiment of the invention, it is advantageous if the input image processed with the proposed processing parameters is passed back as an output image to the image evaluation algorithm, which then outputs an updated image evaluation result, wherein the updated image evaluation result is used to recalculate the proposed processing parameters. Multiple feedbacks can thus be implemented for optimally setting the processing parameters.

[0030] In a special embodiment of the invention, it is advantageous if the newly proposed processing parameters are applied to the image processing processor. The newly proposed processing parameters can at least partially replace the previously proposed and set processing parameters.

[0031] According to the invention, a computer-implemented method according to the invention for image processing an input image of an image sensor is also proposed. This is done by processing the input image by an image processing processor according to processing parameters of the image processing processor set by a computer-implemented method according to the invention for automatically setting processing parameters of an image processing processor. As a result, the image processing can calculate an output image that is more suitable for image analysis.

[0032] According to the invention, an image processing system according to the invention is further provided. The image processing system is used for image processing of an input image of an image sensor, the image processing system having an image processing unit configured to implement the computer-implemented method according to the invention for image processing of an input image of an image sensor, the image processing unit having at least one image processing processor for processing the input image according to processing parameters.

[0033] According to the invention, an image analysis unit according to the invention is also provided. The image analysis unit is used to perform image analysis on an input image processed by the image processing system according to the invention. The image analysis unit can be verified, checked and / or trained, in particular retrained, by means of an output image that was processed starting from the input image by processing parameters of an image processing processor that were set according to the image evaluation result.

[0034] Image analysis can be performed by machine learning to analyze the output image. Machine learning can be performed by applying a neural network and / or a support vector machine (SVM). The neural network can be a convolutional neural network. The neural network can be trained by deep learning. Image analysis can be performed by applying classical algorithms such as optical flow or parallax. to analyze the output image.

[0035] Image analysis may include image interpretation, object recognition - e.g. person recognition, vehicle recognition, traffic sign recognition, including motion trajectory analysis, object classification, scene recognition and / or segmentation, i.e. dividing an image into different regions representing different objects or parts of a scene.

[0036] The first image processing, the second image processing and the image analysis can be implemented in separate software modules or in at least partially common software modules. The software modules can be implemented based on the cloud and / or locally. The software modules can be implemented in each node through distributed computing. Thus, the processing can be performed in parallel and is scalable.

[0037] Further advantages and advantageous embodiments of the invention are apparent from the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention is described in detail below with reference to the accompanying drawings. They are shown respectively:

[0039] Figure 1 : A computer-implemented method for image processing, an image processing system and an image analysis unit, respectively, in a special embodiment of the invention.

[0040] Figure 2 : In a special embodiment of the invention, a computer-implemented method for automated image evaluation.

[0041] Figure 3 : A computer-implemented method for automatically setting processing parameters in a special embodiment of the present invention. DETAILED DESCRIPTION

[0042] Figure 1 A computer-implemented method for image processing, an image analysis unit and an image processing system are shown in each case in a special embodiment of the invention. A computer-implemented method for image processing 10 of an input image 12 is implemented, for example, for image analysis of the input image 12. The input image 12 may be a recording of the surroundings of a vehicle, which is interpreted by means of image analysis.

[0043] An input image 12 is provided as a raw image by an image sensor 14 and is processed by an image processing processor 16 according to processing parameters 18 of the image processing processor 16. The processing parameters 18 may affect the brightness, color depth, etc. of the input image 12. The image processing processor 16 outputs the input image 12 processed in this manner as an output image 20. Therefore, due to the application of the processing parameters 18, the output image 20 is largely distinguished from the input image 12 in terms of the image features affected by the processing parameters 18.

[0044] The image processing system 22 comprises an image processing unit 24 having an image processing processor 16 , which is configured to carry out a computer-implemented method for image processing 10 .

[0045] The output image 20 may be processed by an image analysis unit 26. The image analysis unit 26 performs image analysis 27 on the output image 20 as the input image 12 processed by the image processing system 22, such as image interpretation, object recognition, motion trajectory analysis, object classification, scene recognition and / or segmentation.

[0046] Figure 2A computer-implemented method for automated image evaluation is shown in a special embodiment of the invention. The computer-implemented method for automated image evaluation 28 of an input image 12 processed by an image processing processor 16 first comprises providing 30 an input image 12 by an image sensor 14. Subsequently, a first image processing 32 of the input image 12 is performed by the image processing processor 16 according to processing parameters 18 of the image processing processor 16, and the processed input image 12 is output 33 as an output image 20. With the aid of a subsequent second image processing 34 of the output image 20 by a trained image evaluation algorithm 36, an image evaluation result 38 is formed according to the second image processing 34, wherein the image evaluation result 38 indicates at least one image feature, to which the subsequent image analysis of the output image 20 is dependent. The image evaluation algorithm 36 can process the output image 20 by machine learning.

[0047] For example, the image evaluation algorithm 36 can be trained with input images 12 and reference images processed by the first image processing 32 with predefined processing parameters 18 to create image evaluation results 38. The image evaluation results 38 are then output 40, for example for automated setting of the processing parameters 18.

[0048] Figure 3 A computer-implemented method for automatically setting processing parameters in a particular embodiment of the present invention is shown. A computer-implemented method for automatically setting 42 processing parameters 18 of an image processing processor 16 may be in Figure 2 Extension of the computer-implemented method for automated image evaluation described in .

[0049] In the computer-implemented method for the automated setting 42 of processing parameters 18, first an input image 12 is provided 30 and then a first image processing 32 of the input image 12 by the image processing processor 16 is performed according to the processing parameters 18 of the image processing processor 16. Subsequently, the processed input image 12 is output 33 as an output image 20 and an automated image evaluation is carried out 43 by the computer-implemented method for automated image evaluation, for example in Figure 2 Furthermore, the setting 44 of the processing parameter 18 is performed based on the image evaluation result 38 .

[0050] For example, the image evaluation algorithm 36 can be trained with the input image 12 and the reference image processed by the first image processing 32 with the predefined processing parameters 18 to create an image evaluation result 38. As shown in the figure here, the image evaluation algorithm 36 is assigned to the image analysis 27 and can form the image evaluation result 38 based on the result of the image analysis 27. This allows an application-dependent image evaluation and calculation of the image evaluation result 38. The image analysis 27 preferably corresponds to the image analysis used in a later application, which analyzes the output image processed with the automatically set processing parameters 18.

[0051] Based on the evaluation results 38 , the process parameters 18 are calculated with the aid of a numerical optimizer 46 and output as recommended process parameters 48 .

[0052] The automated setting 44 can have a plurality of optimization steps. The suggested processing parameters 48 are first applied to the image processing processor 16. The input image 12 processed with the suggested processing parameters 48 is passed again as an output image 20' to the image evaluation algorithm 36, which creates an updated image evaluation result 38', from which the numerical optimizer 46 re-outputs the suggested processing parameters 48'. The re-proposed processing parameters 48' are applied again to the image processing processor 16 until an optimal setting of the processing parameter setting 18 is present.

Claims

1. A computer-implemented method for automated image evaluation (28) of an input image (12) processed by an image processing processor (16), the method comprising the steps of: providing (30) the input image (12), performing a first image processing (32) on the input image (12) by the image processing processor (16) according to a processing parameter (18) of the image processing processor (16), outputting (33) the processed input image (12) as an output image (20), performing a second image processing (34) on the output image (20) by means of a trained image evaluation algorithm (36), and outputting (40) an image evaluation result (38) according to the second image processing (34), wherein: The image evaluation result (38) specifies at least one image feature of the output image, with which a subsequent image analysis (27) of the output image (20) is dependent.

2. The computer-implemented method (28) of claim 1, wherein: The image assessment algorithm (36) processes the output image (20) by machine learning.

3. A computer-implemented method for automatically setting (42) processing parameters (18) of an image processing processor (16), the method comprising the steps of: providing (30) an input image (12), performing a first image processing (32) on the input image (12) by an image processing processor (16) according to a processing parameter (18) of the image processing processor (16), outputting (33) the processed input image (12) as an output image (20), carrying out (43) an automated image evaluation (28) according to any one of the preceding claims, and The processing parameters (18) are set (44) based on the image evaluation results (38).

4. The computer-implemented method (42) of claim 3, wherein: The numerical optimizer (46) outputs recommended processing parameters (48) based on the image evaluation results (38).

5. The computer-implemented method (42) of claim 4, wherein: The suggested processing parameters (48) are applied to the image processing processor (16).

6. The computer-implemented method (42) of claim 5, wherein: The input image (12) processed with the aid of the recommended processing parameters (48) is passed back as an output image (20') to the image evaluation algorithm (36), which then outputs an updated image evaluation result (38'), wherein the updated image evaluation result (38') is used to recalculate the recommended processing parameters (48').

7. The computer-implemented method (42) of claim 6, wherein: The re-proposed processing parameters (48') are applied to the image processing processor (16).

8. A computer-implemented method (10) for image processing an input image (12) of an image sensor (14), wherein the input image (12) is processed by an image processing processor (16) according to processing parameters (18) of the image processing processor (16) set by a computer-implemented method (42) according to any one of claims 3 to 7.

9. An image processing system (22) for performing image processing on an input image (12) of an image sensor (14), the image processing system having an image processing unit (24) configured to implement the computer-implemented method (10) according to claim 8, the image processing unit having at least one image processing processor (16) for processing the input image (12) according to processing parameters (18).

10. An image analysis unit (26) for performing an image analysis (27) on an input image (12) processed by the image processing system (22) according to claim 9.

Images