Method and System for Reducing Personal Information in Camera Images

Abstract

While ensuring efficient protection of sensitive data, make the information from the camera images of the camera unit available for use in the execution of vehicle functions such as an assist system. 【Solution】The present invention relates to a method for reducing personal information in a camera image of a camera unit (1), wherein the processed camera image is used to execute a vehicle function, and the original image data of the camera unit (1) is degraded in a plurality of stages before processing. Each stage is degraded in a characteristic manner at that stage and has a variable parameter for determining the intensity of degradation. To determine the distribution of degradation over a plurality of stages by a calculation unit (5), the value of the parameter for each stage is obtained via an optimization algorithm using i) at least the information to be provided for the vehicle function and ii) the desired level of restriction of personal information, and i) and ii) are used as target quantities.

Description

Technical Field

[0001] The present invention relates to a method for reducing personal information in a camera image of a camera unit by degrading the camera image, a system for reducing personal information in a camera image of a camera unit by degrading the camera image, and a vehicle equipped with such a system.

Background Art

[0002] Image data of a camera unit can be automatically processed by an image analysis method in the so-called "computer vision" field. In such processing, for example, it is possible to confirm how many people are currently in the detection area of the camera unit and whether an object is identified in the detection area: In particular, in the application of an automatic driving or driving assistance system for a vehicle, such an automated image analysis method can enhance safety on the road, and in some cases, it is also expected that the implementation of such image analysis will be mandatory for vehicle certification. For example, in many regions of the world, for specific applications, some applications important for safety and certification, such as for vehicle functions such as driver or passenger monitoring, or for operating assist systems that must always be activated while the vehicle is in motion, it is necessary to provide an in-vehicle camera. This basically provides safety-technical advantages by such an in-vehicle camera, while at the same time posing a dilemma that the privacy of the passengers is clearly restricted, especially when the faces of the passengers can be theoretically identified in the original camera image of the camera unit. There is a risk of unauthorized access to the camera image by a hacker attack, while it is impossible to completely stop such a camera unit in order to operate vehicle functions or assist systems. Therefore, it is desired to protect the privacy of people, other delicate situations, and objects that appear in the detection area of the camera unit, and at the same time maintain the functions of automated applications, vehicle functions, or assist systems that depend on the existence of the camera image of the camera unit.

[0003] The following Patent Document 1 relates to making an image area containing personal information unidentifiable. For this purpose, a corresponding image section containing personal information is required, and based on this, an operation is performed to make the area unidentifiable. As personal information, in particular, a human face, a vehicle number plate, a window part of a house, etc. are applicable. In order to make the image area unidentifiable (mask it), the corresponding image area can be decomposed / split in an encrypted form. Furthermore, it is disclosed that in order to restore the original information in the detected image, the corresponding process for making it unidentifiable can also be executed in the reverse direction.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem of the present invention is to ensure efficient protection of confidential data (delicate data), and further, to make the information from the camera image of the camera unit usable for the execution of vehicle functions such as an assist system.

Means for Solving the Problems

[0006] The present invention becomes apparent from the features of the independent claims. Advantageous developments and embodiments are the subject of the dependent claims.

[0007] A first aspect of the present invention relates to a method for reducing personal information in a camera image due to degradation of the camera image of a camera unit. In this method, the camera image processed by image analysis is used to execute a vehicle function, such as an assist system. To reduce the personal information in the camera image, before processing, the original image data of the camera unit is degraded in a plurality of stages of image processing. Each stage of the plurality of stages degrades in a manner characteristic of that stage. Each stage of the plurality of stages has at least one variable parameter for determining the intensity of degradation at that stage. By means of a calculation unit, in order to determine the distribution of degradation over a plurality of stages, the values of the parameters for each stage of the plurality of stages are obtained via an optimization algorithm using i) at least the information to be provided for the vehicle function and ii) the desired level of restriction of personal information. The above i) and ii) are used as target quantities or target values, and the parameters for reaching the target quantity are determined. The information to be provided for the vehicle function at least represents the minimum requirements for the vehicle function to be available within a desired range, i.e., for example, available without restrictions, or available under slight restrictions, or available under major restrictions.

[0008] The camera unit is preferably arranged on or in a vehicle and supplies camera images for vehicle functions, such as for automated use of the vehicle. Vehicle functions of that kind are, for example, observation of the interior of the vehicle to identify the number of occupants in the vehicle, personalization of vehicle functions using face recognition, fatigue warning systems, etc. However, for example, a camera can also be arranged on the exterior of the vehicle to provide visual data for vehicle functions of the vehicle, such as for identifying traffic signs or classifying road users around the vehicle into predetermined categories, such as cyclists, pedestrians, other vehicles, etc.

[0009] For the applications exemplified above, chamber systems of different technical types can be used for the camera unit. In particular, one can be used from each of the following: RGB camera, IR camera, FIR / NIR / thermal imaging camera, Time-of-Flight camera, stereo camera, Structured Light Camera.

[0010] The multi-purpose in-vehicle camera (MPIC) is, for example, an in-vehicle camera arranged on the center console of the vehicle. This camera can supply signals to a number of systems such as Attention Assist (identifying driver fatigue and reduced attention, driver observation for authentication), a driving assistance system with a Hands-Free Driving function, personalization by identifying the driver and passengers, or an Interior Assistant (recognizing humans and gestures), and other systems.

[0011] The camera image of the camera unit not only contains information for vehicle functions but often also contains sensitive data regarding personal privacy. In particular, personal information includes information suitable for human identification, such as information sufficient for face recognition. However, the information itself related to objects other than humans may also contain other sensitive data worthy of protection regarding personal information, such as a vehicle's license plate number or a house number.

[0012] However, depending on the vehicle functions, some personal information of that kind is not necessary for the execution of the vehicle functions. Therefore, according to the present invention, it is proposed to degrade personal information to some extent, if possible, at various stages in the data path from the sensor of the vehicle's camera unit to the computer unit that processes it. At that time, the information provided after degradation must reach at least the target amount defined as the information to be provided in order to provide sufficient quality for executing the vehicle functions.

[0013] The degradation of the camera image is performed by an algorithm or mechanism that characterizes the steps for degrading each camera image by a processing step such as image processing, that is, artificially converting the information interpretable in the whole pixels of each camera image until it becomes difficult to interpret, that is, until it becomes difficult to identify personal data and personal information. The degradation is performed by processing steps such as a process for achieving a reduced resolution of each camera image, a process for applying overexposure, a process using a modified tone value curve, a process using a bilateral filter / guided filter / caricature filter, etc. Furthermore, known methods of computational imaging and known image processing filter methods can also be used for degradation.

[0014] As a result, in one embodiment, the degraded camera image can be completely color-shifted, noise can be included in the degraded camera image, and the resolution of the degraded camera image can be lowered as compared with the original image data. In another embodiment where the original color of the original image data is considered important for properly executing the vehicle functions, preferably, the parameters of each stage are changed so that the original color of the original image data is retained.

[0015] For each of the individual camera images each time, in order to ensure that the vehicle functions can be executed and function properly, there is a need for a balance, i.e., a compromise point to be found, between the one goal of obtaining as much information as possible from the original image data and the other goal of removing as much personal information as possible from the original image data. These are basically competing goals, and in a first variant of the first aspect of the present invention, a selection between these goals is made such that the two goals act in an optimization algorithm as target quantities.

[0016] For this purpose, multi-objective optimization is performed using the aforementioned goals i) and ii) as respective target quantities, and those target quantities are optimized in a common cost function, particularly with weighting.

[0017] In an alternative or additional second variant of the first aspect of the present invention, when both target quantities cannot be achieved simultaneously by changing parameters, the optimization algorithm optimizes the parameters according to the application case to be one of the following: The information provided for the vehicle function reaches target quantity i), and the level of restriction of the personal information achieved is as close as possible to target quantity ii), or Regarding the level of restriction of personal information, target quantity ii) is reached, and the information provided and reached for the vehicle function (the achieved information provided for the vehicle function) is as close as possible to target quantity i). This variant is immediately used when values for parameters that can achieve both target quantities i) and ii) cannot be found, or at least cannot be found within a predetermined time.

[0018] Therefore, according to the present invention, the following alternative forms are possible: - Give priority to privacy by restricting and removing personal information according to target quantity ii), and make the information provided for the vehicle function as close as possible to target quantity i), or - Depending on the target quantity i), prioritize the information to be provided for the vehicle function, and for personal information, approach the desired level of restriction ii) as closely as possible.

[0019] Regardless of whether the optimization algorithm solves the linear optimization problem through analysis, iteratively solves the non-linear optimization problem, or executes a database-based solution (database-based solution: table, lookup table), the result is the distribution of the degradation of the original image data over different stages, using respective characteristic ways for degradation. By determining the values for the parameters of each stage, the intensity of degradation at each stage, and thus in each characteristic way, is determined. However, here, the problem space (determined by the number of parameters in each stage) is typically high-dimensional and typically cannot be solved by a simple compromise such as a one-dimensional parameter restriction. Overall, typically there may be more than a thousand parameters over a single-digit number of processing stages, but there may also be parameters of a larger order (orders of magnitude different numbers).

[0020] Therefore, each stage has at least one parameter that can determine the magnitude and / or manner of degradation of the camera image in a manner specific to that stage. Thus, with all the parameters, after determining the values of the parameters, the distribution of degradation through the degradation manner specific to each stage is determined. That is, in the current situation, the proportion of degradation at each stage is determined particularly depending on the vehicle function.

[0021] The goal of the optimization algorithm is to determine the values of the parameters for the successive stages of processing, and thus to determine the distribution of degradation over multiple stages, as well as to determine the overall intensity of degradation, and in many embodiments (see below), to determine the local intensity distribution of the degradation of the image data within the camera image.

[0022] Therefore, it is an advantageous effect of the present invention that the camera images used to execute vehicle functions are efficiently reduced to some extent with respect to personal information. In particular, the reduction effect is achieved by dividing the reduction into various stages, where each of the stages provides a specific mechanism for reducing personal information. As long as that is the case, the physical and logical characteristics of each stage can be optimally utilized according to what boundary conditions exist with respect to the image characteristics and how much personal information should be removed from the camera image. Furthermore, a compromise point can be defined and implemented between the competing requirements of protecting privacy as much as possible with respect to the personal information contained in the camera image and leaving as much information as possible in the camera image for executing vehicle functions. Therefore, advantageously, in particular, the requirements for privacy can be considered as well as the requirements for vehicle functions such that the vehicle functions do not need to accept losses within their functional scope or only need to accept minor losses, while on the other hand, respect for privacy is significantly improved. Therefore, there is a systematic implementation means for optimizing the balance between data protection and application functions. That is, in principle, an attacker can only obtain the degraded camera image, in which delicate data has already been completely removed or mostly removed.

[0023] According to an advantageous embodiment, a preference measure between at least the information that should be provided to the vehicle function, or a measure of the desired level of restriction of personal information, and / or a prioritization of one of the target amounts is set by the user.

[0024] In other words, the user can set and influence the target amount or target value for the level of functionality of the application based on the quality of the information to be provided, or the target amount or target value for the level of personal information. For example, the user can desire a high level of restriction of personal data, i.e., that the image data contains only a small amount of personal data, or can decide to desire extensive information transmitted by the image data for vehicle functions that are not restricted. This setting by the user is preferably performed in the graphical user interface of the vehicle's operating computer. Here, graphical elements such as sliders can be advantageous, but depending on the situation and application, individual inputs via check boxes in the graphical interface can also be used.

[0025] In the case of safety-critical vehicle functions, or vehicle functions that are legally required by certification regulations, the target amount can only be reduced to a minimum value, or the user's qualifications can also be restricted to developers or factory employees. Therefore, advantageously, various user authorities are provided to enable the setting described above.

[0026] According to another advantageous embodiment, a preference measure between at least the information to be provided for the vehicle function, and the desired level of restriction of personal information, and / or the prioritization of one of the target amounts is set by the calculation unit, for example depending on each vehicle function, speed, driving situation, ambient conditions, etc.

[0027] Unlike the foregoing embodiments, the balance between i) and ii) is taken by the computing unit itself, particularly according to each vehicle function. That is, for example, for safety-critical vehicle functions, the computing unit can define a target amount of information that should be provided at least for the vehicle function, and in the remaining range, the restriction of personal information can be maximized. This can also be applied to individual aspects and can be considered in the distribution of degradation over individual stages. For example, if the eye color is important for the vehicle function, the computing unit autonomously identifies that a color shift (which is useful for the restriction of personal information) should not be performed in this configuration.

[0028] According to another advantageous embodiment, the image processing stage includes the following: the image sensor of the camera unit, particularly the registers of the image sensor, the hardware settings in the control device of the camera unit, particularly calibration data, and the software processing in the control device of the camera unit for image processing executed according to the algorithm.

[0029] Restrictions on personal information are achieved at multiple stages. These multiple stages preferably include the stage of the sensor itself, as described above in general terms, where personal information can be directly removed from the data at the source. Suitable algorithms at the sensor stage include the following applications: 2x2 binning, 8x subsampling, high value saturation, 16x gain, image cropping, automatic exposure control, and tone mapping for optimal privacy. Another suitable stage is realized by hardware settings in the control device of the camera unit connected downstream of the sensor. This stage is realized in particular in the hardware of an ECU (e.g., an infotainment central computer). Here, an algorithm more complex than the algorithm in the sensor can improve data protection without impairing vehicle functions. By being realized in hardware, this stage is advantageously protected from external attackers. Suitable algorithms at the hardware stage include the following applications: tone mapping for optimal privacy, minimum saturation, sharpness reduction with edge preservation. Another suitable stage connected downstream of the hardware settings in the control device of the camera unit is the pre-processing of the software in the control device of the camera unit, which is ideally suitable for flexible image processing executed according to an algorithm, whereby personal data is removed without impairing vehicle functions. Here, personal information can be removed from the data by applying a complex algorithm (e.g., a neural network). However, by realizing this third stage in software, this stage may be more vulnerable to attacks. Suitable algorithms at the software stage include the application of complex wavelet SSIM (CW_SSIM), where the abbreviation SSIM stands for "Structural Similarity Index Measure". CW_SSIM can be used as a measure for information content.For this preferred algorithm, the filters and parameters described below are used, whereby personal information is reduced as much as possible and at the same time, the vehicle functions are guaranteed using CW_SSIM monitoring.

[0030] In another embodiment, in the software processing stage in the control device of the camera unit, filters (in particular, at least one of a noise filter, a sharpening filter, a scaling filter, a tone value curve filter, a luminance filter, a color change filter) for image processing executed according to an algorithm are used, and these filters are implemented by an artificial neural network using parameters for parameterizing them. For this purpose, preferably, general steps, in particular in the form of CNNs (convolutional neuronal networks) are set. A seamless series of CNNs can be comprehensively optimized by a known learning method (using millions of parameters in some cases). The basic premise here is that the camera images obtained at this stage do not necessarily have to look particularly sharp and neutral in order to optimally execute the vehicle functions. Instead, by performing special emphasis (e.g., of edges), due to its non-linear characteristics, the use of information for vehicle functions can be significantly improved. Therefore, this extended embodiment realizes an implementation that is not available in prefabricated "building blocks" (which is still the case in current series projects).

[0031] Each algorithm for each stage is preferably adapted to the maximum available computing power of the computing unit, the data format used, the security requirements, etc. Typically, in the sensor which is the preferred first stage described above, only a small amount of computing power can be utilized, so here preferably a simple algorithm is implemented (for example, saturating the pixels in a specific area to remove personal information). Therefore, the complexity of the applications in the other preferred stages described above tends to be selected to be higher by the computing unit. The data is transmitted between stages, particularly via physical channels (for example, using a cable from the camera unit to the ECU, the transmission from the "sensor" stage to the "hardware setting" stage is performed).

[0032] In another advantageous embodiment, the stage upstream of the sensor is used, in which, for example, an external signal is guided to the sensor as desired to weaken the detection of the sensor itself. In a preferred embodiment, it is achieved by an active light source (especially in the infrared region, by changing the intensity and / or pattern of the existing illumination or additional illumination and / or projector) to artificially reduce the quality of the camera image.

[0033] According to another advantageous embodiment, the desired level of restriction of personal information is defined using a structural similarity index for the camera image. According to this embodiment, the target amount of the desired level of restriction of personal information is defined by the so-called "Structural Similarity Index Measure (SSIM)". The following is used as a suitable measure for quantifying the desired level of restriction: (1 - CW_SSIM), which can be expressed in words as "1 minus CW_SSIM", where "CW" represents "Complex Wavelet" and "SSIM" represents "Structural Similarity Index Measure". Further information regarding this can be obtained from the publication, "Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures" by Z. Wang and A. C. Bovik, IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98 - 117, January 2009, doi: 10.1109 / MSP.2008.930649.

[0034] According to another advantageous embodiment, the information that should be provided at least for vehicle functions includes the average standard deviation or the signal - to - noise ratio regarding the camera image.

[0035] Regarding the average standard deviation (abbreviation "MSE") as well, further information can be obtained from the publication, "Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures" by Z. Wang and A. C. Bovik, IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98 - 117, January 2009, doi: 10.1109 / MSP.2008.930649.

[0036] According to another advantageous embodiment, a computing unit arranged within the vehicle is used, and in this case, the values of the parameters for each stage are continuously updated and obtained by the computing unit in order to continuously update and determine the distribution of degradation over multiple stages.

[0037] Accordingly, the up-to-date parameters at each time, as determined by the computing unit, are determined locally on board, i.e., in the vehicle itself. By continuously updating the determination of the degradation distribution, the degradation distribution over multiple stages, as well as the overall intensity of the degradation, are adapted in real time. Accordingly, advantageously, adaptation to the current conditions can be carried out in order to be able to optimally distribute information continuously between the vehicle functions and the goal of reducing personal information.

[0038] According to another advantageous embodiment, the parameters for each stage are determined by the computing unit depending on the determined situation parameters, where the situation parameters in particular include one of the following: the distance of a person's face to the camera unit, the movement of the person's face relative to the camera unit, ambient conditions such as general (dominant) luminance, the driving situation.

[0039] Accordingly, in particular, the restriction of personal information can be varied. This is because, for example, a face close to the camera is more critical with regard to privacy than a face far away in the rear seat of a dark vehicle. For this purpose, preferably, predictions are made using artificial intelligence or regression methods even if the previously used reference image no longer exists during operation.

[0040] Determining the values of the parameters for each stage for determining the distribution of degradation over a plurality of stages by an optimization algorithm is advantageously performed adaptively, particularly with respect to the geometric ROIs (regions of interest) for vehicle functions, whereby information about the ROI, i.e., this special region, can be obtained to the greatest extent possible. These may vary depending on the scene. The face ROI can be determined using face detection according to known algorithms. There are also face detectors in the prior art (for face recognition) that have very robust characteristics with respect to a reduction in spatial resolution. Furthermore, the characteristics of each vehicle function regarding the sensitivity of the parameter values to specific image characteristics such as noise, defective structures, and defective contrast can be considered. Here, the expected scene or the actual current scene can also be used (e.g., from the perspective of the luminance distribution that can be represented by the dynamic range, histogram) to determine the values of the parameters in the optimization algorithm. Furthermore, in each region of the camera image, combinations of different parameter values can be applied with respect to location, time, and content to perform the degradation of each stage.

[0041] According to another advantageous embodiment, by means of a computing unit, the values of the parameters for each stage are defined for a pre-defined camera image or a camera image derived from a pre-defined scene and stored and held in the control unit of the vehicle.

[0042] Advantageously, according to this embodiment, when a predefined camera image occurs repeatedly, or when a camera image derived from a predefined scene occurs, there is no need to newly determine the value of the parameter. Instead, a predefined set of predefined values of the parameters that have already been determined off-board in the past can be used. That is, advantageously, unnecessary computational costs can be saved. If it can be assumed that the situation detected by the camera unit is a situation that is almost exactly reproduced, a predefined camera image can be used. In contrast, a camera image derived from a predefined scene is used more flexibly, and only the matching of the scene features is required. Once a set of parameter values is obtained, those sets are associated with predefined camera images or predefined scenes and stored in the control unit so that the calculation unit can access them to provide an alternative source to the optimization algorithm.

[0043] According to another advantageous embodiment, the values of the parameters stored in the control unit are used for degradation only when a predefined camera image or a camera image derived from a predefined scene exists during operation, instead of those values of the parameters that are continuously updated by the calculation unit.

[0044] According to another advantageous embodiment, the calculation unit executes the determination of the values of the parameters using a numerical method, in particular an iterative method.

[0045] The iterative and numerical method is advantageously used, in particular, to iteratively approximate a predefined target quantity with respect to i) or ii), i.e., until the required threshold of the target quantity is met, or until at least one threshold i) or ii) of the target quantity is achieved and the corresponding other threshold is achieved in the best possible range. In multi-objective optimization, an iterative search algorithm can be used in a non-linear optimization problem.

[0046] According to another advantageous embodiment, the calculation unit uses a pre-trained artificial neural network to determine the values of the parameters.

[0047] Here, the optimization algorithm utilizes the pre-trained artificial neural network for the specification of the parameters. The possible input variables of the pre-trained artificial neural network are, in particular, the respective camera images as well as the target quantities i) and ii), and the output values are the parameters of each stage.

[0048] According to another advantageous embodiment, the pre-trained artificial neural network is continuously further trained in the server based on data from the vehicle's camera unit, and the update of the artificial neural network is transmitted to the vehicles of the vehicle fleet.

[0049] In another preferred embodiment, the elements of each stage and their parameters are not only combined from a set of pre-manufactured filters, but are completely generated using a deep learning method similar to Generative Adversarial Networks (GANs). In one extended embodiment, furthermore, the hardware components are also taken into account in the objective function according to another aspect. Here, in particular, the resource costs are also reproduced, such that these resource costs are taken into account in the optimization algorithm. For example, when the level of restriction of personal information is the same and the similar (in particular, substantially the same degree of) quality and quantity of the information to be provided for the vehicle functions are the same, the variations of the parameters that are executable in the calculation unit, in particular for selecting the filter module, are selected in an especially efficient manner, or at each respective stage.

[0050] Another aspect of the present invention relates to a system for reducing personal information in a camera image due to degradation of the camera image of a camera unit, in which the camera image processed by image analysis is used, in particular in a vehicle, to execute vehicle functions, and a computing unit is configured to degrade the original image data of the camera unit in a plurality of stages before processing in order to reduce personal information in the camera image, each of the plurality of stages performing degradation in a manner characteristic of that stage, each of the plurality of stages having at least one variable parameter for determining the intensity of degradation in that stage, and the computing unit is configured to determine the values of the parameters for each stage of the plurality of stages by means of an optimization algorithm using i) at least the information to be provided for the vehicle function and ii) the desired level of restriction of personal information, where i) and ii) above are used as target quantities and the parameters for reaching the target quantities are determined.

[0051] Another aspect of the present invention relates to a vehicle equipped with the system described above and described below.

[0052] The advantages and preferred developments of the proposed system become apparent by replacing, in the same way and meaningfully, the explanations made above in connection with the proposed method.

[0053] Other advantages, features, and details will become apparent from the following description, in which at least one embodiment is described in detail, with reference to the drawings as necessary. Identical, similar, and / or functionally identical parts are provided with the same reference signs.

Brief Description of the Drawings

[0054]

Figure 1

Figure 2

Figure 3

Figure 4

MODE FOR CARRYING OUT THE INVENTION

[0055] Figure 1 shows the interior of a vehicle 3 equipped with a camera unit 1 and a computing unit 5. The computing unit 5 is used to execute a method for reducing personal information in each camera image of the camera unit 1 due to degradation of the camera image. The camera images are repeatedly generated at a high frequency by the camera unit 1 for vehicle functions. A Multi-Purpose Interior Cam is used to execute vehicle functions such as an automatic warning regarding fatigue. However, thereby, personal data is also detected as a secondary effect. That is, the data in the camera image, which is in principle sufficient for automated face recognition performed in the same way as by a human, has a security risk in that an attacker may access the data from the outside. Therefore, the goal is to remove as much data as possible that can be used to identify the photographed person in each camera image without impairing the vehicle function. In order to reduce personal information in the camera image before processing by the vehicle function, the original image data is degraded at the processing stage of the camera unit 1, i.e., its quality is artificially reduced. At each stage, the degradation is performed in a manner characteristic of that stage. In order to adjust which of those stages bears which part of the degradation and to what extent the degradation should be performed overall in each scenario and for each vehicle function to obtain a preferable result, each stage has a set of parameters whose values are variable. This distribution is determined by the computing unit 5 by doing the following. That is, in order to determine the distribution of degradation over a plurality of stages, the values of the parameters for each stage are obtained by an optimization algorithm using i) the information to be provided for the use of the vehicle function and ii) the desired level of restriction of personal information. The competing goals i) and ii) are set in advance as the target quantities of an iterative non-linear optimization algorithm for performing multi-objective optimization so that the information provided for the vehicle function reaches the target value i) and the level of restriction of personal information reaches the target value ii).Instead, the optimization of the parameters is performed such that the information provided for the vehicle function reaches the target value i), and the level of restriction of personal information comes as close as possible to the target amount ii), or for the level of restriction of personal information, the target value ii), i.e., the desired level of restriction, is achieved, and the information provided for the vehicle function comes as close as possible to the target amount i).

[0056] The stages are configured to include the following: 1a the image sensor of the camera unit 1, in particular the registers of the image sensor; 1b the hardware settings in the control device of the camera unit 1, in particular the calibration data; 1c the software processing in the control device of the camera unit 1 for image processing executed according to an algorithm. Further, although not described in detail, an optional stage 1d is shown, and this stage 1d includes other parameterizable settings that affect the targets i) and ii), such as illumination of an object, post-processing, etc. This is shown in more detail in FIG. 2. Starting from the camera unit 1, various methods are already applied in the first stage 1a at the sensor to correct the original camera image of the camera unit 1. Each parameter in this stage is related to the above-described image sensor of the camera unit 1. Here, methods such as 2x2 binning, 8x subsampling, high value saturation, 16x gain, exposure control, and tone mapping for optimal privacy can already be implemented. The hardware settings in the control unit of the camera unit 1 represent the second stage 1b and have other parameters in the following ways: tone mapping for optimal privacy, minimum saturation, sharpness reduction with edge preservation. In the third stage 1c, which is provided further upstream of the processing of the image degraded by the vehicle function, complex wavelet SSIM (CW_SSIM) is used. Here, SSIM is an abbreviation for "Structural Similarity Index Measure". The parameter set having the values of the parameters of all stages is determined on-board uniquely for each camera image iteratively by the calculation unit 5 of the vehicle 3 as the overall parameter.

[0057] For this purpose, the images recorded by the camera 1 and processed in each stage are compared with the actual value 7a of the information contained in the image regarding the information i) that should be provided at least for the vehicle function as the target amount 7 for realizing a predetermined function range, and also, the information ii) regarding the desired level of restriction of personal information as the target amount 9 is analyzed by being compared with the actual value 9a of the image regarding the level of personal data. As long as the actual values 7a, 9a are below the target amounts, the calculation unit 5 ideally optimizes the parameters of all stages 1a to 1d in, for example, an iterative optimization process until the actual values of the information provided for the vehicle function and the level of personal data by the images processed in stages S1 to S4 reach at least the target amounts 7, 9, and then outputs the image 11.

[0058] If the actual value of the information provided for the vehicle function and the level of personal data cannot reach the target amounts 7, 9 simultaneously, the parameters of stages 1a to 1d are determined such that the information provided at least for the vehicle function reaches the target amount i), or regarding the level of restriction of personal information, reaches the target amount ii), and the other values in each case are optimized as much as possible. In the case of functions related to safety such as driver observation, the information provided for the vehicle function is prioritized so that the information reaches the target amount required for the function.

[0059] FIG. 3 shows an embodiment in which the optimization algorithm of the calculation unit is implemented as a pre-trained neural network. The neural network is designed to determine the parameters of stages 1a to 1d based on the image recorded by the camera 1, the predetermined target amount i) for the information that should be provided at least for the vehicle function, and the desired level 9 of restriction of personal information, such that the image 11 processed in stages 1a to 1d and then output provides the information 7a required for the vehicle function and complies with the desired level 9a of restriction of personal information.

[0060] Figure 4 shows an exemplary flowchart of the method according to the present invention. In step S1, an image is received. In step S2, the image is processed using predetermined image processing parameters in stages 1a to 1d. In step S3, it is determined whether the information for the vehicle function provided by the image has reached the target amount i), that is, the information (7) that should be provided at least for the vehicle function, and whether the level of restriction of the personal information in the image has reached the desired level of restriction of the personal information. If the result of the determination is an affirmative result, in step S4, the image is output. If the determination in S3 is negative, in step S5, the image processing parameters are changed by the optimization algorithm of the calculation unit 5 and passed to stages 1a to 1d. In step S2, the image is processed using the changed parameters, and then in step 3, it is compared again with the target amounts i) and ii). The optimization in step S5 is performed until the target amounts i) and ii) are reached, and then in step S4, the image can be output.

[0061] The present invention has been illustrated and described in more detail by means of preferred embodiments. However, the present invention is not limited to the disclosed embodiments, and those skilled in the art can derive other modified forms without departing from the protection scope of the present invention. Therefore, it is obvious that there are numerous possible modified forms. Similarly, the exemplified embodiments are merely examples in fact, and it is obvious that they should not be understood as limiting the protection scope, applicability, or configuration of the present invention in any way. Rather, from the foregoing description and the description of the figures, those skilled in the art can specifically implement the exemplified embodiments. At that time, by knowing the idea of the disclosed invention, those skilled in the art can make various changes regarding, for example, the functions or arrangements of the individual elements exemplified in the exemplified embodiments without departing from the protection scope defined by the claims and their legal equivalents, for example, the sufficient description in the specification.

Claims

1. In a method for reducing personal information in a camera image due to degradation of the camera image of a camera unit (1), the camera image processed by image analysis is used, in particular in a vehicle (3), to execute a vehicle function, and in order to reduce the personal information in the camera image, before the processing, the original image data of the camera unit (1) is degraded in a plurality of stages, in each of the plurality of stages, degradation is performed in a manner characteristic of each stage, each of the plurality of stages has at least one variable parameter for determining the intensity of degradation in each stage, in order to determine the distribution of the degradation over the plurality of stages, the values of the parameters for each of the plurality of stages are determined by a calculation unit (5) via an optimization algorithm using i) at least the information (7) to be provided for the vehicle function and ii) the desired level (9) of restriction of personal information, i) and ii) are used as target quantities, and parameters for reaching the target quantities are determined A method, characterized by this.

2. If it is not possible to achieve both target quantities simultaneously by changing the parameters, the optimization algorithm, depending on the application case, so that the information provided for the vehicle function reaches the target quantity i), and the level of restriction of personal information comes as close as possible to the target quantity ii), or for the level of restriction of personal information, the target quantity ii) is reached, and the information provided for the vehicle function is optimized so as to come as close as possible to the target quantity i) A method according to claim 1, characterized by this.

3. A preference measure between the priority rankings of at least the information to be provided for the vehicle function, or the desired level of restriction of personal information, and / or one of the target quantities is set by the user A method according to claim 1 or claim 2, characterized by this.

4. A preference measure between the priority rankings of at least the information to be provided for the vehicle function, and the desired level of restriction of personal information, and / or one of the target quantities is set by the calculation unit (5) A method according to claim 1 or claim 2, characterized by this.

5. The plurality of stages include the image sensor of the camera unit (1), in particular the register of the image sensor, the hardware settings in the control device of the camera unit (1), in particular calibration data, and the software processing in the control device of the camera unit (1) for image processing executed according to an algorithm. The method according to any one of claims 1 to 4, characterized in that.

6. The desired level of restriction of the personal information is defined using a structural similarity index for the camera image. The method according to any one of claims 1 to 5, characterized in that.

7. The information that should be provided at least for the vehicle function includes the average standard deviation or the S / N ratio for the camera image. The method according to any one of claims 1 to 6, characterized in that.

8. A computing unit (5) arranged in the vehicle (3) is used, and the value of the parameter for each stage in the plurality of stages is continuously updated by the computing unit (5) in order to continuously update and determine the distribution of deterioration over the plurality of stages. The method according to any one of claims 1 to 7, characterized in that.

9. The parameter for each stage in the plurality of stages is determined by a computing unit (5) depending on the determined situation parameter. The situation parameter includes, in particular, one of the distance of a person's face to the camera unit (1), the movement of a person's face relative to the camera unit (1), ambient conditions such as general brightness, and driving conditions. The method according to claim 8, characterized in that.

10. The value of the parameter of the plurality of stages is defined by the computing unit for a pre-defined camera image or a camera image derived from a pre-defined scene and stored in the control unit of the vehicle (3). The method according to any one of claims 1 to 9, characterized in that.

11. When a pre-defined camera image or a camera image derived from a pre-defined scene exists during operation, the value of the parameter stored in the control unit of the vehicle is used for the deterioration instead of the value of the parameter continuously updated by the computing unit (5). The method according to claim 10, characterized in that.

12. The determination of the value of the parameter is performed by the calculation unit (5) using a numerical method The method according to any one of claims 1 to 11, characterized in that

13. The determination of the value of the parameter is performed by the calculation unit (5) using a pre-trained artificial neural network The method according to any one of claims 1 to 12, characterized in that

14. The pre-trained artificial neural network is continuously further trained on a server based on data from a camera unit of a vehicle, and an update of the artificial neural network is transmitted to vehicles in a vehicle fleet The method according to claim 13, characterized in that

15. In a system for reducing personal information in a camera image of a camera unit (1) due to degradation of the camera image The camera image processed by image analysis is used, in particular in a vehicle (3), to execute vehicle functions, and a calculation unit (5) is configured to degrade the original image data of the camera unit (1) in a plurality of stages before the processing in order to reduce personal information in the camera image In each stage of the plurality of stages, degradation is performed in a manner characteristic of each stage Each stage of the plurality of stages has at least one variable parameter for determining the intensity of the degradation in that stage The calculation unit (5) is configured to determine the values of the parameters for each stage of the plurality of stages by means of an optimization algorithm using i) at least the information to be provided for the vehicle function and ii) the desired level of restriction of personal information, in order to determine the distribution of the degradation over the plurality of stages i) and ii) are used as target quantities, or one target quantity and one restriction from the set of i) and ii) are used in the optimization algorithm A system, characterized in that

16. A vehicle (3) comprising the system according to claim 15

Images