Method and device for restricting personal information in a camera image

Abstract

The invention relates to a method for reducing personal information in a camera image from a camera unit (1), the processed camera image being used to carry out a vehicle function, wherein: before the processing, original image data from the camera unit (1) are degraded in stages, each of which performs a type of degradation characteristic to the stage in question and has a variable parameter for determining the intensity of the degradation; in order to determine the distribution of the degradation over the stages, a computing unit (5) establishes the values of the parameters for each of the stages by means of an optimization algorithm using i) the minimum information to be provided for the vehicle function and ii) a desired level of restriction of personal information, i) and ii) being used as target variables.

Description

[0001] Method and device for restricting personal information in a camera image

[0002] The invention relates to a method for reducing personal information in a camera image of a camera unit by degrading the camera image, a system for reducing personal information in a camera image of a camera unit by degrading the camera image, and a vehicle with such a system.

[0003] Image analysis methods in the field of so-called "computer vision" allow the automatic processing of image data from a camera unit. This can be used, for example, to determine whether and how many people are currently within the camera unit's detection range, whether objects are detected within the detection range, and similar factors. Particularly in applications involving automated driving of vehicles or driver assistance systems, such automated image analysis methods can increase road safety and are sometimes mandatory for vehicle registration. For example, in some regions of the world, vehicle interior cameras are required for specific, sometimes safety- and registration-relevant, use cases for vehicle functions such as driver or passenger monitoring or for operating assistance systems that must be constantly activated while the vehicle is moving.This fundamentally leads to the dilemma that, in addition to the safety advantages offered by such an interior camera, there are obvious restrictions on the privacy of the occupants, especially if the face of an occupant is identifiable in the original camera images from the camera unit. While there is a risk of unauthorized access to the camera images through hacker attacks, the complete deactivation of such camera units for operating vehicle functions or assistance systems is not possible. It is therefore desirable to protect the privacy of persons appearing within the detection range of a camera unit, as well as other sensitive situations and objects, while simultaneously maintaining the functionality of an automated application, a vehicle function, or an assistance system that relies on the presence of camera images from this camera unit.

[0004] US Pat. No. 8,666,110 B2 relates to the blurring of an image area containing private information. For this purpose, a corresponding image section containing private information is identified, and then this area is manipulated to make it unrecognizable. This particularly applies to a person's face, a vehicle's license plate, a house's window area, or similar. To make it unrecognizable, the corresponding image area can be encrypted, broken down, or divided. It is further disclosed that a corresponding blurring process can also be performed in reverse to restore the original information in the captured image.

[0005] The object of the invention is to ensure efficient protection of sensitive data, while information from a camera image of a camera unit can still be used to execute a vehicle function such as an assistance system.

[0006] The invention is based on the features of the independent claims. Advantageous developments and refinements are the subject of the dependent claims.

[0007] A first aspect of the invention relates to a method for reducing personal information in a camera image of a camera unit by degrading the camera image, wherein the camera image processed by image analysis serves to execute a vehicle function such as an assistance system, and wherein, in order to reduce personal information in the camera image, original image data of the camera unit are degraded in stages of image processing prior to processing, wherein each of the stages carries out a type of degradation characteristic of it and each of the stages has at least one variable parameter for determining the intensity of the degradation in the respective stage,A computing unit determines the parameter values ​​for each of the levels to determine the distribution of degradation across the levels using an optimization algorithm using i) the minimum information to be provided for the vehicle function and ii) a desired level of restriction of personal information, where i) and ii) are used as target values ​​and target values, respectively, and the parameters for achieving the target values ​​are determined. The minimum information to be provided for the vehicle function represents a minimum requirement at which the vehicle function is available to the desired extent, i.e., for example, without restriction, with slight or significant restrictions.

[0008] The camera unit is preferably arranged on or in a vehicle and provides camera images for a vehicle function, for example, an automated application of the vehicle. Such a vehicle function is, for example, the observation of the vehicle interior for the purpose of determining the number of occupants in the vehicle, for personalizing vehicle functions using facial recognition, a fatigue warning system, or similar. However, cameras can also be arranged on the exterior of the vehicle to provide visual data for a vehicle function, for example, for traffic sign recognition or categorizing road users in the vehicle's vicinity into predefined categories such as cyclists, pedestrians, other vehicles, etc.

[0009] For the applications mentioned above, a variety of technically diverse camera systems can be used for the camera unit. In particular, one of the following can be used: RGB camera, IR camera, FIR / NIR / thermal imaging camera, time-of-flight camera, stereo camera, structured light camera.

[0010] A Multi-Purpose Interior Cam (MPIC), for example, is an interior camera located in the vehicle's center console. The camera can provide signals to a variety of systems: Attention Assist (driver monitoring for fatigue and distraction detection, certification), driver assistance systems with "hands-free driving" functions, personalization with driver and passenger identification, or Interior Assistant (person and gesture recognition), among other systems.

[0011] The camera images from the camera unit often contain not only information relevant to the vehicle's function, but may also contain sensitive data relating to individuals' private lives. In particular, personal information includes information suitable for identifying a person, such as information sufficient for facial recognition. However, object-related information beyond the individual may also contain sensitive data, such as vehicle license plates, house numbers, and other sensitive, protected data relating to private information. Depending on the vehicle's function, however, a certain degree of such personal information may not be necessary to perform the vehicle's function.According to the invention, it is therefore proposed to degrade a certain amount of personal information, if possible, in various stages along the data path from the sensor of the camera unit to the processing computer unit, particularly of a vehicle, to execute the vehicle function. The information provided after the degradation must at least achieve the target value defined as the information to be provided in order to be able to provide sufficient quality to execute the vehicle function.

[0012] The degradation of the camera images is carried out using algorithms or mechanisms characteristic of the stage in order to degrade the respective camera image through processing steps such as image processing, i.e. to artificially transform the interpretable information in the entirety of the pixels of the respective camera image into less interpretable information, i.e. to a lower identifiability of personal data and personal information. The degradation is carried out in particular through processing steps such as achieving a reduced resolution of the respective camera image, by imposing overexposure, by means of a modified tonal value curve, by bilateral filters / guided filters / cartoonization filters, or similar. Furthermore, known methods of computational imaging and known image processing filter methods can be used for degradation.

[0013] As a result, in one embodiment, the degraded camera image may be completely color-shifted, noisy, and at a lower resolution compared to the original image data. In another embodiment, in which the original colors of the original image data are considered important for the proper execution of the vehicle's functions, the parameters of the stages are preferably modified so that the original colors of the original image data are retained.

[0014] This results in a necessary balancing act for each individual camera image, i.e. a compromise to be found, between the one objective of obtaining as much information as possible from the original image data in order to ensure flawless execution of the vehicle function, and the other objective of removing as much personal information as possible from the original image data. These are fundamentally competing objectives, between which a selection is made in a first variant of the first aspect of the invention such that both function as target variables in an optimization algorithm. For this purpose, for example, a multi-objective optimization is carried out with the aforementioned objectives i) and ii) as respective target variables, which are optimized in a weighted manner in a common cost function.

[0015] In an alternative or additional second variant of the first aspect of the invention, in the event that both target variables cannot be achieved simultaneously by changing the parameters, the optimization algorithm optimizes the parameters such that, depending on the application, either the target variable i) is achieved for the information provided for the vehicle function and the achieved level of restriction of personal information is as close as possible to the target variable ii), or the target variable ii) is achieved for the level of restriction of personal information and the achieved information provided for the vehicle function is as close as possible to the target variable i). This variant is used as soon as no values ​​can be found for the parameters that enable the simultaneous achievement of both target variables i) and ii), or at least no values ​​can be found within a predetermined time.

[0016] According to the invention, the following alternatives are possible:

[0017] - Prioritising privacy by restricting and removing personal information in line with target ii) and as close as possible to target i) information provided for the vehicle function or

[0018] - Prioritisation of the information to be provided for the vehicle function according to the target size i) and the best possible personal information close to the desired level of restriction ii).

[0019] Regardless of whether the optimization algorithm solves a linear optimization problem analytically, solves a nonlinear optimization problem iteratively, or performs a database-driven solution (table, look-up table), the result is a distribution of the degradation of the original image data across the various stages, each with its own characteristic degradation methods. By determining the values ​​for the parameters of a particular stage, the intensity of the degradation at that stage is determined, thus determining a characteristic method. However, the problem space (determined by the number of parameters in the stages) is typically high-dimensional and cannot typically be solved by a simple compromise such as a 1D parameter limit.In total, there may be over a thousand parameters across a typically single-digit number of processing stages, but there may also be orders of magnitude more parameters.

[0020] Each of the levels thus has at least one parameter, with the aid of which the degree and / or type of degradation of the camera image can be determined in a manner specific to that level. Thus, after determining the parameter values, all parameters as a whole determine a distribution of the degradation using the degradation methods specific to the respective levels, i.e., the proportion of degradation attributable to each of the levels in a current situation, particularly depending on the vehicle function.

[0021] The goal of the optimization algorithm is to determine the values ​​of the parameters for the successive stages of processing and thus to determine the distribution of the degradation across the stages, as well as the intensity of the degradation as a whole, and according to some embodiments (see below) also the local intensity distribution of the degradation of the image data within the camera image.

[0022] It is thus an advantageous effect of the invention that camera images used to perform a vehicle function are efficiently reduced to a certain degree of personal information. The efficiency of the reduction is achieved in particular by dividing the reduction into different stages, with each of the individual stages providing specific mechanisms for reducing the personal information. The physical and logical properties of the respective stage can thus be optimally utilized depending on the boundary conditions regarding the image properties and which and how much personal information is to be removed from the camera images.Furthermore, a compromise can be defined and implemented between the competing requirements of the highest possible privacy protection with regard to the private information contained in the camera images, on the one hand, and the highest possible proportion of remaining information in the camera images for the purpose of executing the vehicle function, on the other. Advantageously, both the privacy requirements and the vehicle function requirements can be taken into account in such a way that the vehicle function suffers little or no loss of functionality, while respect for privacy is significantly increased. This provides a systematic opportunity to optimize the balance between data protection and the application's functionality.In principle, an attacker is only able to identify the degraded camera images in which the sensitive data has already been completely or largely removed.

[0023] According to an advantageous embodiment, a measure of a preference between the minimum information to be provided for the vehicle function or a desired level of restriction of personal information and / or a prioritization of one of the target variables is specified by a user.

[0024] In other words, the user can specify and influence a target variable or target value for the level of functionality of the application through the quality of the information to be provided, or a target variable or target value for the level of personal information. For example, the user can decide that they want a high level of restriction of personal data, i.e. that the image data contains little personal data, or that they want extensive information transmitted via the image data for unrestricted vehicle functionality. This specification by the user is preferably made on a graphical user interface of a vehicle's control computer. A graphical element such as a slider can be advantageous here, but discrete inputs via checkboxes on the graphical interface can also be used, depending on the situation and application.

[0025] In the case of safety-critical vehicle functions or functions that are legally required by approval regulations, the target size can only be reduced to a minimum value, or the user can be restricted to development personnel or workshop staff. It is therefore advantageous to provide various user authorizations to enable the above-mentioned specification.

[0026] According to a further advantageous embodiment, a measure of a preference between the information to be provided for the vehicle function at least and a desired level of restriction of personal information and / or a prioritization of one of the target variables is specified by the computing unit, for example depending on the respective vehicle function, speed, driving situation, environmental conditions, etc.

[0027] In contrast to the previously described embodiment, the trade-off between i) and ii) is made by the computing unit itself, depending on the respective vehicle function. For example, a target value regarding the minimum information to be provided for safety-critical vehicle functions can be set by the computing unit, and the restriction of personal information can be maximized within the remaining scope. This can also apply to individual aspects and be taken into account when distributing the degradation across the individual levels. For example, if eye color is important for the vehicle function, the computing unit will then independently recognize that a color shift (which helps restrict personal information) must not be carried out in this constellation.

[0028] According to a further advantageous embodiment, the stages of image processing comprise the following: the image sensor of the camera unit, in particular registers of the image sensor, hardware settings in the control unit of the camera unit, in particular calibration data, software processing in the control unit of the camera unit for algorithmically executed image processing.

[0029] The restriction of personal information is achieved in several stages. As outlined above, these preferably include the sensor stage itself, as this is where personal information can be removed from the data directly at the source. The preferred algorithm in the sensor stage includes the following applications: 2x2 binning, 8x subsampling, high value saturation, 16x gain, image cropping, auto-exposure control and tone mapping for optimal privacy. Another preferred stage is implemented through the hardware settings in the control unit of the camera unit, which is connected downstream of the sensor. This stage is particularly implemented in hardware in the ECU (e.g. infotainment central computer). Here, more complex algorithms for increasing data protection are possible than in the sensor, without compromising vehicle functionality. Due to its implementation in hardware, this stage is advantageously well protected against external attackers.The preferred algorithm in the hardware stage includes the following applications: tone mapping for optimal privacy, minimal color saturation, and sharpness reduction with edge preservation. Another preferred stage, downstream of the hardware settings in the camera unit's control unit, is software preprocessing in the camera unit's control unit. This is ideally suited for flexible, algorithmic image processing to remove personal data without impairing vehicle functionality. Complex algorithms (e.g., neural networks) can be used here to remove personal information from the data. However, the implementation of this third stage in software makes this stage more vulnerable to attack. The preferred algorithm in the software stage involves the application of Complex Wavelet SSIM (CW_SSIM), where SSIM stands for "Structural Similarity Index Measure."The CW_SSIM can be used as a measure of information content. For this preferred algorithm, the filters and parameters described below are used to minimize personal information while simultaneously ensuring vehicle functionality by monitoring the CW_SSIM.

[0030] In a further embodiment, filters (in particular at least one of: noise filter, sharpening filter, scaling filter, tone curve filter, brightness filter, color change filter) are used in the software processing stage in the control unit of the camera unit for algorithmically executed image processing. These filters are implemented using an artificial neural network with parameters for parameterizing them. For this purpose, generic steps are preferably specified, particularly in the form of CNNs (convolutional neural networks). Using known training methods (sometimes with millions of parameters), a seamless chain of CNNs can thus be globally optimized.The basic assumption here is that for optimal execution of the vehicle's function, the camera image obtained for this stage does not necessarily have to appear particularly brilliant and neutral. Instead, specific highlights (e.g., edges) can significantly improve the use of the information for the vehicle's function due to their nonlinear characteristics. Thus, this extended implementation advantageously enables an implementation that is not even possible with prefabricated "building blocks" (as is still the case in current series projects).

[0031] The respective algorithm for each stage is preferably adapted to the available maximum computing power of the computing unit, the data formats used, security requirements, and the like. Since typically only limited computing power is available in the aforementioned preferred first stage, the sensor, simple algorithms are preferably implemented here (e.g., saturating pixels in certain regions to remove personal information). Accordingly, the complexity of the applications in the aforementioned further preferred stages tends to be higher, depending on the computing unit. Data is transmitted between the stages, particularly via a physical channel (e.g., from the "sensor" stage to the "hardware settings" stage via a cable from the camera unit to the ECU).

[0032] In a further advantageous embodiment, for example, a stage upstream of the sensor is used, in which external signals are specifically directed at the sensor in order to undermine the sensor detection itself. In a preferred embodiment, an artificial deterioration of the camera image is achieved by active light sources (changing the intensity and / or pattern of existing or additional illumination and / or projectors, preferably in the infrared range).

[0033] According to a further advantageous embodiment, the desired level of restriction of personal information is defined using a structural similarity index with respect to the camera image. According to this embodiment, the target value for the desired level of restriction of personal information is defined using a so-called "structural similarity index measure" (SSIM). The preferred measure for quantifying the desired level of restriction is: (1-CW_SSIM), in other words "one minus CW_SSIM" where "CW" stands for "complex wavelet" and "SSIM" stands for "structural similarity index measure"; further information on this can be found in the publication "Z. Wang and AC Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures" in IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98-117, Jan. 2009, doi: 10.1109 / MSP.2008.930649".

[0034] According to a further advantageous embodiment, the information to be provided for the vehicle function at least comprises a mean standard deviation or a signal-to-noise ratio with respect to the camera image.

[0035] Also with regard to the mean standard deviation (abbreviated "MSE"), further information can be found in the publication "Z. Wang and AC Bovik, "Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures" in IEEE Signal Processing Magazine, vol. 26, no. 1 , pp. 98-117, Jan. 2009, doi: 10.1109 / MSP.2008.930649".

[0036] According to a further advantageous embodiment, a computing unit arranged in a vehicle is used, wherein the computing unit continuously determines updated values ​​of the parameters for each of the stages for a continuously updated determination of the distribution of the degradation across the stages.

[0037] The current parameters, as determined by the computing unit, are thus determined onboard, i.e., locally in the vehicle itself. The continuously updated determination of the degradation distribution leads to an adjustment of the degradation distribution across the stages as well as the overall degradation intensity in real time. This advantageously allows for adaptation to prevailing conditions in order to continuously optimize the allocation of information between the vehicle's function and the goal of reducing personal information.

[0038] According to a further advantageous embodiment, the parameters for each of the stages are determined by a computing unit depending on determined situation parameters, wherein the situation parameters include in particular one of the following: distance of a person's face to the camera unit, facial movements of a person relative to the camera unit, ambient conditions such as prevailing brightness, driving situation.

[0039] This allows for varying the restriction of personal information, as, for example, a face close to a camera is more sensitive to privacy than a face further away, such as in the back seat of a vehicle in the dark. For this purpose, a prediction is preferably made using artificial intelligence or regression methods, even if a previously used reference image is no longer available during operation.

[0040] The optimization algorithm advantageously determines the parameter values ​​for each stage to determine the distribution of degradation across the stages, particularly with regard to the geometric regions of interest (ROIs) for the vehicle function, in order to preserve the ROI, i.e., information about this specific area, as much as possible. These regions can change depending on the scene. Facial ROIs can be determined using known algorithmic face detection. State-of-the-art face detectors (for facial recognition) also exist that are highly robust against reductions in spatial resolution. Furthermore, the specific characteristics of the respective vehicle function can be taken into account with regard to the sensitivity of parameter values ​​to certain image properties such as noise, lack of structure, or lack of contrast.Furthermore, the expected or actual current scene (e.g., in terms of dynamic range, brightness distribution, as represented by a histogram) can be used to determine the parameter values ​​in the optimization algorithm, each with respect to specific areas. Furthermore, a different combination of parameter values ​​can be applied spatially, temporally, and content-wise in each area of ​​the camera image, with the respective degradation level.

[0041] According to a further advantageous embodiment, values ​​of the parameters of the stages for predefined camera images or for camera images from predefined scenes are determined by the computing unit and are stored in a control unit of the vehicle.

[0042] Advantageously, according to this embodiment, when predefined camera images or camera images from predefined scenes occur repeatedly, the parameter values ​​do not have to be redetermined, but rather a predefined set of predefined parameter values ​​that have already been determined offboard in the past can be used. This advantageously saves unnecessary computing effort. Predefined camera images can be used when an almost exactly recurring situation captured by the camera unit can be assumed. Camera images from predefined scenes, on the other hand, are more flexible in use and only require matches in the features of the scenes. These sets of parameter values, once they have been determined, are assigned to the predefined camera images or scenes.to the predefined scenes in the control unit and are accessible by the computing unit to provide an alternative source to the optimization algorithm.

[0043] According to a further advantageous embodiment, the values ​​of the parameters stored in the control unit are then used for degradation only when a predefined camera image or a camera image from a predefined scene is present during operation, instead of the values ​​of the parameters continuously updated by the computing unit.

[0044] According to a further advantageous embodiment, the calculation unit determines the values ​​of the parameters by means of a numerical, in particular iterative, method.

[0045] The iterative numerical method is particularly advantageous for iteratively approaching given target variables with respect to i) or ii), i.e., changing the parameter values ​​until the required thresholds of the target variables are met or at least one threshold i) or ii) of the target variable is reached and the correspondingly other threshold is reached as best as possible. In multi-objective optimization, an iterative search algorithm can be used for nonlinear optimization problems.

[0046] According to a further advantageous embodiment, the calculation unit determines the values ​​of the parameters by means of a pre-trained, artificial neural network.

[0047] The optimization algorithm uses a pre-trained artificial neural network to determine the parameters. Possible input variables for the pre-trained artificial neural network include, in particular, the respective camera image and target variables i) and ii; output values ​​are parameters of the stages.

[0048] According to a further advantageous embodiment, the pre-trained artificial neural network is continuously trained on a server based on data from camera units of vehicles, with updates of the artificial neural network being transmitted back to vehicles of a fleet.

[0049] In a further preferred embodiment, the elements of the stages and their parameters are not only combined from a set of prefabricated filters, but are generated entirely using deep learning methods, analogous to Generative Adversarial Networks (GANs). In an extended embodiment, the hardware components are also taken into account in the objective function by means of an additional term. In particular, the resource expenditure is mapped in order to take this into account in the optimization algorithm - for example, with a constant level of restriction of personal information and a similar (in particular to the extent that essentially constant) quality and quantity of information to be provided for the vehicle function, the variant of parameters (in particular for selecting filter modules) is selected that can be executed particularly efficiently on the computing unit or in the respective stage.

[0050] A further aspect of the invention relates to a system for reducing personal information in a camera image of a camera unit by degrading the camera image, wherein the camera image processed by image analysis serves to carry out a vehicle function, in particular in a vehicle, and wherein a computing unit for reducing personal information in the camera image is designed to degrade original image data of the camera unit in stages before processing, wherein each of the stages carries out a type of degradation characteristic of it and each of the stages has at least one variable parameter for determining the intensity of the degradation in the respective stage, wherein the computing unit is designed toto determine the values ​​of the parameters for each of the stages to determine the distribution of degradation across the stages by an optimization algorithm using i) the minimum information to be provided for the vehicle function and ii) a desired level of restriction of personal information, using i) and ii) as target values ​​and determining the parameters to achieve the target values.

[0051] A further aspect of the invention relates to a vehicle with a system as described above and below.

[0052] Advantages and preferred developments of the proposed system result from an analogous and analogous transfer of the statements made above in connection with the proposed method.

[0053] Further advantages, features, and details will become apparent from the following description, which – where appropriate with reference to the drawings – describes at least one embodiment in detail. Identical, similar, and / or functionally equivalent parts are provided with the same reference numerals.

[0054] They show:

[0055] Fig. 1 : A method carried out by a system for reducing personal information in a camera image of a camera unit according to an embodiment of the invention;

[0056] Fig. 2: the degradation stages used in the system according to the embodiment of Fig. 1 in detail;

[0057] Fig. 3: further embodiment of the degradation stages used in the system according to the embodiment of Fig. 1 and

[0058] Fig. 4 Sequence of the method according to the invention.

[0059] Fig. 1 shows the interior of a vehicle 3 with a camera unit 1 and a computing unit 5. With the help of the computing unit 5, a method for reducing personal information in a respective camera image of the camera unit 1 is carried out by degrading the camera image. The camera image is generated by the camera unit 1 in high-frequency repetitions for the purposes of a vehicle function. A multi-purpose interior cam is used to carry out a vehicle function such as an automated drowsiness warning. However, this also has the secondary effect of recording personal data, i.e. the data in the camera image that is in principle sufficient for facial recognition, automated just as it is by a person, represents a security risk because, for example, an attacker from outside could gain access to this data.The aim is therefore to remove as much of the data from the respective camera image that could be used to identify the person being filmed as possible without impairing the vehicle's function. To reduce personal information in the camera image before processing by the vehicle function, the original image data is degraded, i.e. artificially worsened, in processing stages of camera unit 1. Each of the stages performs its own characteristic type of degradation. To adjust which of the stages assumes which share of the degradation in a particular scenario and for a particular vehicle function, and how large the degradation should be overall, each of the stages has a set of parameters with variable values.This distribution is determined by a computing unit 5 by determining the values ​​of the parameters for each of the stages to determine the distribution of degradation across the stages using an optimization algorithm using i) the information to be provided for the vehicle function application and ii) a desired level of restriction of personal information. The competing objectives i) and ii) are specified as target variables of an iterative nonlinear optimization algorithm for performing a multi-objective optimization in order to achieve the target value i) for the information provided for the vehicle function and the target value ii) for the level of restriction of personal information.Alternatively, the parameters are optimized so that either the information provided for the vehicle function reaches the target value i) and the level of restriction of personal information is as close as possible to the target value ii) or for the level of restriction of personal information the target value ii) ie the desired level of restriction is reached and the information provided for the vehicle function is as close as possible to the target value i).

[0060] The stages comprise the following: 1a an image sensor of camera unit 1, in particular the image sensor register; 1b hardware settings in the control unit of camera unit 1, in particular calibration data; and 1c software processing in the control unit of camera unit 1 for algorithmically executed image processing. Furthermore, an optional stage 1d, not explained in detail, is shown, which includes further parameterizable settings influencing objectives i) and ii), such as object lighting, post-processing stages, etc. These are shown in more detail in Fig. 2. Starting with camera unit 1, a variety of methods are already applied in the sensor in the first stage 1a to modify the original camera image of camera unit 1. The respective parameters in this stage relate to the aforementioned image sensor of camera unit 1.Methods such as 2x2 binning, 8x subsampling, high value saturation, 16x gain, exposure control, and tone mapping for optimal privacy can already be implemented here. The hardware settings in the control unit of camera unit 1 represent the second stage 1b and contain additional parameters for the following methods: tone mapping for optimal privacy, minimal color saturation, and sharpness reduction with edge preservation. In the third stage 1c, which is still located upstream of the processing of the degraded image by the vehicle function, a Complex Wavelet SSIM (CW_SSIM) is used, where SSIM stands for "Structural Similarity Index Measure." The parameter set with the values ​​of the parameters of all stages is determined iteratively as a set of parameters onboard the vehicle 3 for each camera image.

[0061] For this purpose, an image processed by the camera 1 is analyzed by comparing the minimum information to be provided for the vehicle function i) as target value 7 to fulfill a predetermined range of functions with an actual value for the information included in the image 7a and that of a desired level of restriction of personal information ii) as target value 9 with an actual value of the image with regard to the level of personal data 9a.If the actual values ​​7a, 9a are below the target values, the computing unit 5 optimizes the parameters of all stages 1a to 1d, for example in an iterative optimization process, until ideally the actual values ​​of the information provided for the vehicle function by the image processed in stages S1 to S4 and the level of the personal data reach at least the target values ​​7, 9 and then outputs the image 11.

[0062] If target values ​​7 and 9 cannot be simultaneously achieved for the actual values ​​of the information provided for the vehicle function and the level of personal data, the parameters of levels 1a to 1d are determined such that target value i) is achieved for at least the information provided for the vehicle function or target value ii) is achieved for the level of restriction of personal information, and the other value is optimized as best as possible. For safety-relevant functions such as driver monitoring, the information provided for the vehicle function is prioritized so that it achieves the target value required for the function.

[0063] Fig. 3 shows an embodiment in which the optimization algorithm of the computing unit is implemented as a pre-trained neural network. The neural network is configured to determine the parameters of stages 1a to 1d based on an image captured by camera 1 and predetermined target variables i) for the minimum information 7 to be provided for the vehicle function and a desired level of restriction of personal information 9 such that the image 11 processed in stages 1a to 1d and subsequently output provides the information 7a required for the vehicle function and maintains the desired level of restriction of personal information 9a.

[0064] Fig. 4 shows an example of a sequence of the method according to the invention, wherein in step S1 an image is received, in step S2 the image is processed in stages 1a to 1d with predetermined image processing parameters. In step S3 a check is made as to whether the information provided by the image for the vehicle function reaches the target value i), i.e. the minimum information (7) to be provided for the vehicle function, and whether the level of restriction of personal information in the image reaches the desired level of restriction of personal information. If the check is positive, the image is output in step S4. If the check in S3 is negative, in step S5 the image processing parameters are changed by the optimization algorithm of the computing unit 5 and transmitted to stages 1a to 1d.In step S2, the image is processed with the modified parameters and then compared again with the target variables i) and i) in step 3. The optimization in step S5 continues until the target variables i) and ii) are reached, and the image can then be output in step 4.

[0065] Although the invention has been illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned by way of example are truly only examples and should not be construed as limiting the scope, possible applications, or configuration of the invention in any way.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Claims

Patent claims Method for reducing personal information in a camera image of a camera unit (1) by degrading the camera image, wherein the camera image processed by image analysis serves to carry out a vehicle function, in particular in a vehicle (3), and wherein, in order to reduce personal information in the camera image, original image data of the camera unit (1) are degraded in stages before processing, wherein each of the stages carries out a type of degradation characteristic of it and each of the stages has at least one variable parameter for determining the intensity of the degradation in the respective stage,wherein a computing unit (5) determines the values ​​of the parameters for each of the stages for determining the distribution of the degradation across the stages by an optimization algorithm using i) the information (7) to be provided for the vehicle function at least and ii) a desired level of restriction of personal information (9), wherein i) and ii) are used as target variables and the parameters for achieving the target variables are determined. Method according to claim 1, wherein, in the event that both target variables cannot be achieved simultaneously by changing the parameters, the optimization algorithm optimizes the parameters tothat, depending on the application, either the target value i) is achieved for the information provided for the vehicle function and the level of restriction of personal information is as close as possible to the target value ii), or the target value ii) is achieved for the level of restriction of personal information and the information provided for the vehicle function is as close as possible to the target value i). Method according to claim 1 or 2, wherein a measure for a preference between the information to be provided for the vehicle function at least or a desired level of restriction of personal information and / or a prioritization of one of the target variables is specified by a user. Method according to claim 1 or 2, wherein a measure for a preference between the information to be provided for the vehicle function at least and a desired level of restriction of personal information and / or a prioritization of one of the target variables is specified by the computing unit (5). Method according to one of the preceding claims, wherein the stages comprise the following: the image sensor of the camera unit (1), in particular registers of the image sensor, hardware settings in the control unit of the camera unit (1), in particular calibration data, software processing in the control unit of the camera unit (1) for algorithmically executed image processing.Method according to one of the preceding claims, wherein the desired level of restriction of personal information is defined by means of a structural similarity index with respect to the camera image. Method according to one of the preceding claims, wherein the at least information for the vehicle function comprises a mean standard deviation or a signal-to-noise ratio with respect to the camera image. Method according to one of the preceding claims, wherein a computing unit (5) arranged in a vehicle (3) is used, wherein continuously updated values ​​of the parameters for each of the stages are determined by the computing unit (5) for a continuously updated determination of the distribution of the degradation across the stages. Method according to claim 8. wherein a computing unit (5) determines the parameters for each of the stages depending on determined situation parameters, wherein the situation parameters include, in particular, one of the following: distance of a person's face from the camera unit (1), facial movements of a person relative to the camera unit (1), ambient conditions such as prevailing brightness, driving situation. Method according to one of the preceding claims, wherein values ​​of the parameters of the stages are determined by the computing unit for predefined camera images or for camera images from predefined scenes and are stored in a control unit of the vehicle (3).Method according to one of claims 1 to 10, wherein the parameter values ​​stored in the vehicle's control unit are used for degradation instead of the parameter values ​​continuously updated by the computing unit (5) when a predefined camera image or a camera image from a predefined scene is present during operation. Method according to one of the preceding claims, wherein the computing unit (5) determines the parameter values ​​using a numerical method. Method according to one of the preceding claims, wherein the computing unit (5) determines the parameter values ​​using a pre-trained, artificial neural network.The method according to claim 13, wherein the pre-trained artificial neural network is continuously trained on a server based on data from camera units of vehicles, wherein updates of the artificial neural network are transmitted back to vehicles of a fleet. A system for reducing personal information in a camera image of a camera unit (1) by degrading the camera image, wherein the camera image processed by image analysis is used to execute a vehicle function, in particular in a vehicle (3), and wherein a computing unit (5) for reducing... ization of personal information in the camera image is designed to degrade original image data of the camera unit (1) in stages prior to processing, wherein each of the stages carries out a type of degradation characteristic of it and each of the stages has at least one variable parameter for determining the intensity of the degradation in the respective stage, wherein the computing unit (5) is designed to determine the values ​​of the parameters for each of the stages for determining the distribution of the degradation across the stages by means of an optimization algorithm using i) the information to be provided for the vehicle function at least and ii) a desired level of restriction of personal information, wherein i) and ii) are used as target variables or from the set of i) and ii) a target variable and a restriction are used in the optimization algorithm. Vehicle (3) with a system according to claim 15.

Images