Method and device for anonymising vehicle surroundings in a camera image

Abstract

Embodiments of the present invention provide a method (100) for anonymising vehicle surroundings in a camera image. The method (100) comprises obtaining (110) a camera image from an in-vehicle camera, wherein the camera image comprises a region of the vehicle surroundings through a transparent region of the vehicle. The method (100) further comprises obtaining (120) the position and orientation of the in-vehicle camera in the vehicle coordinate system and obtaining (130) coordinates of the transparent region of the vehicle in the vehicle coordinate system. Furthermore, the method (100) comprises projecting (140) the 3D coordinates of the transparent region from the vehicle coordinate system into a 2D image coordinate system of the camera image. Finally, the method (100) comprises anonymising (150) the region of the vehicle surroundings in the camera image.

Description

[0001] 24-3408 1

[0002] METHOD AND DEVICE FOR ANONYMIZING A VEHICLE'S ENVIRONMENT IN A CAMERA IMAGE

[0003] Description

[0004] Modern vehicles can be equipped with camera systems that capture the interior or the vehicle's surroundings. The resulting camera images could be used for a variety of applications, including safety features, comfort improvements, or data collection for analysis. Several camera-based functions already exist in the automotive industry, allowing users to take snapshots or videos, make video calls, or remotely inspect the vehicle's interior. However, in certain scenarios, particularly when capturing images of the vehicle's surroundings through transparent areas such as windows, personal or other sensitive data may be captured. This data could include faces, license plates, or other identifying characteristics, the collection and processing of which may be subject to legal or ethical restrictions.In some countries, such as China, specific regulations require that personal data visible in outdoor footage be anonymized. This includes faces, license plates, and other identifying information. Such regulations prohibit, among other things, the transmission of non-anonymized data via backend systems to ensure privacy protection.

[0005] Previous approaches to anonymizing such data may be based on generic image processing methods that are either insufficiently precise or inefficient. Furthermore, existing systems often lack the ability to dynamically adapt to different vehicle configurations or camera positions. Therefore, there may be a need for a solution that enables improved anonymization.

[0006] The method, the device, the computer program and the vehicle meet this need in accordance with the independent claims.

[0007] Exemplary embodiments relate to a method for anonymizing a vehicle's surroundings in a camera image. The method includes obtaining a camera image from an interior vehicle camera. The camera image includes a portion of the vehicle's surroundings through a transparent area of ​​the vehicle. The method further includes obtaining the position and orientation of the interior vehicle camera in a vehicle coordinate system. The method also includes obtaining the coordinates of the transparent area of ​​the vehicle within the vehicle coordinate system. The method includes projecting the 3D coordinates of the [24-3408 2]

[0008] The process converts the transparent area from the vehicle's coordinate system into a 2D image coordinate system of the camera image. Finally, the method includes anonymizing the area of ​​the vehicle's surroundings within the camera image. This enables precise anonymization of image areas captured through transparent sections of the vehicle. It contributes to compliance with data protection requirements by making sensitive data such as faces or license plates unrecognizable. At the same time, the functionality of the vehicle's interior camera is maintained, which is particularly crucial for safety-critical applications. Furthermore, the method is flexible and adaptable, allowing it to be applied to various vehicle configurations and camera positions.

[0009] In one embodiment, the method can include determining a filter kernel. Anonymization can then be achieved by applying the filter kernel to the area of ​​the vehicle's surroundings within the camera image. This enables precise and efficient processing of image data for targeted anonymization.

[0010] In one embodiment, anonymizing the area surrounding the vehicle in the camera image can include at least one of the following techniques: blurring the area surrounding the vehicle, pixelating the area surrounding the vehicle, masking the area surrounding the vehicle by complete coverage, geometric distortion of the area surrounding the vehicle, and noise overlaying the area surrounding the vehicle. This allows the anonymization to be adapted to various requirements.

[0011] In one embodiment, the method can include calibrating the vehicle's interior camera. The calibration can determine at least one of the following parameters: the position of the vehicle's interior camera in the vehicle's coordinate system, the orientation of the vehicle's interior camera, any distortion of the vehicle's interior camera, and a principal point of the vehicle's interior camera. This enables a precise mapping of the 3D coordinates to the image space and increases the accuracy of the anonymization.

[0012] In one embodiment, the method can include obtaining design data of the vehicle. Furthermore, it can include determining the coordinates of the transparent area of ​​the vehicle in the vehicle coordinate system based on the obtained design data. This improves the identification of relevant image areas, particularly in the case of complex vehicle geometries.

[0013] In one embodiment, the method can include object recognition of the camera image using machine learning. The coordinates of the transparent area of ​​the 24-3408 3 can be used for this purpose.

[0014] The vehicle's coordinate system is determined based on the object detection performed. This allows for dynamic adjustment of the anonymization to the detected objects and their positions.

[0015] In one embodiment, the intensity of anonymization can depend on a detected object in the vehicle's surroundings within the camera image. This allows for targeted adjustment of the anonymization level to the protection requirements of individual objects.

[0016] In one embodiment, the intensity of anonymization can depend on the size and / or position of the transparent area of ​​the vehicle. This allows for different requirements regarding image quality and data protection to be taken into account.

[0017] In one embodiment, the transparent area of ​​the vehicle can comprise at least one of the following parts of the vehicle: a windshield, a side window, a rear window, a panoramic roof, or transparent vehicle cladding.

[0018] Another embodiment relates to a device for anonymizing a vehicle's surroundings in a camera image. The device comprises a communication interface configured to receive a camera image from an interior vehicle camera. The camera image includes a portion of the vehicle's surroundings through a transparent area of ​​the vehicle. The communication interface is further configured to receive the position and orientation of the interior vehicle camera within a vehicle coordinate system. The communication interface is also configured to receive the coordinates of the transparent area of ​​the vehicle within the vehicle coordinate system. The device further comprises a processor circuit configured to project the 3D coordinates of the transparent area from the vehicle coordinate system into a 2D image coordinate system of the camera image.The processor circuit is also designed to anonymize the area of ​​the vehicle's surroundings in the camera image.

[0019] Another embodiment relates to a vehicle that includes a device for anonymizing image content.

[0020] In another embodiment, the method comprises a computer program configured to execute one of the described methods. 24-3408 4

[0021] Another embodiment relates to a non-volatile machine-readable medium on which such a program is stored, which is configured to carry out one of the described methods.

[0022] Another embodiment relates to a non-volatile machine-readable medium on which such a program is stored, which is configured to carry out one of the described methods.

[0023] In another embodiment, a program with program code for carrying out one of the described methods relates to the program being executed on a processor or a programmable hardware component.

[0024] Examples of implementation are explained in more detail below with reference to the accompanying figures. These show:

[0025] Fig. 1 shows a schematic representation of a method 100 for controlling a voice assistant based on a sign language utterance;

[0026] Fig. 2 schematically shows a device 200 for controlling a voice assistant based on a sign language utterance; and

[0027] Fig. 3 schematically shows a vehicle 300 comprising a device 200 for controlling a voice assistant based on a sign language utterance.

[0028] Several embodiments are now described in more detail with reference to the accompanying drawings, in which some of these embodiments are illustrated. For the sake of clarity, the thickness dimensions of lines, layers, and / or regions may be exaggerated in the figures.

[0029] Fig. 1 schematically shows a method 100 for anonymizing a vehicle's surroundings in a camera image. The method is carried out, for example, by a device 200 (see Fig. 2). The method 100 comprises receiving 110 a camera image from a vehicle's interior camera. The camera image encompasses an area of ​​the vehicle's surroundings through a transparent section of the vehicle. Receiving 110 the camera image from the vehicle's interior camera can include receiving or retrieving the camera image by a device. Receiving 110 the camera image can involve the physical reception of the camera image and / or the provision of the camera image for further processing.

[0030] Processing includes. In some examples, receiving the camera image 110 may involve the device 200 receiving the camera image, for example, through the device 200's communication interface 210. Receiving the camera image can occur via a data connection, such as a wireless connection, a wired connection, or an internal connection within the vehicle system. For example, the camera image can be received from the vehicle's interior camera. In some other examples, receiving the camera image 110 may involve the device 200 retrieving the camera image, for example, from the device's memory or from a connected storage medium.For example, the camera image may have been previously recorded by the vehicle's interior camera and stored in the memory of the device 200, in order to subsequently be received by the communication interface 210 of the device 200 and processed by a processor circuit 220.

[0031] In some examples, the vehicle could be a passenger car, a commercial vehicle, a truck, or a bus. The vehicle could be powered by an internal combustion engine, or electrically or as a hybrid.

[0032] For example, an interior vehicle camera can be a camera that captures an image of the vehicle interior, such as the cabin and / or the occupants. This image can be a visual representation captured by the interior camera and stored or processed as digital or analog data. In some cases, the image can depict both the vehicle's interior and its surroundings. This means the image can include the occupants, such as the driver, front passenger, or passengers in the rear. Furthermore, the image captured through a transparent part of the vehicle can depict various areas of the vehicle's environment.

[0033] For example, the vehicle environment is an area extending at a certain distance around the vehicle's outer body. In some examples, the vehicle environment may encompass a radius around the vehicle ranging from the immediate vicinity of the vehicle body to a larger area detected by sensors or interior and / or exterior cameras. In other examples, the vehicle environment may include both the ground area on which the vehicle is driving or parked and the airspace above and to the sides of the vehicle. For example, the vehicle environment is an area within the range of a vehicle sensor, such as a camera system. In some examples, the vehicle environment may include the road and traffic in front of, beside, or behind the vehicle, including other vehicles, pedestrians, cyclists, and traffic signs.The vehicle environment can also include static objects such as buildings, parking lots or vegetation, as well as dynamic objects such as moving24-3408 6.

[0034] This includes vehicles or people. Depending on the position and orientation of the vehicle's interior camera, the vehicle's surroundings may also include adjacent or more distant areas of public and private space.

[0035] This allows the area of ​​the vehicle's surroundings depicted in the camera image to include, for example, other vehicles, including their license plates and brand emblems, as well as the faces of drivers or passengers. Pedestrians, cyclists, and other road users can also be captured. Furthermore, stationary objects such as buildings, street signs, or traffic lights can be visible in the camera image. In some cases, the camera image can also record safety-relevant information such as accidents or critical traffic situations. In addition to these visible objects, personal information, such as the identity of individuals, vehicle registration, or the vehicle's location, can be derived by combining the recorded data. The areas of the vehicle's surroundings captured depend significantly on the field of view of the vehicle's interior camera, specifically its position and / or orientation.

[0036] For example, the vehicle's interior camera can be mounted inside the vehicle and capture both still images and video data. However, the interior camera can also capture images of areas surrounding the vehicle. This means that the image captured by the interior camera encompasses a portion of the vehicle's surroundings. This can be achieved, for instance, by the interior camera capturing an area of ​​the vehicle's surroundings through a transparent section of the vehicle. This transparent section of the vehicle, through which the interior camera captures an area of ​​the surroundings, can encompass one or more sub-areas of the vehicle. These sub-areas can be contiguous or separate.For example, the transparent area of ​​the vehicle includes at least one of the following vehicle components: a windshield, a side window, a rear window, a panoramic roof, or transparent vehicle cladding. For example, transparent vehicle cladding can include a surface of a vehicle made of a transparent material such as glass or plastic. In some examples, transparent vehicle cladding can serve to allow light or visibility to pass through, and it may be located outside the vehicle's traditional window areas. For example, transparent vehicle cladding can include a cover for sensors, cameras, or other technical components of the vehicle.

[0037] The vehicle interior camera can be mounted in various locations within the vehicle interior (vehicle cabin) and, depending on its installation location, capture different areas of the vehicle interior and the vehicle's surroundings within its field of view. In some examples, 24-3408 7

[0038] The vehicle's interior camera system can comprise multiple cameras to capture the vehicle interior and / or an area of ​​the vehicle's surroundings from various angles within its field of view. For example, the vehicle's interior camera system can be installed with a fixed, predefined orientation, or it can be freely movable within the vehicle interior.

[0039] For example, the vehicle's interior camera can be mounted on the rearview mirror. In such cases, the camera can either be installed with a fixed, predefined orientation or be freely adjustable, as the rearview mirror is often movable. This flexibility allows the captured area of ​​the vehicle's surroundings to be adapted to specific requirements. In some examples, the vehicle's interior camera can be positioned on the center console, the dashboard, and / or the vehicle's ceiling. Depending on the specific position and design of the interior camera, different areas of the vehicle's surroundings can fall within the camera's field of view, thus capturing various areas of the environment through transparent parts of the vehicle.When the vehicle's interior camera is mounted, for example, on the rearview mirror, it can project images of the vehicle's surroundings through the windshield, side windows, and rear window, including areas to the sides and behind the vehicle. This makes it possible to detect objects such as other vehicles, pedestrians, cyclists, or stationary elements like road signs and buildings within the camera's field of view.

[0040] For example, the vehicle's interior camera could be an RGB camera that records color information, or a 3D camera that captures depth information, thus enabling more precise detection of movements and objects. In one embodiment, the camera could be based on lidar technology to measure distances and create three-dimensional images of the captured field of view. In another embodiment, the vehicle's interior camera could include an infrared camera that provides usable data even in low light or darkness. In some examples, the vehicle's interior camera could incorporate a combination of one or more of the aforementioned sensor technologies to ensure versatile and robust capture of sign language utterances.

[0041] Method 100 further comprises obtaining 120 the position and orientation of the vehicle interior camera in a vehicle coordinate system. For example, obtaining the position and orientation of the vehicle interior camera may include receiving or retrieving this information by a device, such as device 200 (see Fig. 2), which further processes this information. In some examples, obtaining the position and orientation of the vehicle interior camera may be accomplished by calibration, in which the 24-3408 8

[0042] The vehicle's interior camera determines its own position and orientation within the vehicle's coordinate system. This calibration can be performed while the vehicle is in operation, with the interior camera calculating its spatial position and orientation relative to defined reference points within the vehicle. In other examples, the position and orientation of the vehicle's interior camera can be determined by a calibration device connected to the camera. The information obtained by the calibration device can then be transmitted to and received by a device, for example, Device 200 (see Fig. 2). In another example, the position and orientation of the vehicle's interior camera can be determined by a third party, such as a manufacturer of the vehicle or the vehicle's interior camera, and stored in a memory of Device 200.In this case, obtaining the position and orientation of the vehicle's interior camera may involve retrieving the data from a memory within the device 200.

[0043] For example, the vehicle coordinate system may comprise a three-dimensional Cartesian coordinate system used to describe positions and orientations within a vehicle and its immediate surroundings. In some examples, the vehicle coordinate system may have an origin located at the vehicle's centerline or at a defined reference point, such as the vehicle's axle or center of gravity. The coordinate system axes may be oriented in a defined relationship to the vehicle's geometry, with one axis aligned along the vehicle's longitudinal axis, a second along its transverse axis, and a third along its vertical axis.

[0044] The vehicle coordinate system can be used to precisely define the position and orientation of components or objects relative to the vehicle. In some examples, the position and orientation of the vehicle's interior camera can be specified within this coordinate system to determine its exact elevation and viewing direction (i.e., field of view). Using the vehicle coordinate system simplifies the integration of sensors, cameras, and other components because all relevant data can be referenced to a unified frame of reference. This also allows for the calculation of fields of view and the mapping of camera images to specific areas of the vehicle or its surroundings.

[0045] For example, the position and orientation of the vehicle's interior camera can be mathematically described by six parameters in the vehicle's coordinate system. The position of the vehicle's interior camera can be specified, for example, by the three coordinates (x, y, z).

[0046] which define their position relative to the origin of the vehicle coordinate system. The camera's orientation can be described, for example, by three rotation angles: the roll angle (p), the pitch angle (9), and the yaw angle (ip). These angles specify the camera's rotations relative to the axes of the vehicle coordinate system.

[0047] For example, the vehicle's interior camera is mounted on the rearview mirror. Its position in the vehicle's coordinate system could be described by the vector (x = 1.5 m, y = 0 m, z = 1.2 m), where the x-axis runs along the vehicle's longitudinal axis, the y-axis along the transverse axis, and the z-axis along the vertical axis. The camera's orientation is specified by the roll angle (p = 0°), the pitch angle (9 = -15°), and the yaw angle (i = 0°). This mathematical description allows for a precise and unambiguous definition of the camera's position and orientation within the vehicle's coordinate system.

[0048] In some examples, Procedure 100 may include performing the calibration of the vehicle's interior camera. The calibration may determine at least one of the following parameters: the position of the vehicle's interior camera in the vehicle's coordinate system, the orientation of the vehicle's interior camera, any distortion of the vehicle's interior camera, a principal point of the vehicle's interior camera, or the focal length of the vehicle's interior camera. For example, calibration may be used to determine the exact geometric and optical properties of a camera to enable precise measurements and imaging. In some examples, the calibration may determine parameters such as the position and orientation of the camera in the vehicle's coordinate system, optical properties of the camera, or systematic deviations in image acquisition. The calibration may be performed, for example, by the vehicle's interior camera itself, by an external device, or by special software.In some cases, the calibration of the vehicle's interior camera can be performed using physical reference objects or specialized calibration equipment that provides the camera with defined data or patterns for analysis. Alternatively, calibration can be carried out by external parties such as the vehicle manufacturer, and the corresponding parameters stored in memory from which the camera can later retrieve them. The goal of calibration is to set the camera to a defined state and ensure accurate reproduction of the captured camera images. For example, the focal length of the vehicle's interior camera can be an optical property that specifies the distance between the lens plane and the image sensor at which a sharp image of an object is produced. The focal length is usually specified in millimeters, but in digital cameras, it can also be expressed in pixels to describe the image size on the image plane.In some examples, the focal length can influence the perspective and magnification level of the vehicle's interior camera, with a short focal length providing a wider field of view (wide angle) and a long focal length 24-3408 10.

[0049] It offers a smaller field of view (telephoto lens). Precise knowledge of the focal length is essential for accurate projections and correct modeling of the geometry of the depicted scenes.

[0050] For example, distortion in a vehicle's interior camera can include deviations from an ideal, linear image caused by the camera's optical properties or mechanical influences. Distortion can manifest as geometric deformations in the camera image, such as barrel distortion, where straight lines appear curved outwards, or pincushion distortion, where they appear curved inwards. In some cases, such distortions can arise from the camera lens design, the position of the optical center, or incorrect alignment. Camera calibration can measure these distortions and calculate appropriate correction parameters to correct the camera image and enable precise geometric alignment of the image content.

[0051] For example, the principal point of a vehicle's interior camera can be the point on the image sensor where the camera's optical axis intersects the image plane perpendicularly. In an ideal camera system, the principal point is located exactly in the center of the image sensor. In practice, however, the principal point can deviate from this ideal position due to manufacturing tolerances, assembly errors, or mechanical deviations. Accurate determination of the principal point during calibration can be crucial for correcting distortions and ensuring precise geometric alignment of the image data within the vehicle's coordinate system.

[0052] Method 100 further comprises obtaining 130 the coordinates of the vehicle's transparent area in the vehicle coordinate system. For example, obtaining 130 the coordinates of the vehicle's transparent area in the vehicle coordinate system can involve receiving or retrieving these coordinates by a device, such as device 200 (see Fig. 2), for further processing. In some examples, the coordinates of the vehicle's transparent area can be stored in the device's memory and retrieved from there. In another example, device 200 can receive the coordinates of the vehicle's transparent area from an external source for further processing. For example, the device may have stored or received design data (such as a CAD model) of the vehicle.The vehicle's design data can, for example, contain the coordinates of the vehicle's transparent area (see below). In another example, the device or an external source can determine the coordinates of the vehicle's transparent area based on the camera image, for example, using machine learning (see below). Die24-3408 11.

[0053] Device 200 can then receive data from specific coordinates of the transparent area of ​​the vehicle for further processing.

[0054] For example, the coordinates of the vehicle's transparent area within the vehicle coordinate system can provide a precise geometric description of its spatial location and extent. This means that the coordinates of the vehicle's transparent area within the vehicle coordinate system include, for all its constituent parts, such as a windshield, side window, rear window, or panoramic roof, a precise geometric description of the spatial location and extent of these parts. The coordinates can specify the position of the corners or edges of the transparent area (or all its constituent parts) using numerical values ​​(x, y, z) defined relative to the origin of the vehicle coordinate system. In some examples, the coordinates can also specify the orientation of the area by defining the normal direction or by specifying angles within the coordinate system.These coordinates make it possible to precisely locate the transparent area and analyze its relationship to other vehicle components, such as the vehicle's interior camera.

[0055] For example, the transparent area encompasses a side window of a vehicle. The coordinates of the vehicle's side window in the vehicle coordinate system are, for example, as follows: Top front: (x = 1.5 m, y = 0.6 m, z = 1.2 m); Top rear: (x = 1.8 m, y = 0.6 m, z = 1.2 m); Bottom front: (x = 1.5 m, y = 0.6 m, z = 0.7 m); Bottom rear: (x = 1.8 m, y = 0.6 m, z = 0.7 m). For example, the x-axis runs along the vehicle's longitudinal axis, the y-axis along the vehicle's transverse axis, and the z-axis describes the height relative to the vehicle floor. The given coordinates precisely define the position and extent of the side window in the vehicle coordinate system.

[0056] Method 100 involves projecting the 3D coordinates of the transparent area from the vehicle's coordinate system into a 2D image coordinate system of the camera image. For example, the 2D image coordinate system of the camera image is a two-dimensional Cartesian coordinate system defined on the image plane of the vehicle's interior camera. This coordinate system serves, for example, to precisely describe the position of points or pixels in the camera image. In some examples, the origin of the 2D image coordinate system may be located in the upper left corner of the image plane. The horizontal axis u may run from left to right along the width of the camera image, while the vertical axis v may run from top to bottom along the height of the camera image. The units in the 2D image coordinate system may be defined by the physical resolution of the vehicle's interior camera's image sensor, for example, in pixels.Each point in the camera image can be 24-3408 12.

[0057] The image can be described by the coordinates (u, v), where u represents the horizontal pixel position and v the vertical pixel position. In some examples, the 2D image coordinate system can also be normalized, with the origin located in the center of the camera image and the axis values ​​between -1 and +1 relative to the width and height of the image.

[0058] For example, projecting 140 of the 3D coordinates of the transparent area from the vehicle coordinate system into the 2D image coordinate system of the camera image involves mapping the points of the transparent area, described in the three-dimensional vehicle coordinate system by their coordinates (x, y, z), onto the two-dimensional image plane of the vehicle's interior camera (u, v). This mapping is achieved, for example, through a mathematical transformation that takes into account the spatial properties of the camera, such as its position and orientation in the vehicle coordinate system, as well as its optical properties. The goal is to assign each point (x, y, z) of the transparent area from the vehicle coordinate system to a point (u, v) in the 2D image coordinate system of the camera image.

[0059] Projection 140 can be achieved by applying a so-called projection matrix, which models the properties of the vehicle's interior camera. The projection matrix can contain parameters such as the focal length of the vehicle's interior camera, the position of the principal point, distortion parameters, and the position and orientation of the camera within the vehicle's coordinate system. The transformation can first recalculate the 3D coordinates relative to the vehicle's interior camera and then map these onto the 2D image plane using a perspective projection. In some examples, this can be implemented by multiplying a 3D point in a homogeneous coordinate format by the projection matrix. The result is the coordinates (u, v) in the 2D image coordinate system of the camera image. For example, the projection matrix can be...The information for calculating the projection matrix can be determined by calibrating the vehicle's interior camera, for example by device 200 or by an external source.

[0060] The projection matrix P can be represented, for example, as follows:

[0061] f x 0 c x 0

[0062] P = 0 fy Cy 0

[0063] 0 0 1 024-3408 13

[0064] Here are f x and f y the focal lengths of the vehicle's interior camera in pixels along the x- and y-axes, c x and c y the coordinates of the principal point on the image plane, and the last line represents the homogenization of the coordinates.

[0065] A point (x, y, z) in the vehicle coordinate system can be represented in homogeneous coordinates [xyz l T written and multiplied by the projection matrix P to calculate the 2D coordinates (u, v) in the image coordinate system:

[0066]

[0067] The final coordinates in the image (u, v) are obtained by normalization with the scaling factor w:

[0068] UV

[0069] u = — wv = — w

[0070] In some examples, projecting 140 of the 3D coordinates can be performed using known mathematical algorithms or specialized software. For example, well-known software libraries or frameworks such as OpenCV or MATLAB® can be used, which offer comprehensive functions for camera calibration and projection calculation. In another example, projecting 140 can also be done directly on the device controlling the vehicle's interior camera using embedded software. Such software solutions are often optimized to perform projections in real time and with minimal resource consumption.

[0071] After projecting 140 of the 3D coordinates of the transparent area from the vehicle coordinate system into the 2D image coordinate system of the camera image, the points of the transparent area, which were originally described by three-dimensional coordinates (x, y, z) in the vehicle coordinate system, are uniquely mapped onto two-dimensional pixel coordinates (u, v) of the image plane of the vehicle's interior camera, i.e., in the camera image. This results in a precise mapping of the real geometry of the transparent area to its representation in the camera image. These mapped pixel coordinates (u, v) can then be used to identify and further process the position and extent of the transparent area in the camera image. 24-3408 14

[0072] Finally, the procedure includes anonymizing the area of ​​the vehicle's surroundings in the camera image. Using the previously projected knowledge of the 2D coordinates of the transparent area of ​​the vehicle in the camera image, the areas of the vehicle's surroundings captured by these transparent areas (e.g., windshield, side windows, or other transparent vehicle parts) can be precisely located. Based on these 2D coordinates in the camera image, targeted anonymization techniques can be applied to the corresponding pixels in the camera image that depict the vehicle's surroundings. For example, anonymizing the area of ​​the vehicle's surroundings in the camera image includes obscuring the image content visible through the transparent area of ​​the vehicle in the camera image. In some cases, this obscuring can be achieved by altering, pixelating, blurring, or covering the affected image areas.This process pixelates or completely obscures areas containing recognizable faces or license plates, while the rest of the image content, particularly the vehicle interior, remains unchanged. This selective processing enables privacy-compliant image data processing without affecting the usability of the remaining information.

[0073] The described method enables the precise capture and processing of areas of the vehicle's surroundings visible through transparent sections of the vehicle in the camera image, ensuring that this image content can be anonymized in compliance with data protection regulations. Determining the position and orientation of the vehicle's interior camera within the vehicle's coordinate system allows its exact location and viewing direction to be established, thus enabling the precise assignment of the captured image data to actual vehicle areas. Furthermore, projecting the 3D coordinates of the transparent area onto the 2D image coordinate system of the camera image ensures that the captured areas of the vehicle's surroundings can be accurately located and efficiently processed.The process ensures that information visible in the camera image in the vehicle's surroundings, such as faces, license plates, or other identifying features, is reliably obscured without affecting the remaining image content. This not only guarantees data protection-compliant use of the image data but also preserves the usability of the remaining image content, which remains available for safety-related or functional purposes, for example.

[0074] In some examples, the procedure 100 may include determining a filter kernel. Anonymization 150 can be achieved by applying the filter kernel to the area of ​​the vehicle's surroundings in the camera image. For example, the filter kernel is a matrix of values ​​used to selectively modify specific areas of the camera image, such as the area of ​​the vehicle's surroundings. The filter kernel determines how the 24-3408 15

[0075] Pixel values ​​in this area are processed by combining or modifying neighboring pixels. For example, a 3x3 matrix filter kernel achieves subtle anonymization effects such as gentle blurring, while an 10x10 filter kernel produces more extensive changes, such as coarse pixelation. The size of the filter kernel determines how strongly neighboring pixels are combined and thus influences the strength of the anonymization.

[0076] Granularity describes the level of anonymization, which is controlled by the choice and application of the filter kernel. High granularity means that fine details in the camera image are largely preserved because a small filter kernel is used, causing only minor changes. This is useful when only small or nearby areas, such as a face or license plate, need to be precisely anonymized. Low granularity, on the other hand, is achieved with larger filter kernels that combine larger image areas and deliberately blur or eliminate details. This is suitable for more distant or less relevant objects where a coarser distortion is sufficient. By adjusting the filter kernel size to the specific area of ​​the vehicle's surroundings, the granularity can be flexibly adapted to the requirements of the application.

[0077] For example, applying the filter gem to the vehicle's surroundings in the camera image can mean selectively modifying the pixels visible through transparent areas such as the windshield. These areas have been precisely located through projection. For instance, the vehicle's surroundings might include a face visible through the windshield. This area is defined by the 2D coordinates (ul, ul) and (u2, u2). A suitable filter gem, such as a 5x5 filter gem, can then be applied to this area to modify the pixel values, rendering the face unrecognizable, for example, by pixelation. For more distant areas, such as a license plate in the background, a larger filter gem with lower granularity could be used to apply strong anonymization to larger areas.This method ensures that anonymization is selective and context-dependent, while preserving data privacy and retaining relevant information outside the anonymized area.

[0078] In some examples, anonymizing the vehicle surroundings area in the camera image may involve at least one of the following techniques: blurring the vehicle surroundings area, pixelating the vehicle surroundings area, masking the vehicle surroundings area by full coverage, geometric distortion of the vehicle surroundings area, and noise overlaying the vehicle surroundings area. For example, blurring the vehicle surroundings area may mean that the pixel values ​​in the anonymized area are modified by smoothing, so that sharp details are not lost.24-3408 16

[0079] Blurring can be achieved, for example, by applying a filter that blends adjacent pixels and smooths transitions. Blurring can obscure subtle details such as facial features or license plates, while the overall area remains recognizable.

[0080] Pixelation of the vehicle's surroundings can involve dividing the area into large, square blocks, each with the same color. This can result in a complete loss of detail, leaving the area as an abstract pixelated structure. Pixelation can be used to quickly and efficiently anonymize faces or license plates. Complete masking involves replacing the entire vehicle's surroundings with a uniform color or pattern. For example, a face could be completely obscured by a black or white surface. This technique eliminates all detail and prevents any identification of the original content.

[0081] Geometric distortion of the vehicle's surroundings, for example, is an optical alteration of the area. This can be achieved through stretching, compression, or other transformations that make the area appear unnatural and obscure identifying features. Noise overlay can involve inserting random pixel values ​​into the vehicle's surroundings to distort the original image content. By overlaying this with random visual noise, such as graininess or color variations, details are obscured while the area's structure remains intact.

[0082] In some examples, Procedure 100 may involve obtaining vehicle design data. It may further involve determining the coordinates of the vehicle's transparent area within the vehicle coordinate system based on this obtained design data. For example, vehicle design data may include digital information describing the geometry, structure, and position of the vehicle's components. This data serves to define the precise dimensions and spatial arrangements of vehicle components and to enable their integration. Design data can exist in various formats created by computer-aided design (CAD) software and often includes three-dimensional models (3D CAD data) or two-dimensional technical drawings.

[0083] In some examples, the design data may include a precise description of the vehicle's transparent areas in the vehicle coordinate system. For example, the 24-3408 17

[0084] Design data encompasses components such as windshields, side windows, or panoramic roofs, including their shape, size, and position relative to the vehicle's coordinate system. This design data can be, for example, CAD data and include a CAD model of the vehicle containing the entire vehicle geometry, including the body, interior structure, and the arrangement of sensors or cameras. Furthermore, design data can include information about material properties, mounting points, or assembly tolerances relevant for the precise definition of the components. By using a standardized vehicle coordinate system, the design data can be used precisely to define, for example, the location of transparent areas or the fields of view of cameras, and to facilitate technical processes such as calibrations or the mapping of sensor information.

[0085] In some examples, procedure 100 can include object recognition of the camera image using machine learning. For example, object recognition can involve a process in which specific objects are identified and located within the camera image. Object recognition can include both the classification of an object (e.g., face, vehicle, or license plate) and its position and size within the image using bounding boxes or masks. For example, object recognition using machine learning receives the camera image as input data. The machine learning model analyzes the camera image and provides output data such as a list of the detected objects, e.g., the transparent areas of the vehicle, faces, trees, license plates, etc., located in the image coordinate system with the coordinates (ul, vl) and (u2,u2). The machine learning model has, for example, been trained to recognize these objects.This method utilizes the efficient processing and generalization capabilities of modern algorithms to enable reliable and scalable object recognition. The goal of object recognition is to automatically analyze relevant image content and make the detected objects available for further processing.

[0086] Object recognition through machine learning can involve a process in which a model trained on large datasets is used to recognize specific patterns and features in images. In some examples, this can be achieved through deep neural networks, such as convolutional neural networks (CNNs). These networks analyze image data in multiple layers to extract characteristic features of objects and determine their classes. Such models can be specifically trained to recognize objects like faces, license plates, or vehicles in a camera image.24-3408 18

[0087] In some examples, the procedure can further include determining the coordinates of the vehicle's transparent area within the vehicle coordinate system, based on the object recognition performed. For example, the position and extent of transparent areas, such as windshields or side windows, within the vehicle coordinate system are determined by analyzing camera image data. In this case, the machine learning model recognizes the outlines of the transparent areas in the camera image based on image patterns, edges, or other features. In some examples, the recognized image coordinates of the transparent areas can then be transformed into the three-dimensional vehicle coordinate system, taking into account the camera position and orientation as well as the vehicle's geometric properties. This allows the 3D coordinates of the transparent areas to be precisely determined, even if no prior design data is available.Transforming the data back into the 3D vehicle coordinate system allows, for example, the comparison of the detected areas with vehicle-specific data such as camera position or geometry. This can ensure more precise and context-specific anonymization, especially with multiple interior vehicle cameras or complex vehicle structures, before the anonymization is applied directly to the camera image.

[0088] Object detection can also be used in conjunction with anonymizing the area of ​​the vehicle's surroundings within the camera image. In some examples, the intensity of anonymization can depend on a detected object within the vehicle's surroundings in the camera image. For instance, the intensity of anonymization can describe the degree to which an area in the camera image is obscured. It indicates how strongly an area of ​​the vehicle's surroundings is rendered unrecognizable through anonymization to protect identifying information. The intensity can be influenced by the choice of anonymization technique (e.g., pixelation, blurring, complete masking) and its parameters. While a low intensity causes only subtle changes, a high intensity leaves no original information recognizable.Intensity refers to the overall anonymization effect and is related, for example, to granularity, as the latter describes a specific aspect of the fineness or coarseness within the applied technique. High intensity could be achieved, for instance, through complete coverage, regardless of the granularity. With pixelation, however, granularity influences intensity: a large filter kernel with low granularity leads to higher intensity because details are heavily obscured. For example, a nearby face could be completely anonymized by coarse pixelation, while with subtle blurring using small granularity, the intensity remains lower, allowing contours to remain recognizable. The choice of intensity thus enables flexible adjustment of the 24-3408 19.

[0089] Anonymization is adapted to data protection requirements and the specific characteristics of the identified objects.

[0090] In some examples, the intensity of anonymization can depend on the size and / or position of the vehicle's transparent area. The intensity of anonymization can vary depending on the size of the transparent area because larger areas could potentially reveal more information about the vehicle's surroundings. For example, a small transparent area, such as a side window, might require less intensive anonymization because the visible environment is more limited. Conversely, a larger area, such as a windshield, which captures a large portion of the vehicle's surroundings, might require stronger anonymization to ensure that no sensitive information, such as faces, license plates, or detailed objects, remains unrecognizable. This adjustment allows for more efficient anonymization without unnecessarily slowing down image processing.The position of the transparent area within the vehicle can also influence the intensity of anonymization. A transparent area directly in the line of sight of the vehicle's interior camera, such as the windshield, may require more intensive anonymization, as this area captures the vehicle's main surroundings. Areas like side windows or a panoramic sunroof might require less anonymization, as they often only capture peripheral areas or specific perspectives. For example, a stronger filter might be used to pixelate a windshield, while a lesser degree of anonymization, such as subtle blurring, would suffice for a side window.

[0091] Further details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may include one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one of the examples described below (e.g., Figs. 2-3).

[0092] Fig. 2 schematically shows the device 200 for anonymizing a vehicle's surroundings in a camera image. The device 200 comprises a communication interface 210. The device further comprises a processor circuit 220. The communication interface 210 is, for example, a hardware and software unit that exchanges data between the processor circuit 220 and a vehicle interior camera, a memory, and / or an external source. For example, the processor circuit 210 can be a single dedicated processor, a single shared processor, or a plurality of single processors, some or all of which can be used jointly, a microcontroller, 24-3408 20

[0093] The device 200 may be or comprise an application-specific integrated circuit (ASIC), an integrated circuit (IC), a system-on-a-chip (SoC), a programmable logic element, or a field-programmable gate array (FPGA) with a microprocessor. The processor may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a neuromorphic processor, and / or a tensor processor. The processor circuit 210 may also be connected to a memory, such as a read-only memory (ROM) for storing software, a random access memory (RAM), and / or a non-volatile memory. The device 200 may, for example, include a memory or...to be coupled with this, which is configured to store instructions which, when executed by the processor circuit 210, cause the processor circuit 210 to perform the steps and procedures for context-dependent background generation described herein.

[0094] Communication interface 210 is configured to receive a camera image from a vehicle interior camera, whereby the camera image encompasses a portion of the vehicle's surroundings through a transparent area of ​​the vehicle. Communication interface 210 is further configured to receive the position and orientation of the vehicle interior camera within a vehicle coordinate system. Communication interface 210 is also configured to receive the coordinates of the transparent area of ​​the vehicle within the vehicle coordinate system.

[0095] The processor circuit 220 is configured to project the 3D coordinates of the transparent area from the vehicle coordinate system into a 2D image coordinate system of the camera image. The processor circuit 220 is also configured to anonymize the area of ​​the vehicle's surroundings in the camera image.

[0096] Further details and aspects are mentioned in connection with the examples described above or below. The example shown in Fig. 2 may include one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or an example described above (e.g., Fig. 1) or below (e.g., Fig. 3).

[0097] Fig. 3 schematically shows a vehicle 300 comprising the device 200 for anonymizing the vehicle's surroundings in a camera image. The device 200 is, for example, the device as described in Fig. 2. 24-3408 21

[0098] Further details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 3 may include one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one of the examples described above (e.g., Figs. 1-2).

[0099] Another embodiment is a computer program for carrying out one of the methods described herein, when the computer program runs on a computer, a processor or a programmable hardware component.

[0100] Another example of implementation is a non-volatile, machine-readable medium on which a program is stored that includes program code for carrying out one of the procedures described herein, when the program is executed on a processor or programmable hardware component. 24-3408 22

[0101] Reference symbol list

[0102] 100: Method for anonymizing a vehicle's surroundings in a camera image

[0103] 110: Receiving a camera image from a vehicle interior camera

[0104] 120: Obtaining the position and orientation of the vehicle's interior camera in the vehicle coordinate system

[0105] 130: Obtaining coordinates of the transparent area of ​​the vehicle in the vehicle coordinate system

[0106] 140: Projecting the 3D coordinates of the transparent area from the vehicle coordinate system into a 2D image coordinate system of the camera image

[0107] 150: Anonymizing the area of ​​the vehicle's surroundings in the camera image

[0108] 200: Device

[0109] 210: Communication interface

[0110] 220: Processor circuit

[0111] 300: Vehicle

Claims

24-3408 23 Patent claims 1. Method (100) for anonymizing a vehicle environment in a camera image, comprising: Obtaining (110) a camera image from a vehicle interior camera, wherein the camera image includes an area of ​​the vehicle environment through a transparent area of ​​the vehicle; Obtain (120) the position and orientation of the vehicle interior camera in a vehicle coordinate system; Obtain (130) coordinates of the transparent area of ​​the vehicle in the vehicle coordinate system; Projecting (140) the 3D coordinates of the transparent area from the vehicle coordinate system into a 2D image coordinate system of the camera image; and anonymizing (150) the area of ​​the vehicle environment in the camera image.

2. Device according to claim 1, further comprising determining a filter kernel, wherein the anonymization is carried out by applying the filter kernel to the area of ​​the vehicle environment in the camera image.

3. A method according to any of the preceding claims, wherein anonymizing the area of ​​the vehicle environment in the camera image comprises applying at least one of the following techniques: blurring the area of ​​the vehicle environment, pixelating the area of ​​the vehicle environment, masking the area of ​​the vehicle environment by complete coverage and geometric distortion of the area of ​​the vehicle environment, noise superimposition of the area of ​​the vehicle environment.

4. Method according to one of the preceding claims, further comprising performing a calibration of the vehicle interior camera, wherein the calibration determines at least one of the following parameters: the position of the vehicle interior camera in the vehicle coordinate system, the orientation of the vehicle interior camera, a distortion of the vehicle interior camera and a principal point of the vehicle interior camera.

5. A method according to any one of the preceding claims, further comprising: Obtaining design data for the vehicle; and Determining the coordinates of the vehicle's transparent area in the vehicle coordinate system based on the obtained design data.

6. A method according to any of the preceding claims, further comprising performing object recognition in the camera image using machine learning. 24-3408 24 7. Method according to claim 6, further comprising determining the coordinates of the transparent area of ​​the vehicle in the vehicle coordinate system based on the object recognition performed.

8. Method according to claim 6 or 7, wherein the intensity of anonymization depends on a detected object in the area of ​​the vehicle environment of the camera image.

9. Method according to one of the preceding claims, wherein the intensity of anonymization depends on the size and / or position of the transparent area of ​​the vehicle.

10. Method according to one of the preceding claims, wherein the transparent area of ​​the vehicle comprises at least one of the following parts of the vehicle: a windshield, a side window, a rear window, a panoramic roof, or a transparent vehicle fairing.

11. Device (200) for anonymizing a vehicle's surroundings in a camera image, comprising: a communication interface (210) is set up for: Receiving a camera image from a vehicle interior camera, wherein the camera image through a transparent area of ​​the vehicle includes an area of ​​the vehicle's surroundings; Obtaining the position and orientation of the vehicle's interior camera in a vehicle coordinate system; Obtaining coordinates of the transparent area of ​​the vehicle in the vehicle coordinate system; a processor circuit (220) that is configured to: Projecting the 3D coordinates of the transparent area from the vehicle coordinate system into a 2D image coordinate system of the camera image; and Anonymizing the area of ​​the vehicle's surroundings in the camera image.

12. Vehicle (300) comprising a device (200) for anonymizing image content of a vehicle according to claim 11.

13. Non-volatile machine-readable medium on which a program is stored containing program code for performing one of the methods (100) according to any one of claims 1 to 10, when the program is executed on a processor or a programmable hardware component. 24-3408 25 14. Program comprising program code for carrying out one of the methods (100) according to any one of claims 1 to 10, when the program is executed on a processor or a programmable hardware component.

Images