Vehicle, device, computer program and method for monitoring an environment of a vehicle

Abstract

Embodiments of the present disclosure relate to a vehicle (210), an apparatus (400), a computer program and a method (100) for monitoring an environment of the vehicle (210). The method (100) comprises obtaining (110) object-specific information about an object (220) associated with a third party (230) using at least one sensor of the vehicle (210) and monitoring criteria. Furthermore, the method (100) comprises providing (120) the object-specific information to the third party (230).

Description

Technical Field

[0001] Embodiments of the present disclosure relate to vehicles, apparatus, computer programs and methods for monitoring the environment of a vehicle. In particular, but not exclusively, embodiments relate to the concept of privacy protection when using a vehicle to monitor the environment. Background Art

[0002] Monitoring a vehicle's environment plays an increasingly important role in automotive applications. In practice, an autonomous or semi-autonomous vehicle may monitor its environment for the purpose of autonomous driving. However, monitoring the environment and / or sharing sensor data of the monitored environment (e.g., with a third party) may be problematic due to privacy concerns. Summary of the invention

[0003] Therefore, there may be a need for an improved concept for monitoring a vehicle's environment.

[0004] This need may be met by the subject matter of the attached independent and dependent claims.

[0005] Embodiments are based on the discovery that information collected by a vehicle about an environment can also be used for purposes unrelated to the vehicle, for example, for monitoring infrastructure objects in the environment for maintenance and / or surveillance purposes, and can be shared with third parties responsible for the maintenance of the infrastructure objects. In this way, monitoring infrastructure objects may be less time-consuming and costly. Another discovery is that, by using appropriate monitoring criteria, such information can be reduced or limited to object-specific information about these objects (e.g., the property of a third party) related to a third party involved in one or more objects to provide privacy protection. For example, the monitoring criteria can ensure that the object-specific information provided to the third party represents a predefined location, object, and / or scene in order to protect privacy. In this way, the third party can only receive object-specific information about its object, thereby respecting the privacy of others. In practice, the object-specific information can be obtained from the sensor data of the vehicle (for autonomous or semi-autonomous driving) and can include only its part related to the third party's object and / or meta-information from the sensor data, as described in more detail later.

[0006] Embodiments of the present disclosure provide a method for monitoring the environment of a vehicle. The method includes obtaining object-specific information about an object related to a third party using at least one sensor of the vehicle and monitoring standards. In addition, the method includes providing the object-specific information to the third party. For example, this allows the third party to monitor the object based on the object-specific information, for example, so as to perform less time-consuming and costly maintenance of the object. The monitoring standards can provide privacy protection, for example, so that the third party only receives object-specific information of objects related to the third party (for example, objects that the third party has the right or is authorized to monitor). In practice, the object is, for example, the property of the third party or the object that the third party is designated or authorized to monitor. The third party can be a (natural) person, an administrative agency, a company, etc. For example, the third party is the owner of the object or a person / company entrusted by the owner to monitor and / or maintain the object.

[0007] Optionally, obtaining object-specific information comprises monitoring the environment of the vehicle in order to obtain sensor data of the environment, and obtaining object-specific information from the sensor information of the environment based on the monitoring criteria. This allows sensor-based monitoring of the object. In some embodiments, the sensor data comprises or corresponds to sensor data for (semi-)autonomous driving. In this case, the sensor data serves a multifunctional use of sensor data for (semi-)autonomous driving and other purposes of a third party.

[0008] In some embodiments, the object-specific information obtained includes an object-specific portion of the sensor data of the environment. In this way, a third party can receive the sensor data related to the object recorded by the vehicle as is. A skilled person having the benefit of this disclosure will appreciate that in some applications, (the portion of) the sensor data may be superior to the meta-information and / or other information for use by the third party.

[0009] Alternatively or additionally, the object-specific information includes meta-information about the object from the sensor data. It will be appreciated by those skilled in the art that the meta-information may be less detailed than the sensor data and thus may provide better privacy protection.

[0010] Thus, obtaining object-specific information may comprise obtaining meta-information from the sensor data using an aggregator and a privacy assessor procedure for checking the aggregator by comparing the aggregator with at least one reference software architecture and / or based on training data for training the aggregator. In this way, the privacy assessor may ensure that the aggregator works as expected, in particular that the aggregator complies with and / or does not violate (predefined) privacy protection requirements. In particular, the proposed use of a privacy assessor allows the use of an aggregator provided and / or trained by a third party or any other external party (e.g. other than the manufacturer of the vehicle).

[0011] In some embodiments, the monitoring criteria indicates a predefined location where the object is located, and obtaining the object-specific information includes determining whether the object-specific information represents the predefined location, and obtaining the object-specific information if the object-specific information represents the predefined location. For example, this can ensure that a third party only receives the object-specific information of the predefined location, thereby facilitating further improving the privacy protection of others.

[0012] Optionally, the monitoring criteria indicate one or more predefined scenarios of the environment. Thus, obtaining object-specific information may include determining whether at least one of the predefined scenarios exists in the environment, and obtaining object-specific information if at least one of the predefined scenarios exists. For example, the scenario defines when or in what event a third party intends to receive object-specific information. In an application for maintenance of an object, for example, the scenario (only) includes scenarios in which the object requires repair or other maintenance measures. In this way, the privacy of others can be better protected and unnecessary communications with third parties can be avoided.

[0013] In some embodiments, determining whether at least one of the predefined scenarios exists includes determining whether the object-specific information indicates at least one of the predefined scenarios, and wherein obtaining the object-specific information based on the monitoring criteria includes obtaining the object-specific information if the object-specific information indicates at least one of the predefined scenarios. In practice, the object-specific information includes, for example, sensor data, and determining that the sensor indicates that the object requires repair or other maintenance measures. In this way, additional information about the existence or non-existence of such scenarios can be omitted.

[0014] Optionally, determining whether the object-specific information indicates at least one of the predefined scenes comprises using a machine learning-based program to determine whether the object-specific information indicates at least one of the predefined scenes, wherein the machine learning-based program is trained to detect the one or more predefined scenes in the object-specific information. In this way, the scene can be detected in the object-specific information more reliably.

[0015] The method may further comprise determining information about false positive detections and / or detection frequencies of at least one of the scenarios from the object-specific information, and determining a fault based on the information about the detection frequencies and / or false positive detections. In this way, for example, an abuse of the application of the proposed concept may be detected. In an embodiment, for example, one or more predefined scenarios are detected unexpectedly frequently, for example more frequently than a predefined maximum frequency of the one or more scenarios, and / or false positive detections (false positive detections of such scenarios or other false positive detections) occur unexpectedly frequently, for example more frequently than a predefined maximum frequency of false positive detections, and a fault is determined by comparison with the corresponding maximum frequency.

[0016] In practice, the (vehicle's) environment comprises, for example, at least one infrastructure object, and the object-specific information is therefore indicative of said at least one infrastructure object. In this case, the proposed method allows for a more efficient monitoring and maintenance of the infrastructure objects.

[0017] In some embodiments, the monitoring criteria indicates at least one predefined infrastructure object. Thus, obtaining the object-specific information may include determining whether the object-specific information represents the at least one predefined infrastructure object, and obtaining the object-specific information if the object-specific information represents the at least one predefined infrastructure object. For example, the monitoring criteria specifies a specific infrastructure object, such as a specific building and / or streetlight, such that the third party only receives object-specific information about the specified building and / or streetlight. Thus, in this way, specific predefined objects may be monitored.

[0018] Optionally, determining whether the object-specific information indicates the at least one predefined infrastructure object comprises using a machine learning-based program to determine whether the object-specific information indicates the at least one predefined infrastructure object, the machine learning-based program being trained to detect the at least one predefined infrastructure object in the object-specific information. A skilled person having the benefit of the present disclosure will appreciate that a machine learning-based program can more reliably determine whether the object-specific information indicates an infrastructure object than other programs (not based on machine learning), and thus can provide more reliable privacy protection and more reliable object monitoring.

[0019] In some embodiments, obtaining the object-specific information includes clearing a portion of the object-specific information based on the at least one monitoring criterion. In doing so, sensitive portions of the object-specific information may be deleted or obscured to achieve better privacy protection.

[0020] In practice, the at least one monitoring criterion may specify at least one object type to be cleared for privacy protection. Thus, clearing the portion of the object-specific information may include determining from the object-specific information a portion of information indicative of the at least one object type specified by the at least one monitoring criterion, and clearing the determined portion of information. Examples of such object types may include faces, license plates, and / or other sensitive objects. Clearing information about such objects provides greater privacy protection.

[0021] A skilled person having the benefit of this disclosure will appreciate that, in application, the proposed method may be implemented on the vehicle, or may be implemented outside / separate from the vehicle by a separate entity capable of retrieving object-specific information. Thus, the method may be performed by the vehicle and / or an external server separate from the vehicle. A skilled person will appreciate that execution of the method by the vehicle may avoid interception of sensitive information, for example when such information is transmitted with a separate entity, and therefore, may provide a higher protection for sensitive information. On the other hand, execution by an external server / separate entity may reduce the computing power required at the vehicle and allow for centralized review of shared information.

[0022] Other embodiments of the present disclosure provide a computer program having a program code for performing at least one of the methods of the preceding claims when the computer program is executed on a computer, a processor or a programmable hardware component.

[0023] Further embodiments provide an apparatus comprising one or more interfaces for communication and a data processing circuit configured to control the one or more interfaces and to perform embodiments of the proposed method using the one or more interfaces.

[0024] Other embodiments of the present disclosure provide a vehicle including the proposed device. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Some examples of apparatus, methods and / or computer programs will be described below, by way of example only, with reference to the accompanying drawings, in which

[0026] Figure 1 shows a flow chart schematically illustrating an embodiment of a method for monitoring an environment of a vehicle;

[0027] Figure 2 Schematically illustrate use cases of the proposed concept;

[0028] Figure 3 shows a flow chart schematically illustrating how to generate and deploy a machine learning based program for the proposed concept; and

[0029] Figure 4 Shown is a block diagram schematically illustrating an embodiment of the proposed arrangement. DETAILED DESCRIPTION

[0030] Various example embodiments are now described more fully with reference to the accompanying drawings, in which some example embodiments are illustrated. In the drawings, the thickness of lines, layers, or regions may be exaggerated for clarity. Optional components may be illustrated using dashed, dashed, or dotted lines.

[0031] Thus, although the example embodiments are capable of various modifications and alternative forms, embodiments thereof are shown in the drawings by way of example and will be described in detail herein. However, it should be understood that the example embodiments are not intended to be limited to the specific forms disclosed, but rather, the example embodiments should encompass all modifications, equivalents, and alternatives falling within the scope of the present invention. Throughout the description of the drawings, the same reference numerals refer to the same or similar elements.

[0032] As used herein, the term "or" refers to a non-exclusive or unless otherwise specified (e.g., "otherwise" or "or as an alternative"). In addition, unless otherwise specified, words used herein to describe relationships between elements should be broadly interpreted to include direct relationships or the presence of intervening elements. For example, when an element is referred to as being "connected" or "coupled" to other elements, the element may be directly connected or coupled to the other elements, or there may be intervening elements. Conversely, when an element is referred to as being "directly connected" or "directly coupled" to other elements, there are no intervening elements. Similarly, terms such as "between," "adjacent," and the like should also be interpreted in a similar manner.

[0033] The terms used herein are only used to describe specific embodiments and are not intended to limit the exemplary embodiments. Unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" used herein are intended to also include plural forms. It should also be understood that the terms "comprise" or "include" when used herein specify the presence of stated features, integers, steps, operations, elements or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components or groups thereof.

[0034] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the example embodiments belong. It should also be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0035] Currently, infrastructure providers have to regularly check the status of their infrastructure in order to identify parts of the infrastructure that need maintenance, such as broken street lights, potholes, bicycle parking lots, etc. Automated cars will be able to observe such infrastructure using their onboard sensors. In some automotive applications of highly automated driving systems, vehicles are equipped with a large number of powerful sensors that operate at many locations that may be of interest to infrastructure providers. Therefore, infrastructure providers can use sensor data from such sensors to monitor their objects in a more practical, less time-consuming, and less costly way. However, due to privacy protection concerns, infrastructure providers may not own or have access to sensor data from such sensors to monitor their infrastructure. Therefore, the vehicle OEM (Original Equipment Manufacturer) that owns the sensor data may not be able to or is not authorized to grant access to the vehicle's sensor data.

[0036] Therefore, an object of the present disclosure may be to solve the above problems.In particular, but not exclusively, embodiments of the present disclosure provide a system that can allow infrastructure providers to access information from vehicle sensor data for monitoring purposes.

[0037] Various aspects and features of the claimed subject matter are described in more detail below with reference to the accompanying drawings.

[0038] Figure 1 A flow chart schematically illustrating an embodiment of a method 100 for monitoring an environment of a vehicle is shown.

[0039] In the context of the present disclosure, a vehicle may be any ground-based vehicle (e.g., car, truck, motorcycle, etc.), boat, airplane, etc. In practice, the vehicle may be an autonomous or semi-autonomous driving vehicle equipped with one or more sensors.

[0040] As can be seen from the flowchart, the method 100 includes obtaining 110 object-specific information about an object associated with a third party using at least one sensor of the vehicle and monitoring criteria. Furthermore, the method 100 includes providing 120 the object-specific information to the third party.

[0041] A sensor can be any device for obtaining information about the environment (surroundings) of a vehicle. In practice, the sensor, for example, includes or corresponds to an imaging sensor, a distance sensor and / or any measurement system for sensing the environment. Examples of such sensors include, for example, LIDAR sensors, RADAR sensors, (RGB) cameras, infrared sensors, etc. The skilled person will appreciate that information about the environment can also be received from other vehicles. Therefore, in addition or alternatively, the sensor may include any interface for communicating information about the environment, for example, with other vehicles and / or (communication) infrastructure objects. For example, the sensor may include any wireless communication device, such as an antenna and / or a signal processing circuit. Thus, object-specific information may indicate sensor data of at least one (on-board) sensor of the vehicle and / or information communicated with other vehicles or infrastructure objects (e.g., sensor data from one or more sensors of a vehicle and / or infrastructure object).

[0042] The use of monitoring criteria can ensure that privacy protection is provided, as described in more detail below. A skilled person having the benefit of the present disclosure will appreciate that the monitoring criteria can be any criteria suitable for determining whether information about the environment is related to an object involving a third party (e.g., whether the information is related to an object that the third party is authorized to monitor). In this way, only (privacy-protecting) object-specific information that the third party is allowed or authorized to receive and / or use is obtained, for example, information that is explicitly dedicated to the object or exclusively includes information about the object. Therefore, the monitoring criteria can also be understood as privacy criteria. Method 100 provides a privacy-protecting approach that allows information collected by a vehicle to be shared for external / non-vehicle related purposes, particularly for maintenance and / or monitoring by third parties.

[0043] A skilled person having the benefit of this disclosure will appreciate that the proposed concept can be applied to any object. In automotive applications, for example, the method 100 can be applied to any object that is "visible" to the vehicle. Thus, examples of objects include infrastructure objects, buildings, street lights, traffic lights, (bicycle) parking lots, other vehicles, etc. In particular, the proposed concept can be applied by or for infrastructure providers who may need to monitor infrastructure objects / buildings, for example for maintenance purposes.

[0044] In order to obtain 110 the object-specific information, the vehicle may monitor an environment using sensors in order to obtain sensor data of the environment. The object-specific information may then be obtained from the sensor data of the environment based on a monitoring criterion. In doing so, for example, an object-specific portion corresponding to the object-specific information is extracted from the sensor data of the environment. For example, the monitoring criterion provides for (only) extracting the object-specific information from the (continuously recorded) sensor data. In particular, the object-specific information may be extracted from the sensor data for vehicle-related purposes, such as for autonomous driving, semi-autonomous driving or other driver assistance functions.

[0045] Alternatively or additionally, the monitoring criteria may provide that the sensor only records information about the object. To this end, for example, the monitoring criteria are adapted so that the sensor is only activated when the sensor is able to record the object. For example, the monitoring criteria are configured so that the imaging sensor is only activated when the vehicle is in a suitable position so that the imaging sensor records the object. As a result, the sensor data may only include object-specific sensor data that may serve as object-specific information.

[0046] Thus, the object-specific information may comprise sensor data output by a sensor.In an embodiment, the object-specific information comprises, for example, image data, LIDAR data, RADAR data.

[0047] Alternatively or additionally, the object-specific information may indicate a (high-level) interpretation or evaluation of the sensor data. In an example, the object-specific information comprises, for example, meta-information about the object from the sensor data. The meta-information may, for example, be understood as abstract or high-level information about the object. In practice, the meta-information comprises, for example, information about the properties of the object. In applications for maintenance purposes, the meta-information, for example, indicates (only) whether and / or which measures are necessary to repair or maintain the object. To obtain the object-specific information comprises obtaining the meta-information from the sensor data using an aggregator. In practice, the aggregator may be configured to interpret or evaluate the sensor data to extract the meta-information.

[0048] In order for the meta-information to indicate the expected type of information or attribute of the object, the aggregator can be tailored to the expected use of the meta-information. As described above, the aggregator is configured, for example, to extract meta-information indicating whether the object requires maintenance or repair measures. To this end, the aggregator can be configured by a third party or an external software engineering company entrusted to configure the aggregator. In this case, method 100 can specify measures to ensure that the aggregator works properly, that is, so that the privacy of others is protected. Therefore, regulations and code / model review measures can be taken for this design. For example, method 100 can optionally specify the use of a privacy assessor program to check the aggregator. In doing so, the privacy assessor can compare the aggregator with at least one reference software architecture. For example, the reference software architecture is one of a set of (limited) acceptable software architectures that meet predefined security and privacy protection requirements or regulations. By comparing the reference software architecture with the aggregator, it can be determined whether the software architecture of the aggregator or some (critical) part thereof matches or corresponds to the acceptable software architecture to check whether the aggregator works properly and / or meets predefined security and privacy protection requirements or regulations. A skilled artisan having the benefit of this disclosure will appreciate that the reference software architecture may be adapted to various security and / or privacy protection requirements or regulations, for example, to account for different privacy protection regulations in different countries.

[0049] In some applications, machine learning may be used to train the aggregator. The skilled person will appreciate that in this case, whether the aggregator is working properly may be particularly dependent on the training data used to train the aggregator. Therefore, alternatively or additionally, the privacy assessor may be configured to check whether the aggregator is working properly based on the training data used to train the aggregator. To this end, the training data may be fed to the privacy assessor to check whether the training data is actually relevant to the purpose of the object to be monitored and / or the object-specific information, and / or is reasonable for the purpose of the object to be monitored and / or the object-specific information. In an embodiment, method 100 is applied for monitoring purposes, and the object is, for example, a street lamp. In this case, the training data may include an image, and the privacy assessor may, for example, use image recognition to check whether at least a portion of the training data indicates a street lamp. The skilled person will appreciate that this can be similarly applied to any type of training data, object and / or object-specific information purpose.

[0050] A skilled person having the benefit of this disclosure will appreciate that, in order to protect privacy, information about the environment may be limited, for example, to a specific area or space that a third party is authorized to monitor. To this end, the monitoring criteria may indicate a predefined location where the object is located. Thus, in some embodiments, obtaining 110 object-specific information includes determining whether the object-specific information represents a predefined location, and if the object-specific information represents a predefined location, obtaining the object-specific information.

[0051] In this way, other locations can be excluded from the object-specific information so as not to infringe on the privacy of others. To this end, for example, geo-fencing technology is applied so that only the location information of the object is recorded and / or included in the object-specific information. To this end, the information about the location can indicate a certain area of ​​the street in which the vehicle can record the object, and only the sensor data that have been recorded when the vehicle is in this area are included or represented in the object-specific information. To this end, for example, GPS (Global Positioning System) data or other information about the location of the vehicle can be used. For example, in geo-fencing technology, a so-called "geofence" can be specified that includes (only) the location of the object, and it is stipulated that the vehicle only records sensor data when it is within the geofence, and / or only includes sensor data or information related to the location surrounded by the geofence.

[0052] In other embodiments, the sensor data may include image data, and image recognition may be used to determine information included in the object-specific information.

[0053] In an exemplary use case, for example, method 100 is applied to monitor a (specific) bicycle parking lot for maintenance and / or surveillance, and the information about the location indicates (only) the location of the bicycle parking lot, such that other locations outside the bicycle parking lot (e.g., buildings surrounding the bicycle parking lot) are excluded from the object-specific information.

[0054] Those skilled in the art who have the benefit of this disclosure will appreciate that the various aspects and features described above may be similarly applied and / or adapted to other use cases or objects.

[0055] In some use cases, scenario-based monitoring may be preferred, e.g. in order to avoid unnecessary communications. This scenario-based monitoring aims at third parties receiving object-specific information only in a certain scenario, e.g. only when a bicycle parking lot requires repair or maintenance measures.

[0056] To this end, the monitoring criteria may indicate one or more predefined scenarios of the environment. For example, the monitoring criteria specifies that object-specific information is obtained only when / if the one or more scenarios are present in the environment. For example, an exemplary scenario is a scenario where a bicycle parking lot is damaged or dirty, such that repair and cleaning are required, respectively.

[0057] Thus, obtaining object-specific information may include determining whether at least one predefined scene is present in the environment, and obtaining the object-specific information if at least one predefined scene is present. In practice, for example, object-specific information indicating at least one predefined scene is (only) extracted from the sensor data, and / or sensor data of the object is recorded (only) when at least one predefined scene is present.

[0058] A skilled person having benefit of this disclosure will appreciate that there may be different ways to determine whether one or more scenarios exist. In some use cases, for example, the determination of whether one or more scenarios exist is based on a message received from a person or user. Such a message is, for example, received from a user of a bicycle parking lot and may indicate that the bicycle parking lot is damaged or dirty.

[0059] Additionally or alternatively, the object-specific information may be used to determine whether at least one scene exists. In an embodiment, for example, determining whether at least one predefined scene exists includes determining whether the object-specific information indicates at least one predefined scene. In practice, the object-specific information, for example, indicates or includes image data of the object, and image recognition may be used to determine whether at least one scene exists from the image data. In an embodiment, for example, image recognition is used to determine whether a bicycle parking lot is damaged or dirty from the image data.

[0060] Then, the object-specific information may be obtained (only) if it indicates at least one predefined scenario. In practice, for example, (only) the part indicating at least one scenario is extracted from the sensor data in order to forward it to a third party. For example, (only) the part indicating that a bicycle parking lot is damaged and / or how it is damaged may be obtained and provided to a third party.

[0061] The skilled person will appreciate that, in practice, various routines or programs may be used to determine whether the object-specific information indicates one or more predefined scenes. In some embodiments, it may be preferred to use a machine learning-based program that is trained to detect one or more predefined scenes in the object-specific information. In order to enable the machine learning-based program to detect one or more scenes from the object-specific information, it may be trained based on training data (training data pairs) indicating one or more predefined scenes. In practice, for example, the machine learning-based program is an image recognition program, and the training data may include image data of damaged or dirty bicycle parking lots to detect such scenes.

[0062] In some use cases, a procedure for determining whether object-specific information indicates at least one scenario may be manipulated and / or may not function properly. In some cases, the procedure, for example, erroneously detects that the object-specific information indicates at least one scenario and / or (other) false positive detections may occur. In order to detect such a failure, the method may further include determining information about false positive detections and / or detection frequencies of at least one scenario from the object-specific information, and determining the failure based on the information about false positive detections and / or detection frequencies of at least one scenario.

[0063] In practice, for example, false positive detections of wrong / false objects (other than the above-mentioned objects) and / or too many false detections of one of the scenarios may occur. In such cases, the program may detect that the bicycle parking lot is damaged more often than expected. One idea of ​​the present disclosure is then to determine a fault based on the detection frequency. To this end, for example, the detection frequency is compared with a maximum expected detection frequency of one of the predefined scenarios. Then, if the detection frequency exceeds the maximum expected frequency, a fault may be assumed / determined.

[0064] Alternatively or additionally, false detections can be determined. To this end, object-specific information can be manually and / or automatically reviewed by other appropriately designed software and / or independent entities. In practice, object-specific information is, for example, associated with the wrong object. For example, the object to be monitored includes a building, and the object-specific information may be erroneously associated with a person. In this case, automatic and / or manual review may show that the wrong object has been monitored, and thus, information about the false detection may indicate that the object-specific information is associated with the wrong object. Thus, a failure can be assumed in this case. In addition, this information can be excluded from the object-specific information. The review of positive detection can ensure that the information included in the object-specific information is actually reasonable for the described purpose and prevent the object-specific information from being associated with other things. The review can be based on a random selection among all positive detections, that is, based on a random selection of information included in the object-specific information.

[0065] In some applications of the proposed concept, it may be necessary to monitor one or more very specific objects. Thus, the monitoring criteria may indicate at least one predefined infrastructure object. In this case, obtaining the object-specific information may include determining whether the object-specific information represents the at least one predefined infrastructure object, and obtaining the object-specific information if the object-specific information represents the at least one predefined infrastructure object.

[0066] For example, the monitoring criteria specifies that (only) object-specific information of one or more (very) specific objects or infrastructure objects is to be obtained in order to provide it to a third party. In an embodiment, the monitoring criteria specifies, for example, that (only) information about several specific bicycle parking lots is to be obtained in order to provide it to a third party. In practice, for example, they are owned and / or maintained by the same company or person. Thus, object-specific information of (only) these bicycle parking lots can be obtained and provided to said company or person. Thus, a monitoring criterion indicating at least one predefined infrastructure object can provide that a third party can monitor very specific objects.

[0067] In some embodiments, a suitably trained machine learning-based program is used to determine whether the object-specific information indicates at least one predefined infrastructure object. In some use cases, the object-specific information includes, for example, image data, and the machine learning-based program is trained to detect infrastructure objects in the image data by image recognition. Those skilled in the art will appreciate that, to this end, the machine learning-based program may be trained using training data that includes images of infrastructure objects.

[0068] In some cases, the object-specific information obtained may include sensitive information. In practice, the object-specific information may include image data of a license plate or a person's face (near or in front of the monitored object). To this end, the monitoring criteria may specify the removal of such information, and thus, obtaining the object-specific information may include removing a portion of the object-specific information based on at least one monitoring criterion. In an embodiment, the monitoring criteria specify the removal of image data of license plates or faces. Thus, such information in the object-specific information may be removed. Removing such information may include any measures suitable for removing sensitive information from the object-specific information or making it unrecognizable. To this end, for example, image data of license plates or faces is removed or blurred. To do this, such sensitive information can be detected using image recognition and then automatically removed.

[0069] In some embodiments, at least one monitoring standard specifies at least one object type to be cleared for privacy protection. Examples of such object types include, for example, sensitive objects such as license plates, faces, people, vehicles, etc. Thus, clearing the portion of the object-specific information may include determining from the object-specific information an information portion indicating at least one object type specified by at least one monitoring standard, and clearing the determined information portion. In this way, information about sensitive objects can be excluded from the object-specific information. In an embodiment, for example, the object-specific information includes image data, and information about such object types in the image data is detected by image recognition and then blurred or deleted. Those skilled in the art will appreciate that for this purpose, a suitably trained machine learning-based program can be used.

[0070] In an application, the method 100 may be performed by the vehicle and / or by an external server separate from the vehicle.

[0071] The external server, for example, comprises or corresponds to an OEM backend of a vehicle manufacturer and can be configured to communicate with the vehicle and a third party. In this case, the OEM backend can execute the method 100 to obtain object-specific information and provide it to a third party. For example, this allows the method 100 to be applied to vehicles that are not particularly adapted to the proposed concept, for example, vehicles adapted to obtain object-specific information at the vehicle. Likewise, this allows the method 100 to be centrally executed for a plurality of vehicles (e.g., a fleet of vehicles) that are configured to communicate with the external server. Furthermore, this can save additional computing resources on the vehicle.

[0072] Likewise, for example, the method 100 may be performed by a vehicle so as to dispense with such a separate external server. To this end, the vehicle may be appropriately configured to perform the method 100 and communicate with a third party.

[0073] A skilled person having the benefit of this disclosure will appreciate that the program and / or routine used to perform the proposed method 100 can be protected from attack to improve the security of sensitive data stored and / or processed during the method 100. In particular, a so-called "sandbox" can be applied for this purpose, so that the software used for the proposed concept is separated from other codes running in the vehicle to prevent mutual interference. Alternatively or additionally, other data protection measures can also be taken and implemented.

[0074] In some applications, for example, method 100 may also be performed collaboratively by an external server and a vehicle such that the vehicle obtains object-specific information and the external server provides the object-specific information to a third party, as described in more detail below.

[0075] Refer to the following Figure 2 Some aspects and features of embodiments of the present disclosure are described in more detail.

[0076] Figure 2 The schematic diagram illustrates a use case of the proposed concept. In the illustrated use case, for example, an infrastructure provider 230 may want to monitor an infrastructure object 220 such as a bicycle parking lot. In this example, the infrastructure provider 230 may own the infrastructure object 220 or the bicycle parking lot, or may otherwise be authorized to monitor it. However, monitoring the infrastructure may require separate cameras and / or human resources.

[0077] For example, a parking lot is adjacent to a street so that a passing vehicle 210 can see the parking lot. According to the basic idea of ​​the present disclosure, it is proposed to monitor infrastructure objects using a vehicle 210 that passes by the infrastructure object randomly or purposefully. The vehicle 210 can be equipped with one or more sensors for monitoring its environment. In practice, the vehicle 210 is, for example, a (semi-)autonomous driving vehicle, and the sensors may include one or more sensors of an autonomous driving system or a driver assistance system. When passing by, the vehicle 210 can record the infrastructure object 220, thereby generating sensor data of the infrastructure object 220. In doing so, the vehicle 210 applies monitoring criteria in order to obtain object-specific information from the sensor data, so that the obtained object-specific information can be shared with the infrastructure provider while protecting the privacy of others.

[0078] For example, the monitoring standard stipulates that the sensor is activated only when passing through the infrastructure object 220, so that, in particular, object-specific sensor data of the infrastructure object is recorded and included in the object-specific information. Optionally, the sensor data can be processed to obtain object-specific information. In an example, the sensor data can be evaluated to obtain meta-information about the infrastructure object 220, as described above. The meta-information, for example, indicates whether the infrastructure needs repair, maintenance and / or cleaning, such as whether the infrastructure object 220 is damaged or dirty.

[0079] Optionally, the sensors of the vehicle 210 are permanently active, for example, in order to continuously sense the environment for an autonomous driving system or a driver assistance system. In this case, monitoring criteria can be applied to extract object-specific information from the sensor data, so that (only) sensor data specifically representing the infrastructure object 220 is included in the object-specific information, while other (sensitive) sensor data is not included in the object-specific information / is excluded from the object-specific information.

[0080] In the next step, the (privacy-preserving) object-specific information is sent to an external server 240, for example, the backend of the vehicle 210 (schematically indicated by the dashed arrow numbered 212). The external server 240 then provides the object-specific information to the infrastructure provider 230 (as indicated by the dashed arrow numbered 242).

[0081] In some other embodiments, the vehicle 210 may provide sensor data to the external server 240, and the external server 240 obtains object-specific information from the sensor data. To this end, the infrastructure provider 220 may submit an appropriate request to the external server 240 for object-specific information.

[0082] A skilled artisan having the benefit of this disclosure will appreciate that Figure 2The use cases illustrated in the diagrams may also be combined with other aspects and features described herein. Likewise, the skilled person will appreciate that the use cases illustrated in the diagrams may also be applied to other applications, vehicles and / or objects.

[0083] In particular, the skilled person will appreciate that the proposed concept can be applied to use cases with multiple third parties and / or can be tailored specifically for the purpose of the application. In practice, for example, different infrastructure providers may have different specially designed machine learning programs in order to obtain object-specific information that is tailored specifically for the purpose for which the object-specific information is used (see above). For this purpose, a separate machine learning-based program can be deployed.

[0084] Reference below Figure 3 Other aspects and features of this machine learning-based procedure for obtaining object-specific information are described in more detail.

[0085] Figure 3 A flow chart schematically illustrating how to generate and deploy a machine learning based program for the proposed concept is shown.

[0086] In particular, Figure 3 It may involve an application where a fleet backend manager 330 (typically an OEM) provides a service to identify problems visible from the street, and some independent infrastructure provider 310 requests that service.

[0087] As shown in the flow chart, a (separate) machine learning based program may be ordered from a machine learning engineering office 320 by a (independent) infrastructure provider 310 for obtaining sensor data and / or meta information for object-specific information, e.g. in the above-described object type-based, scene-based and / or location-based manner. In particular, the machine learning based program may be configured for infrastructure state recognition, which is why the machine learning (ML) based program may also be referred to as an “ML recognition module”. The machine learning engineering office 320 may customize the ML recognition module taking into account the object-specific information or the purpose of the ML recognition module, a predefined and / or selected reference software or ML architecture, a predefined and / or selected training data, and / or a desired geo-fence (see above).

[0088] The machine learning engineering office then provides the ML recognition module to the vehicle's backend, for example, a so-called "fleet connected driving software manager". The backend then deploys the ML recognition module. To this end, the backend installs a program, for example, in the vehicle and / or in the backend.

[0089] The proposed concept can also be implemented in an apparatus, as shown below Figure 4 Described in more detail.

[0090] Figure 4 Shown is a block diagram schematically illustrating an embodiment of an apparatus 400 implementing the proposed concept.

[0091] from Figure 4 As can be seen in FIG. 4 , the device 400 includes one or more interfaces 410 for communication and a data processing circuit 420 configured to control the one or more interfaces 410. The data processing circuit 420 is also configured to control the one or more interfaces and use the one or more interfaces to perform one of the proposed methods. That is, the device 400 can be configured to perform the method 100. Thus, the device 400 can be implemented in an external server of the vehicle and / or in the vehicle.

[0092] In an embodiment, one or more interfaces 410 may include a device for communicating with a vehicle or other server. To this end, one or more interfaces 410 may include or correspond to any device for obtaining, receiving, sending or providing analog or digital signals or information, for example, any connector, contact, pin, register, input port, output port, conductor, channel, etc. that allows providing or obtaining signals or information. The interface may be wireless or wired, and it may be configured to communicate with other internal or external components, i.e., send or receive signals, information. One or more interfaces 410 may include additional components for implementing corresponding communications, such components may include transceiver (transmitter and / or receiver) components, such as one or more low noise amplifiers (LNA), one or more power amplifiers (PA), one or more duplexers, one or more diplexers, one or more antenna duplexers, one or more filters or filtering circuits, one or more converters, one or more mixers, correspondingly adapted radio frequency components, etc. One or more interfaces 410 may be coupled to one or more antennas, which may correspond to any transmitting and / or receiving antennas, such as horn antennas, dipole antennas, patch antennas, sector antennas, and / or the like. The antennas may be arranged in defined geometric configurations, such as uniform arrays, linear arrays, circular arrays, triangular arrays, uniform field antennas, field arrays, combinations thereof, and / or the like.

[0093] like Figure 4As shown in , one or more interfaces 410 are coupled to a data processing circuit 420. In an embodiment, the data processing circuit 420 may include any means for processing information according to one of the proposed methods. The data processing circuit 420 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer, or a programmable hardware component that can operate with corresponding adapted software. In other words, the functions of the data processing circuit 420 may also be implemented with software that is then executed on one or more programmable hardware components. Such hardware components may include a general-purpose processor, a digital signal processor (DSP), a microcontroller, and / or the like.

[0094] Those skilled in the art will readily appreciate that the steps of various above methods can be performed by a program-controlled computer. In some embodiments, the apparatus may also include a program storage device, such as a digital data storage medium, which is machine or computer readable, and encodes a machine executable or computer executable program of an instruction, wherein the instruction performs some or all of the steps of the method described herein. The program storage device may be, for example, a digital memory, a magnetic storage medium such as a disk and a tape, a hard drive, or an optically readable digital data storage medium. Embodiments are also intended to encompass a computer programmed to perform the steps of the method described herein, or a (field) programmable logic array ((F) PLA) or a (field) programmable gate array ((F) PGA) programmed to perform the steps of the above method.

[0095] The description and drawings merely illustrate the principles of the present invention. Therefore, it should be appreciated that those skilled in the art will be able to design various arrangements that, although not explicitly described or shown herein, embody the principles of the present invention and are included in its spirit and scope. In addition, all examples listed herein are primarily intended to be clearly used for teaching purposes only to help the reader understand the principles of the present invention and the concepts contributed by the inventors to further develop the art, and should be interpreted as not being limited to such specifically listed examples and conditions. In addition, all statements describing the principles, aspects and embodiments of the present invention herein, and their specific examples are intended to cover their equivalents.

[0096] When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or multiple separate processors, some of which may be shared. In addition, the explicit use of the term "processor" or "controller" should not be interpreted as referring only to hardware capable of executing software, but may implicitly include, but not limited to, digital signal processor (DSP) hardware, network processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read-only memories (ROMs), random access memories (RAMs), and non-volatile storage devices for storing software. Other hardware, conventional or customized, may also be included. Their functions may be implemented by the operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or even manually, and the specific technology may be selected by the implementer based on a more specific understanding of the context.

[0097] Those skilled in the art will appreciate that any block diagrams herein represent conceptual views of illustrative circuits embodying the principles of the present invention. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocodes, etc. represent various processes that can be substantially represented in a computer-readable medium and thus executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0098] In addition, the following claims are hereby incorporated into the Detailed Description section, where each claim can stand on its own as a separate embodiment. Although each claim can stand on its own as a separate embodiment, it is noted that - although in the claims, a dependent claim may refer to a specific combination with one or more other claims - other embodiments may also include a combination of the subject matter of the dependent claim with each other dependent claim. Unless it is stated that a specific combination is not intended to be used, such combinations are proposed herein. In addition, it is intended to also include features of a certain claim into any other independent claim, even if the claim is not directly dependent on the independent claim.

[0099] It should also be noted that the methods disclosed in the specification or claims may be implemented by a device having means for performing each of the respective steps.

[0100] Reference numerals

[0101] 100 Methods

[0102] 110 Get object-specific information

[0103] 120 Providing subject-specific information to third parties

[0104] 210 Vehicles

[0105] 212 dotted arrow

[0106] 220 objects

[0107] 230 Third Party / Infrastructure Providers

[0108] 240 External Server / Backend

[0109] 242 dotted arrow

[0110] 310 Infrastructure Providers

[0111] 320 Machine Learning Engineering Office

[0112] 330 Fleet Connected Driving Software Manager / Backend

[0113] 400 Devices

[0114] 410 One or more interfaces

[0115] 420 Data Processing Circuit

Claims

1. A method (100) for monitoring an environment of a vehicle (210), the method comprising: obtaining (110) object-specific information about an object (220) associated with a third party (230) using at least one sensor of the vehicle (210) and monitoring criteria; The object-specific information is provided (120) to a third party (230).

2. The method (100) of claim 1, wherein obtaining (110) the object-specific information comprises: Monitoring the environment of the vehicle (210) to obtain sensor data of the environment; as well as Based on the monitoring criteria, the object-specific information is obtained from sensor data of the environment.

3. The method (100) according to claim 2, wherein the object-specific information obtained comprises an object-specific part of sensor data of the environment.

4. The method (100) according to claim 2 or 3, wherein the object-specific information comprises meta-information about the object (220) from sensor data.

5. A method (100) according to claim 4, wherein obtaining (110) the object-specific information includes obtaining the meta-information from sensor data using an aggregator and a privacy assessor program, wherein the privacy assessor program is used to check the aggregator by comparing the aggregator with at least one reference software architecture and / or based on training data used to train the aggregator.

6. The method (100) of any one of the preceding claims, wherein the monitoring criteria indicates a predefined location at which the object (220) is located, and wherein obtaining (110) the object-specific information comprises: determining whether the object-specific information represents the predefined location; as well as If the object-specific information represents the predefined location, the object-specific information is obtained.

7. The method (100) according to any of the preceding claims, wherein the monitoring criteria indicate one or more predefined scenarios for the environment, and wherein obtaining (110) the object-specific information comprises: Determining whether there is at least one of the predefined scenarios in the environment; as well as If at least one of the predefined scenarios exists, the object-specific information is obtained.

8. A method (100) according to claim 7, wherein determining whether at least one of the predefined scenarios exists includes determining whether the object-specific information indicates at least one of the predefined scenarios, and wherein obtaining (110) the object-specific information based on the monitoring criteria includes obtaining the object-specific information if the object-specific information indicates at least one of the predefined scenarios.

9. The method (100) according to claim 7 or 8, wherein the method (100) further comprises: determining information about false positive detections and / or detection frequencies of at least one scene from the object-specific information; as well as A fault is determined based on information regarding false positive detections and / or detection frequencies of at least one scenario.

10. The method (100) according to any one of the preceding claims, wherein the environment comprises at least one infrastructure object (220), and wherein the object-specific information indicates the at least one infrastructure object (220).

11. The method (100) of claim 10, wherein the monitoring criteria indicates at least one predefined infrastructure object (220), and wherein obtaining (110) the object-specific information comprises: determining whether the object-specific information represents the at least one predefined infrastructure object (220); as well as If the object-specific information represents the at least one predefined infrastructure object (220), the object-specific information is obtained.

12. The method (100) according to any of the preceding claims, wherein obtaining (110) the object-specific information comprises clearing a portion of the object-specific information based on the at least one monitoring criterion.

13. A computer program having a program code for carrying out at least one of the methods (100) according to the preceding claim when the computer program is executed on a computer, a processor or a programmable hardware component.

14. An apparatus (400), comprising: one or more interfaces (410) for communication; and A data processing circuit (420), wherein the data processing circuit (420) is configured to: controlling the one or more interfaces (410); and One of the methods (100) according to claims 1 to 12 is performed using the one or more interfaces (410).

15. A vehicle (210) comprising the device according to claim 14.

Images