Method for operating an assistance system of a mobile unit, assistance system of a mobile unit and mobile unit with an assistance system

Abstract

Method for operating an assistance system (10) of a mobile unit (20), wherein the mobile unit performs the following steps: a) Acquiring metadata (41, 61) of an external sensing device (30, 32) to the mobile unit (20), which is associated with an external sensor data set (45) of the external sensing device (30, 32), wherein the metadata (41, 61) describe properties about features of the external sensor data set (45) and / or the external sensing device (30, 32), b) Checking the metadata (41, 61) for at least one relevance criterion (70) for an operating mode (16) of the mobile unit (20), which specifies properties or characteristics of the mobile unit in relation to a current and / or future routing or route that the mobile unit is currently following, and c) only if the metadata (41, 61) exhibits at least one relevance criterion (70): • Requesting and capturing the external sensor data set (45), • Use of at least the external sensor data set (45) to operate the assistance system (10).

Description

[0001] The invention relates to a method for operating an assistance system of a mobile unit. The invention also relates to an assistance system of the mobile unit, as well as a mobile unit with the assistance system. The invention focuses in particular on using external sensor data to operate the assistance system.

[0002] A mobile unit is understood to be a movable or non-stationary device or apparatus, such as a motor vehicle. An assistance system of the mobile unit is understood to be a system, such as a driver assistance system, by means of which an assistance function, for example, a driver assistance function, can be performed for a user of the mobile unit. Thus, the assistance system supports the user in operating or using the mobile unit. The assistance system intervenes independently in an operation or operating mode of the mobile unit without user input. Accordingly, the aforementioned driver assistance function of the driver assistance system can be, for example, steering intervention, braking intervention, or acceleration intervention.This intervention can prevent accidents and / or unstable driving dynamics of the vehicle, for example, if the driver's reaction time is no longer sufficient.

[0003] An example of a driver assistance system is an emergency braking assistant. This system automatically initiates emergency braking without driver intervention if it detects an imminent collision with another road user, such as a vehicle or pedestrian. Another example of a driver assistance system is an electronic stability program, which counteracts vehicle skidding by selectively braking individual wheels. A further example of a driver assistance system is a system that enables piloted or automated driving, i.e., autonomous driving.

[0004] To perform its assistance function, an assistance system uses sensor data or a sensor dataset to decide on a necessary reaction or intervention in the operation of the mobile unit. The sensor data could, for example, be an environmental image or video of the surroundings, captured or recorded by a sensor device on the mobile unit. The sensor device could be, for example, a camera sensor or radar sensor. The sensor device is specifically an internal sensor device, meaning at least one sensor assigned to the mobile unit and, in particular, a component of the mobile unit. The internal sensor device can thus provide internal sensor data or an internal sensor dataset for operating the assistance system.

[0005] However, the detection range or field of view of the internal sensor is often limited. For example, an optical obstacle might be present in the detection range, preventing the internal sensor from fully capturing the environment. In such cases, it can be advantageous to use sensor data from an external sensor to capture the environment. The external sensor can thus provide external sensor data or a complete sensor dataset to the mobile unit. The term "external" in this context describes a sensor outside the mobile unit, meaning it is not part of the mobile unit. To clarify the difference between the internal and external sensor, the external sensor will also be referred to as the external detection device.The external sensing device can capture the environment from a different perspective than the internal sensor device, ensuring that the field of view or detection range of the external sensing device remains unaffected by the optical obstacle. This allows the external sensing device to capture an extended or enlarged area of ​​the environment, which can then be used to operate the assistance system.

[0006] The external detection device can be, for example, a stationary or mobile device, such as road traffic infrastructure, for example a surveillance camera at a traffic light or building, a sensor device of another motor vehicle, or smart glasses with a sensor device.

[0007] In connection with the acquisition of external sensor data, for example, DE 10 2008 042 565 A1 discloses a method for operating a driver assistance system in a vehicle, in which the driver assistance system receives external sensor data about a traffic situation from at least one external device, and this external sensor data is used for the operation of the driver assistance system. By taking the external sensor data into account, a comprehensive representation of the traffic situation can thus be provided, and the vehicle's field of vision is extended.

[0008] Furthermore, EP 3 404 638 A1 discloses a vehicle system which has a detection unit that captures information about a blind spot of an infrastructure.

[0009] Furthermore, US patent 7 102 496 B1 discloses a sensor system for a motor vehicle that captures sensor data from various other road users, such as a motor vehicle or a stationary highway sensor, and uses this data to avoid collisions.

[0010] One disadvantage of using external sensor data for an assistance system is that receiving and processing this data, in addition to the internal sensor data, increases the computational load for the system. With increasing connectivity, especially in cities, the number of potential sensor data providers for a mobile unit is constantly growing. This further increases the computational effort required to process the external sensor data and thus to capture the expanded surroundings. Therefore, either expensive and powerful processors are needed, or not all available external sensor data can be utilized.

[0011] Finally, DE 10 2008 037 883 A1 discloses a traffic light system for generating metadata based on input information about the traffic light system's surroundings. The traffic light system acts as a data pool and can independently recreate a road segment using information it measures or receives from neighboring vehicles. From the input information, the traffic light system can also generate metadata that is communicated to other vehicles, thus ensuring a constant traffic flow.

[0012] The object of the present invention is to further develop an assistance system in such a way as to increase road safety.

[0013] The problem is solved by the independent patent claims. Advantageous embodiments of the invention are disclosed in the dependent patent claims, the description, and the figures.

[0014] The invention is based on the understanding that, due to the abundance of external sensing devices for the mobile unit, it is often complex to determine which of the external sensor data provided by the respective sensing device are relevant for the current operating mode of the mobile unit. The operating mode, or mode of travel, indicates properties or characteristics of the mobile unit with regard to a current and / or future route or the route the mobile unit is currently following. The current operating mode of the mobile unit can be determined, for example, using a navigation data set from a navigation device of the mobile unit. In the case of a motor vehicle as the mobile unit, the current operating mode can also be determined by whether a turn signal or indicator is activated or deactivated.

[0015] To assess the relevance of external sensor data depending on the operating mode, the mobile unit, and especially its assistance system, would first have to evaluate all available external sensor data, also known as the external sensor data set. This would mean that a large portion of the computing power of the computer, i.e., the assistance system's processor, would be used for evaluation instead of for executing assistance functions. This could lead to a reduction in the assistance system's reaction time and consequently a delay in executing the assistance function, thus increasing the risk of accidents.

[0016] To enhance the traffic safety of the assistance system, the invention provides that the mobile unit first acquires and processes metadata from the respective external sensing device to verify whether the external sensor data set possesses properties, such as a field of view, that are relevant to the operating mode of the mobile unit. The metadata is associated with the external sensor data set provided by the external sensing device. It describes information or properties about features of the external sensor data set and the external sensing device.The metadata can include, for example, the following criteria or information: the time the external sensor captured the environment, the position of the external sensor at the time of capture, a direction of view or perspective, a cone of view or field of view, a zoom level, lens information, and, if applicable, a cost for transferring the external sensor data set to the mobile unit. The amount of data or file size of the metadata is therefore much smaller than the amount of data or file size of the external sensor data set. Consequently, the computational effort required to process the metadata is also much lower than the computational effort required to process the external sensor data set.

[0017] After the metadata is captured, it is checked for its relevance to the operating mode of the mobile unit. This means that a relevance criterion for the metadata is determined. This relevance criterion is preferably selected from the aforementioned metadata criteria. For example, it can be checked whether the position and / or the field of view of the external detection device correlate with the route of the mobile unit. That is, it can be checked whether the position and / or field of view lie on a current route or movement trajectory of the mobile unit. If the position and / or field of view correlate with the route, the respective criterion can be classified as a relevance criterion. It can thus be confirmed that the metadata has at least one relevance criterion for the operating mode.

[0018] In this case, the external sensor data set can thus be classified as relevant for expanding the mobile unit's detection range. Only then is the external sensor data set requested. This can involve a one-time or repeated request or transmission of an external sensor data set from the external detection device. "Repeated" in this context means that the external detection device transmits an external sensor data set—that is, a current image or video of the environment—at predetermined intervals until a communication link to the mobile unit is lost or terminated. Subsequently, the respective external sensor data set is evaluated, and an environmental situation is recognized based on this evaluation, which ultimately triggers the execution of the assistance system's function.

[0019] If the mobile unit is designed as a motor vehicle, for example, the aforementioned emergency braking assistant can be activated when it receives an image from an external traffic monitoring camera showing a person intending to cross the road and who is outside the detection range of the vehicle's internal or vehicle-integrated sensor system. As an assistance function, a braking system can be put into an activation mode, such as positioning the brake pedal just before it reaches its maximum braking force. This reduces the time the emergency braking assistant needs to fully depress the brake pedal if the person actually moves onto the road.

[0020] Alternatively, the mobile unit can also be configured as an aircraft, for example a drone, a mobile device, or smart glasses, such as augmented reality or virtual reality glasses. The assistance system can accordingly be configured, for example, as an automatic flight assistant, a navigation assistance system, an information system, or something similar.

[0021] In contrast, the external sensor data set can be classified as irrelevant if, for example, the position and / or field of view are outside or deviate from the current route. The relevance criterion for the metadata can therefore be negated. The external sensor data set from the external detection device is then not requested.

[0022] The invention offers the advantage that a more cost-effective processor can be used to evaluate the metadata and then request only the relevant external sensor data sets. This allows all relevant available external sensor data sets to be utilized without impairing the response time of the assistance system or compromising the road safety of the mobile unit. For example, it prevents a sensor data set from being acquired and processed by an external detection device located behind or to the side of the mobile unit in the direction of travel, even though the mobile unit is moving straight ahead.

[0023] The following summarizes the individual steps of the procedure for operating the mobile unit's assistance system. In step a), the metadata for the mobile unit's external sensing device, which is associated with the external sensor data set of the external sensing device, is first collected. In step b), the metadata is then checked against the relevance criterion for the mobile unit's operating mode. Subsequently, only if the metadata exhibits at least one relevance criterion for the operating mode, in step c), the external sensor data set is requested and collected, and at least the external sensor data set is used to operate the assistance system.

[0024] Preferably, the aforementioned process steps, in particular steps b) and c), are repeated periodically, i.e., at predetermined time intervals, and / or event-driven, i.e., depending on an operational event of the mobile unit. An operational event describes an intervention or a change in the current operating mode. For example, an operational event could be a steering intervention that causes the mobile unit to turn. This is because the operational event can change at least one relevance criterion for the operating mode, potentially rendering the metadata irrelevant for operating the assistance system.

[0025] To utilize at least the external sensor data set for the assistance system, the system preferably includes a computing unit or processor. This unit can process at least the external sensor data set and, depending on the results, generate an environmental model. This environmental model encompasses all objects, such as people, other road users, and traffic signs, that are within the field of view of the external detection device. The computing unit can then evaluate the environmental model to determine the environmental situation. This allows the computing unit to execute an object recognition algorithm. Depending on the environmental situation, the computing unit can then control an actuator, such as the braking system of the mobile unit, to perform the assistance function.

[0026] The following describes embodiments of the invention which offer additional advantages.

[0027] In one embodiment, it is provided that, before acquiring the external sensor data set, the metadata is checked to determine whether the assistance system possesses an internal sensor data set that already encompasses or covers the environmental area represented by the external sensor data set. That is, before step c) is executed, the metadata is checked for a further relevance criterion for the mobile unit's internal sensor data set.

[0028] The internal sensor data set may, for example, be a sensor data set that was acquired by the mobile unit's internal sensor device, or that is provided by an external acquisition device and stored in a storage device of the mobile unit.

[0029] To verify compliance with the further relevance criterion, the metadata is compared with known environmental data of the mobile unit. This environmental data is associated with the mobile unit's internal sensor dataset. The environmental data describes the properties of the internal sensor dataset and can therefore be understood as metadata of the internal sensor dataset.

[0030] If, for example, the comparison reveals that the mobile unit has an internal sensor data set covering a field of view that, according to the metadata, corresponds to the field of view of the external sensing device, the external sensor data set can be classified as irrelevant. However, if the field of view of the external sensing device is larger or lies outside the field of view of the mobile unit, the external sensor data can be classified as relevant. The field of view criterion of the metadata can thus be considered the additional relevance criterion, and it can be confirmed that the metadata exhibits at least one further relevance criterion. In this case, step c) is then executed.

[0031] Another embodiment provides that, before executing step c), the computing load of the assistance system is compared with a load threshold, whereby the procedure is only executed if the computing load is below the load threshold. This has the advantage that it is first checked whether the computing device of the mobile unit has sufficient processing capacity to evaluate an external sensor data set in addition to operating the assistance system using the internal sensor data set. Only if sufficient processing capacity is available, i.e., if the computing load is below the load threshold, are the metadata and the external sensor data set from the external detection device acquired at all. In other words, the mobile unit can be checked against a load criterion, for example, if the metadata exhibits at least one relevance criterion.The utilization criterion could be, for example, a predetermined computing capacity or a predetermined data transfer rate that is the minimum required to process the external sensor data set. Step c) is only executed if the utilization criterion is met, i.e., if sufficient computing capacity or a sufficient data transfer rate is available.

[0032] Advantageously, it is provided that, in addition to the computing load, the network load of a communication network or communication link between the assistance system and the external acquisition device for transmitting the metadata and / or the external sensor data set is also checked. For this purpose, before executing step b), the network load can be compared with a load threshold, whereby the procedure is only executed if the network load is below the load threshold.

[0033] In a further embodiment, the assistance system can be remotely controlled by an external computing device to perform the assistance function. For this purpose, steps a) to c) are executed by the external computing device. The method comprises the following additional steps. In step f), the external computing device first generates an operating command data set for the mobile unit based on the external sensor data set. The operating command data set is a remote control command, i.e., a control command for an actuator of the mobile unit. In step g), the operating command data set is then transmitted from the external computing device to the mobile unit.Upon receipt of the operating command data set by the mobile unit, the aforementioned actuator of the mobile unit is finally controlled in step h) with the operating command data set to execute the assistance function.

[0034] This offers the advantage that the evaluation of the external sensor data set can be outsourced from the mobile unit's processing unit to an external computing device, such as a server. This allows for a more cost-effective processing unit with less computing capacity to be installed in the mobile unit. Furthermore, the workload on the computing device can also be reduced.

[0035] In a further embodiment, it is provided that more than one sensor data set, i.e., at least two sensor data sets, are used to operate the assistance system. By combining several sensor data sets, a more precise and / or a wider environmental area can be used to operate the assistance system. For this purpose, step i) provides that at least one further sensor data set is acquired depending on its respective metadata according to steps a) to c). This at least one further sensor data set can, in particular, be an internal or an external sensor data set. To acquire the at least one further sensor data set, steps a) to c) can be carried out again.According to step j), the at least one further sensor data set and the external sensor data set are then combined and the at least two sensor data sets are used to operate the assistance system according to step c).

[0036] Preferably, in this case, it can be provided that, before requesting and acquiring the at least one further sensor data set according to step c), the respective metadata of the at least two sensor data sets are first compared against each other. That is, the respective metadata can be compared to determine the quality or reliability of the associated sensor data sets. For example, a second sensor data set can replace a first sensor data set that was requested earlier if, upon comparison of the metadata, it is determined that the second sensor data set has, for example, a larger field of view than the first sensor data set.

[0037] To ensure the trustworthiness of the sensor or acquisition device when using the at least two sensor data sets for the assistance system, a further embodiment provides in step k) that, before step j), each sensor data set is weighted according to information about the provider contained in the respective metadata. This provider information thus constitutes an additional criterion of the metadata. The provider information indicates the origin, i.e., the sender of the metadata. The at least two sensor data sets are then used to operate the assistance system, depending on their weighting.

[0038] This allows, for example, an internal sensor data set acquired by the mobile unit itself using its internal sensor equipment to be weighted more heavily than an external sensor data set acquired by an external acquisition device. Furthermore, an external sensor data set sent by a known or verified external acquisition device could also be weighted more heavily than an external sensor data set sent by an unknown or unverified external acquisition device. "Verified" or "known" in this context means that the external acquisition device sends its sensor data set to the mobile unit encrypted, for example, using asymmetric cryptography. For instance, sensor data could be exchanged between designated traffic infrastructures and the mobile unit in a signed or encrypted form.These sensor data sets would then, for example, have a high degree of trustworthiness, whereas sensor data sets from private data providers, such as a surveillance camera in a residential building or a camera in augmented reality glasses, would be viewed more skeptically, especially for protection against hackers.

[0039] Furthermore, in another embodiment, it is provided that in step I), before step j) is executed, the at least two sensor data sets are checked for a difference with respect to an object detected in the respective sensor data set. For this purpose, the respective sensor data sets are evaluated and then compared with each other. Subsequently, the respective sensor data set containing the at least one object is only used to operate the assistance system if this sensor data set, according to step k), has a weighting that exceeds a weighting threshold. The weighting threshold specifies how highly the respective sensor data set must be weighted to be considered trustworthy and used to operate the assistance system. Thus, for example, an internal sensor data set of the mobile unit could be compared with an external sensor data set, for example, from smart glasses.If the external sensor data set of the smart glasses detects an object, such as a person, at a location where the internal sensor data set indicates that no object is present, this object would not be used to operate the assistance system. In other words, all information or data relating to this object would be removed from the external sensor data set, and only then would the external sensor data set be used to operate the assistance system. This prevents, for example, a virtual object generated by the augmented reality glasses from influencing the assistance system.

[0040] In a further embodiment, the mobile unit not only acquires the metadata and / or sensor data set from the external acquisition device, but also proactively requests it. To this end, step m) provides that, before executing step a), the mobile unit sends a connection request to the external acquisition device. If the connection request is accepted, a communication connection is then established with the external acquisition device to transmit the respective metadata and / or sensor data. This communication connection can be, in particular, a wireless connection, such as a WLAN connection, a Bluetooth connection, a cellular connection, or similar.

[0041] The invention also relates to an assistance system for the mobile unit. The assistance system comprises a communication device, such as an antenna module, configured to acquire the metadata and, optionally, the external sensor data set from the external sensing device. Furthermore, the assistance system also comprises the aforementioned computing device, such as a processor or microprocessor, configured to check the metadata for at least one relevance criterion for the operating mode of the mobile unit. If the computing device confirms that the metadata exhibits at least one relevance criterion for the operating mode, the computing device is further configured to request the external sensor data set from the external sensing device. For this purpose, the computing device controls the communication device to acquire the external sensor data set.Furthermore, the computing device is designed to use at least the external sensor data set in the aforementioned manner to operate the assistance system.

[0042] The invention also relates to the mobile unit with the aforementioned assistance system. As mentioned above, the mobile unit can be designed, for example, as a motor vehicle or aircraft, such as a drone, or as smart glasses, i.e., augmented reality or virtual reality glasses, or as a mobile device, such as a smartphone. The motor vehicle is preferably designed as a car, in particular as a passenger car or truck, or as a bus or motorcycle.

[0043] The invention also includes further developments of the assistance system according to the invention, which have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the assistance system according to the invention are not described again here.

[0044] The invention also includes combinations of the features of the described embodiments.

[0045] The following are exemplary embodiments of the invention described. This is illustrated by: Fig. 1. A schematic representation of a motor vehicle as a mobile unit with an assistance system which uses metadata from an external recording device to perform an assistance function depending on an operating mode; and Fig. 2. A flowchart of the process steps that are performed to operate the assistance system.

[0046] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0047] In the figures, identical reference symbols denote functionally equivalent elements.

[0048] Fig. Figure 1 shows an exemplary situation in which an assistance system 10 of a mobile unit is operated depending on an external sensor data set 45 of a recording device 30 external to the mobile unit. Fig. Figure 1 shows the mobile unit as an example motor vehicle 20. The assistance system 10 is designed as a driver assistance system, in particular as a brake assistance system.

[0049] The function of a brake assist system is to brake the motor vehicle 20, depending on the environmental situation, without any input from the driver or user, and if necessary, to bring it to a complete stop. To recognize the environmental situation in which braking of the motor vehicle 20 is required, the assistance system 10 has an internal sensor device 13. Fig. In this case, the sensor device 13 is configured as a camera sensor. Alternatively, the sensor device 13 could also be configured as a radar sensor. The sensor device 13 can detect and record the environment of the motor vehicle 20 within a field of view 21 or detection range defined by the properties of the sensor device 13. The environment detected by the sensor device 13 is available to the assistance system 10 as an internal sensor data set 15.

[0050] In order to recognize an environmental situation from the internal sensor data set 15, the assistance system 10 also includes a computing unit 11. Fig. In this case, the computing unit 11 is configured as a central processing unit (CPU). Among other things, the computing unit 11 is configured to process and evaluate the internal sensor data set 15, and in doing so, for example, to generate an environmental model of the field of view 21 detected by the internal sensor unit 13. This environmental model identifies all objects located within the field of view 21 of the sensor unit 13. By evaluating the environmental model, the computing unit 11 can deduce the environmental situation that determines when the brake assist system is activated.

[0051] An example of such an environmental situation is a person 52 who, while a motor vehicle 20 is moving in the direction of travel, moves onto a roadway 50 in front of the motor vehicle 20. In this case, the computing unit 11 can thus recognize as an environmental situation that the person 52 is an obstacle for the motor vehicle 20 on the roadway 50 and can then control an actuator 14 to perform the assistance function. The actuator 14 is in Fig. 1 is designed as a braking system. By activating the actuator 14, the motor vehicle 20 can be brought to a standstill in good time before reaching person 52, thus avoiding a collision with person 52.

[0052] As in Fig. As shown in Figure 1, it is possible that person 52 is outside the field of view 21 of the sensor device 13. In Fig. In case 1, the person 52 is obscured for the sensor device 13 by a motor vehicle 51 parked at the edge of the roadway 50. The motor vehicle 51 thus represents an optical obstacle to the field of view 21 of the sensor device 13. As a result, the sensor device 13 cannot detect the pedestrian and the brake assist system may not be activated in time.

[0053] To avoid this, it is provided that at least one additional external sensor data set 45 is used to operate the assistance system. This allows the environmental area considered by the assistance system to be expanded or enlarged. The external sensor data set 45 is acquired by an external detection device 30 of the vehicle 20. The detection device 30 is in Fig. 1. This is exemplified by traffic infrastructure, such as a traffic monitoring camera. The external detection device 30 can capture the environment from a different perspective than the internal sensor device 13. Thus, as shown in Fig. Figure 1 shows a viewing area 31 of the external detection device 30 from a viewing area 21 of the internal sensor device 13.

[0054] As in Fig. As shown in Figure 1, person 52, who intends to cross lane 50 in the direction of travel towards motor vehicle 20, is within the field of view 31 of the external detection device 30. If the external sensor data set 45 is now transmitted from the external detection device 30 to motor vehicle 20 via a communication link 40, the evaluation of the external sensor data set 45 allows for early detection that person 52 intends to move onto lane 50. This enables the actuator 14, i.e., the braking system of motor vehicle 20, to be activated in time so that motor vehicle 20 comes to a stop before person 52 reaches it, thus preventing an accident.

[0055] Especially in highly networked cities, there is often more than one external detection device 30, 32, whose respective sensor data sets the assistance system 10 can use to expand its detected environmental area. Fig. Figure 1 shows, in addition to the external sensing device 30, another external sensing device 32, which is designed as augmented reality smart glasses. To capture the environment, the smart glasses have a camera sensor (not shown) with which an environmental image can be captured within the field of view 50. Analogous to the external sensing device 30, the camera sensor of the smart glasses also captures the environment from a different perspective than the internal sensor device 13, so that the field of view 50 of the smart glasses also differs from the field of view 21 of the internal sensor device 13.

[0056] However, a disadvantage of acquiring multiple external sensor data sets is that the processing load on the computing unit 11 increases due to the evaluation and processing of these external sensor data sets. If the processing load exceeds a certain limit, the assistance system 10 may only be able to react to the detected environmental situation with a time delay.

[0057] To avoid this, it is provided that metadata 41, 61 are first transmitted to the assistance system 10 of the motor vehicle 20 before the external sensor data set 45 is transmitted. The metadata 41, 61 describe the properties of the external detection device 30, 32. Thus, the metadata 41, 61 include, among other things, the following information or criteria: position information specifying the location of the external detection device 30, 32; field of view information specifying the respective field of view 31, 33; and provider information identifying the sender or provider of the metadata 41, 61.

[0058] Since the metadata 41, 61 are property information, the amount of data in the metadata 41, 61 is reduced compared to the amount of data in an external sensor dataset. Therefore, processing the metadata 41, 61 generally requires much less computing power or capacity than processing the external sensor dataset. To acquire the metadata 41, 61 and also the external sensor dataset 45, the assistance system 10 further includes a communication device 12. Fig. Figure 1 shows the communication device 12 as an antenna module. This communication device 12 can, for example, be controlled by the computer 11 to establish a communication connection 40, 60 with the respective external acquisition device 30, 32. The respective metadata 41, 61 can then be requested and acquired via the communication connection 40, 60.

[0059] After acquiring the metadata 41, 61, the computing unit 11 can check the respective metadata 41, 61 against at least one relevance criterion 70 for an operating mode 16 of the motor vehicle 20. The operating mode describes the properties that the motor vehicle 20 exhibits in its current driving operation. These include, among other things, position information, field of view information, and route information of the motor vehicle 20. The position information describes the current position of the motor vehicle 20, the field of view information describes the field of view 21 of the sensor device 13 in the environment, and the route information describes a current and / or future vehicle route or trajectory of the motor vehicle 20.

[0060] To check the metadata 41, 61 against at least one relevance criterion 70 for the operating mode 16, the information contained in the metadata 41, 62 can, for example, be compared with the properties described by the operating mode. For example, it could be checked whether the position and field of view 31, 33 of the respective external detection device 30, 32 lie on the current and / or future vehicle route.

[0061] As in Fig. As shown in Figure 1, the external detection device 30 is located in front of the vehicle 20 in the direction of travel, and the external detection device 32 is located to the side of the vehicle 20 in the direction of travel. By comparing the operating mode with the respective metadata 41, 61, it can therefore be determined that the position and field of view 31 of the external detection device 30 lie on the future route of the vehicle 20. Accordingly, both the position information and the field of view information of the metadata 41 can be classified as relevance criteria by the computing unit 11. At the same time, it can also be determined that the position and field of view 33 of the external detection device 32 lie outside the current and future route of the vehicle. Accordingly, neither the position information nor the field of view information of the metadata 61 is classified as relevance criteria by the computing unit 11.

[0062] Thus, the metadata 41 of the external detection device 30 exhibits at least one relevance criterion 70 for the operating mode. Consequently, the computing unit 11 can control the communication unit 12 to transmit the external sensor data set 45 from the external detection device 30 to the assistance system 10 via the communication link 40. Subsequently, the assistance system 10 can use the external sensor data in a known manner to expand its detected environmental area and thereby activate the braking system early to decelerate the vehicle 20.

[0063] In contrast, the metadata 61 of the external acquisition device 32 does not contain a relevance criterion 70 for the operating mode. Therefore, in this case, the external sensor data set of the external acquisition device 32 is not requested.

[0064] By checking the metadata 41, 61 beforehand, it is thus possible to avoid overloading the assistance system 10 with the evaluation of the external sensor data sets.

[0065] Fig.Section 3 summarizes the individual procedural steps of the procedure for operating the assistance system 10 of the motor vehicle 20. The procedure is started with a start step ST. Subsequently, in step S1, the metadata 41, 61 from the external detection device 30, 32 of the motor vehicle 20, which are assigned to an external sensor data set 45 of the external detection device 30, 32, are acquired. In step S2, the metadata 41, 61 are then checked for at least one relevance criterion 70 for the operating mode 16 of the motor vehicle. If it is determined that the metadata 41, 61 do not have a relevance criterion 70 for the operating mode 16, the procedure is terminated with a final step SE. The external sensor data set 45 acquired by the respective external detection device 30, 32 is thus discarded as irrelevant.If, however, it is determined that the metadata 41, 61 exhibits at least one relevance criterion 70 for the operating mode 16, the external sensor data set 45 of the external acquisition device 30, 32 is requested and acquired in step S3. Finally, in step S4, at least the external sensor data set 45 is used to operate the assistance system 10. This means, for example, that the aforementioned actuator of the assistance system 10 can be controlled by the computer 11 so that the desired assistance function is executed. The procedure is then terminated with a final step SE.

[0066] Overall, the examples show how the invention enables the reception and processing of external sensor data sets.

Claims

[1] Method for operating an assistance system (10) of a mobile unit (20), wherein the mobile unit carries out the following steps: a) capturing metadata (41, 61) of a capturing device (30, 32) external to the mobile unit (20) which is associated with an external sensor data set (45) of the external capturing device (30, 32), wherein the metadata (41, 61) describe properties about features of the external sensor data set (45) and / or the external capturing device (30, 32), b) checking the metadata (41, 61) for at least one relevance criterion (70) for an operating mode (16) of the mobile unit (20), which indicates properties or features of the mobile unit with respect to a current and / or future route guidance or route that the mobile unit is currently following, and c) only in the event that the metadata (41, 61) have at least one relevance criterion (70): • Requesting and recording the external sensor data set (45), • Using at least the external sensor data set (45) to operate the assistance system (10). [2] Method according to one of the preceding claims, comprising the following further step: d) before carrying out step c): checking the metadata (41, 61) for at least one further relevance criterion (70) for an internal sensor data set (15) of the mobile unit (20) by comparing the metadata (41, 61) with known environmental data of the mobile unit (20) which are associated with the internal sensor data set (15), wherein step c) is only carried out if the checking shows that the metadata (41, 61) have the at least one further relevance criterion (70). [3] Method according to one of the preceding claims, comprising the following further step: e) before executing step a): comparing a computing load of the assistance system (10) with a load limit, wherein the method is only executed if the computing load is below the load limit. [4] Method according to one of the preceding claims, wherein steps a) to c) are carried out by a computing device external to the mobile unit (20), wherein the use of at least the external sensor data set (45) for operating the assistance system (10) comprises the following steps: f) generating an operating command data set for the mobile unit (20) in dependence on the external sensor data set (45) by the external computing device, g) transmitting the operating command data set from the external computing device to the mobile unit (20), and h) upon receipt of the operating command data set by the mobile unit (20): controlling an actuator device (14) of the mobile unit (20) with the operating command data set to execute an assistance function of the assistance system (10). [5] Method according to one of the preceding claims, comprising the following further steps: i) capturing at least one further sensor data set (15) depending on its respective metadata (41, 61) according to steps a) to c), and j) combining the at least one further sensor data set (15) and the external sensor data set (45) to use the at least two sensor data sets (15, 45) to operate the assistance system (10) according to step c). [6] Method according to claim 5, comprising the following further step: k) before carrying out step j): weighting the respective sensor data set (15, 45) depending on provider information (41.3, 61.3) which is included in the respective metadata (41, 61), and using the at least two sensor data sets (15, 45) to operate the assistance system (10) depending on the weighting. [7] Method according to claim 6, comprising the following further step: l) before performing step j): • Checking the at least two sensor data sets (15, 45) for a difference with respect to an object detected in the respective sensor data set (15, 45), • Using the determined object of the respective sensor data set (15, 45) to operate the assistance system (10) only if this sensor data set (15, 45) has a weighting that is above a weighting limit value according to step k). [8] Method according to one of the preceding claims, comprising the following further steps: m) before performing step a): • Sending a connection request to the external detection device (30, 32), and • in the event that the connection request is confirmed: establishing a communication connection (40, 60) with the external detection device (30, 32) for transmitting the respective metadata (41, 61) and / or the respective external sensor data set (45). [9] Assistance system (10) of a mobile unit (20), comprising a communication device (12) which is designed to capture metadata (41, 61) of a capture device (30, 32) external to the mobile unit (20), which metadata is assigned to an external sensor data set (45) of the external capture device (30, 32), wherein the metadata (41, 61) describe properties about features of the external sensor data set (45) and / or the external capture device (30, 32), a computing device (11) which is designed to check the metadata (41, 61) for at least one relevance criterion (70) for an operating mode (16) of the mobile unit (20), which indicates properties or features of the mobile unit with regard to a current and / or future route guidance or route which the mobile unit is currently following, wherein in the event that the computing device confirms that the metadata (41,61) which has at least one relevance criterion (70) for the operating mode (16), the computing device (11) is further designed:, • to request the external sensor data set (45) from the external detection device (30, 32) and to control the communication device (12) for detecting the external sensor data set (45), and • to use at least the external sensor data set (45) to operate the assistance system (10). [10] Mobile unit (20) with an assistance system (10) according to claim 9, wherein the mobile unit (20) is designed as a motor vehicle or aircraft or data glasses or mobile terminal.

Images