Evaluating measurement data using an abstraction layer

Abstract

In order to create a method (20) for controlling at least one actuator (14) of a vehicle (100), which is designed, for example, as an automated or semi-automated vehicle (100), and which can reduce the effort involved in subsequently replacing sensors (11), especially in vehicles (100) with higher levels of automation, it is proposed to receive measurement data from at least one sensor (11) from an intermediate unit (12) in the form of an abstraction layer, wherein the abstraction layer (12) evaluates the received measurement data to create a standardized overview of a traffic situation and transmits it to a downstream processing unit (13), and the processing unit (13) analyzes the standardized overview of the traffic situation.wherein, based on an analysis of the standardized overall view of the traffic situation, control commands are generated to control at least one actuator (14) of the vehicle (100).

Description

[0001] The invention relates to a method for controlling at least one actuator of a vehicle, which is configured, for example, as an automated or semi-automated vehicle. Furthermore, the invention relates to a sensor system and a vehicle with a sensor system.

[0002] Depending on their classification, vehicles can be designed with different levels of automation to implement different automation functions. Typically, SAE level or BASt level are used to represent the degree of vehicle automation. For example, a traffic jam assistant or an emergency braking assistant can be implemented in an SAE Level 2 vehicle. An SAE Level 5 vehicle, on the other hand, can be operated completely driverless. The operation and implementation of such automation functions requires on-board sensors that are connected to corresponding control units via a data-conducting network. Especially in vehicles with higher levels of automation, the sensor data is evaluated using complex algorithms or AI.

[0003] Repaired or replaced sensors must be retrained regularly. Depending on the design, the entire software or corresponding artificial intelligence can be retrained and certified. This method of replacing defective sensors creates considerable effort during the development and release phase and also increases the computing load for the artificial intelligence, as it has to laboriously convert raw sensor data into objects, etc.

[0004] US 2022 / 0215675 A1 describes a lane keeping system for vehicles in which measurement data from at least one camera sensor is received and evaluated.

[0005] From DE 10 2021 100 526 A1 a method for operating a radar sensor and a method for providing an artificial intelligence evaluation algorithm for such an operating method are known.

[0006] The invention is based on the object of creating a method and a sensor arrangement that can reduce the effort involved in subsequently replacing sensors, particularly in vehicles with higher levels of automation.

[0007] This object is achieved by the features specified in claim 1. Further advantageous embodiments of the invention are described in the subclaims.

[0008] According to one aspect of the invention, a method is provided for controlling at least one actuator of a vehicle, which is designed, for example, as an automated or semi-automatic vehicle.

[0009] In a first step, measurement data from at least one sensor is acquired and received by an intermediate unit in the form of an abstraction layer. By evaluating the received measurement data, the abstraction layer creates a standardized overall view of the traffic situation and transmits it to a downstream processing unit. The standardized overall view of the traffic situation is then analyzed by the processing unit. Based on the analysis of the standardized overall view of the traffic situation, the processing unit generates control commands to activate at least one actuator of the vehicle.

[0010] Using the generated control commands, a vehicle's actuators can be controlled to implement longitudinal and / or lateral control. Due to the interposition of the abstraction layer, the evaluation of the measurement data for direct control of the actuators takes place independently of the measurement data from the sensors. This step can essentially be handled by the abstraction layer, meaning that replacing the sensors has no direct impact on the processing unit. This at least avoids having to modify or adapt the processing unit when replacing sensors.

[0011] According to a further aspect of the invention, a sensor system is provided which is configured to carry out the method according to the invention. The sensor system comprises at least one sensor and an intermediate unit in the form of an abstraction layer. The abstraction layer is coupled to the at least one sensor in a data-conducting manner. Furthermore, at least one processing unit is provided, which is connected to the intermediate unit or the abstraction layer.

[0012] The abstraction layer can receive and process the sensor measurement data in digital and / or analog form. Depending on the design, the abstraction layer can receive and process the unprocessed signals, for example, in the form of voltage values, current values, or electrical resistances. To do this, the abstraction layer can transform the measurement data into a digital form in a single step. Depending on the design of the sensor system, digital measurement data, for example, in the form of point clouds, digital image information, and the like, can also be received and processed.

[0013] Advantageously, the abstraction layer can be designed for a variety of different types and forms of measurement data. This allows the abstraction layer to check in a single step what type of measurement data is being received. The measurement data can then be transformed or directly processed to provide a holistic and sensor-independent view of a traffic situation.

[0014] The at least one sensor can be configured, for example, as a camera sensor, LIDAR sensor, radar sensor, ultrasonic sensor, laser knife, electrical contact, electrical resistance and the like.

[0015] According to a further aspect of the invention, a vehicle is provided. The vehicle has a sensor system according to the invention and at least one actuator. A processing unit of the sensor system is directly or indirectly connected to the actuator and is configured to control and / or regulate the at least one actuator using control commands.

[0016] The method and sensor system according to the invention allow the abstraction layer to take over object recognition and the immediate evaluation of the measurement data or raw data from the sensors instead of the processing unit. In one possible definition, measurement data may have already undergone internal sensor processing, while the raw data from the sensors may not have undergone any internal sensor processing.

[0017] The information obtained during the processing of the measurement data or raw data can then be transmitted to the processing unit and, for example, to an AI of the processing unit via a standardized interface. This measure also reduces the processing unit's computing power, allowing the processing unit's computing capacity to be used for more essential algorithms or tasks, such as braking maneuvers, evasive maneuvers, and the like.

[0018] The abstraction layer can be implemented particularly easily in the processing of the measurement data if the measurement data from at least one sensor is received and evaluated by an abstraction layer in the form of middleware. The intermediate unit or abstraction layer, designed as so-called middleware, can represent a platform in a complex software system that, as a "service provider," enables data exchange with other otherwise decoupled software components, such as the processing unit.

[0019] The processing of sensor data can be carried out, in particular, within the framework of a so-called ISO / OSI reference model for network protocols as a layered architecture. This allows defined network protocols with clear interfaces to be easily exchanged within the same layers.

[0020] According to a further embodiment, the abstraction layer determines a standardized overall view of a traffic situation in the form of a third-person perspective of the vehicle's sensor-detected surroundings. Thus, the abstraction layer can detect all relevant objects in the field of view and make them available to the processing unit via a standardized overall view. This occurs independently of the sensors or sensor type used. For example, an object with various attributes, such as position, size, motion vector, probability of being present after a certain period of time, and the like, can be determined from a cluster of pixels. This information can be used directly by the processing unit to control actuators.

[0021] The at least one processing unit can optimally assess a traffic situation if the abstraction layer determines a standardized overall view of the traffic situation in the form of a 360° perspective of the vehicle's surroundings as recorded by sensors. This measure enables the abstraction layer to aggregate measurement data from all sensors, regardless of their installation position, detection direction, or detection range.

[0022] According to a further embodiment, the abstraction layer determines a standardized overall view of a traffic situation in the form of a bird's-eye view of the vehicle's surroundings as detected by sensors. This visualizes all detected static and / or dynamic objects in the vehicle's surroundings together with the vehicle in an overall view of the traffic situation and / or represents them with data in such a way that the processing unit can utilize the overall view directly and with the least possible effort.

[0023] Similar to a 360° perspective, a traffic situation can be presented particularly clearly if the abstraction layer determines a standardized overall view of the traffic situation in the form of a top-down view of the vehicle and the vehicle's surroundings as recorded by sensors. This allows the vehicle, or so-called ego-vehicle, to be created as an overall view together with the objects recorded relative to the vehicle and made available to the processing unit.

[0024] According to a further embodiment, object recognition is performed by the abstraction layer based on the received measurement data. This measure allows the processing unit to be relieved of the burden by the intermediate unit. The intermediate unit takes over direct communication with the sensors and can thus decouple the sensors from the processing unit.

[0025] The objects identified by the intermediate unit can be used particularly optimally by the processing unit for evaluation and / or assessment if, by standardizing the overall view of a traffic situation, all objects identified by object recognition are projected or located on a road surface of the vehicle. The projection of the objects and the vehicle relative to the objects onto the road surface enables the creation of an overview of the traffic situation regardless of the viewing angle, resolution, and refresh rate. For example, the overall view can be represented visually and / or through data from an areal view or a view from above of the vehicle or from a centered perspective in the direction of travel behind the vehicle. This means that an AI (artificial intelligence), for example, can always receive a standardized image of the traffic situation.

[0026] If, for example, a vehicle sensor is replaced with another, repaired or new, sensor, this adjustment to the vehicle does not result in any change in the overall view, meaning the processing unit does not have to consider the different characteristics of the sensors. Therefore, the processing unit's AI does not need to be retrained.

[0027] When replacing a sensor, for example, a camera sensor with a higher or lower resolution, a LIDAR sensor or radar sensor with a different sampling frequency, a sensor with an enlarged or reduced detection range or angle, and the like can be installed.

[0028] According to a further embodiment of a vehicle, the measurement data received by the abstraction layer is evaluated or processed independently of the number and type of sensors. For this purpose, the intermediate unit can have means for receiving and processing a plurality of different types of measurement data and / or raw data. As a result, the respective measurement data from the sensors with different coordinate systems and reference points are subjected not only to object recognition but also to standardization with respect to the coordinate systems. For example, a vehicle-side point or a vehicle-side axis can be used as an origin for the standardized results of the abstraction layer's evaluation.

[0029] Depending on the design, when a sensor is replaced, the corresponding interfaces or drivers of the intermediate unit can be adapted or updated to enable error-free processing.

[0030] The evaluation of the overall view of the traffic situation created by the intermediate unit can be used particularly efficiently if, based on this overall view, the processing unit generates control commands to control at least one actuator for longitudinal and / or lateral guidance of the vehicle. Depending on the configuration, the overall view of the traffic situation created can run in real time and thus be updated at a predefined or variable frequency to achieve automated and / or semi-automated vehicle control.

[0031] According to a further embodiment, the at least one intermediate unit and the at least one processing unit are designed as hardware modules or software modules and integrated into a common control unit. This measure allows the intermediate unit and the processing unit to be arranged centrally within a computer component. This enables a space-saving arrangement of the intermediate unit and the processing unit in the vehicle.

[0032] The intermediate unit and the processing unit can be optimized for maintenance and repair if the intermediate unit is configured as a first control unit and the at least one processing unit is configured as a second control unit, with the first control unit being connected in a data-conducting manner between the at least one sensor and the second control unit. This allows the intermediate unit and the processing unit to be housed in separate components or configured as separate components. If the intermediate unit requires modification, it can also be replaced or temporarily disassembled for updates.

[0033] According to another embodiment of a vehicle, the vehicle is configured as an automated or semi-automated vehicle, wherein at least one control unit is configured to implement longitudinal and / or lateral guidance of the vehicle based on generated control commands. This allows the processing unit to analyze the traffic situation based on the standardized overall view of the traffic situation created by the intermediate unit and control the vehicle accordingly.

[0034] Exemplary embodiments of the invention are explained in more detail below with reference to the drawings. They show: Fig. 1 a schematic representation of a vehicle according to the invention with a sensor system according to an embodiment, and Fig. 2 a schematic flow diagram illustrating a method according to the invention according to an embodiment.

[0035] In the figures, the same structural elements have the same reference numerals.

[0036] Fig. 1 shows a schematic representation of a vehicle 100 according to the invention with a sensor system 10 according to an embodiment. Fig. 1 shows, in particular, a side view of the vehicle 100. In the direction of travel X, in front of the vehicle 100, there is, for example, an obstacle 200 in the form of a red deer, which is detected by sensors. The vehicle 100 and the sensor system 10 are described in order to illustrate a method according to the invention, which Fig. 2 is visualized in a flow chart.

[0037] The sensor system 10 is configured to carry out the method according to the invention. The sensor system 10 comprises at least one sensor 11 and an intermediate unit 12 in the form of an abstraction layer. The abstraction layer 12 is data-conductingly coupled to the at least one sensor 11. Furthermore, at least one processing unit 13 is provided, which is connected to the intermediate unit 12 or the abstraction layer.

[0038] The at least one sensor 11 can be configured, for example, as a camera sensor, an NIR and / or FIR camera, a radar sensor, a LIDAR sensor, an ultrasonic sensor, a laser rangefinder, and the like. Advantageously, multiple sensors 11 with different installation positions and scanning directions can be provided to scan a vehicle's surroundings U.

[0039] The abstraction layer 12 can receive and process the measurement data from the sensors 11 in digital and / or analog form. Depending on the configuration, the abstraction layer 11 can receive and process the unprocessed signals, for example in the form of voltage values, current values, or electrical resistances. For this purpose, the abstraction layer 12 can transform the measurement data into a digital form in one step. Depending on the configuration of the sensor system 10, digital measurement data, for example in the form of point clouds, digital image information, and the like, can also be received and processed. Thus, for example, an object 200 in the form of a red deer with various attributes, such as position, size, motion vector, probability of presence in xx ms, etc., can be generated by the abstraction layer 12 from a cluster of pixels or a point cloud.

[0040] Advantageously, the abstraction layer 12 can be designed for a variety of different types and forms of measurement data. Thus, the abstraction layer 12 can check in a single step what type of measurement data is being received. Subsequently, the measurement data can be transformed or directly processed to depict a holistic and sensor-independent view of a traffic situation.

[0041] In the illustrated embodiment, the overall view of a traffic situation is determined and displayed as a top view or 360° perspective with the vehicle 100 and in the form of a bird's eye view of a sensor-detected environment U of the vehicle 100. In this overall view, the determined object 210 is also depicted in a standardized form.

[0042] In standardized form, for example, means independent of a sensor type or sensor property. Furthermore, in the standardized representation, all objects 210 and the vehicle 100 can be projected onto a road plane F of the vehicle's surroundings U in order to reduce the number of possible dimensions when processing the overall view by the processing unit 13.

[0043] Thus, all determined static and / or dynamic objects 210 in the vehicle environment U are visualized together with the vehicle 100 in an overall view of the traffic situation and / or represented by data in such a way that the processing unit 13 can utilize the overall view directly and with the minimum possible effort.

[0044] Based on this overall view, the processing unit 13 generates control commands for controlling at least one actuator 14 for longitudinal and / or lateral guidance of the vehicle 100. The control commands can be generated directly by the processing unit 13 or with the aid of an interposed vehicle control system 15. The at least one actuator 14 can, for example, accelerate, brake, steer, and / or adjust the vehicle's damping effect.

[0045] In the illustrated embodiment, the vehicle 100 comprises the sensor system 10 and at least one actuator 14. The processing unit 13 of the sensor system 10 is connected to the actuator 14 directly or indirectly via a vehicle controller 15 and is configured to control and / or regulate the at least one actuator 14 using control commands.

[0046] In the Fig. 2 shows a schematic flow diagram illustrating a method 20 according to the invention according to one embodiment. The method 20 serves to control at least one actuator 14 of a vehicle 100, which is configured, for example, as an automated or semi-automated vehicle 100. The Fig. 1 is also used to illustrate procedure 20.

[0047] In a step 21, measurement data from at least one sensor 11 is determined and received by an intermediate unit 12 in the form of an abstraction layer. Without receiving measurement data, no action is taken by the abstraction layer 12 24.

[0048] The abstraction layer's evaluation of the received measurement data results in a standardized overall view 22 of a traffic situation, which is transmitted to a downstream processing unit 13. This also checks whether the received measurement data can be standardized. If standardization of the measurement data is not possible, an error message 23 can be output.

[0049] The standardized overall view of the traffic situation is then received and analyzed 25 by processing unit 13. If action is required or an intervention in the vehicle control system 15 is necessary, control commands for controlling 26 at least one actuator 14 of vehicle 100 are generated based on the analysis of the standardized overall view of the traffic situation by processing unit 13. Alternatively, no action 24 is performed, and the traffic situation continues to be monitored.

[0050] Using the generated control commands 26, actuators 14 of vehicle 100 can be controlled to implement longitudinal and / or lateral control. Due to the interposition of abstraction layer 12, the evaluation of the measurement data for the direct control of actuators 14 takes place independently of the receipt of measurement data from sensors 11. This step can essentially be performed by abstraction layer 12, meaning that replacing sensors 11 has no direct impact on processing unit 14. This at least avoids having to modify or adapt processing unit 14 when sensors are replaced. Thus, complex AI can be used in processing unit 14, which does not need to be retrained each time a sensor is replaced. List of reference symbols 100 motor vehicles 200 sensory detected object 210 standardized object / object reduced to road level 10 Sensor system 11 sensors 12 Intermediate unit / abstraction layer 13 Processing unit 14 actuators 15 Vehicle control 20 procedures 21 Receiving measurement data 22 Creating a standardized overall view of a traffic situation 23 Error output 24 no action 25 Receiving and analyzing a standardized overview 26 Controlling actuators F Road level U Vehicle environment X Direction of travel / longitudinal direction QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 2022 / 0215675 A1

[0004] DE 10 2021 100 526 A1

[0005]

Claims

[1] Method (20) for controlling at least one actuator (14) of a vehicle (100), which is designed, for example, as an automated or semi-automatic vehicle (100), wherein measurement data from at least one sensor (11) are determined and received by an intermediate unit (12) in the form of an abstraction layer, wherein by evaluating the received measurement data by the abstraction layer (12) a standardized overall view of a traffic situation is created and transmitted to a downstream processing unit (13), wherein the standardized overall view of the traffic situation is analyzed by the processing unit (13) and, based on an analysis of the standardized overall view of the traffic situation, control commands for controlling at least one actuator (14) of the vehicle (100) are generated. [2] Method according to claim 1, wherein the measurement data of the at least one sensor (11) are received and evaluated by an abstraction layer (12) in the form of a middleware. [3] Method according to claim 1 or 2, wherein the abstraction layer (12) determines a standardized overall view of a traffic situation in the form of a third-person perspective of a sensor-detected environment (U) of the vehicle (100). [4] Method according to one of claims 1 to 3, wherein the abstraction layer (12) determines a standardized overall view of a traffic situation in the form of a 360° perspective of a sensor-detected environment (U) of the vehicle (100). [5] Method according to one of claims 1 to 4, wherein the abstraction layer (12) determines a standardized overall view of a traffic situation in the form of a bird's eye view of a sensor-detected environment (U) of the vehicle (100). [6] Method according to one of claims 1 to 5, wherein the abstraction layer (12) determines a standardized overall view of a traffic situation in the form of a plan view of the vehicle (100) and a sensor-detected environment (U) of the vehicle (100). [7] Method according to one of claims 1 to 6, wherein object recognition is carried out by the abstraction layer (12) based on the received measurement data. [8] Method according to claim 7, wherein by normalizing the overall view of a traffic situation, all objects (210) determined by the object recognition are projected and / or reduced into a roadway plane (F) of the vehicle (100). [9] Method according to one of claims 1 to 8, wherein the measurement data received by the abstraction layer (12) are evaluated or processed independently of a number and type of sensors (11). [10] Method according to one of claims 1 to 9, wherein control commands for controlling at least one actuator (14) for longitudinal guidance and / or transverse guidance of the vehicle (100) are generated by the processing unit (13). [11] Sensor system (10) which is configured to carry out the method (20) according to one of the preceding claims, comprising at least one sensor (11), comprising an intermediate unit (12) in the form of an abstraction layer which is coupled to the at least one sensor (11) and at least one processing unit (13) which is connected to the intermediate unit (12). [12] Sensor system according to claim 11, wherein the at least one intermediate unit (12) and the at least one processing unit (13) are designed in the form of hardware modules or software modules and are integrated in a common control unit. [13] Sensor system according to claim 11, wherein the intermediate unit (12) is designed as a first control unit and the at least one processing unit (13) is designed as a second control unit, wherein the first control unit is connected in a data-conducting manner between the at least one sensor (11) and the second control unit. [14] Vehicle (100) comprising a sensor system (10) according to one of claims 11 to 13 and at least one actuator (14), wherein a processing unit (13) of the sensor system (10) is connected directly or indirectly to the actuator (14) and is configured to control and / or regulate the at least one actuator (14) by means of control commands. [15] Vehicle according to claim 14, wherein the vehicle (100) is designed as an automated or semi-automated vehicle, wherein at least one control unit is configured to implement a longitudinal guidance and / or transverse guidance of the vehicle based on generated control commands.

Images