Methods for evaluating vehicle operating information

Abstract

Method for evaluating vehicle operating information measured and stored by a vehicle during its operation, wherein a filter unit (4) filters out raw vehicle data from the vehicle operating information, wherein the vehicle operating information and the raw vehicle data include position data of the vehicle, wherein a fusion unit (6) adds environmental data from the surroundings of the position data to the raw vehicle data in order to generate simulation data (11), wherein the simulation data (11) is made accessible, in particular displayed, by means of a user interface (12, 13), characterized in that the addition of environmental data from the surroundings of the position data to the raw vehicle data in order to generate simulation data (11) and / or the making accessible, in particular display, of the simulation data (11) is carried out by means of a game engine, wherein the user interface (12,13) a virtual reality user interface (12) such that the simulation data (11) are displayed in an interactive virtual environment, wherein the display and operation of the system is carried out by means of a virtual reality GUI, comprising a virtual reality headset, wherein the display can be changed by movements of the user of the virtual reality user interface (12), namely by movements of the head.

Description

Field of the invention

[0001] The present invention relates to a method for evaluating vehicle operating information measured and stored by a vehicle during its operation. State of the art

[0002] Particularly during vehicle development, during test drives or bench tests, for example in a workshop, but also, depending on the vehicle, during normal operation by a vehicle owner, vehicle operating information can be read out from a wide variety of data sources on the vehicle and stored in order to enable later analysis of the stored data. Such vehicle operating information also includes, for example, the data from all sensors installed in the vehicle. Such information can be stored, for example, using so-called "data loggers" that can record trace data on a wide variety of vehicle parameters, in particular trace data on all control units installed or retrofitted in the vehicle or trace data on communication between the control units, via various data bus systems or between control units and sensors or actuators.

[0003] Such trace data or stored vehicle operating information is usually laboriously analyzed manually by experts when necessary, for example after completion of tests or after a vehicle accident, and converted into text evaluations or simple graphic diagrams and the like in order to gain a better understanding of the vehicle's behavior and, for example, to enable optimization of the vehicle or vehicle type in question.

[0004] From US 2010 / 0 030 540 A1, a system for reconstructing a traffic accident is known, comprising: a data collection unit that collects / transmits accident-related data corresponding to the occurrence of a traffic accident; a geographical data provision unit that transmits geographical data of a surrounding area of ​​an accident scene; and an accident analysis unit that reconstructs the traffic accident based on the data output by the data collection unit and the geographical data provision unit.

[0005] US 2015 / 0 112 504 A1 teaches a method in an electronic processing system comprising a first memory and one or more processors, the method comprising: receiving sensor data in the electronic processing system representing information collected by one or more sensors that are (i) located on or in a first vehicle and (ii) configured to sense an environment external to the first vehicle; storing the received sensor data in the first memory; determining, via the one or more processors and based on the stored sensor data, one or both of (i) driving habits of a driver of a second vehicle that is different from the first vehicle, and (ii) conditions under which the second vehicle was driven, and a license plate number of the second vehicle;Identifying the driver of the second vehicle using the determined license plate number; and determining a risk level for the driver of the second vehicle using (i) the driving habits of the driver of the second vehicle and (ii) the conditions under which the second vehicle was driven, or both.

[0006] From US 2010 / 0 076 646 A1, an in-vehicle recording system is known, comprising: a plurality of occupant sensors in the vehicle that acquire occupant data; a video acquisition module that records video data inside and outside the vehicle; a data recorder in the vehicle, the data recording the occupant data and the video data and synchronizing the occupant data with the video data.

[0007] Document US 2015 / 0 155 007 A1 discloses a method comprising: receiving sensor data from one or more sensors on board a vehicle at an input module; receiving mapping data at the input module corresponding to locations traversed during a trip of the vehicle; processing the sensor data and the mapping data in a generation module comprising at least one computer processor to correlate the sensor data and the mapping data using location information in the sensor data and the mapping data; generating, at the generation module, a video playback of the trip using the correlated sensor data and mapping data; and outputting, by an output module, the video playback of the trip to a display device.

[0008] US 2013 / 0 116 855 A1 shows an apparatus for documenting an incident involving a first vehicle at an incident location, the apparatus comprising: a communications interface; a memory for storing processor-executable instructions; and a processing unit communicatively coupled to the communications interface and the memory, wherein upon execution of the processor-executable instructions by the processing unit, the processing unit controls the communications interface to electronically receive at least one input image of a geographical area including the accident area; capture vehicle-based information relating to the first vehicle at a time during or near the incident;based at least in part on the vehicle-based information, render a tagged image containing a first representation of at least a portion of the incident overlaid on the at least one input image; and further control the communication interface and / or the memory to electronically transmit and / or electronically store information related to the tagged digital image to document the incident with respect to the geographic area.

[0009] From US 2014 / 0 046 550 A1, a device for reproducing vehicle event data is known, comprising: a logic processor; a graphical display device; and a graphical user interface system, wherein the graphical user interface system comprises a plurality of control objects, wherein at least one control object is coupled to a vehicle event data set, wherein the visual nature of the coupled control elements depends on data values ​​in the vehicle event data set. Summary of the invention

[0010] It is an object of the invention to provide a method for evaluating vehicle operating information that enables a clear and intuitive analysis of vehicle operating information. In particular, this should enable simpler, faster, and higher-quality improvements to vehicles or vehicle types and / or individual components of a vehicle.

[0011] The object is achieved by a method for evaluating vehicle operating information which is measured and stored by a vehicle during its operation, with the further features according to claim 1.

[0012] According to the invention, a filter unit filters raw vehicle data from the vehicle operating information, wherein the vehicle operating information and the raw vehicle data comprise position data of the vehicle. A fusion unit adds or fuses environmental data, in particular geographical environmental data, from the vicinity of the position data to the raw vehicle data to generate simulation data. The simulation data is made accessible, in particular displayed, via a user interface (i.e., via an MMI - human-machine interface). In this case, the user can preferably be enabled to interact with the accessible simulation data.

[0013] According to the invention, vehicle operating information, i.e., data that is generated during vehicle operation and changes during operation—such as position, speed, vehicle acceleration or other accelerations such as lateral acceleration, etc., transmission status, engine characteristics, sensor data from radar or lidar, etc.—is stored, for example, in the form of a trace file. The recording or method for recording such trace files is already known. However, such trace files or vehicle operating information typically contain too much data and measured values ​​to easily identify the actual and significant events during vehicle operation.

[0014] According to the invention, this vehicle operating information is therefore first filtered to initially reduce the complexity of the data. The remaining operating data, which is further used for analysis, forms the raw vehicle data, which is merged with vehicle-external data, i.e., data relating to the respective environment of the vehicle, by a fusion unit. The vehicle-external data contains information that does not originate from the vehicle in question, but rather from external sources, such as traffic infrastructure, databases, telematics services, other vehicles, etc. The totality of the compiled vehicle-internal and vehicle-external data forms the simulation data, which is presented to a user of the method graphically and / or acoustically and / or haptically.This makes it much easier for a user of the method to identify relevant factors in the operation of a specific vehicle and thereby to carry out optimizations on this vehicle or vehicle type and / or on individual components of the vehicle.

[0015] In addition to geographical data, the environmental data from the area surrounding the position can also include, for example: the status of traffic lights or traffic monitoring devices, the status of roads or bridges, traffic signs, etc.

[0016] Preferably, the simulation data is displayed in a three-dimensional perspective view, which provides the most realistic picture possible of the events that occurred. "User interface" is also understood to mean a system that, in addition to displaying data, also enables user interaction.

[0017] According to the invention, the system is displayed and operated using a virtual reality GUI, comprising virtual reality glasses. However, the system is particularly preferably displayed and / or operated using a virtual reality MMI (VR-MMI) or an augmented reality MMI.

[0018] In the context of this invention, "augmented reality MMI" refers to all intermediate stages between real environments and purely virtual environments according to Milgrim and Kishino, such as merged reality, mixed reality, and augmented virtuality. The term "virtual reality user interface" also encompasses such augmented reality user interfaces within the scope of the invention.

[0019] According to the invention, the user interface is therefore a virtual reality user interface or is implemented, for example, by means of virtual reality, in which the simulation data is displayed in an interactive, virtual environment, so that the display can be changed by movements of the user of the virtual reality user interface, namely movements of the head.

[0020] The simulation data can be displayed in a way that represents the operation of the vehicle from a perspective inside the vehicle, i.e. in particular corresponding to the view of an occupant of the vehicle and / or a sensor attached to the vehicle. Alternatively or optionally, the display can also be from a perspective outside the vehicle, for example from a view outside the vehicle’s lane of the road and / or the road or the vehicle’s surroundings can be displayed in a room, for example on a table and / or the perspective can be displayed within the vehicle’s surroundings, e.g. at the side of the road, and / or can be within or outside the displayed scene. For example, the vehicle and the vehicle’s surroundings can be displayed in a three-dimensional constructed world as a scenario on a table in a room and “experienced” using virtual reality.The user interface or the MMI can also contain control elements for influencing the simulation or the user interface, for example a selection list for various menu items, pause / resume buttons, arrows and / or a small map for influencing the displayed image section, etc. The simulation data are preferably synchronized in time during fusion, which makes it possible to run the simulation in real time or to delay or accelerate the simulation process.

[0021] The vehicle operating information and the vehicle raw data preferably include vehicle time data, i.e., in particular, time stamps in which the actual date and time of the processes during vehicle operation are recorded. The timing of the simulation data display can correspond to the timing of the time data, i.e., it can occur in real time and / or slower, i.e., with a time delay, and / or faster, i.e., in fast motion.

[0022] The vehicle operating information is preferably recorded as trace data during the operation of the vehicle, in particular as a recording or extract from a communication in the bus system of the vehicle and / or in particular includes the data communication of the bus systems of the vehicle.

[0023] According to the invention, environmental data, preferably geographical data, from the vicinity of the position data to the raw vehicle data in order to generate simulation data and / or the display of the simulation data is carried out using a game engine. Such game engines are specifically designed for the construction of three-dimensional scenarios, are inexpensively available, and can also automatically take into account physical laws, for example, the mechanics of collisions between objects.

[0024] The environmental data can include data about the terrain and / or data about roads and / or data about objects in road traffic, such as traffic lights, barriers, buildings, other vehicles, pedestrians, and / or bridges, etc., whereby the environmental data preferably also includes dynamically changing data, such as the status of a traffic light or a barrier. Terrain data can be taken from a data source such as Bing™ Maps, for example, while road layouts can be taken from another data source, such as OpenStreetMap™.

[0025] Preferably, the fusion unit can add data about other road users, in particular other vehicles, from the vicinity of the position data to the raw vehicle data to generate the simulation data, so that other road users in the vicinity of the simulated vehicle can be represented via the user interface. This allows the representation of relationships and influences between the movement of other road users and the vehicle in question. Brief description of the drawings

[0026] The invention is described below by way of example with reference to the drawings. Fig. 1 is a schematic representation of program modules for carrying out a method according to the invention. Fig. 2 is a schematic representation of a possible user interface of a method according to the invention in a first embodiment. Fig. 3 is a schematic representation of a possible user interface of a method according to the invention in a second embodiment. Fig. 4 is a schematic representation of a possible user interface of a method according to the invention in a third embodiment. Detailed description of the invention

[0027] Fig. Figure 1 shows a schematic representation of program modules or program units, for example, processes or objects, for executing a method according to the invention, with communication connections between the program modules. The program modules are preferably executed on a vehicle-external computer or backend. However, the program modules can also be implemented in a control unit of the analyzed vehicle and executed by the control unit.

[0028] The program modules are functionally assigned to the areas of data input 14, data preparation 15, data processing 16 and data user interface 17, i.e. representation and interaction with the simulation data.

[0029] The method according to the invention enables automatic or semi-automatic reconstruction of driving situations from a combination of trace data 1 recorded in the vehicle and external environmental data, in particular data from map services and / or road map services 5, 8. The trace data 1 and external geographical environmental data 5, 8 can be collectively referred to as data input 14 of a method according to the invention.

[0030] The stored trace data 1 can be processed, for example parsed - reference numeral 2 in Fig. Figure 1 depicts the process of "parsing trace data" and converting it into a generic format, reference numeral 3, "converting data." From this data, environmentally relevant information, such as GPS coordinates and timestamps, is then extracted using a filter unit 4, i.e., information is extracted from the data that is used as input for a map service 5 and / or a road map service 8. Vehicle-related data is also reported directly to the simulation data 11 to be displayed later.

[0031] On the basis of the extracted data, a landscape within a predetermined radius, for example 20 meters or 200 meters around the position data of the vehicle under investigation or around the route to be analyzed, is created as a three-dimensional model using geographical environmental information from an external source, namely a map service 5, such as “Bing™ Maps”, using a fusion unit 6, in particular a terrain generator, for example “WorldComposer™” or a game engine, such as “Unity 3D™”, in order to obtain a “terrain” 7.

[0032] For this landscape, terrain 7, information on roads and other stationary objects, such as traffic lights, buildings, rails, etc., is obtained from another external source, namely a road map service 8 such as “OpenStreetMap”™. This external information can include, for example: type, position, orientation, color, size of the object, number of lanes, etc. Three-dimensional objects are also generated from this information and combined with the previously created terrain 7 by the game engine or fusion unit 6 to obtain a “terrain + roads” 9.

[0033] The created objects can contain dynamic properties, for example, traffic lights can have switching times, drawbridges or barriers can have changing states, etc. Other external parameters such as light incidence depending on the time of day, or weather information that can also be retrieved from an external data source can also be integrated into the simulation, for example by the same fusion unit 6, in order to create a data set “Terrain + Streets + Environment” - in Fig. 1 with reference number 10. The combination of the aforementioned objects with the terrain creates a virtual image of the vehicle's surroundings when recording trace data 1.

[0034] For example, a manually selected vehicle model can be positioned in the created virtual image of the environment. Positioning is performed based on the position data originally contained in trace data 1 (GPS position data or corrected position data).

[0035] Other road users can also be reconstructed, preferably from the trace data 1, for example, from information from camera data, a radar, or Lidar (light detection and ranging) system. Other road users can also be added to the environment model using other sources.

[0036] The sequence of events, in particular the position and movement of the vehicle recording trace data, for example using a data logger, or of the vehicle under observation, is then automatically reconstructed using the chronologically ordered and filtered trace data, or data extracted in filter 4, to create the simulation data 11. The result can be viewed and controlled by a user of the system via a user interface 12, 13, in particular via a virtual reality (VR) user interface 12 (graphical user interface (GUI), VR-GUI and / or VR-MMI), or a classic screen user interface 13 (desktop GUI and / or desktop MMI) - for example by starting, pausing, fast-forwarding and rewinding, adjusting camera settings or perspectives, annotating the display, etc.

[0037] The representation of additional information, such as the vehicle's engine speed, speeds, trajectory, etc. from the vehicle's trace data 1 in the simulation is preferably possible and in particular selectable and / or configurable.

[0038] The Fig. 2, Fig. 3 and Fig. 4 show different variants of a possible representation of a method according to the invention.

[0039] In this case, Fig. 2 the simulation data 11 are displayed by a user interface 12, in particular in a VR-GUI and / or MMI, in a manner that represents the operation of the analyzed vehicle in a perspective from the interior of the vehicle.

[0040] In addition to elements of the vehicle from the view from the inside, such as the windshield, steering wheel, dashboard, the respective reconstruction of the surroundings of the vehicle at the time the vehicle operating information is stored is displayed in the windshield 22 of the vehicle, possibly also in other windows of the vehicle.

[0041] For example, a small map display 18 can also be displayed, allowing the selection of the displayed section of the simulation data 11. Past and / or future events can be displayed in an event list 19 from the perspective of the simulated point in time. Additionally, information can be shown and / or functions for controlling the simulation can be selected, for example, on a displayed screen 20. Dedicated buttons 21 can also be displayed in the VR GUI and / or VR MMI to control the simulation, such as pause, fast-forward, and rewind buttons.

[0042] Similarly, Fig. 3 an alternative representation of the same system or an alternative system, taking a perspective outside the vehicle and displaying the vehicle and its surroundings from a point adjacent to the route of the vehicle being analyzed. Elements such as a map display 18, event list 19, screen 20, and / or buttons 21 can again be displayed.

[0043] In Fig. 4, the display also takes place from outside the vehicle, with the vehicle and its surroundings being projected onto a table 23 or a similar plane. Control elements and additional information, such as screen 20 and buttons 21, can then be displayed in the vicinity of the table or plane, and can, for example, depict a room as a whole.

[0044] Additional tools can be stored in or retrieved from environmental objects corresponding to the room. The tools preferably have a form that corresponds to their intended purpose, so that the user can intuitively use them, particularly in the VR-MMI 12. Examples of this are a trash can for disposing of simulation data or a door through which other rooms can be entered. Each room can represent an individually furnished environment for specific, easily recognizable test purposes (e.g., workshop, garage, living room). In addition, non-interactive environmental objects such as plants or windows preferably ensure a sense of well-being for the user, making using the VR-MMI more pleasant than without them. These environmental objects and their position in the virtual space can be predefined or placed manually by the user.

[0045] Preferably, the simulation data 11 can be displayed on a table in the user's virtual space.

[0046] Whether this table is physically located in the user's actual room (if the user is in a room while using the program) is optional.

[0047] The representation of the simulation data 11 on the virtual table can be rotated and / or scaled, in particular live during use, and / or be limited horizontally to the extent of the virtual table.

[0048] Scaled data on a fixed-size table may exceed the dimensions of the table. In this case, the user interface may restrict the data display to a virtual cage, with said cage being invisible to the user. This means that only those objects in the simulation (or the corresponding parts of the objects) whose position lies on or above / below the virtual table are displayed. All other objects are simulated but are, at least visually, invisible to the user via the HMI.

[0049] The user interface can have more than one room. Users can switch between them manually while using the MMI. This doesn't require the program to be closed or completely reloaded.

[0050] Each room can contain interactive objects that belong to the tool but not to the simulation data. For example, a magnifying glass or a trash can could be placed in the room, which the user can pick up, touch, or put down. Ideally, they serve an analytical or management function in addition to their aesthetic appeal.

[0051] These interactive objects are designed as naturally as possible, such as drawers and shelves that can be used to store interactive objects. For example, such an object could be a virtual file folder containing the scenario to be investigated. This could then be placed in a filing cabinet, retrieved and opened by the user, thereby loading the simulation data onto the table.

[0052] Some interactive objects can also be duplicated or removed as needed. For example, a magnifying glass could be present in the room, allowing specific objects or locations in the simulation data to be selectively enlarged. If more than one location or object in the simulation data is to be magnified, the user can "create" a second magnifying glass. If this is no longer needed, it can be deleted, for example, using the virtual trash can or by pressing a key. List of reference symbols 1 Trace data 2 Prepare trace data, especially parse 3 Convert data 4 Filter unit (extract information from the data) 5 Map service (Bing Maps) 6 Fusion Unit (Terrain Generator (World Composer)) 7 terrain 8 Street map service (Open Street Maps) 9 terrain + roads 10 Terrain + Roads + Environment 11 Simulation data 12 Virtual Reality User Interface (VR MMI, VR-GUI, Virtual Reality MMI or Augmented Reality MMI) 13 Screen user interface (desktop GUI, desktop MMI) 14 Data input 15 Preparing data 16 Process data 17 Data User Interface 18 Map display 19 Event list 20 screen 21 Button 22 Windshield 23 Table

Claims

[1] Method for evaluating vehicle operating information measured and stored by a vehicle during its operation, wherein vehicle raw data are filtered out of the vehicle operating information by a filter unit (4), wherein the vehicle operating information and the vehicle raw data comprise position data of the vehicle, wherein environmental data from the environment of the position data are added to the vehicle raw data by a fusion unit (6) in order to generate simulation data (11), wherein the simulation data (11) are made accessible, in particular displayed, by means of a user interface (12, 13), characterized bythat the addition of environmental data from the environment of the position data to the raw vehicle data in order to generate simulation data (11) and / or the making accessible, in particular displaying the simulation data (11) takes place by means of a game engine, wherein the user interface (12, 13) is a virtual reality user interface (12), so that the simulation data (11) is displayed in an interactive virtual environment, wherein the display and operation of the system takes place by means of a virtual reality GUI, comprising virtual reality glasses, wherein the display can be changed by movements of the user of the virtual reality user interface (12), namely by movements of the head. [2] Method according to claim 1, characterized bythat the simulation data (11) are presented in a manner that represents the operation of the vehicle from a perspective from inside the vehicle and / or from a perspective from outside the vehicle. [3] Method according to at least one of the preceding claims, characterized by that the vehicle operating information and the vehicle raw data comprise time data of the vehicle, wherein the time sequence of the representation of the simulation data (11) corresponds to the sequence of the time data, i.e. takes place in real time and / or slower, i.e. with a time delay and / or faster, i.e. with time lapse. [4] Method according to at least one of the preceding claims, characterized bythat the vehicle operating information is recorded as trace data (1) during the operation of the vehicle, wherein the trace data (1) is recorded in particular from a communication in the bus system of the vehicle and / or comprises the data communication of the bus systems of the vehicle. [5] Method according to at least one of the preceding claims, characterized by that the environmental data comprise data about the terrain and / or the geography and / or data about roads and / or data about objects in road traffic such as traffic lights, barriers and / or bridges, wherein the environmental data preferably also comprise dynamically changing data, such as in particular operating states, for example of traffic lights. [6] Method according to at least one of the preceding claims, characterized bythat the fusion unit (6) adds data about other road users from the environment of the position data to the raw vehicle data in order to generate the simulation data (11), so that other road users can be displayed by means of the user interface (12, 13). [7] Method according to claim 4, characterized by that the trace data (1) are measured and stored during operation of the vehicle by control devices conventionally present in the vehicle and / or by at least one device additionally installed in the vehicle for such measuring purposes, in particular a data logger. [8] Method according to at least one of the preceding claims, characterized by that the virtual position of the user of the user interface (12, 13) in the simulation data (11) is in constant relation to the vehicle or is in constant relation to the surroundings of the vehicle. [9] Method according to at least one of the preceding claims, characterized by that the virtual position of the user of the user interface (12, 13) is within the simulation data (11). [10] Method according to at least one of the preceding claims, characterized by that the virtual position of the user of the user interface (12, 13) is in a virtual space outside the simulation data (11). [11] Method according to at least one of the preceding claims, characterized by that the virtual position of the user of the user interface (12, 13) can be influenced by the user by means of an input device. [12] Method according to claim 10, characterized by that the simulation data (11) are displayed on a table in the user's virtual space. [13] Method according to claim 12, characterized bythat the representation of the simulation data (11) on the virtual table can be rotated and / or scaled and / or is limited horizontally to the extent of the virtual table. [14] Method according to claim 10, characterized by that the user can switch between different virtual spaces without leaving the user interface, wherein preferably at least one virtual space, preferably each virtual space, contains interactive objects with analysis functions. [15] Method according to claim 14, characterized by that at least one virtual room, preferably every virtual room, has storage or storage options for the interactive objects with analysis functions. [16] Method according to claim 14 or 15, characterized by that instances of the interactive objects with analysis functions can be created and / or destroyed by user input.

Images