Methods for evaluating vehicle data and vehicle data evaluation systems for carrying out such a method

Abstract

Method for evaluating vehicle data that characterizes at least one driving maneuver and / or at least one state of at least one motor vehicle (10) over a specified period of time and from which at least one motor vehicle (10) is recorded, comprising the steps: - Operating the motor vehicle (10) for the specified period of time, wherein during the operation of the motor vehicle (10) the vehicle data are recorded by at least one recording device (20) of the motor vehicle (10) and stored in a storage device (22) of the motor vehicle (10) (S1); - Transmitting the stored vehicle data to an external computing device (30) (S2); - Recognition of at least one data pattern in the vehicle data by applying a neural network to the provided vehicle data by the vehicle-external computing device (30) (S3); - Check whether the detected data pattern corresponds to a predefined reference pattern stored in the vehicle-external computing unit (30) which is assigned to at least one driving maneuver and / or at least one state of the motor vehicle (10) (S4); - Applying a predefined quality criterion to the vehicle data underlying the detected data pattern and / or other vehicle data acquired at the same time (S5), wherein the quality criterion is met if the other vehicle data acquired at the same time exhibits a predefined minimum diversity of driving situations in which it was acquired, and the other vehicle data that meet the predefined quality criterion are stored as verified test vehicle data for the detected data pattern in the vehicle-external computing device (30); - If the detected data pattern matches the stored reference pattern and / or the vehicle data meets the quality criterion, a corresponding message is issued (S6).

Description

[0001] The invention relates to a method for evaluating vehicle data and a vehicle data evaluation system designed to carry out such a method.

[0002] Interconnected vehicle components in a motor vehicle, such as networked control units, sensors, and / or actuators, communicate with each other via bus systems within the vehicle. The bus system can be, for example, a Local Interconnect Network (LIN) bus, Controller Area Network (CAN) bus, FlexRay, Ethernet, Peripheral Component Interconnect Express (PCIe), or Media Oriented Systems Transport (MOST) bus. The data exchanged via one or more bus systems is often organized as protocol data units (PDUs). Each PDU contains a predefined set of information and signals that can be exchanged via one or more of the aforementioned bus systems. The individual networked vehicle components can receive and process at least subsets of the PDUs.External recording devices, known as data loggers, are designed to receive PDUs transmitted within a vehicle and store them on a medium such as a hard disk or a solid-state drive (SSD). This allows for the analysis of all, or at least subsets, of PDUs both inside and outside of vehicles. The same applies to simulation and test equipment, such as hardware-in-the-loop (HIL) simulators, software-in-the-loop (SIL) simulators, or test vehicles, which can be used to simulate or operate at least subsystems of a vehicle's electronics. Such simulation and test equipment also typically records its PDU traffic.

[0003] A cyclic PDU with a size of, for example, 8 bytes, generates a data rate of 80 bytes per second, or 288,000 bytes per hour, with a typical cycle time of 100 milliseconds. Thus, for example, 100 PDUs would generate a data volume of 659 megabytes in 24 hours. However, evaluating such a data volume is only possible to a limited extent using traditional analytical methods. Furthermore, the data volume increases with a growing number of measured vehicles and, consequently, a growing number of recording PDUs, as well as with a longer recording period. Evaluating such a data volume with traditional analytical methods also requires detailed knowledge of the functionality of the corresponding vehicle equipment and the communication relationships within the vehicle in order to implement a complex search strategy.For example, signal values ​​typical for specific driving maneuvers can only be recognized within the PDUs if all vehicle-internal data processing steps related to that signal value are fully known. Furthermore, the system's intended behavior can only be verified during data analysis if the expected system response has been modeled in detail beforehand. A malfunction of a vehicle software component will only be detected during such data analysis if the system is specifically searching for such a malfunction.

[0004] Methods of artificial intelligence (AI), such as the use of artificial neural networks, are suitable for evaluating large amounts of data.

[0005] DE 103 54 322 A1 discloses a method and a system for determining the driving situation of a motor vehicle using data provided in the vehicle. This involves, among other things, an artificial neural network provided in the vehicle by a suitably programmed computer. The neural network ultimately enables the recognition of the current driving situation, whereupon measures are taken to relieve the driver.

[0006] German patent DE 10 2016 121 691 A1 discloses a method for operating at least one motor vehicle. This method compares the actual driving behavior of a driver of the motor vehicle with a predefined driving behavior in an adaptable artificial neural network, and the adaptable artificial neural network is adjusted based on the comparison using a computing unit in the motor vehicle. The aim is to make the behavior of human drivers in motor vehicles usable in a particularly high level of detail.

[0007] DE 10 2011 012 238 A1 discloses a method for controlling an internal combustion engine, whereby engine operation is monitored. An artificial neural network can be used to estimate a steady-state nitrogen oxide value.

[0008] German patent application DE 102 35 525 A1 discloses a method and system for an improved vehicle monitoring system, in which machine learning and data mining technologies are applied to data collected from multiple vehicles to generate models. Frequent collection of vehicle sensor and diagnostic data now enables a comparison with the generated models to provide vehicle analysis with regard to repair, maintenance, and diagnostics.

[0009] The object of the invention is to provide a solution by means of which data recorded in a vehicle can be processed and evaluated quickly and thoroughly.

[0010] This problem is solved by the subject matter of the independent patent claims. Advantageous embodiments with expedient and non-trivial further developments of the invention are specified in the dependent claims, the present description, and in the figures.

[0011] The method according to the invention serves to evaluate vehicle data. The vehicle data characterizes at least one driving maneuver and / or at least one state of at least one motor vehicle over a predetermined period of time and is acquired by the at least one motor vehicle. The vehicle data thus includes, for example, sensor data from a detection device of the motor vehicle, position data from a global positioning system (GPS) transmitted to the motor vehicle by an external device, data exchanged between individual vehicle functions, such as individual driver assistance systems of the motor vehicle, and / or already evaluated sensor data, such as data concerning an object detected by a camera system of the motor vehicle in the vicinity of the motor vehicle.

[0012] The method according to the invention comprises the following steps: First, the at least one motor vehicle is operated for a predetermined period of time, during which time the vehicle data is recorded by at least one of the vehicle's acquisition devices and stored in a storage device of the vehicle. The vehicle data is thus recorded during a period in which the motor vehicle is activated. For example, the motor vehicle can be manually controlled by a driver for the predetermined period, who drives a predetermined test route to acquire vehicle data. This test route includes both sections in a rural environment and sections in an urban environment. This test drive is conducted, for example, to test a sensor system of the motor vehicle in different driving environments.The sensor system could, for example, consist of a camera system and an associated evaluation unit for analyzing the camera system data. During the test drive, various data points are collected, determined, and made available by both the camera system and the evaluation unit, and stored in the storage device. Furthermore, during the described drive, other vehicle sensors, such as a speedometer, turn signals, and / or a lane departure warning system, also collect and store current vehicle data in the storage device. The stored vehicle data can, for example, be organized as PDUs (Power Distribution Units).

[0013] In the next step, the vehicle data stored in the storage device is transmitted to an external computing unit. This computing unit is, for example, a server belonging to a vehicle manufacturer, which collects vehicle data in order to test the functionality of a specific detection device, such as a new camera system, and, if necessary, to optimize it using the insights gained from data analysis.

[0014] The vehicle's external computing unit is designed to analyze the transmitted vehicle data. In the next step, it identifies at least one data pattern within the vehicle data by applying an artificial neural network to the data provided to the computing unit. Such an artificial neural network is a network of artificial neurons. In neuroinformatics, artificial neural networks are used as a form of artificial intelligence because they are suitable for analyzing data without requiring detailed knowledge of its function or the communication relationships between individual data sets. An artificial neural network is therefore defined as a self-learning computer program that can recognize complex patterns and relationships in large datasets within a relatively short time.An artificial neural network is based on an abstracted model of interconnected neutrons, meaning that an artificial neural network, at least to some extent, mimics the structure of a biological brain. In simplified terms, the structure and function of an artificial neural network can be described as a network of neurons, each of which receives information from the outside or from other neurons and forwards it, modified, to other neurons or outputs it as a final result. For example, positive or negative weights can be assigned, representing an excitatory or inhibitory influence between the neurons. The knowledge, and thus the artificial intelligence, of an artificial neural network is ultimately stored in the connections between the individual neurons and their weights.During training of the neural network, these weights of the connections change depending on the applied learning rules and the results achieved.

[0015] In other words, by applying an artificial neural network to the vehicle data, it is possible to identify conspicuous data packets, known as data patterns, within this data in a particularly short time. Once at least one data pattern has been detected, the external computing unit checks whether the detected data pattern corresponds to a predefined reference pattern. This predefined reference pattern is stored in the external computing unit and is assigned to at least one driving maneuver and / or at least one state of the vehicle. For example, a reference pattern might be stored that corresponds to a braking process of the vehicle.The neural network can thus be used to evaluate whether the vehicle data contains at least one data pattern that corresponds to the reference pattern for the braking process or, taking predefined limits into account, at least resembles it, so that ultimately all braking maneuvers performed within the specified time period during which the vehicle was operated and the vehicle data was recorded are recognized. This makes it possible, for example, to extract all data patterns indicating a braking process from vehicle data provided by several vehicles, each of which has undergone multiple multi-hour test drives. The method according to the invention thus enables the detection of a learned pattern of a driving maneuver and / or vehicle state within large datasets of vehicle data.

[0016] In a further process step, a predefined quality criterion is applied to the vehicle data underlying the recognized data pattern and / or other vehicle data recorded simultaneously. The quality criterion can, for example, be chosen such that it is met if vehicle data was acquired under different lighting conditions, such as both daytime and nighttime driving. Based on such a quality criterion, it can then be determined, for example, with regard to the camera system, how useful a test drive with a vehicle is for optimizing the camera system based on the acquired vehicle data. The rationale behind this is that, for example, test drives conducted only in daylight do not provide any information about what sensor data recorded at night might look like.Regarding data acquisition for investigating the functionality of the camera system, the quality criterion might only be met if vehicle data was recorded both during daylight and at night. However, if the vehicle's braking behavior is being investigated, such information regarding lighting conditions during the test drive has only a minor influence on the data quality. Therefore, even if the test drives were only conducted in daylight, the vehicle data would still be of sufficiently high quality to meet the specified quality criterion.

[0017] If the detected data pattern matches the stored reference pattern and / or the vehicle data meets the quality criterion, a corresponding message is issued. This message can be displayed, for example, on a screen of the vehicle's external computing unit. This message can include detailed information about the detected data pattern and the applied quality criterion. Ultimately, this allows the data analysis results to be displayed to, for example, an employee of the vehicle manufacturer who provides the external computing unit, and also made available for further data analysis.This ultimately makes it possible to process recorded, received, and thus, in a sense, overheard vehicle data for evaluation purposes using artificial intelligence methods, without requiring detailed functional knowledge or information about communication relationships within the vehicle. The described method is based on the fact that individual driving maneuvers, such as braking, as well as vehicle states, such as waiting at a red light, lead to characteristic patterns within the recorded vehicle data, which is organized, for example, as PDUs (Power Distribution Units). These patterns can be recognized in the data and processed via neural networks. This enables, for example, the quality evaluation of test data and the rapid identification of desired data patterns within a large amount of data.Ultimately, this provides an efficient, thorough, fast data analysis strategy applicable to any vehicle data.

[0018] Furthermore, the quality criterion is considered fulfilled if the additional vehicle data recorded over time exhibits a predefined minimum diversity of driving situations in which it was recorded. This quality criterion is based on the understanding that the quality of test and verification data can be assessed by quantifying the number of different driving maneuvers or by providing evidence that certain driving maneuvers were actually performed.For example, if 5000 km of driving test data were requested, but all of it was recorded during daytime driving on a highway, and it would be beneficial to have a mix of vehicle data from both highway and rural road driving, as well as from driving in daylight and driving without daylight, then the quality criterion can be adjusted so that a minimum level of diversity in the described driving situations is required for the criterion to be met. This quality criterion is preferably applied to other vehicle data, as it is useful, for example, when evaluating the quality of vehicle data regarding braking behavior, to obtain and analyze information about the actual driving situation of each braking maneuver based on this additional vehicle data.The specified minimum diversity is designed depending on the desired application area of ​​the acquired vehicle data, since different minimum diversities should be chosen depending on which vehicle's acquisition device is to be optimized or which software and / or hardware component of the vehicle is to be analyzed and possibly optimized in its functionality.

[0019] The remaining vehicle data that meet the specified quality criteria are stored as verified test vehicle data for the identified data pattern in the vehicle's external computing unit. This makes it possible to select precisely those vehicle data from a large dataset, collected, for example, from the journeys of over 100 vehicles, that should be further analyzed and extracted in more detail for optimization purposes, such as optimizing a driver assistance system. One suitable method for this is determining a so-called toggle coverage value. This value is determined during a test that ultimately outputs a value dependent on whether various eventualities of driving maneuvers and / or vehicle states are reflected in the data. The toggle coverage value thus represents a quality metric.Ultimately, checking the quality criterion makes it possible to analyze and verify vehicle data provided by test vehicles from test drives based on the evaluation of the vehicle data volume and the vehicle data content using artificial intelligence.

[0020] The invention also includes embodiments that offer additional advantages.

[0021] In an advantageous embodiment of the invention, reference vehicle data from at least one driving maneuver and / or at least one state of the at least one motor vehicle are provided to the vehicle-external computing unit. This reference vehicle data is generated, for example, during predetermined test drives on a test track. For instance, it may be provided that, during a test drive on this test track, the motor vehicle brakes to a standstill five times after reaching a predetermined speed. The neural network then learns the reference pattern for the corresponding driving maneuver and / or state based on the provided reference vehicle data. Thus, based on, for example, the five braking maneuvers performed during the test drive, the artificial neural network can learn what the reference pattern for a braking maneuver looks like.The reference pattern is then stored in the vehicle's external computing unit. The artificial neural network is therefore initially trained using vehicle data from test vehicles and / or a testing facility, whereby the test vehicle performs at least one reference maneuver or the testing facility simulates at least one reference maneuver. Subsequently, the neural network is able to recognize this reference maneuver, such as a braking process, in any vehicle data. Consequently, a training procedure for the neural network is initially provided, so that ultimately a reliable evaluation result regarding the assignment of the recognized data patterns to the stored reference patterns is possible.

[0022] In a further particularly advantageous embodiment of the invention, it is provided that, if the detected data pattern does not correspond to a stored reference pattern, a check is performed to see whether at least one of the following situations is detected based on the data pattern: a cyberattack on the computing device and / or the motor vehicle, driver error during the acquisition of the vehicle data, and / or a malfunctioning software program in the motor vehicle. If one of the aforementioned situations is detected, the quality criterion is considered fulfilled, and a corresponding notification is issued. Thus, even if no stored reference pattern can be used to assign the detected data pattern to one of the driving maneuvers and / or states of the motor vehicle, statements regarding the evaluation of the vehicle data can still be made based on the data pattern.Because with the help of the artificial neural network, not only known data patterns but also anomalies in vehicle data can be automatically detected. Such anomalies, which can also be referred to as vehicle data irregularities, can, for example, indicate driver error. If, for instance, the driver performs an illegal maneuver, such as activating autopilot while on a stretch of road where autopilot activation is prohibited, this behavior can be detected, and the corresponding data can be excluded from further data analysis.Furthermore, a malicious attack on the vehicle's external computing unit and / or the vehicle itself—a so-called cyberattack—can be detected based on the data pattern, allowing countermeasures to be taken to protect the computing unit and / or the vehicle. Such situations can be identified because the corresponding data patterns do not match the trained behaviors, i.e., the stored reference patterns. Therefore, the method is not only suitable for recognizing already known driving maneuvers and / or vehicle states, which are stored as reference patterns in the vehicle's external computing unit, but can also be used to detect further, potentially unexpected, manipulations of the vehicle data.This function is particularly interesting in practice with regard to malfunctioning software programs in motor vehicles, as it can detect, for example, a vehicle function malfunction. For instance, a driver assistance system might activate an emergency stop even though there is no obstacle in the vehicle's path. One reason for this could be that the camera system under test mistakenly identified a shadow as a person walking in front of the vehicle. Such a malfunctioning software program could thus be detected by analyzing the data using the artificial neural network, as the associated data pattern allows for inferences about such a situation.Ultimately, this enables the vehicle data to be checked for cyberattacks, driver error, and faulty software programs in the vehicle, thus allowing a comprehensive analysis of the vehicle data regarding possible sources of error that can be rectified in subsequent steps.

[0023] Another embodiment of the invention provides for checking, based on the vehicle data, whether at least one measured variable included in the vehicle data is recorded under multiple designations. For example, it is possible that the vehicle may use different designations for the measured variable "vehicle speed" in the vehicle, such as v0, v, Speed, and Velocity. This results in redundancy within the vehicle data, as the same measured variable is stored multiple times. Such redundancy can arise, for example, if different measured variable designations are used in different software components of the vehicle. The artificial neural network can detect redundancies, for example, by observing that individual data packets behave synchronously, thus creating a corresponding data pattern.This makes it possible to identify redundancies in the measurement parameter designations, thus enabling a quick and thorough check of the vehicle software.

[0024] Another embodiment of the invention provides that if at least one measured variable is recorded under several designations, one of the designations is selected and the vehicle software is corrected so that only the selected designation is used for the measured variable in the vehicle. This standardizes the redundantly used designation for one and the same measured variable and then makes the corresponding adjustment in the affected vehicle software. This enables a standardization of the designations for individual measured variables within the vehicle software, which may reduce additional and ultimately unnecessary intermediate evaluation steps and, for example, accelerate the vehicle software.

[0025] In a further embodiment of the invention, it is provided that a test maneuver and / or a test state of the motor vehicle is generated based on vehicle data that is generated according to one of the recognized data patterns. In other words, targeted generation of desired test cases can be enabled, taking into account the recognized data patterns, i.e., ultimately the acquired vehicle data. This allows, for example, a simulation to be performed, which can be used to thoroughly test, for instance, the newly developed camera system and the associated evaluation software, and to examine their expected reaction to predefined test events within the framework of simulations. This simplifies product development and product improvement with regard to motor vehicles.

[0026] Furthermore, one embodiment provides for the vehicle data to be transmitted from the vehicle's external computing unit to a simulation unit. This simulation unit performs a resimulation using the transmitted vehicle data, after which the simulation data obtained during the resimulation is compared with the transmitted vehicle data. Depending on the result of this comparison, a corresponding output signal is generated and displayed. In other words, a reaction to a communication pattern associated with a known driving maneuver can be investigated. For example, data for reproduction, i.e., for resimulation, is transmitted to the simulation unit, which is designed as a test device. For instance, vehicle data from a braking maneuver is transmitted to the simulation unit.This data is then fed into the simulation system, which, for example, simulates another braking maneuver. However, this re-simulation may generate additional vehicle data beyond what was originally transmitted. This allows, for instance, the testing of the new camera system to determine whether it recognizes lane markings and people in the vicinity of the vehicle, as indicated by the collected vehicle data, or whether it detects other objects during the re-simulation. The corresponding output signal can include an evaluation comparing the results of the re-simulation with the results obtained during actual driving, which are derived from the stored vehicle data.This allows the evaluation of the large amount of vehicle data to contribute to a detailed investigation and subsequent optimization of individual vehicle functions within the framework of simulations.

[0027] Furthermore, the implementation of a communication definition can be verified. This communication definition encompasses, for example, relationships stored within the vehicle between a measured quantity, such as vehicle speed, a storage location for this measured quantity within the vehicle, and at least one conversion factor related to the corresponding measured quantity. The extent to which the vehicle data indicates that the stored communication definitions are meaningful and correspond to the actual computational steps implemented within the vehicle software can be verified using data analysis with the artificial neural network. Ultimately, this data analysis will determine a concrete communication matrix that includes the aforementioned information for each individual measured quantity.This enables a further meaningful step towards optimal vehicle software, as any error occurring within the described communication definitions can be detected or subsequently rectified if necessary.

[0028] In a further advantageous embodiment of the invention, the vehicle data is organized as protocol data units, i.e., as so-called PDUs. This is a common concept from the field of network protocols, where a complete set of data and management information of a hierarchical data layer is designated as a PDU. Such an organization of vehicle data is particularly common in motor vehicles and is especially suitable for organizing large amounts of data, such as those generated and accumulated in motor vehicles.

[0029] The invention further comprises a vehicle data evaluation system, which includes an external computing unit and at least one motor vehicle with a central storage unit. Vehicle data, acquired by the motor vehicle and characterizing at least one driving maneuver and / or at least one state of the motor vehicle over a predetermined period, is stored in the central storage unit of the motor vehicle. The motor vehicle of the vehicle data evaluation system is designed to operate for the predetermined period and, during operation, to acquire the vehicle data from at least one acquisition device of the motor vehicle and to store it in the motor vehicle's storage unit. Furthermore, the motor vehicle is designed to transmit the stored vehicle data to the external computing unit.The external computing unit is designed to recognize at least one data pattern in the vehicle data by applying an artificial neural network to the provided vehicle data and to verify whether the recognized data pattern corresponds to a predefined reference pattern stored in the external computing unit, which is assigned to at least one driving maneuver and / or at least one state of the vehicle. The external computing unit is also designed to apply a predefined quality criterion to the vehicle data underlying the recognized data pattern and / or other simultaneously acquired vehicle data and, if the recognized data pattern corresponds to the stored reference pattern and / or the vehicle data meets the quality criterion, to issue a corresponding notification.The quality criterion is met if the simultaneously acquired additional vehicle data exhibit a predefined minimum diversity of driving situations in which they were acquired. The vehicle-external computing unit is also designed to store the additional vehicle data that meets the predefined quality criterion as verified test vehicle data for the recognized data pattern in the vehicle-external computing unit (30). The preferred embodiments and their advantages presented in connection with the method according to the invention apply accordingly, insofar as applicable, to the vehicle data evaluation system according to the invention. For this reason, the corresponding embodiments of the vehicle data evaluation system according to the invention are not described again here.

[0030] The motor vehicle is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0031] The invention also includes combinations of the features of the described embodiments.

[0032] An embodiment of the invention is described below. The following is shown: Fig. 1. A schematic representation of several motor vehicles on a test drive to generate vehicle data; and Fig. Figure 2 shows a schematic representation of a signal flow graph for a method for evaluating vehicle data.

[0033] The embodiment described below is a preferred embodiment of the invention. In this embodiment, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiment other than those shown. Furthermore, the described embodiment can also be supplemented by further features of the invention already described.

[0034] In the figures, identical reference symbols denote functionally equivalent elements.

[0035] In Fig. Figure 1 shows a motor vehicle 10 driving on a road 12 within a city 14. The motor vehicle 10 is driving through the city 14 during daylight hours, that is, while the sun 15 is shining. Within the city 14, the motor vehicle 10 moves through relatively dense traffic, frequently stopping and starting again, for example, because it is required by a traffic light 18 to wait at a red light. During the test drive, the motor vehicle 10 leaves the city 14 and drives to a mountainous region 16, that is, it is driving on a country road with relatively little traffic. Meanwhile, the lighting conditions have also changed, and the motor vehicle 10 is driving through the mountainous region 16 at night, which is indicated by a moon 17 in Fig. Figure 1 is outlined. The motor vehicle 10, which drives at night, is shown in Fig. 1 with the reference symbol 10'.

[0036] The motor vehicle 10 comprises a recording device 20, a storage device 22, and a communication device 24. The motor vehicle 10 can establish a communication link 25 with the communication link 24 of an external computer 30 via the communication device 24. The motor vehicle 10, together with the external computer 30, forms a vehicle data evaluation system 40. The vehicle data evaluation system 40 can also include further motor vehicles 10, which can likewise be connected to the external computer 30 via a communication link 25. Fig. The communication link 25 shown in Figure 2 is purely symbolic and is intended to illustrate that the individual vehicles 10 are designed to transmit data to the vehicle-external computing unit 30. The transmission itself takes place, for example, via a cable connection between the vehicle 10 and the vehicle-external computing unit 30, at a time when the test drive has already ended.

[0037] Within the vehicle 10, data from the acquisition device 20, for example a driver assistance system and / or a sensor device such as a camera system, is transmitted via an internal vehicle bus system to the storage device 22 and stored therein. This data is referred to as vehicle data, characterizes at least one driving maneuver and / or at least one state of the vehicle 10 over a specified period of time, and was acquired by the acquisition device 20. The vehicle data can be organized as protocol data units, i.e., as so-called PDUs.

[0038] In Fig.Figure 2 shows the individual process steps in a signal flow diagram, which are carried out either by the vehicle 10 or by the vehicle-external computing unit 30. In a first step S1, the vehicle 10 is operated for the specified duration, during which time the vehicle data is recorded by the acquisition unit 20 and stored in the storage unit 22. In a second step S2, the stored vehicle data is transmitted to the vehicle-external computing unit 30.

[0039] In step S3, the vehicle-external computing unit 30 then recognizes at least one data pattern in the vehicle data by applying an artificial neural network to the provided vehicle data. This allows, for example, the detection of whether the vehicle 10 performed a braking maneuver during the test drive, perhaps due to the red phase of traffic light 18. This is possible because such a braking maneuver results in a characteristic data pattern in the vehicle data. In a subsequent step S4, it is checked whether the recognized data pattern corresponds to a predefined reference pattern stored in the vehicle-external computing unit 30, which is assigned to at least one driving maneuver and / or at least one state of the vehicle 10.For example, a vehicle data set describing a typical braking process can be stored in the vehicle-external computing unit 30 as such a reference pattern, so that the braking process can be recognized based on the recognized data patterns after comparison with this reference pattern.

[0040] The reference pattern is first provided to the vehicle-external computing unit 30 by providing reference vehicle data from at least one driving maneuver and / or at least one state of the vehicle 10 to the vehicle-external computing unit 30. The neural network then learns the reference pattern for the corresponding driving maneuver and / or state based on the provided reference vehicle data, and the reference pattern is stored in the vehicle-external computing unit 30. However, it is possible that the recognized data pattern does not correspond to any stored reference pattern.

[0041] In step S5, a predefined quality criterion is applied to the vehicle data underlying the recognized data pattern. Alternatively, or in addition, the predefined quality criterion is applied in step S5 to other vehicle data recorded simultaneously. If the other vehicle data recorded simultaneously exhibits a predefined minimum diversity of driving situations in which it was captured, the quality criterion is considered fulfilled. The vehicle data provided by vehicle 10, for example, exhibits high diversity because it was recorded both during the day and at night and also includes driving maneuvers within city 14 as well as within a mountainous landscape 16, i.e., a non-urban environment. If vehicle 10 were to record vehicle data only during daylight hours within city 14, such diversity of driving situations might not be present.

[0042] If the specified quality criterion is met and / or the detected data pattern matches the stored reference pattern, a corresponding message is issued. This message can, for example, include a detailed report of the completed data analysis. The message is issued in step S6. Furthermore, the additional vehicle data that meets the specified quality criterion are stored as verified test vehicle data of the detected data pattern in the vehicle's external computing unit 30.

[0043] If the detected data pattern does not match any stored data pattern, the system also checks whether at least one of the following situations is detected based on the data pattern: a cyberattack on the vehicle-external computing device 30 and / or the vehicle 10, driver error by the vehicle 10 during the acquisition of vehicle data, and / or a malfunctioning software program in the vehicle 10. If one of these situations is detected, the quality criterion is also considered fulfilled, and a corresponding message is issued in step S6.

[0044] Furthermore, the vehicle data can be used to check whether at least one measured variable included in the vehicle data is recorded under multiple designations. If at least one measured variable is recorded under multiple designations, one of the designations is selected, and the vehicle software is corrected so that only the selected designation is used for the measured variable in vehicle 10. Additionally, a test maneuver and / or a test state of vehicle 10 is generated based on vehicle data that was generated according to one of the recognized data patterns.The described data evaluation also enables the vehicle data to be transmitted from the vehicle-external computing unit 30 to a simulation unit, which performs a resimulation with the transmitted vehicle data, whereupon the simulation data determined during the resimulation are compared with the transmitted vehicle data and a corresponding output signal is generated.

[0045] Overall, the examples demonstrate a vehicle matrix using artificial intelligence, that is, they show how a learned data pattern can be found in large datasets in order to evaluate vehicle data during test drives of motor vehicles particularly quickly, thoroughly and reliably.

Claims

[1] Method for evaluating vehicle data that characterizes at least one driving maneuver and / or at least one state of at least one motor vehicle (10) over a specified period of time and from which at least one motor vehicle (10) is recorded, comprising the steps: - Operating the motor vehicle (10) for the specified period of time, wherein during the operation of the motor vehicle (10) the vehicle data are recorded by at least one recording device (20) of the motor vehicle (10) and stored in a storage device (22) of the motor vehicle (10) (S1); - Transmitting the stored vehicle data to an external computing device (30) (S2); - Recognition of at least one data pattern in the vehicle data by applying a neural network to the provided vehicle data by the vehicle-external computing device (30) (S3); - Check whether the detected data pattern corresponds to a predefined reference pattern stored in the vehicle-external computing unit (30) which is assigned to at least one driving maneuver and / or at least one state of the motor vehicle (10) (S4); - Applying a predefined quality criterion to the vehicle data underlying the detected data pattern and / or other vehicle data acquired at the same time (S5), wherein the quality criterion is met if the other vehicle data acquired at the same time exhibits a predefined minimum diversity of driving situations in which it was acquired, and the other vehicle data that meet the predefined quality criterion are stored as verified test vehicle data for the detected data pattern in the vehicle-external computing device (30); - If the detected data pattern matches the stored reference pattern and / or the vehicle data meets the quality criterion, a corresponding message is issued (S6). [2] Method according to the preceding claim, wherein reference vehicle data of the at least one driving maneuver and / or the at least one state of the at least one motor vehicle (10) are provided to the vehicle-external computing device (30), wherein the neural network learns the reference pattern for the corresponding driving maneuver and / or the corresponding state based on the provided reference vehicle data and the reference pattern is stored in the vehicle-external computing device (30). [3] Method according to one of the preceding claims, wherein, if the detected data pattern does not correspond to a stored reference pattern, it is checked whether at least one of the following situations is detected based on the data pattern: - a cyber attack on the computer equipment and / or the motor vehicle (10); - misconduct by the driver of the motor vehicle (10) during the recording of the vehicle data; - a malfunctioning software program in a motor vehicle (10); wherein, if one of the situations is detected, the quality criterion is met. [4] Method according to one of the preceding claims, wherein the vehicle data is used to check whether at least one measurement variable included in the vehicle data is recorded under several designations. [5] Method according to the preceding claim, wherein if the at least one measured quantity is recorded under several designations, one of the designations is selected and a vehicle software is corrected such that only the selected designation is used for the measured quantity in the motor vehicle (10). [6] Method according to one of the preceding claims, wherein a test maneuver and / or a test state of the motor vehicle (10) is generated based on vehicle data generated according to one of the recognized data patterns. [7] Method according to one of the preceding claims, wherein the vehicle data is transmitted from the vehicle-external computing device (30) to a simulation device which performs a resimulation with the transmitted vehicle data, whereupon the simulation data determined during the resimulation are compared with the transmitted vehicle data and a corresponding output signal is generated. [8] Method according to any of the preceding claims, wherein the vehicle data are organized as protocol data units. [9] Vehicle data evaluation system comprising a vehicle-external computing device (30) and at least one motor vehicle (10) with a central storage device in which vehicle data is stored that has been recorded by the motor vehicle (10) and characterizes at least one driving maneuver and / or at least one state of the motor vehicle (10) over a specified period of time, wherein the motor vehicle (10) is designed to - to be operated for the specified period of time and to record the vehicle data from at least one recording device of the motor vehicle (10) during operation and to store it in a storage device of the motor vehicle (10); - to transmit the stored vehicle data to the vehicle-external computing unit (30); and the vehicle-external computing unit (30) is designed to, - to recognize at least one data pattern in the vehicle data by applying a neural network to the provided vehicle data; - to check whether the detected data pattern corresponds to a predefined reference pattern stored in the vehicle-external computing unit (30) which is assigned to at least one driving maneuver and / or at least one state of the motor vehicle (10); - to apply a predefined quality criterion to the vehicle data underlying the recognized data pattern and / or other vehicle data recorded at the same time, whereby the quality criterion is met if the other vehicle data recorded at the same time exhibits a predefined minimum diversity of driving situations in which it was recorded; - to store the further vehicle data that meet the specified quality criterion as verified test vehicle data for the recognized data pattern in the vehicle-external computing unit (30); and, - if the detected data pattern matches the stored reference pattern and / or the vehicle data meets the quality criterion, to issue a corresponding notification.

Images