Method and device for evaluating quality of automation function of motor vehicle

By implementing computer methods in motor vehicles, obtaining and transmitting predetermined parameters to create digital maps, the problem that the existing technology cannot evaluate the quality of new functions of automation functions and functions other than ODD is solved, and self-evaluation and ODD expansion of automation functions are realized. , improves the quality and performance of functions.

CN119032294APending Publication Date: 2024-11-26BMW AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380033364.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-05-05
Filing Date
2023-03-28
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art cannot effectively analyze and evaluate the new functions of motor vehicle automation functions and their functional quality outside the operating design domain (ODD), and cannot evaluate the quality of new automation functions through conventional methods.

Method used

By implementing a computer-implemented method in a motor vehicle, an analysis function is provided as a software module of an automation function, and a predetermined parameter is obtained through an algorithm, including uncertainty in the association between the object identified by the sensor and the trajectory, hypothetical change frequency and error detection post-test. Measurements, etc., transmit these parameters to the backend, and create digital maps to evaluate the quality of automation capabilities and ODD extensions.

Benefits of technology

Self-evaluation and ODD expansion of motor vehicle automation functions are realized, enabling the evaluation of the quality and risk of functions in specific situations, providing data for software updates to improve the performance of automation functions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119032294A_ABST
    Figure CN119032294A_ABST
Patent Text Reader

Abstract

The invention relates to a method for evaluating the quality of an automation function of a motor vehicle, said method comprising providing at least one self-evaluation function as a software module for the automation function, ascertaining predetermined variables by means of the self-evaluation function on the basis of input data and / or output data of the automation function, and transmitting the predetermined variable from the motor vehicle to the rear end.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for assessing the quality of an automated function of a motor vehicle. Additionally or alternatively, a data processing device or a data processing system designed to at least partially execute the method is provided. Additionally or alternatively, a (optionally automated) motor vehicle having a portion of the data processing device is provided. Additionally or alternatively, a computer program is provided, which includes commands that, when the computer executes the program, cause the computer to at least partially execute the method. Additionally or alternatively, a computer-readable medium is provided, which includes commands that, when the computer executes the commands, cause the computer to at least partially execute the method. Background Art

[0002] Via a mobile data connection, data or information from a (customer) vehicle can be transmitted to a backend or cloud in order to collect and analyze the data there. Conventionally, existing data interfaces are tapped and the data is sent via a stream or in a buffered manner. For cost reasons, only small amounts of signals with low frequencies can be sent.

[0003] Typical signals provided upstream are information about the availability of customer functions or automation functions (or automation systems), or information whether and at which location the customer has activated (or deactivated) such a function, or information about where problems with such a function have occurred. Said information allows an analysis of existing functions, i.e. limited to the extent that existing functions are active or activatable according to their ODD (operational design domain).

[0004] ODD can be understood as a description or determination of a specific operating area where an automated function or automated system should operate normally, including but not limited to road type, speed range, environmental conditions (weather, time of day, etc.) and / or other restrictions on the operating area. In other words, ODD can be understood as the specific conditions under which a specific driving automation system or its function should operate, including but not limited to driving mode. This can include various restrictions, such as geographical, traffic-dependent, speed-dependent and / or road-dependent restrictions.

[0005] However, new functions or functions outside their ODD cannot be analyzed because the analytical capability is linked to the activation or activatability of the function. Summary of the invention

[0006] Against the background of the prior art, the object of the present disclosure is to specify a device and a method which are each suitable for expanding the prior art.

[0007] This object is achieved by the features of the independent claims. The coordinated claims and the dependent claims relate to possible developments of the disclosure.

[0008] The object is then achieved by a method for assessing the quality of an automated function of a motor vehicle.

[0009] Quality may be understood as an objective variable or objective measure of the functional effect or quality of an automated function of a motor vehicle, in particular a subsystem of such a function (eg fusion) or a software module thereof.

[0010] The method may be a computer-implemented method, ie one, several or all steps of the method may be executed by a computer or a data processing device.

[0011] An automated function may be understood as a function or a (software and / or hardware) module which automatically or without human intervention takes over one or more predetermined tasks in a motor vehicle. In this context, an automated function may be part of a driver assistance system or represent such a driver assistance system. Examples of automated functions include object recognition, trajectory planning and / or direct and / or indirect intervention or direct and / or indirect control of the lateral and / or longitudinal guidance of a motor vehicle.

[0012] The method comprises providing at least one self-analysis function as a software module for an automated function.

[0013] The method includes determining a predetermined variable by means of a self-analyzing function, based on input data and / or output data of an automation function.

[0014] In other words, the algorithm, the so-called self-evaluation function, determines the variable or information that is determined by the algorithm design. It is not the value of the variable or information that is predetermined or predefined, but rather which variable is to be determined. In this case, it can be determined in the corresponding algorithm which signal or which data stream in the motor vehicle is to be tapped and how the tapped signal is to be analyzed in order to obtain the (predetermined) variable.

[0015] The method includes transmitting the (ascertained) predetermined variable from the motor vehicle to a rear end.

[0016] The sending or transmission of the parameters can be carried out wirelessly, for example via the Internet and / or a mobile phone network. The transmission of the parameters and the execution of the method can be carried out continuously, in particular cyclically or continuously. The transmission can also be referred to as streaming (so-called upstream).

[0017] In other words, a client function or an automated function usually consists of a number of modules, such as object recognition, road recognition, trajectory planner, etc. It is proposed to supplement existing modules (e.g. recognition functions, such as object recognition) or modules already present in the motor vehicle by self-analysis functions or "self-evaluation" functions. The task is to determine characteristic values ​​(within the scope of determining so-called predefined variables) which can determine the quality of the module.

[0018] The method can be used to analyze function expansions and function risks based on a self-assessment or self-analysis of an onboard algorithm. It is thus possible to evaluate and ultimately implement new automation functions and / or ODD expansions of existing automation functions.

[0019] If signals or information provided for and / or by an automated function already installed in a motor vehicle are used in a new way (e.g. a specific new response to a pedestrian who shows a particular behavior pattern, which is detected by an already existing automated function), the new function can be built on existing modules. Since the new automated function is not present in the vehicle, it cannot be evaluated by the customer fleet using conventional methods. However, it is possible to evaluate using the above methods to what extent the module on which the new automated function is based delivers signals or information of a quality that is sufficient for the new automated function (e.g. is pedestrian recognition good enough to be able to recognize a particular behavior pattern?). In this case, it is possible to gain an understanding that these signals are available in the required quality, for example, comprehensively or with specific restrictions (depending on the situation, location, etc.).

[0020] As described above, the method can also be used to expand the ODD of existing automation functions. The ODD is intentionally limited in the functional design, in the expectation that in certain cases the automation function cannot be provided with sufficient quality or quality (for example, depending on the road category, the automation function for adjusting objects is not provided on small roads, because in this road category, particularly difficult-to-identify objects must be considered and the installed sensor device is nominally unable to achieve this). If it is now desired to evaluate whether a function can operate outside its nominal ODD or what measures need to be taken to expand its ODD, then an evaluation of the behavior of the individual functional modules in the ODD area to be checked is necessary and can be achieved with the help of the above method by appropriately limiting the predetermined parameters or data or information that can be obtained by the self-analysis function.

[0021] The above-described method is explained in detail below with reference to possible further developments.

[0022] The predetermined variable may be an uncertainty in the association between an object recognized by the sensors of the motor vehicle and an object trajectory, a frequency of hypothesis changes in ambiguous detections of objects recognized by the sensors of the motor vehicle, a frequency of class changes of objects recognized by the sensors of the motor vehicle, and / or an a posteriori Maβ measure of erroneous detections of objects recognized by the sensors of the motor vehicle.

[0023] In other words, one may consider measures of uncertainty / ambiguity in the association of sensor objects with object trajectories, measures of the frequency of assumed changes in ambiguous sensor detections, measures of the frequency of changes in the ID of tracked entities and / or a posteriori measures of false detections (e.g., how often an object in a far region is misclassified, but this is only recognizable by observation in a near region after approaching).

[0024] The method may include: providing multiple self-analysis functions as software modules of automation functions, respectively determining predetermined parameters based on input data and / or output data of the automation functions through the self-analysis functions, determining data vectors based on the predetermined parameters determined by the self-analysis functions, and transmitting the data vectors from the motor vehicle to the back end.

[0025] The method may include ascertaining a position of the motor vehicle from which the predetermined variable is ascertained and transmitting the ascertained position together with the predetermined variable from the motor vehicle to a rear end.

[0026] The method may include: creating a digital map in the back end based on the transmitted predetermined parameters and the location transmitted together with the predetermined parameters. The digital map may then include, for example, a description of functional restrictions, risks, etc. to be expected in a specific geographical location or in a specific location, section and / or in a specific area of ​​the digital map.

[0027] The method may include executing the method a plurality of times in order to obtain a plurality of predetermined variables or a plurality of data vectors in the back end, which are determined at the same position of the motor vehicle and / or in the same driving situation. Furthermore, the method may include aggregating a plurality of predetermined variables or a plurality of data vectors in the back end, wherein the creation of the digital map may be performed based on the aggregated plurality of predetermined variables or a plurality of data vectors.

[0028] Furthermore, the creation of the digital map can also be based on the availability of the automation functions at the respective locations of the motor vehicle and / or information about the ODD areas of the automation functions. The digital map can then contain information about the availability of the automation functions at specific geographic locations or in specific locations, sections and / or in specific areas of the digital map.

[0029] In other words and with regard to the specific design described as not limiting the present disclosure, the above can be summarized as follows: a plurality of such "self-assessment" functions can be run, and the resulting data vectors can be streamed to the data backend, for example, via a permanent uplink channel. There, a map can now be created that shows the data vectors aggregated by many customer or developer fleet vehicles as well as the functional ODD areas and customer function availability information. Thus, the subsequent possibility is also generated: the correlation between the "self-assessment" vectors and the customer function behavior can be evaluated. For example, a higher ambiguity in object recognition can be associated with an erroneous braking on an object. Additionally or alternatively, risky locations and situations can be identified in which the "self-assessment" vectors indicate poor or defective behavior, which may not yet have led to a loss of functionality. For example, in a narrow single-lane curve, changes in the ID of the tracked lane can be observed primarily, but this has never led to complaints in testing because no errors occurred when resetting the lane due to the single-lane nature. Thus, risks that may be associated with narrow curves with multiple lanes are now identified, which can be quantified and evaluated using fleet data. Additionally or alternatively, it is possible to identify locations and situations where there may be opportunities, in which the "self-assessment" vector indicates good enough behavior to run existing functions or provide new functions outside its ODD. For example, a function based on good object recognition but never released for small road types can be considered. Now, what can be confirmed from these data is that the recognition quality is actually enough to expand the function to small road types. Additionally or alternatively, logical conditions can be created from the links of the "self-assessment" vector. These logics can be provided to the on-board system within the scope of a specific data collection activity through a downlink channel (i.e., from the back end to the vehicle). Thus, the on-board system creates a trigger condition, which starts uploading specific (or all available) sensor data in the time period before and / or after the triggering event. These data can then be analyzed in detail in the back end, for example, in order to analyze the reasons for the quality loss in the recognition function in a specific context. The driving function with errors in rare cases in roundabout traffic can be used as an example. Subsequently, it can be seen from the "self-assessment" vector that these situations are associated with abnormalities in lane ID changes, higher uncertainty in object recognition, and slow opposite traffic. The combination of these three elements of the "self-assessment" vector (ID change, uncertainty, oncoming traffic speed) and the location information "in roundabout" is provided to the on-board system as a trigger condition. Therefore, data from the field with these specific anomalies can be collected and analyzed, and improvements can be made in the form of software updates.

[0030] Furthermore, a data processing system is provided, which comprises means for executing the above-mentioned computer-implemented method.

[0031] The system may be a distributed system which may include a data processing device external to the vehicle (ie, a backend), for example operated by the vehicle manufacturer, and / or a data processing device connected thereto (in particular wirelessly) and arranged in or on the vehicle.

[0032] The latter data processing device can therefore be part of a motor vehicle. The data processing device of a motor vehicle can be, for example, a data logger and / or an electronic control unit (ECU). The data processing device of a motor vehicle can be an intelligent, processor-controlled unit, which can communicate with other modules, for example, via a central gateway (CGW) and, if necessary, can form a vehicle on-board network via a fieldbus, such as a CAN bus, LIN bus, MOST bus and FlexRay or via automotive Ethernet, for example together with a telematics control unit. The data processing device can be connected to the sensor system, for example, via the above-mentioned on-board network.

[0033] It is conceivable that the data processing device comprises a storage unit in which the collected data and / or the ascertained variables are buffered before they are sent to the external data processing device. It is conceivable that the data processing device sends, in particular streams (optionally without buffering), the collected data to a data processing device operated by the vehicle manufacturer.

[0034] The above description regarding the method also applies analogously to the data processing system, and vice versa.

[0035] Furthermore, a motor vehicle having a data processing device may be provided, wherein the data processing device is designed to execute a method for evaluating the quality of an automation function of a motor vehicle, the method comprising: determining a predetermined variable by a self-analysis function based on input data and / or output data of the automation function, and transmitting the predetermined variable from the motor vehicle to a back end.

[0036] The motor vehicle may be a passenger vehicle, in particular a car. The motor vehicle may be an automated motor vehicle, which may be designed to at least partially and / or at least temporarily take over longitudinal guidance and / or transverse guidance during automatic driving of the motor vehicle. Automatic driving may be performed such that the movement of the motor vehicle is performed (largely) autonomously.

[0037] The motor vehicle may be a motor vehicle of autonomy level 0, ie the driver takes over the task of dynamic driving even when supporting systems (eg ABS or ESP) are present.

[0038] The motor vehicle may be an autonomy level 1 motor vehicle, ie, has a specific driver assistance system, such as an adaptive cruise control system (ACC), which supports the driver in the operation of the vehicle.

[0039] The motor vehicle may be an autonomy level 2 motor vehicle, ie semi-autonomous, so that some functions, such as automatic parking, lane keeping or lateral guidance, general longitudinal guidance, acceleration and / or deceleration, are taken over by the driver assistance system.

[0040] The motor vehicle may be a motor vehicle of autonomy level 3, i.e. conditionally automated, so that the driver does not have to continuously monitor the vehicle systems. The motor vehicle automatically performs some functions, such as triggering turn signals, lane changes and / or lane keeping. The driver can be engaged in other matters, but is asked by the system to take over the steering if necessary within the warning time.

[0041] The motor vehicle may be a motor vehicle of autonomy level 4, ie highly automated, so that the control of the vehicle is continuously taken over by the vehicle system. If the system is no longer competent for the driving task, the driver may be asked to take over the control.

[0042] The motor vehicle may be a motor vehicle of autonomy level 5, ie fully automated, so that the driver does not need to perform driving tasks. No human intervention is required except for determining the target and activating the system.

[0043] The above descriptions regarding the method and the data processing system also apply analogously to the motor vehicle and vice versa.

[0044] A computer program may also be provided. The computer program is characterized in that the computer program includes commands, which prompt the computer to at least partially execute the above method when the computer executes the program.

[0045] The program code of the computer program may be present as any desired code (including, for example, LUA scripts), in particular as a code suitable for controlling a motor vehicle.

[0046] In addition, a computer-readable medium, in particular a computer-readable storage medium, is provided. The computer-readable medium is characterized in that the computer-readable medium comprises a command, which causes the computer to at least partially perform the above method when the computer executes the program.

[0047] That is, a computer readable medium comprising the computer program defined above may be provided. The computer readable medium may be any digital data storage, such as a USB stick, hard disk, CD-ROM, SD card or SSD card.

[0048] The computer program does not necessarily have to be stored on such a computer-readable storage medium in order to be provided to the motor vehicle, but can also be obtained externally via the Internet or other means.

[0049] The above descriptions with respect to the method, the data processing system and the motor vehicle also apply analogously to the computer program and the computer-readable medium and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Then, refer to Figure 1 and Figure 2 An implementation scheme is described.

[0051] Figure 1 A data processing system is schematically shown, which comprises means for carrying out a method for assessing the quality of an automated function of a motor vehicle, and

[0052] Figure 2 A flow chart of the method is schematically shown. DETAILED DESCRIPTION

[0053] Figure 1 The system 1 shown has a backend 2 and a motor vehicle 3 connected to the backend via a wireless data connection 4. The motor vehicle 3 has a data processing device 31 and a module or a part 32 connected to the data processing device (e.g. a driver assistance system) which is designed to execute automated functions (e.g. object recognition based on sensor data of a sensor system (not shown) of the motor vehicle 3). The system 1 is designed to execute the following also referred to Figure 2 Describes a method for assessing the quality of automated functions.

[0054] In a first step S1 of the method, a plurality of self-analyzing functions are provided for the automation function, each as a software module or collected as a (single) software module. For this purpose, the self-analyzing functions can be transmitted from the backend 2 to the motor vehicle 3, more precisely to the data processing device 31, for example, via a wireless data connection 4.

[0055] In a second step S2 of the method, at least one predetermined variable is determined by a self-analysis function executed by the data processing device 31, based on input data and / or output data of the automation function. The input data can be sensor data, such as image data. The output data can be, for example, the result of the sensor data analysis performed by the automation function. The predetermined variable can be an uncertainty in the association of an object recognized by the motor vehicle's sensors with an object trajectory, a frequency of assumed changes in ambiguous detections of objects recognized by the motor vehicle's sensors, a frequency of class changes of objects recognized by the motor vehicle's sensors, and / or an a posteriori measure of erroneous detections of objects recognized by the motor vehicle's sensors. In addition, the position of the motor vehicle at which the predetermined variable is determined or determined. In addition, the availability of the automation function at the corresponding position of the motor vehicle 3, i.e. at the time of determining the predetermined variable, and information about the ODD area of ​​the automation function are determined.

[0056] In a third step S3 of the method, a data vector is ascertained by data processing device 31 , which includes the predetermined variables ascertained by the self-evaluation function, the ascertained position of the motor vehicle, the availability of the automation function at the time of ascertaining the position and the predetermined variables, and information about the ODD range of the automation function.

[0057] In a fourth step S4 of the method, the data vector is transmitted from the motor vehicle 3 to the back end 2 .

[0058] Here, the above steps S1-S4 are repeated or performed multiple times by multiple motor vehicles in the motor vehicle fleet to which motor vehicle 3 belongs, so that predetermined parameters and data vectors based on the predetermined parameters are determined multiple times at the same position (at different time points) and / or in the same driving situation (for example, driving through a roundabout), and the predetermined parameters and data vectors are transmitted from the corresponding motor vehicles to the back end 2.

[0059] In a fifth step S5 of the method, a plurality of data vectors are combined in the back end 2. This can be achieved, for example, by forming a mean value of the variables determined and contained in the data vectors.

[0060] In a sixth step S6 of the method, an existing digital map is created or supplemented based on the aggregated plurality of data vectors.

[0061] In a seventh step S7 of the method, the quality of the automation function is evaluated using the digital map according to predetermined or predefined criteria and the automation function is adapted, for example by expanding its ODD area and / or further automation functions are ascertained based on the evaluation results.

[0062] In an eighth step S8 of the method, adapted and / or additional automation functions can be implemented in motor vehicles 3 , in particular in the entire motor vehicle fleet to which motor vehicle 3 belongs, by means of a software update via wireless data connection 4 .

[0063] Reference numerals list

[0064] 1 Data Processing System

[0065] 2 Backend

[0066] 3 Motor Vehicles

[0067] 31 Data processing device

[0068] 32 modules with automation functions

[0069] 4 Wireless data connection

[0070] S1-S8 Method Steps

Claims

1. A method for evaluating the quality of an automated function of a motor vehicle (3), wherein: The method comprises: - providing at least one self-analysis function as a software module of said automation function, - determining a predetermined variable by means of the self-analysis function based on input data and / or output data of the automation function, and - transmitting the predetermined variable from the motor vehicle (3) to the rear end (2).

2. The method according to claim 1, wherein: The predetermined parameters are: - uncertainty in the association of objects detected by sensors of the motor vehicle (3) with object trajectories, - a frequency of hypothetical changes in the ambiguous detection of objects detected by sensors of the motor vehicle (3), - the frequency of changes in the category of objects detected by sensors of the motor vehicle (3), and / or - an a posteriori measure of an erroneous detection of an object detected by a sensor of the motor vehicle (3).

3. The method according to claim 1 or 2, wherein: The method comprises: - providing a plurality of said self-analysis functions as software modules of said automation functions, - determining a predetermined variable in each case by means of the self-analysis function based on the input data and / or the output data of the automation function, - determining a data vector based on the predetermined parameters determined by the self-analysis function, and - transmitting the data vector from the motor vehicle (3) to the back end (2).

4. The method according to any one of claims 1 to 3, wherein: The method comprises: - determining the position of the motor vehicle (3) from which the predetermined variable is determined, and - the ascertained position is transmitted together with the predetermined variable from the motor vehicle (3) to the rear end (2).

5. The method according to any one of claims 1 to 4, wherein: The method comprises creating a digital map in the back end based on the transmitted predetermined parameter and the position transmitted together with the predetermined parameter.

6. The method according to claim 5, wherein: The method comprises: - executing the method according to any one of claims 1 to 4 several times in order to obtain a plurality of predetermined variables or a plurality of data vectors in the rear end (2), said predetermined variables or said data vectors being ascertained at the same position of the motor vehicle (3) and / or in the same driving situation, and - aggregating a plurality of said predetermined parameters or a plurality of said data vectors in said back end (2), - wherein the creation of the digital map is performed based on an aggregated plurality of the predetermined parameters or a plurality of the digital vectors.

7. The method according to claim 5 or 6, wherein: The digital map is also created based on the availability of the automated functions at the respective locations of the motor vehicle (3) and / or information about the ODD areas of the automated functions.

8. A data processing system (1) comprising means for executing the method according to any one of claims 1 to 7.

9. A computer program comprising instructions, which, when a computer executes the program, cause the computer to execute the method according to any one of claims 1 to 7.

10. A computer-readable medium comprising instructions, which, when a computer executes the instructions, cause the computer to perform the method according to any one of claims 1 to 7.