Method, device and computer program product for operating a vehicle
By designing a confidence model to evaluate the confidence of sensor data and verify vehicle sensor data, the problem of insufficient reliability and safety of vehicle sensor data is solved, and the safety and reliability of vehicle object detection is improved.
Patent Information
- Application Number
- CN201980087893.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-02-06
- Filing Date
- 2019-10-29
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-10-29
AI Technical Summary
The reliability and safety of existing vehicle sensor data is difficult to effectively verify and improve, resulting in insufficient safety of vehicles in object detection.
By designing a confidence model to evaluate the confidence of sensor data, and verify and verify the sensor data based on confidence values, improving the quality marking of object detection, thereby enhancing the safety of the vehicle. The confidence model combines sensor type, object state and vehicle state to fuse multiple sensor data to improve detection reliability.
It improves the safety and reliability of the vehicle in object detection, and verifies sensor data through confidence models, enhancing the efficiency and safety of vehicle functions.
Smart Images

Figure CN113302108B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for operating a vehicle. The invention also relates to a device for operating a vehicle. The invention also relates to a computer program product for operating a vehicle. Background Art
[0002] Sensors in a vehicle may provide the vehicle with data regarding object detection that may be used by vehicle functions. Summary of the invention
[0003] The object of the present invention is to contribute to an increased safety of vehicles.
[0004] This object is achieved by the features of the invention.
[0005] According to a first aspect, the invention features a method for operating a vehicle.
[0006] In a method according to the first aspect, sensor data of an object detection by a sensor of a vehicle is provided, the object detection representing an identified object. In addition, a confidence model is provided, the confidence model being designed to model the confidence of the object detection. A confidence value for the identified object is determined based on the sensor data and the confidence model, the confidence value for the identified object representing how high the confidence in the identified object is. The vehicle is operated based on the confidence value for the identified object.
[0007] The sensor data can thus be checked with the aid of a confidence model and a confidence value for the identified object and verified and / or checked. A quality mark can thus be assigned to the object detection in order to increase or decrease its reliability. Furthermore, if the sensor data are used by a vehicle function of the vehicle, the safety of the vehicle can thereby be increased, for example. For example, the sensor data can also be used by a plurality of vehicle functions of the vehicle.
[0008] The sensor data may include, for example, one or more data records, which include object detections of sensors of the vehicle. The sensor data may also include, for example, a plurality of object detections of a sensor, each of which represents a corresponding identified object. The sensor data may also include, for example, one or more object detections of different sensors of the vehicle, for example, the method being performed for the corresponding object detections. For example, a plurality of object detections of different sensors of the vehicle may be fused with the aid of a confidence model and confidence values, for example in a sensor fusion system. The sensor fusion system is designed, for example, to fuse a first object detection of a first sensor of the vehicle with a second object detection of a second sensor of the vehicle, if the first and second object detections represent the identified object.
[0009] For example, sensor data are provided for a data processing cycle respectively. The data processing cycle depends, for example, on the sampling frequency of the sensor and / or the processing time of the sensor. For example, the sensors of the vehicle and / or the different sensors are heterogeneous sensors. The sensors of the vehicle and / or the different sensors include, for example, camera sensors and / or radar sensors and / or sensors for navigation, such as GPS receivers, and / or laser sensors, such as Lidar sensors and / or Ladar sensors, and / or ultrasonic sensors and / or infrared sensors, etc.
[0010] The vehicle has, for example, sensors of the vehicle and / or different sensors of the vehicle. For example, the vehicle may also have a plurality of sensors of the same type.
[0011] The identified object may represent, for example, a moving object. In addition, the identified object may also represent, for example, a stationary object. The identified object may represent, for example, an object in road traffic, such as an animal, an obstacle, a road boundary, a car, a passerby, a wall, etc.
[0012] The confidence model may include, for example, one or more confidence sub-models. The one or more confidence sub-models are designed to model the confidence of object detection or to model at least one corresponding partial aspect of the confidence of object detection. For example, the confidence of object detection is modeled according to the type of sensor and / or the state of object detection and / or the state of the vehicle, etc.
[0013] For example, the confidence value of the identified object is determined based on the confidence model and / or the confidence sub-model. The confidence value of the identified object may, for example, include a value on a continuous scale. In addition, the confidence value of the identified object may also have multiple values, which, for example, respectively represent one of the confidence sub-models.
[0014] For example, operating the vehicle includes executing a vehicle function of the vehicle, which uses sensor data as a function of the confidence value of the detected object. For example, the vehicle function uses a tracking object detection and / or object tracking and / or tracking management method for this purpose, as described in "Y. Bar-Shalom, XR Li, 1995, Multi-Object-Multi-Sensor Tracking: Principles and Techniques, ISBN-13: 978-0964831209".
[0015] For example, an object list provided to a vehicle function of the vehicle is determined based on sensor data and / or a confidence model and / or a confidence value of the recognized object. The object list includes, for example, information and status of the recognized object.
[0016] For example, an initial confidence value of the identified object is determined based on standardized parameters used in association with object tracking and / or tracking management methods. The initial confidence value of the identified object represents, for example, how high the initial confidence in the identified object is when the identified object is first identified. For example, the initial confidence value of the identified object is additionally determined based on sensor data of the first identification of the identified object. In addition, for example, an extended confidence value of the identified object can be determined based on the confidence value of the identified object and the initial confidence value of the identified object. For example, when determining the extended confidence value of the identified object, the initial confidence value of the identified object is fused with the confidence value of the identified object in a sensor fusion system. For example, sensor data records of multiple object detections including different sensors can be fused in a sensor fusion system based on the confidence value of the identified object and / or the extended confidence value of the identified object.
[0017] According to an optional embodiment, information about the sensor type is provided. In addition, the confidence model includes a sensor confidence model, which is designed to model the confidence of the object detection according to the type of the corresponding sensor. The confidence value of the identified object is determined based on the sensor data, the information about the sensor type and the confidence model.
[0018] Thus, the object detection of the sensor can be considered in a sensor-type dependent manner using the confidence value of the detected object according to the sensor confidence model. Thus, for example, different sensors can have different weights in sensor fusion.
[0019] For example, depending on the respective sensor type and the sensor confidence model, object detection is taken into account to varying degrees when determining the confidence value for a detected object.
[0020] For example, the sensor confidence model comprises a simulation model and / or a mathematical model and / or characteristic data or the like of the corresponding sensor.
[0021] According to another optional embodiment, object state information is provided, the object state information comprising information about the state of the identified object during object detection. In addition, the confidence model comprises a state confidence model, the state confidence model being designed to model the confidence of the object detection based on the corresponding state of the identified object during object detection. A confidence value of the identified object is determined based on the sensor data, the object state information and the confidence model.
[0022] Thus, object detection by the sensor can be taken into account to different degrees depending on the state of the identified object in relation to different states of the identified object and / or other objects using the confidence value of the identified object according to the state confidence model.
[0023] For example, object detection is taken into account to varying degrees when determining the confidence value of the identified object, depending on the respective state of the identified object and the state confidence model.
[0024] For example, the object status information includes the position of the identified object and / or the orientation of the identified object and / or the speed of the identified object, etc. For example, the object status information additionally includes information about a predetermined area in the vehicle environment. The predetermined area in the vehicle environment depends on the detection range of the sensor and / or represents a statistical accumulation of false alarms.
[0025] According to another optional embodiment, vehicle state information is provided, the vehicle state information comprising information about the state of the vehicle during object detection. In addition, the confidence model comprises an environmental confidence model, which is designed to model the confidence of the object detection based on the corresponding state of the vehicle during object detection. The confidence value of the identified object is determined based on the sensor data, the vehicle state information and the confidence model.
[0026] Thus, object detection by the sensor can be taken into account to varying degrees depending on the state of the vehicle using the confidence values of the detected objects according to the environmental confidence model, depending on the state of the vehicle.
[0027] For example, depending on the respective state of the vehicle and the surrounding confidence model, object detection is taken into account to varying degrees when determining the confidence value for a detected object.
[0028] For example, the vehicle status information includes information about the type of road currently being traveled on and / or current visibility conditions and / or vehicle speed.
[0029] According to another optional embodiment, sensor range information is provided, the sensor range information including range-related information of the sensor. In addition, an initial position of the identified object is provided, the initial position of the identified object representing the position of the identified object when it is first identified. In addition, the confidence model includes an initial confidence model, the initial confidence model is designed to model the confidence of the object detection based on the corresponding initial position of the identified object and the corresponding range-related information of the sensor. A confidence value of the identified object is determined based on the sensor data, the sensor range information, the initial position of the identified object and the confidence model.
[0030] Thus, the object detection of the sensor may be considered to different extents in dependence on different initial positions of the identified object by means of the confidence value of the identified object according to the initial confidence model depending on the range-related information of the sensor and the initial position of the identified object.
[0031] For example, object detection is taken into account to varying degrees when determining the confidence value of the identified object, depending on the respective initial position of the identified object and the respective range-related information of the sensor and the initial confidence model.
[0032] For example, the initial confidence model includes information about the detection range of the sensor and / or the statistical false detection probability related to the range of the sensor. In addition, the initial confidence model may also include one or more assumptions about the vehicle environment. For example, the assumption about the vehicle environment includes that at least two identified objects cannot completely or partially overlap each other.
[0033] According to another optional embodiment, a property of the object detection is determined based on the sensor data and / or the confidence value of the identified object. The property of the object detection represents the quality and / or the confidence level of the object detection. The vehicle is operated based on the property of the object detection.
[0034] For example, the object detection property corresponds to one of the standardized parameters used in connection with the object tracking and / or tracking management method. In addition, for example, a plurality of properties of the object detection can also be determined.
[0035] Thus, standardized parameters used in connection with object tracking and / or tracking management methods may be determined based on sensor data and / or confidence values of identified objects.
[0036] Standardized parameters used in connection with object tracking and / or tracking management methods are not determined from a unified database and / or depend to a large extent on the algorithms of sensor manufacturers, for example. This has a negative impact, for example, in sensor fusion systems. By determining object detection attributes based on confidence values of identified objects, sensor fusion systems can be improved and contribute to increasing vehicle safety.
[0037] For example, an object list is additionally determined based on the properties of the object detection and is provided to a vehicle function of the vehicle. The object list additionally includes the properties of the object detection, for example. The properties of the object detection may also include a plurality of properties.
[0038] According to another optional embodiment, the attributes of the object detection include a probability of existence, which represents how high the statistical probability is that the identified object exists, and / or a quality level or quality standard of the sensor, and / or one or more standard deviations from the corresponding state of the identified object during object detection.
[0039] Thus, object detection properties determined from sensor data and / or confidence values of identified objects may be used in connection with object tracking and / or tracking management methods.
[0040] For example, vehicle functions of the vehicle including object tracking and / or tracking management methods may operate the vehicle based on sensor data and / or confidence values of identified objects.
[0041] According to another optional embodiment, when the identified object is identified for the first time, a first object data record of the identified object is created. The first object data record includes a first object confidence value and a unique first object identification number representing the identified object. In addition, the first object confidence value is determined based on the confidence value of the identified object. The vehicle is operated based on the first object data record.
[0042] As a result, the detected object can be taken into account during the first detection during operation of the vehicle.
[0043] For example, the first object data record is additionally stored after the determination. For example, the first object confidence value is determined in such a way as to correspond to the confidence value of the identified object as a function of the confidence value of the identified object. For example, the stored first object data record is additionally taken into account when determining the confidence value in a further object detection, which is provided temporally after the object detection. For example, the stored first object data record is adjusted in the further object detection. For example, the stored first object data record is deleted in the further object detection. For example, the first object data record is additionally determined as a function of properties of the object detection.
[0044] For example, an object list is additionally determined based on the first object data record and is made available to a vehicle function of the vehicle.
[0045] According to another optional embodiment, a second object data record of a second object is provided. The second object data record includes a second object confidence value and a unique second object identification number representing the second object. In addition, it is determined whether the identified object is assignable to the second object. If the identified object is assignable to the second object, the second object confidence value is adjusted according to the confidence value of the identified object. For example, if the identified object is assignable to the second object, the second object confidence value is additionally adjusted according to the properties of the object detection. If the identified object cannot be assigned to the second object, the second object confidence value is adjusted. The vehicle is operated according to the second object data record.
[0046] As a result, the provided second object data record can be taken into account when operating the vehicle.
[0047] For example, the second object data record is additionally stored after the adjustment. For example, the second object data record additionally includes other information, such as the observation time of the identified object. The observation time represents the duration from the first recognition of the identified object to the detection of the object. For example, if the identified object can be assigned to the second object, the second object confidence value is adjusted in such a way that the second object confidence value is determined based on the confidence value of the identified object and the observation time of the identified object. For example, if the identified object cannot be assigned to the second object, the second object confidence value is adjusted in such a way that the second object confidence value is reduced, for example, in value. For example, if the identified object cannot be assigned to the second object, the second object data record is deleted. For example, the second object data record is deleted based on a comparison of the second object confidence value with a threshold value.
[0048] For example, the second object data record may also include a plurality of second object data records representing different second objects.
[0049] For example, an object list is additionally determined based on the second object data record and is made available to a vehicle function of the vehicle.
[0050] According to a second aspect, the invention is characterized by a device for operating a vehicle. The device is designed to carry out the method for operating a vehicle according to the first aspect.
[0051] According to another aspect, the invention features a computer program comprising instructions which, when executed by a computer, cause the computer to perform a method for operating a vehicle when the program is executed on a data processing device. According to another aspect, the invention features a computer program product comprising an executable program code which, when executed by a data processing device, performs a method for operating a vehicle.
[0052] The computer program product comprises, in particular, a medium which can be read by a data processing device and on which the program code is stored. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The embodiments of the present invention are described in detail below with reference to the schematic diagrams.
[0054] Figure 1 A flow chart for operating a vehicle is shown. DETAILED DESCRIPTION
[0055] Figure 1 A flow chart showing a program for operating a vehicle.
[0056] The program begins in step S1 , in which variables are initialized, if necessary.
[0057] The procedure continues in step S3. In step S3, object detections of the vehicle sensors are determined, the object detections representing recognized objects.
[0058] The procedure continues in step S5. In step S5, sensor data comprising object detections of vehicle sensors are provided.
[0059] The procedure continues in step S7. In step S7, a confidence model is provided, which is designed to model the confidence of the object detection.
[0060] Optionally, information about the sensor type is provided in step S7. In addition, the confidence model comprises a sensor confidence model, which is designed to model the confidence of the object detection according to the type of the corresponding sensor.
[0061] Optionally, in step S7, object state information is provided, the object state information comprising information about the state of the identified object during object detection. In addition, the confidence model comprises a state confidence model designed to model the confidence of object detection according to the corresponding state of the identified object during object detection.
[0062] Optionally, vehicle state information is provided in step S7, the vehicle state information comprising information about the state of the vehicle during object detection. In addition, the confidence model comprises an environmental confidence model, which is designed to model the confidence of object detection according to the corresponding state of the vehicle during object detection.
[0063] Optionally, in step S7, sensor range information is provided, the sensor range information comprising range-related information of the sensor. In addition, an initial position of the identified object is provided, the initial position of the identified object representing the position of the identified object when it is first identified. In addition, the confidence model comprises an initial confidence model, the initial confidence model being designed to model the confidence of the object detection based on the corresponding initial position of the identified object and the corresponding range-related information of the sensor.
[0064] The program continues in step S9. In step S9, a confidence value of the identified object is determined based on the sensor data and the confidence model, and the confidence value of the identified object represents how high the confidence of the identified object is.
[0065] Optionally, in step S9 , a confidence value for the identified object is additionally determined based on information about the sensor type.
[0066] For example, the false detection rate of the sensor is determined with the aid of a sensor confidence model. For example, there is a first sensor of a first type, for which a very high first false detection rate is determined. Object detection of the identified object by the first sensor determines a first confidence value for the identified object. For example, there is a second sensor of a second type, for which a second, lower false detection rate is determined. Object detection of the identified object by the second sensor determines a second confidence value for the identified object. In this case, the first confidence value for the identified object is numerically lower than the second confidence value for the identified object.
[0067] Optionally, in step S9 , a confidence value of the identified object is additionally determined based on the object status information.
[0068] For example, the confidence value of the identified object is determined by means of a state confidence model depending on whether the position of the identified object is located in a predetermined area in the vehicle environment. If the position of the identified object is located in the predetermined area in the vehicle environment, the confidence value of the identified object is, for example, numerically greater than when this is not the case. For example, the sensor predetermines the predetermined area specifically for the respective type of sensor. For example, when determining the confidence value of the identified object, the near-field sensor is correspondingly weighted more in the detection of objects at close range than in the detection of objects at long range.
[0069] Optionally, in step S9 , a confidence value of the identified object is additionally determined based on the vehicle state information.
[0070] For example, the vehicle state information additionally includes information about the current weather conditions, which represents heavy rain, for example. For example, the confidence value of the detected object determined by the camera sensor in heavy rain is smaller in value than that determined by the radar sensor based on the current weather conditions using the environmental confidence model.
[0071] Optionally, in step S9 , a confidence value of the identified object is additionally determined based on the sensor range information and the initial position of the identified object.
[0072] For example, it is possible to deduce with the aid of the confidence model and the confidence value whether there is a false detection. A false detection means, for example, that the sensor has recognized an object, which is not actually located in the vehicle's environment. In addition, it is also possible to deduce, for example, whether there is a missed detection. A missed detection means, for example, that the sensor has not recognized another object, although the other object is actually located in the vehicle's environment.
[0073] Optionally, the procedure continues in step S11. In optional step S11, properties of the object detection are determined based on the sensor data and / or the confidence value of the identified object. The properties of the object detection represent the quality and / or the confidence of the object detection.
[0074] The attributes of an object detection include a presence probability representing how high the statistical probability is that the identified object is present, and / or a quality level or quality standard of the sensor and / or one or more standard deviations from the corresponding state of the identified object during object detection.
[0075] For example, the object detection attribute corresponds to one of the standardized parameters used in connection with object tracking and / or tracking management methods.
[0076] Optionally, the procedure continues in step S13. In optional step S13, a second object data record of a second object is provided. The second object data record comprises a second object confidence value and a unique second object identification number representing the second object.
[0077] For example, the second object data record represents a second track used in connection with an object tracking and / or track management method. The second track represents a map or image of the recognized objects actually located in the vehicle environment and includes information about the recognized objects.
[0078] For example, the second object data record corresponds to a previously stored first object data record.
[0079] Optionally, the program continues in step S15. In optional step S15, it is determined whether the identified object can be assigned to a second object. If this is the case, the processing continues in step S17, otherwise it continues in step S19.
[0080] In step S17, the second object confidence value is adjusted according to the confidence value of the identified object. After step S17, the program continues in step S23.
[0081] Optionally, a second object data record is additionally stored after the adaptation.
[0082] In step S19, a first object data record of the identified object is created when the identified object is identified for the first time. The first object data record includes a first object confidence value and a unique first object identification number representing the identified object. In addition, the first object confidence value is determined based on the confidence value of the identified object.
[0083] For example, the first object data record represents a first trajectory used in connection with the object tracking and / or tracking management method. The first trajectory represents a map or image of the recognized object actually located in the vehicle environment and includes information about the recognized object.
[0084] Step S19 may also be performed, for example, after step S21 in terms of time.
[0085] In step S21, the second object confidence value is adjusted. Optionally, the second object data record is additionally stored after the adjustment. After step S19, the program continues in step S21. After step S21, the program continues in step S23.
[0086] In step S23, the vehicle is operated according to the confidence value of the identified object.
[0087] Optionally, in step S23 , the vehicle is additionally operated depending on the properties of the object detection.
[0088] Optionally, in step S23 , the vehicle is additionally operated according to the first object data record.
[0089] Optionally, in step S23 , the vehicle is additionally operated according to a second object data record.
[0090] For example, operating the vehicle includes executing a vehicle function of the vehicle that uses sensor data as a function of a confidence value for a detected object.
[0091] For example, the vehicle function of the vehicle is a longitudinal control vehicle function of the vehicle, such as adaptive speed control, intersection assistance, distance keeping function, etc. Alternatively or additionally, the vehicle function of the vehicle is, for example, a lateral control vehicle function of the vehicle, such as side collision assistance, steering guidance assistance, lane guidance assistance, etc. The longitudinal control vehicle function of the vehicle is designed to mainly affect the movement along the vehicle extension plane. The lateral control vehicle function of the vehicle is designed to mainly affect the movement transverse to the vehicle extension plane. The longitudinal control vehicle function of the vehicle and / or the lateral control vehicle function of the vehicle, for example, include requirements including corresponding false detection rates and / or corresponding missed detection rates. The requirements including the corresponding false detection rate, for example, represent the corresponding predetermined number of false detections that should not be exceeded within the corresponding predetermined time interval and / or on the corresponding predetermined route. The requirements including the corresponding missed detection rate, for example, represent the corresponding predetermined number of missed detections that should not be exceeded within the corresponding predetermined time interval and / or on the corresponding predetermined route. For example, the requirements including the corresponding false detection rate and / or the corresponding missed detection rate can be checked based on the confidence value of the identified object and / or the sensor data. The check determines, for example, whether a longitudinal control vehicle function of the vehicle and / or a transverse control vehicle function of the vehicle can use the sensor data, for example to ensure and / or increase the safety of the vehicle. The check determines, for example, whether a longitudinal control vehicle function of the vehicle and / or a transverse control vehicle function of the vehicle can be used sequentially.
[0092] Then, the program starts again in step S3.
Claims
1. A method for operating a vehicle, wherein: - providing sensor data, the sensor data comprising object detections of sensors of the vehicle, the object detections representing identified objects, - provide a confidence model designed to model the confidence of object detections, - determining a confidence value of the identified object based on the sensor data and the confidence model, wherein the confidence value of the identified object represents how high the confidence in the identified object is, and - operating the vehicle as a function of the confidence values of the identified objects, wherein the sensor data can be checked and verified with the aid of a confidence model and the confidence values of the identified objects, - determining a property of the object detection based on the sensor data and / or the confidence value of the identified object, the property of the object detection representing the quality and / or confidence of the object detection, and - operating the vehicle based on attributes of object detection, wherein the attributes of object detection include: a probability of presence representing how high a statistical probability the identified object is present, and / or a quality level of a sensor, and / or one or more standard deviations from a corresponding state of the identified object during object detection.
2. The method according to claim 1, wherein: - provide information about the sensor type, - the confidence model comprises a sensor confidence model designed to model the confidence of the object detection according to the type of the corresponding sensor, and - Determining a confidence value for the identified object based on the sensor data, information about the sensor type and the confidence model.
3. The method according to claim 1, wherein: - providing object state information comprising information about the state of an object identified during object detection, - the confidence model comprises a state confidence model designed to model the confidence of the object detection according to the corresponding state of the object being identified during the object detection, and -Determine a confidence value for the identified object based on sensor data, object state information, and a confidence model.
4. The method according to any one of claims 1 to 3, wherein: - providing vehicle state information, said vehicle state information comprising information about the state of the vehicle during object detection, - the confidence model includes an environment confidence model designed to model the confidence of the object detection based on the corresponding state of the vehicle during the object detection, and -Determine a confidence value for the identified object based on sensor data, vehicle state information, and a confidence model.
5. The method according to any one of claims 1 to 3, wherein: - providing sensor range information, the sensor range information comprising range-related information of the sensor, - providing an initial position of the identified object, where the initial position of the identified object represents the position of the identified object when it is first identified, - the confidence model comprises an initial confidence model designed to model the confidence of the object detection based on the corresponding initial position of the identified object and the corresponding range-related information of the sensor, and -Determine a confidence value for the identified object based on the sensor data, the sensor range information, the initial position of the identified object, and the confidence model.
6. The method according to any one of claims 1 to 3, wherein: When the object is first identified - creating a first object data record for the identified object, said first object data record comprising a first object confidence value and a unique first object identification number representing the identified object, - determining a first object confidence value based on the confidence value of the identified object, and - operating the vehicle according to the first object data record.
7. The method according to claim 6, wherein: - providing a second object data record for a second object, the second object data record comprising a second object confidence value and a unique second object identification number representing the second object, - determining whether the identified object is assignable to a second object, - if the identified object is assignable to the second object, adjusting the second object confidence value according to the identified object confidence value, - if the identified object cannot be assigned to the second object, adjusting the second object confidence value, and - operating the vehicle according to the second object data record. 8 . A device for operating a vehicle, the device being designed to carry out the method according to claim 1 .
9. Computer program product comprising an executable program code which, when executed by a data processing device, performs the method according to any one of claims 1 to 7.
