Method for formulating control instructions for a motor vehicle
The calculation unit processes sensor data, evaluates the reliability of control instructions, solves the problem of misunderstanding of motor vehicle environmental detection, realizes a safer driving assistance system, and meets the requirements of high safety levels.
Patent Information
- Application Number
- CN201980071677.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-11-02
- Filing Date
- 2019-10-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2039-10-29
AI Technical Summary
The driver assistance system of existing motor vehicles is prone to misunderstandings in environmental testing, which causes the vehicle to mistake dangerous objects as harmless obstacles, and then fails to take emergency braking and other measures, resulting in safety risks.
The calculation unit receives the original data obtained by the sensor, processes it to derive environmental information and calculate the probability coefficient of errors in each information, estimate the reliability of the control instructions based on the sensor quality coefficient, and decide whether to correct the control instructions based on the reliability to ensure that the sensor is Safe use under working conditions.
It effectively reduces environmental detection errors, improves the accurate understanding of the environment of motor vehicles, ensures safe driving under unsafe conditions, and meets the ASIL D safety level of ISO26262 standard.
Smart Images

Figure CN112930290B_ABST
Abstract
Description
Technical Field
[0001] The present invention generally relates to driver assistance for motor vehicles.
[0002] The invention relates more particularly to a method for formulating control commands for one or more transmission components of a motor vehicle, such as a steering system, a braking system or a powertrain.
[0003] The invention also relates to a motor vehicle comprising a computing unit and a plurality of sensors suitable for acquiring raw data relating to the environment of the motor vehicle.
[0004] The invention is more particularly applicable to vehicles equipped with an autonomous driving mode. Thus, the invention may be applicable in the automotive, aeronautical and aerospace fields. Background Art
[0005] In order to facilitate the driving of motor vehicles and make them safer, it is known to equip the latter with driver assistance systems. These may be systems that allow the vehicle to be driven autonomously (without human intervention), or systems that allow the vehicle to be driven partially autonomously (generally systems suitable for temporarily controlling the vehicle, for example to apply emergency brakes or to return the vehicle to the center of its lane).
[0006] In order for these systems to understand the environment around the vehicle, multiple sensors (such as cameras and RADAR sensors, LIDAR sensors, SONAR sensors, etc.) are placed on the vehicle.
[0007] Each sensor has its qualities and drawbacks. In order to minimize environmental detection errors, it is then known to perform "data fusion", that is to say to consider the data transmitted by different sensors in order to derive a single environmental data therefrom. Thus, it is possible to benefit from the qualities of each sensor.
[0008] Unfortunately, even then, errors can still occur, i.e., the vehicle misinterprets a situation: for example, the vehicle may mistake a hazardous object for a harmless obstacle and therefore fail to command emergency braking.
[0009] So try to reduce these errors. Summary of the invention
[0010] The invention provides a new method and a new system which meet the functional safety level ASIL D (acronym for “Automotive Safety Integrity Level D”) as defined in the standard ISO 26262.
[0011] More specifically, according to the present invention, a method for controlling a motor vehicle is proposed, the method comprising the following steps:
[0012] - a computing unit of the vehicle receives raw data acquired by sensors of the vehicle and relating to the environment of the vehicle,
[0013] the calculation unit processes the raw data in order to derive therefrom information relating to the environment of the motor vehicle and probability coefficients relating to the probability of an error in deriving each item of information,
[0014] - formulating a control command for the motor vehicle according to said information and said probability coefficient,
[0015] - determining, for at least a first sensor of the sensors, a quality factor related to the quality of raw data acquired by the first sensor,
[0016] - estimating the reliability of the control instruction based on the quality coefficients and the probability coefficients, and
[0017] -determining whether to correct the control instruction according to the estimated reliability of the control instruction.
[0018] Thus, by means of the invention, it is possible to take into account the operating conditions of the sensors (by determining the quality factor of these sensors) in order to decide whether control commands for the motor vehicle can be used as in complete safety.
[0019] By way of example, it can be determined whether the brightness is good enough to consider the data acquired by the camera to be of high quality. It can also be determined whether the vehicle is passing through spray to know whether the data acquired by the LIDAR sensor is of high quality.
[0020] Other advantageous and non-limiting features of the control method according to the invention are as follows:
[0021] - in the determination step, the quality coefficient of at least a first sensor of the sensors is determined based on raw data acquired by at least another sensor of the sensors and / or based on third-party data acquired by a third-party detector and related to measurement conditions of the raw data acquired by the first sensor;
[0022] - the third party detector is a light sensor, or a rain sensor, or a sensor suitable for detecting the state of the road on which the motor vehicle is travelling;
[0023] - at least one of the sensors is an image sensor, or a RADAR sensor, or a LIDAR sensor;
[0024] - during the processing step, the raw data transmitted by each sensor are processed separately from the raw data transmitted by the other sensors in order to detect objects in the environment of the motor vehicle and to classify said objects, each probability coefficient being associated with a classified object and with said sensor;
[0025] - In the processing step, after processing the raw data, the processed data are fused by taking into account each probability coefficient;
[0026] - In the processing step, after processing the raw data, the processed data are fused by taking into account each quality coefficient;
[0027] - in the estimation step, the reliability of the control instruction is also estimated according to the fusion result of the processed data; and
[0028] - The decision whether to correct the control command is also made based on redundant information from sensors different from said sensor.
[0029] The invention also relates to a motor vehicle comprising a plurality of sensors suitable for acquiring raw data relating to the environment of the motor vehicle and a computing unit suitable for implementing a control method as described above. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The following description, with reference to the accompanying drawings, given by way of non-limiting examples, will provide a better understanding of what the invention is and how it can be implemented.
[0031] In the accompanying drawings, Figure 1 is a diagram illustrating a control system suitable for implementing the method according to the invention. DETAILED DESCRIPTION
[0032] The invention applies more particularly to motor vehicles equipped with a control system that allows the vehicle to be driven autonomously, that is to say without human intervention.
[0033] More precisely, the invention relates to a method for controlling at least one transmission element of a motor vehicle.
[0034] This transmission can be formed, for example, by the power train of the motor vehicle, or by a steering device or by a braking device. In the remainder of this specification, it will be considered that all these transmissions are controlled by the computing unit of the vehicle.
[0035] Figure 1 The computing unit 20 shown in part in FIG. 1 comprises a processor, a memory and various input and output interfaces.
[0036] The computing unit is suitable for implementing different but interdependent algorithms (represented here in the form of blocks).
[0037] By means of its memory, the computing unit 20 stores a computer application consisting of a computer program comprising instructions that are executed by the processor to allow the implementation of the method that will be described below.
[0038] By means of its output interface, the computing unit 20 is connected to the drive elements 30 in such a way that it can transmit control commands C1 to these.
[0039] By means of its input interfaces, the computing unit 20 is connected to several sensors 10 , 11 , 12 , 13 (at least two sensors, but preferably more).
[0040] These can be any type of sensors.
[0041] By way of example, a motor vehicle may be equipped with a digital camera 10, a RADAR sensor 11, a LIDAR sensor 12, and a light sensor 13, which are oriented to cover all orientations around the vehicle (ie, 360 degrees).
[0042] The presence of the light sensor 13 enables the conventional function of automatically switching on the lights of the vehicle to be provided.
[0043] For their part, the presence of the other sensors 10 , 11 , 12 , hereinafter referred to as environmental sensors, is intended to ensure the function of autonomously controlling the vehicle.
[0044] Each of these environmental sensors 10, 11, 12 has qualities and drawbacks. For example, a camera can perform good obstacle detection in clear weather, but will perform poorly in low or bright light. Conversely, a RADAR sensor or LIDAR sensor can perform good obstacle detection regardless of the light, but will provide inaccurate data in the presence of water mist or bad weather (rain, fog, snow).
[0045] Here, the control command C1 transmitted to the transmission element is formulated mainly based on the raw data transmitted by the environmental sensors 10 , 11 , 12 .
[0046] Then you can refer to Figure 1 The method of formulating the control instruction C1 is described in detail.
[0047] In practice, the calculation unit 20 is programmed to implement the method described below in a recursive manner, that is to say in a cyclic manner, at fixed time intervals.
[0048] The method consists of seven main steps.
[0049] During a first step, the computing unit 20 reads the raw data already acquired by all sensors 10 , 11 , 12 , 13 .
[0050] In the example considered here, the computing unit 20 reads the raw data transmitted by the camera 10 , the RADAR sensor 11 , the LIDAR sensor 12 and the light sensor 13 .
[0051] By way of example, in the case of camera 10, the raw data is formed by the color and brightness characteristics of each pixel of the camera's light-sensitive sensor. In the case of light sensor 13, the raw data is formed by the light level measured over time.
[0052] During a second step, the acquired raw data are processed in order to derive therefrom information relating to the environment of the motor vehicle.
[0053] In practice, the raw data transmitted by the environmental sensors 10 , 11 , 12 are processed separately from one another.
[0054] The aim is to detect objects in the environment of the motor vehicle on the basis of these raw data, to classify these objects (obstacles, traffic signs, third-party vehicles, pedestrians, etc.), and to assign to each classified object S1, S2, S3 a probability coefficient P1, P2, P3 related to the probability of an error occurring during detection and in the classification of the object.
[0055] To implement this step, classification methods based on machine learning techniques such as, for example, “CNN” (“convolutional neural network”) can be used.
[0056] As an alternative or in addition, filters or any other type of suitable treatment may be used.
[0057] In general, if Figure 1 As shown, the computing unit 20 comprises three boxes B10, B11, B12, which receive as input the raw data from the camera 10, from the RADAR sensor 11 and from the LIDAR sensor 12, respectively, and transmit as output the description S1, S2, S3 of each object that has been detected and classified, respectively, and each object that has been detected and classified is associated with a probability coefficient P1, P2, P3.
[0058] During a third step, the calculation unit 20 determines a quality factor Q1, Q2, Q3 for each of the environmental sensors 10, 11, 12. This quality factor Q1, Q2, Q3 is related to the quality of the raw data acquired by the sensor in question.
[0059] In practice, this quality factor Q1 , Q2 , Q3 makes it possible to know the extent to which the external conditions are suitable for allowing the sensor in question to function correctly.
[0060] In other words, these quality coefficients Q1, Q2, Q3 make it possible to determine:
[0061] - whether the camera 10 is able to detect the object correctly, taking into account, for example, the ambient light, and
[0062] - Whether the RADAR sensor 11 and the LIDAR sensor 12 can correctly detect objects, taking into account e.g. the weather.
[0063] Each quality coefficient Q1 , Q2 , Q3 is determined from raw data acquired by the sensor in question (as indicated by the solid arrows) and also from raw data acquired by other sensors (as indicated by the dashed arrows).
[0064] Thus, the weather may be determined from the image acquired by the camera 10 and the ambient light may be acquired by the light sensor 13 .
[0065] Other sensors may of course be used, in particular to determine the weather. Thus, a rain sensor and / or an accelerometer, possibly located in the wheels of the vehicle and suitable for detecting the state of the road on which the motor vehicle is travelling, may be used.
[0066] Each quality factor Q1, Q2, Q3 is determined using the raw data from the sensors 10, 11, 12, 13 by applying the following method here:
[0067] - statistical methods (in the case of raw data from camera 10 , in particular the “BRIQUE” method or the “NIQUE” method can be used), and / or
[0068] -Frequency method (in the case of raw data from camera 10, the “clarity / blurriness” method or the “high-low frequency index” method can also be used to determine the clarity of the image; in the case of raw data from a LIDAR sensor, the “RMSE with reference” method, the “HDMAP and GPS” method or the “covariance matrix / entropy measurement” method can be used).
[0069] In general, if Figure 1As shown, the calculation unit 20 comprises three boxes B10', B11', B12', which receive as input raw data from the camera 10 and / or from the RADAR sensor 11 and / or from the LIDAR sensor 12 and / or from the light sensor 13, and each transmits as output a quality coefficient Q1, Q2, Q3 associated with one of the environmental sensors 10, 11, 12 and related to the level of accuracy of the measurements made by this sensor taking into account the driving conditions.
[0070] As will become apparent below, estimating a quality coefficient for each environmental sensor 10 , 11 , 12 will make it possible to favor one or more sensors whose working conditions are estimated to be the best and which therefore deliver the most reliable raw data.
[0071] During a fourth step, the data from the various environmental sensors 10 , 11 , 12 are fused.
[0072] For this purpose, the raw data acquired by the sensor can be fused on the one hand and the data from blocks B10 , B11 , B12 can be fused on the other hand.
[0073] However, it will be considered here that only the data from blocks B10 , B11 , B12 , ie descriptions S1 , S2 , S3 , will be fused.
[0074] These data are merged here by taking into account each probability factor P1 , P2 , P3 and possibly also each quality factor Q1 , Q2 , Q3 .
[0075] “Data fusion” refers to a mathematical method that applies several data coming from heterogeneous sensors and that enables improved detection and classification of objects present in the surroundings of a motor vehicle.
[0076] For example, data from images acquired by the camera 10 may be fused with data from the RADAR sensor 11 and the LIDAR sensor 12 to better estimate the exact position and dynamics (velocity and acceleration) of objects detected in the images acquired by the camera 10 .
[0077] The probability coefficients P1, P2, P3 and the quality coefficients Q1, Q2, Q3 are then used to dynamically adjust the weight of each environmental sensor 10, 11, 12 to detect and classify the object.
[0078] In general, if Figure 1As shown, the computing unit 20 comprises a box B1 which receives as input descriptions S1, S2, S3 of detected objects as well as probability coefficients P1, P2, P3 and quality coefficients Q1, Q2, Q3, and transmits as output a result D2 comprising a description (category, position and dynamics) of each object which has been detected by several environmental sensors and which has been checked by the data fusion algorithm.
[0079] With the aid of this result D2 , during a fifth step, the computing unit 20 formulates control commands C1 for the individual transmission elements 30 of the motor vehicle.
[0080] For this reason, Figure 1 As shown, the calculation unit 20 comprises a block B2 which receives the result D2 from the block B1 as input and delivers the control instruction C1 as output.
[0081] The control command C1 is therefore generated taking into account the evaluation of the vehicle surroundings by the computing unit 20 .
[0082] To prevent any errors in this assessment from having dangerous consequences for the vehicle occupants, two additional steps are provided to make the method safe.
[0083] During a sixth step, the calculation unit 20 estimates the reliability of the control command C1 as a function of the quality factors Q1 , Q2 , Q3 and the probability factors P1 , P2 , P3 .
[0084] In fact, the reliability of the control instruction C1 is estimated by the reliability coefficient D3.
[0085] The algorithm for calculating the reliability coefficient D3 may be based, for example, on a method of associating quality coefficients Q1 , Q2 , Q3 with probability coefficients P1 , P2 , P3 .
[0086] Preferably, the reliability coefficient D3 will be determined mainly according to the quality coefficients Q1, Q2, Q3.
[0087] In particular, if these quality coefficients Q1 , Q2 , Q3 indicate that the operating conditions of most of the environmental sensors 10 , 11 , 12 do not allow the vehicle to have a good understanding of its environment, this information will be primarily taken into account when determining the reliability coefficient D3 (regardless of the value of the probability coefficient).
[0088] In other words, the probability coefficients P1, P2, P3 have lower statistical weight than the quality coefficients Q1, Q2, Q3.
[0089] Furthermore, the greater the number of sensors used to determine the quality factor of a given sensor, the greater the weight of this quality factor in the calculation of the reliability factor D3.
[0090] The algorithm for calculating the reliability coefficient D3 will be able to take other data into account. Therefore, preferably, the reliability coefficient D3 will also be estimated based on the fused result D2. In this way, if the fused result D2 is inconsistent, the inconsistency will be able to be taken into account when calculating the reliability coefficient D3.
[0091] In general, if Figure 1 As shown, the calculation unit 20 comprises B3, which receives the probability coefficients P1, P2, P3 and the quality coefficients Q1, Q2, Q3 and the fused result D2 as input, and transmits the reliability coefficient D3 as output.
[0092] During a seventh step, the computing unit 20 will then make a decision whether to correct the control command C1 (before sending this control command to the transmission 30 ).
[0093] This decision is made primarily by considering the reliability coefficient D3.
[0094] Preferably, this decision will also be able to be made based on redundant information D1 from sensors different from the sensors 10 , 11 , 12 considered until this moment.
[0095] In practice, it is envisaged that the calculation unit 20 will request a correction of the control instruction C1 if the reliability coefficient D3 is below a threshold value and / or if the redundant information D1 indicates an inconsistency between the data considered.
[0096] The action resulting from the correction may be, for example, deactivating the vehicle's autonomous driving mode or ceasing to consider raw data from one or more previously determined sensors.
[0097] In general, if Figure 1 As shown, the calculation unit 20 comprises a block B4 which receives as input the reliability coefficient D3 and the redundant information D1 and which possibly delivers as output an instruction for correcting the control instruction C1 .
[0098] This block B4 is formed by an algorithm whose purpose is to ensure the ASIL-D safety level within the meaning of the ISO 26262 standard.
Claims
1. A method for controlling a motor vehicle, the motor vehicle comprising a computing unit (20) and a plurality of sensors (10, 11, 12) suitable for acquiring raw data relating to the environment of the motor vehicle, the method comprising the following steps: - the computing unit (20) receives the raw data acquired by the sensors (10, 11, 12), the calculation unit (20) processes the raw data in order to derive therefrom information (S1, S2, S3) relating to the environment of the motor vehicle and probability coefficients (P1, P2, P3) relating to the probability of an error in deriving each item of information (S1, S2, S3), and - formulating a control instruction (C1) for the motor vehicle based on said information (S1, S2, S3) and said probability coefficients (P1, P2, P3), Characterized in that the method further comprises the following steps: - determining, for at least a first sensor of the sensors (10, 11, 12), a quality factor (Q1, Q2, Q3) related to the quality of raw data acquired by the first sensor (10, 11, 12), - estimating the reliability of the control command (C1) based on the quality coefficients (Q1, Q2, Q3) and the probability coefficients (P1, P2, P3), and -determining whether to correct the control instruction (C1) according to the estimated reliability of the control instruction (C1), The reliability of the control instruction (C1) is estimated by a reliability coefficient (D3), wherein the reliability coefficient (D3) is calculated based on an algorithm that associates a quality coefficient (Q1, Q2, Q3) with a probability coefficient (P1, P2, P3), wherein in the process of calculating the reliability coefficient (D3), the weight of the quality coefficient (Q1, Q2, Q3) is higher than the weight of the probability coefficient (P1, P2, P3), and the more sensors are used to determine the quality coefficient (Q1, Q2, Q3) of the first sensor, the greater the weight of the quality coefficient (Q1, Q2, Q3) in the calculation of the reliability coefficient (D3).
2. The method of claim 1, wherein: In the determination step, the quality coefficient (Q1, Q2, Q3) of at least a first sensor among the sensors (10, 11, 12) is determined based on raw data acquired by at least another sensor among the sensors (10, 11, 12) and / or based on third-party data, which is acquired by a third-party detector (13) and is related to the measurement conditions of the raw data acquired by the first sensor (10, 11, 12).
3. The method of claim 2, wherein: The third party detector (13) is a light sensor, or a rain sensor, or a sensor suitable for detecting the state of the road on which the motor vehicle is travelling.
4. The method of claim 1, wherein: At least one of the sensors (10, 11, 12) is an image sensor, or a RADAR sensor, or a LIDAR sensor.
5. The method of claim 1, wherein: In the processing step, the raw data transmitted by each sensor (10, 11, 12) are processed separately from the raw data transmitted by the other sensors (10, 11, 12) in order to detect objects in the environment of the motor vehicle and classify the objects, each probability coefficient (P1, P2, P3) being associated with the classified object and with the sensor (10, 11, 12).
6. The method of claim 5, wherein: In this processing step, after the raw data is processed, the processed data is fused by considering each probability coefficient (P1, P2, P3).
7. The method of claim 5, wherein: In the processing step, after the raw data is processed, the processed data is fused by considering each quality coefficient (Q1, Q2, Q3).
8. The method according to claim 6 or 7, wherein: In the estimation step, the reliability of the control instruction (C1) is also estimated based on the fusion result of the processed data.
9. The method of claim 1, wherein: The decision whether to correct the control command (C1) is also made based on redundant information from sensors other than the sensors (10, 11, 12).
10. A motor vehicle comprising: a plurality of sensors (10, 11, 12) adapted to acquire raw data relating to the environment of the motor vehicle; And a computing unit (20), characterized in that the computing unit (20) is suitable for implementing the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Method and device for operating an environment sensor of a vehicle
DE102016202805A1
Sensor fusion system and vehicle control system therewith
US20050125154A1
Use of Multiple Internal Sensors for Measurement Validation
US20120016623A1
Apparatus and method for recognizing driving environment for autonomous vehicle
US20160061612A1
Cross-validating sensors of an autonomous vehicle
US9221396B1
