Method and system for creating training data for learning an artificial neural network of a driver assistance system for a motor vehicle

Abstract

The invention relates to a method for generating training data for learning an artificial neural network of a driver assistance system for a motor vehicle (1). The invention also relates to a computer system (100) implementing such a method. Figure for the abstract: 1

Description

Title of the invention: Method and system for creating training data for learning an artificial neural network of a driver assistance system for a motor vehicle. Technical field of the invention

[0001] The present invention relates to the field of driver assistance systems for motor vehicles. The invention relates in particular to a method for generating training data for learning an artificial neural network of a driver assistance system for a motor vehicle, the method being implemented by a computer system comprising a data generation module embedded in a motor vehicle and a data processing module located remotely from the vehicle. The invention further relates to such a computer system that implements such a method. The invention is applicable to motor vehicles, in particular motor vehicles such as cars. Prior art

[0002] Modern automotive driver assistance systems now use artificial neural networks to provide driver assistance functionalities, particularly for autonomously managing driving in certain specific driving environments. To achieve this, training artificial neural networks requires large volumes of training data. Most often, the training data used for network training consists of video sequences of a vehicle's driving environment, generated by cameras that are now standard equipment in most vehicles. To acquire this training data, car manufacturers generally use dedicated vehicles for testing, during which the training data is generated.Alternatively, private vehicles equipped with appropriate electronic and computer components can be used. However, in both cases, expensive computing resources must be available onboard the vehicle to manage the processing of large volumes of data generated primarily by video footage of the driving environment filmed by the vehicle's cameras. Furthermore, since the processing of this training data is not performed onboard the generating vehicle, it is necessary, particularly in the case of private vehicles, to transmit these large volumes of data via conventional mobile communication networks to remote data processing systems. This... This constraint also imposes increased needs in terms of data transmission capacity and bandwidth, which in turn increases the congestion of communication networks as well as the costs associated with data transmission, which ultimately increases the cost of manufacturing vehicles. Summary of the invention

[0003] The invention aims to overcome these drawbacks. In particular, it aims to provide a solution for generating training data for learning an artificial neural network of a driver assistance system for a motor vehicle in a less expensive and faster manner, thereby maximizing the availability of such training data. By these means, the invention aims to contribute to improving the safety of driver assistance systems that use artificial neural networks and, consequently, the safety of the motor vehicles that incorporate them.

[0004] In order to achieve these goals, the invention relates, according to a first aspect, to a method of constructing training data for learning an artificial neural network of a driver assistance system for a motor vehicle, the method being implemented by a computer system comprising a data generation module embedded in a motor vehicle and a data processing module remote from the vehicle, the method comprising the steps of: i. obtaining, by the data generation module, data characterizing at least one video sequence generated by means of at least one camera of a driver assistance system for the vehicle; ii. determine, by the data generation module, based on the data obtained during step i), data characterizing at least one description of at least one driving environment of the vehicle; iii. cause, by the data generation module, a transmission of the data determined during step ii) to the data processing module; iv. determine, by the data processing module, data characterizing at least one video sequence reconstructed by means of a generative adversarial network fed with the data transmitted during step iii); and v. generate, by the data transmission module, said training data as a function of the data determined during step iv).

[0005] According to one variant, said data characterizing a description of at least one driving environment of the vehicle can establish at least one position of the vehicle, a number of traffic lanes, information relating to an obstacle present in the driving environment, information relating to a weather condition and / or information relating to a road surface.

[0006] According to another variant, said data characterizing a description of at least one driving environment of the vehicle may contain a three-dimensional representation of said driving environment.

[0007] According to yet another variant, said data characterizing a description of at least one driving environment of the vehicle may contain at least one image of said environment.

[0008] According to yet another variant, step i) may include a step of determining whether the video sequence is of interest for training an artificial neural network of a driver assistance system for a motor vehicle.

[0009] According to a second aspect, the invention relates to a system for constructing training data for learning an artificial neural network of a driver assistance system for a motor vehicle, said system comprising a data generation module embedded in a motor vehicle and a data processing module remote from the vehicle which jointly implement a method as described above. Brief description of the figures

[0010] Other features and advantages of the invention will become apparent from an examination of the detailed description below, and the accompanying figures, in which:

[0011] [Fig-1] is a diagram illustrating a system according to the invention; and

[0012] [Fig.2] is a diagram illustrating the steps of a process according to the invention. Detailed description of the invention

[0013] Figure 1 schematically illustrates a training data generation system 100 for training an artificial neural network of a driver assistance system for a motor vehicle according to the invention. It is fundamentally a computer system comprising, firstly, a data generation module 200 mounted on board a motor vehicle 1. The data generation module 200 is a computer device comprising at least one information processing unit 201, with one or more processors, a data storage medium 202, and an input / output interface 203 enabling the reception and transmission of data. Preferably, the data generation module 200 is hosted on a computer and interacts via its input / output interface 203 and by means of a wired communication network of the vehicle 1 (e.g., CAN, Ethernet, MOST, etc.).) with a vehicle 1 driver assistance system 2, which classically includes at least one camera 3 for taking pictures and generating video sequences of the vehicle's driving environment, and with a . vehicle 1 radio frequency signal communication system 4, which allows communication with remote communicating entities by means of conventional mobile communication networks and protocols (e.g. 5G, ITS-G5, C-V2X, etc.).

[0014] The system 100 according to the invention comprises, secondly, a data processing module 300 located remotely from the vehicle 1, for example, a server hosted within a cloud computing infrastructure. Advantageously, the data processing module 300 integrates with or interacts with a generative adversarial network 301, in other words, a generative artificial intelligence.

[0015] According to the invention, all the elements described above contribute to enabling the implementation of a method for constructing training data for learning an artificial neural network of a driver assistance system for a motor vehicle, as described below in relation to [Fig.2].

[0016] Figure 2 illustrates, by means of a flowchart, the steps of the process according to the invention.

[0017] According to a first step E1 of the method according to the invention, the data generation module 200 obtains data characterizing at least one video sequence generated by means of at least one camera of the vehicle's driver assistance system 2. In other words, the data generation module 200 acquires, during this first step, video sequences of the driving environment of the vehicle 1 that are conventionally generated by the camera 3 of the vehicle's driver assistance system 2.

[0018] Preferably, when obtaining video sequences, the data generation module 200 determines whether a video sequence is of interest for training an artificial neural network of a driver assistance system for a motor vehicle, and it obtains exclusively data that corresponds to such video sequences of interest. To do this, it interacts with the driver assistance system 2 of vehicle 1. Indeed, the neural networks of driver assistance systems are generally equipped with capabilities for evaluating the quality of their output and for detecting that input data is unknown or poorly represented. The data generation module 200 advantageously relies on these evaluation data and these self-critical capabilities to determine whether a video sequence is of interest.Alternatively or cumulatively, a video sequence can be classified as interesting if two neural networks of the driver assistance system 2 reach contradictory conclusions (e.g., different results between a main network and a so-called "phantom" network). As can be understood, this advantageously reduces, even at this stage, the volume of data processed by the data generation module 200 and, consequently, the load on the onboard computing resources of the vehicle 1.

[0019] Thus, at the end of this first step, the data generation module 200 holds data which is similar to that which is usually used for training an artificial neural network of a driver assistance system for a motor vehicle and whose large volume generates the problems mentioned in the preamble.

[0020] According to a second step E2 of the method according to the invention, the data generation module 200 determines, based on the data obtained during the first step, data characterizing at least one description of at least one driving environment of the vehicle. In other words, the data generation module 200 uses the previously acquired video sequences at this stage to generate descriptions of the vehicle's driving environment. As will be seen, this significantly reduces the load on computing and data storage resources and, consequently, the need for expensive computer components.

[0021] Preferably, the data thus determined during this step by the data generation module 200 establishes at least one position of the vehicle 1, a number of traffic lanes, information relating to an obstacle present in the driving environment, information relating to a weather condition and / or information relating to a road surface. Alternatively or cumulatively, it contains a digital image of the driving environment.

[0022] To determine this data, the data generation module 200 preferentially uses an image encoder to extract images from the video sequences, then it performs a projection of segmented images in a three-dimensional space.

[0023] Advantageously, the volume of this data characterizing at least one description of at least one driving environment of the vehicle, which is determined during this second step E2 of the method, is significantly smaller (on the order of 10⁵) than the volume of corresponding video sequences. Preferably, this data is recorded at regular intervals, the recording frequency being able to vary depending on the nature of the data (e.g., low frequency when a static obstacle is present in the driving environment or high in the case of a moving obstacle) and / or the complexity of the driving environment (e.g., low frequency for an environment with few obstacles). This advantageously minimizes the data storage resources required.

[0024] Next, according to a third step E3 of the process according to the invention, the data generation module 200 causes a transmission of the data determined during the second step to the data processing module 300, by interacting for this purpose with the radio frequency signal communication system 4. In other words, at this stage, it is the descriptions of the driving environment, determined from the relevant video sequences, that are transmitted to the 300 data processing module, and no longer the video sequences themselves. This significantly minimizes the need for computing resources and bandwidth, thereby reducing the cost of obtaining training data and, consequently, increasing its availability.

[0025] According to a fourth step E4 of the method according to the invention, the data processing module 300 determines data characterizing at least one reconstructed video sequence by feeding the generative adversarial network 301 with the data transmitted by the data generation module 200 during the previous step. In other words, the data processing module 300 reconstructs video sequences of a vehicle's driving environments at this stage from the driving environment descriptions that have been transmitted to it. To do this, it feeds the generative adversarial network 301, which conventionally provides video sequences that correspond to the driving environment descriptions with which it is fed.Thus, at the end of this fourth step, the 300 data processing module identified exactly the same type of data typically used for training artificial neural networks, namely video sequences of a vehicle's driving environments during one or more driving sessions. However, unlike the known implementations described in the introduction, the 300 data processing module manages to generate such data without requiring the use of costly computing resources and communications.

[0026] Finally, according to a fifth and final step E5 of the process, the data processing module 300 generates the training data that can then be used to train an artificial neural network of a driver assistance system for a motor vehicle based on the data determined during the previous step. It does this, for example, by aggregating received descriptions of driving environments and corresponding reconstructed video sequences determined as explained above.

[0027] Thus, thanks to the method and system according to the invention described above, a solution is provided for generating training data for the learning of an artificial neural network of a driver assistance system for a motor vehicle in a faster and less expensive manner, thereby maximizing the availability of such data. In this way, the invention contributes to improving the learning capabilities of artificial neural networks in driver assistance systems and, ultimately, to contributing to driving safety.

Claims

Claims

1. Method for constituting training data for learning an artificial neural network of a driving assistance system of a motor vehicle, the method being implemented by a computer system (100) comprising a data generation module (200) on board a motor vehicle (1) and a data processing module (300) remote from the vehicle, characterized in that the method comprises the steps of: i. obtaining, by the data generation module (200), data characterizing at least one video sequence generated by means of at least one camera of a driving assistance system (2) of the vehicle; ii. determining, by the data generation module (200), as a function of the data obtained during step i), data characterizing at least one description of at least one driving environment of the vehicle; iii.causing, by the data generation module (200), a transmission of the data determined during step ii) to the data processing module (300); iv. determining, by the data processing module (300), data characterizing at least one video sequence reconstructed by means of a generative adversarial network (301) supplied with the data transmitted during step iii); and v. generating, by the data processing module (300), said training data as a function of the data determined during step iv).

2. Method according to claim 1, characterized in that said data characterizing a description of at least one environment of the vehicle establishes at least one position of the vehicle, a number of traffic lanes, information relating to an obstacle present in the driving environment, information relating to a meteorological condition and / or information relating to a road surface.

3. Method according to one of the preceding claims, characterized in that that said data characterizing a description of at least one driving environment of the vehicle contains data characterizing a three-dimensional representation of said driving environment.

4. Method according to one of the preceding claims, characterized in that said data characterizing a description of at least one driving environment of the vehicle contains at least one image of said environment.

5. Method according to one of the preceding claims, characterized in that step i) comprises a step consisting of determining whether the video sequence is of interest for the training of an artificial neural network of a driving assistance system for a motor vehicle.

6. System (100) for constituting training data for learning an artificial neural network of a driving assistance system for a motor vehicle, characterized in that said system comprises a data generation module (200) on board a motor vehicle (1) and a data processing module (300) remote from the vehicle which jointly implement a method according to one of the preceding claims.

Images