Method and device for determining an off-road driving environment in which a vehicle is traveling

Abstract

The present invention relates to a method and device for determining the type of off-road driving a vehicle is experiencing. To this end, the method comprises receiving (41) first data from a set of onboard sensors and second data representing a map, the first data being representative of images depicting the vehicle's surroundings and a geographical position; classifying (43a) the first data into a set of classes representing a type of environment according to two methods; and selecting (43b) a class by comparing the classifications. The method also comprises determining (44a) a zone type by projecting the geographical position onto the map and comparing (44b) the zone type to a result of an image classification.The off-road driving type is then determined (45) based on the selected class and the result of the zone type comparison. Figure for the summary: Figure 4.

Description

Title of the invention: Method and device for determining an off-road driving environment in which a vehicle is traveling. Technical field

[0001] The invention relates to methods and devices for determining the driving environment of a vehicle, particularly, but not exclusively, motor vehicles. The invention also relates to a method and device for processing vehicle data. In particular, the invention relates to a method and device for determining and validating the type of off-road environment in which the vehicle is operating. The invention also relates to a method and device for controlling a set of on-board vehicle systems when the type of environment in which the vehicle is operating is off-road. Technological background

[0002] Modern vehicles are made up of a large number of components or parts developed and designed to operate in different types of environments, primarily on-road environments including roads with asphalt pavements. However, vehicles sometimes operate in so-called off-road or all-terrain environments, that is, on unpaved surfaces such as sand, gravel, riverbeds, mud, snow, rocks, or other natural terrain.

[0003] Detecting the type of environment in which the vehicle is operating is an important issue, particularly for identifying data associated with driving a vehicle in an off-road environment, the data thus collected being able to be used to develop and properly size the various components and parts of the vehicles.

[0004] Gaining a better understanding of vehicle use in real-world conditions is an important issue for car manufacturers, for example to improve mission profile calculations to better size components according to their actual use by customers or to learn failure prediction models on the training database reflecting the actual use of vehicles. Summary of the present invention

[0005] One object of the present invention is to solve at least one of the problems of the technological background described above.

[0006] Another object of the present invention is, for example, to improve the detection of an off-road type environment when a vehicle is traveling in such an environment.

[0007] Another object of the present invention is to improve the relevance of vehicle usage data, in particular for the design and modeling of vehicle components or organs.

[0008] According to a first aspect, the present invention relates to a method for determining the type of off-road driving in which a vehicle is operating, the method being implemented by at least one processor and comprising the following steps: - reception of initial data from a set of sensors onboard the vehicle and second data representing a map, the initial data being representative: • of vehicle component behavior, • parameters of a vehicle environment and images representing that environment, and • the geographical location of the vehicle; - classifications of the first data into a first set of classes, each representative of a type of environment, according to two distinct methods; - selection of at least one class in the first set of classes by comparing classifications; - determination of a type of zone from a set of zone types by projecting the geographical position onto the map; - comparison of the zone type to a result of an image classification by a zone type prediction model; - determination of the type of off-road driving based on at least one selected class and a result of the comparison of the type of zone.

[0009] Determining the type of off-road driving the vehicle is experiencing, according to the first aspect of the invention, is achieved in several stages, including two successive validation stages based on the use of classifications employing, for example, machine learning techniques or even deep learning techniques, fed by data acquired from sensors or devices onboard the vehicle. This improves confidence in determining the type of off-road driving the vehicle is experiencing, without driver intervention.

[0010] The data thus collected can be associated with the type of off-road driving determined and then, for example, used to establish various mission profile calculations necessary for the sizing of components or on-board systems. in vehicles operating in off-road environments, under real-world conditions.

[0011] According to one variant of the method, one of the methods includes a comparison of at least a part of the first data to reference characterization matrices, each reference characterization matrix representing a type of environment associated with a class of the first set of classes.

[0012] According to one variant, the method includes a step of storing the first data associated with the type of off-road driving in a database.

[0013] According to another variant of the process, one of the methods includes a first deep neural network learned in a first learning phase from the database.

[0014] According to a further variant of the method, the area type prediction model is implemented by a second deep neural network learned in a second learning phase from the database.

[0015] According to yet another variant, the method further includes a step of generating a set of control instructions for a set of on-board systems, the instructions being adapted to the type of off-road driving.

[0016] According to a further variant, the process further comprises the following steps: - displaying a set of graphic objects on a touchscreen of a human-machine interface embedded in the vehicle, each graphic object in said set of graphic objects being representative of a type of off-road driving determined according to the result of the classifications of the initial data in the first set of classes and the result of the image classification, and - reception of third data representing a touch press on one of the graphic objects in the set of graphic objects, the determination of the type of off-road driving being a function of third-party data.

[0017] According to another embodiment, the method further comprises a step of processing the first data implemented before the classifications of the first data, the processing including filtering of at least part of the first data and / or smoothing of at least part of the first data.

[0018] According to a second aspect, the present invention relates to a device for determining a type of off-road driving, the device comprising a memory associated with a processor configured for implementing the steps of the process according to the first aspect of the present invention.

[0019] According to a third aspect, the present invention relates to a vehicle, for example a motor vehicle, comprising the device according to the second aspect of the present invention

[0020] According to a fourth aspect, the present invention relates to a computer program which includes instructions adapted for carrying out the steps of the process according to the first aspect of the present invention, in particular when the computer program is executed by at least one processor.

[0021] Such a computer program may use any programming language, and be in the form of source code, object code, or an intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0022] According to a fifth aspect, the present invention relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the process according to the first aspect of the present invention.

[0023] On the one hand, the recording medium can be any entity or device capable of storing the program. For example, the medium can include a storage means, such as a ROM, a CD-ROM or a microelectronic circuit-type ROM, or a magnetic recording means or a hard disk drive.

[0024] On the other hand, this recording medium can also be a transmissible medium such as an electrical or optical signal, such a signal being able to be transmitted via an electrical or optical cable, by conventional or radio frequency, by self-directing laser beam, or by other means. The computer program according to the present invention can, in particular, be downloaded from an Internet-type network.

[0025] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to execute or to be used in the execution of the process in question. Brief description of the figures

[0026] Other features and advantages of the present invention will become apparent from the description of the specific and non-limiting embodiments of the present invention below, with reference to the attached Figures 1 to 6, in which:

[0027] [Fig.1] schematically illustrates a type of off-road driving in which a vehicle operates, according to a particular and non-limiting embodiment of the present invention;

[0028] [Fig.2] schematically illustrates a process for determining the type of off-road driving in which the vehicle of [Fig.1] operates, according to a first particular and non-limiting embodiment of the present invention;

[0029] [Fig.3] illustrates a device configured to determine the type of off-road driving of the [Fig.1], according to a particular and non-limiting embodiment of the present invention;

[0030] [Fig.4] illustrates a flowchart of the different stages of a process for determining the type of off-road driving in which the vehicle of [Fig.1] operates, according to a second particular and non-limiting embodiment of the present invention;

[0031] [Fig. 5] illustrates environmental characterization matrices, according to a particular and non-limiting embodiment of the present invention; and

[0032] [Fig.6] schematically illustrates classification results, according to a particular and non-limiting embodiment of the present invention. Description of examples of achievements

[0033] A method and device for determining a vehicle driving environment will now be described in what follows with joint reference to Figures 1 to 6. The same elements are identified with the same reference signs throughout the description that follows.

[0034] The terms "first," "second" (or "firsts," "seconds"), etc., are used in this document by arbitrary convention to allow for the identification and distinction of different elements (such as operations, means, etc.) implemented in the embodiments described below. Such elements may be distinct or correspond to a single element, depending on the embodiment.

[0035] Fig. 1 schematically illustrates a vehicle 10 evolving in an environment 1, according to a particular and non-limiting embodiment of the present invention.

[0036] Vehicle 10 corresponds, for example, to a vehicle with an internal combustion engine, with electric motor(s), or even a hybrid vehicle with an internal combustion engine and one or more electric motors. Vehicle 10 thus corresponds, for example, to a land vehicle, such as a car, a truck, a bus, or a motorcycle.

[0037] The vehicle 10 corresponds to a so-called connected vehicle in that it carries a communication system configured to communicate with one or more remote devices 101 via a wireless communication network infrastructure. The remote device 101 corresponds, for example, to a server or a computer in the "cloud" 100.

[0038] The communication system of a connected vehicle includes, for example, one or more communication antennas connected to a telematics control unit, known as a TCU (Telematic Control Unit), which is itself connected to one or more computers of the connected vehicle's embedded system. The antenna(s), the TCU, and the computer(s) form, for example, a multiplexed architecture for providing various services necessary for the proper functioning of the vehicle. connected and to assist the driver and / or passengers of the connected vehicle in controlling the connected vehicle and / or to diagnose the operation of one or more components of the connected vehicle. The computer(s) and the TCU communicate and exchange data with each other via one or more computer buses, for example a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data-Rate), FlexRay (according to ISO 17458) or Ethernet (according to ISO / IEC 802-3) type communication bus.

[0039] The mobile communication infrastructure enabling wireless data communication between the vehicle 10 and the remote device 101 includes, for example, one or more communication devices 102 of the type relay antenna (cellular network) or roadside unit, known as RSU. In a communication mode using such a network architecture, the data is, for example, transmitted by the vehicle connected to the remote device 101 from the "cloud" 100 via a relay antenna 102 (the antenna 102 being, for example, connected to the "cloud" 100 via a wired link and the remote device 101 being itself connected to the network infrastructure of the "cloud" 100 via a wired and / or wireless network).

[0040] The wireless communication system enabling data exchange between the connected vehicle and the remote device 101 corresponds, for example, to: - a vehicle-to-infrastructure (V2I) communication system, for example based on the 3GPP LTE-V or IEEE 802.1 lp standards of ITS G5; or - a cellular network communication system, for example an LTE (Long-Term Evolution) network, LTE-Advanced (also called LTE 3G, 4G or 5G); or - a Wifi type communication system according to IEEE 802.11, for example according to IEEE 802.1 In or IEEE 802.1 lac.

[0041] The vehicle 10 advantageously carries a set of sensors and devices configured to detect or determine the type of environment 1 in which the vehicle 10 is operating. Off-road driving corresponds to driving off-road as defined in the first type, the surface 11 on which the vehicle 10 is operating corresponding to an unpaved surface such as sand, gravel, a riverbed, mud, snow, rocks or other natural terrain.

[0042] The vehicle 10 carries, for example, one or more of the following sensors: - one or more millimeter-wave radars arranged on the vehicle 10, for example at the front, at the rear, on each front / rear corner of the vehicle; each radar is adapted to emit electromagnetic waves and to receive the echoes of these waves reflected by one or more objects, in order to detect any object or obstacle and their distances from the vehicle 10; and / or - one or more LIDAR(s) (Light Detection and Ranging), a LIDAR sensor corresponding to an optoelectronic system composed of a laser emitter, a receiver including a light collector (to collect the portion of the light emitted by the emitter and reflected by any object located in the path of the light rays emitted by the emitter) and a photodetector that transforms the collected light into an electrical signal; a LIDAR sensor thus makes it possible to detect the presence of objects located in the emitted light beam and to measure the distance between the sensor and each detected object; and / or - one or more cameras, for example a front camera, a camera mounted on each corner of the vehicle, a rear camera; the camera(s) may be configured, for example, for RGB (Red, Green, Blue) image acquisition and / or for thermal (infrared) cameras; and / or - one or more external temperature sensors; and / or - an altimeter; and / or - one or more vibration sensors, for example arranged in different locations on the vehicle chassis 10.

[0043] The vehicle 10 corresponds, for example, to a vehicle configured to drive in a two-wheel drive mode and in a four-wheel drive mode, also called AWD mode (from the English "All-Wheel Drive"), the driver of the vehicle 10 selecting one mode or the other from a control device provided for this purpose in the passenger compartment of the vehicle 10.

[0044] According to one variant, vehicle 10 corresponds to a vehicle configured to drive only in two-wheel drive mode.

[0045] The vehicle 10 may also include, for example, one or more ADAS systems (Advanced Driver-Assistance System), such as, for example: - an electronic stability control system fitted to the vehicle, known by the acronyms ESC (from the English "Electronic Stability Control" or in French "Contrôle électronique de la tranquillité"), DSC (from the English "Dynamic Stability Control" or in French "Contrôle dynamique de la tranquillité") or ESP (from the English "Electronic Stability Program" or in French "Programme électronique de la tranquillité"), - a lane keeping assist system, known as the LKA system (from the English "Lane Keep Assist"), and / or - a traction control system, known as DST (Dynamic Steering Torque), and / or - an electronic traction control system, known as ASR (Anti-Slip Regulation), such an ASR system regulating acceleration to limit the loss of traction of the drive wheels, and / or - a system called ABS (from the German "Antiblockiersystem" or in French "système anti-blocage des roues"), and / or - an adaptive cruise control system, also known as ACC (Adaptive Cruise Control), and / or - a geolocation system enabling the vehicle 10 to obtain data or information representative of its geographical position at any time, for example in the form of coordinates (latitude and longitude), via a satellite link with a set of satellites (not shown on [Fig.1]), the geolocation system corresponding for example to a system of type GPS (from the English “Global Positioning System” or in French “Système de emplacement global”), Galileo or GLONASS.

[0046] Fig. 2 schematically illustrates a process for determining the type of off-road driving in which the vehicle 10 operates, according to particular and non-limiting embodiments of the present invention.

[0047] The process is for example implemented by one or more processors of one or more computers of the vehicle 10.

[0048] This process is implemented in particular when off-road or all-terrain driving conditions are detected.

[0049] In a first operation 21 of the process, a first set of data is obtained or received from a set of sensors, devices or systems on board the vehicle 10 and a second set of data is obtained or received from one or more remote devices such as the server 101 via the wireless communication unit or interface (e.g. TCU unit) of the vehicle 10 and / or a mobile communication device (e.g. a smart phone or "Smartphone") connected wirelessly (e.g. via Bluetooth® or Wifi®) or wired (e.g. via USB (Universal Serial Bus)) to the vehicle 10.

[0050] The initial data are representative: • of vehicle component behavior 10, • parameters of environment 1 of vehicle 10 and images representing this environment 1, and • the geographical location of the vehicle.

[0051] Thus, the initial data include, for example: - representative data of an outside temperature; and / or - representative data on the presence of frost; and / or - representative data of the presence of rain or the activation of the vehicle's windshield wipers 10; - representative image data of the vehicle's external environment 10, i.e., image data of the driving environment received from the vehicle's camera(s) 10; and / or - representative vehicle speed data 10; and / or - data representing accelerations along three directions defining a reference frame linked to the vehicle 10 and / or vibrations experienced by the vehicle 10, for example by one or more components of the vehicle 10 such as the chassis, shock absorbers, etc.; and / or - data representing the topography of the surface 11 on which the vehicle 10 is moving, for example obtained from a LiDAR of the vehicle 10; and / or - representative data of gear changes via the vehicle's gearbox 10; and / or - representative torque data sent to each wheel or set of wheels of vehicle 10; and / or - representative data on the activation or deactivation of on-board systems such as the ABS system, which activates in the event of a loss of traction of a wheel of the vehicle 10; and / or - representative data on switching between two-wheel drive and four-wheel drive modes; and / or - representative data on energy consumption (electricity or fuel depending on the type of engine(s) of the vehicle 10); and / or - representative data on altitude variation; and / or - representative location data of vehicle 10, for example in the form of a pair of longitude and latitude data.

[0052] The above list is provided as a purely illustrative example and is not exhaustive, the first data set comprising all data received from the sensors, devices or systems on board the vehicle 10, via the vehicle 10 on-board network.

[0053] The second set of data includes, for example: - representative mapping data of the environment in which vehicle 10 is traveling, this data being, for example, of the OpenStreetMap® type; and / or - representative location data for vehicle 10, for example obtained from a geolocation system implemented in the form of a mobile application executed on a mobile communication device connected in communication to vehicle 10.

[0054] The above list is provided as a purely illustrative example and is not exhaustive, the second data set comprising all data received from devices external to the vehicle 10 via a wired or wireless connection.

[0055] According to a particular embodiment, in a second operation 22, the first data are processed, the processing including in particular a filtering of at least part of the first data and / or a smoothing of at least part of the first data, that is to say that one or more processing operations is / are applied to all or part of the first data.

[0056] The treatment(s) correspond, for example: • to filtering a portion of the initial data to remove erroneous or aberrant values, • to a smoothing of some of the initial data, and • to any appropriate processing to improve the consistency and coherence of the measured or received data.

[0057] In a third operation 23a of the process, the initial data are classified into a first set of classes, each representing a type of environment, according to two distinct methods. The classification is notably multi-class, meaning that several types of environment can be combined.

[0058] The classes represent for example types of environment such as: • a river, • mud, • snow, • sand, • gravel, • rocks, • etc.

[0059] According to the first method, each of these types of environment is associated with reference characterization matrices as illustrated in [Fig. 5]. These reference characterization matrices are, for example, cross-tabulations comprising a first entry relating to a first parameter and a second entry relating to a second parameter, the first and second parameters being homogeneous with at least part of the first data, that is to say, these parameters and part of the first data are comparable either directly because they have the same unit or indirectly after conversion to the same unit, for example the first parameter represents the speed of the vehicle 10 and the second parameter represents a vibration level measured on a component of the Vehicle 10. The arguments of the reference characterization matrix are then predetermined values ​​representing, for example, the expected vibration level of the vehicle component at a given speed as a function of a type of environment: vibration = f(speed; type of environment). It should be noted, however, that reference characterization matrices include at least two entries but are not limited to two entries; the number of entries can be greater than or equal to two, for example, 5, 10, or 20, depending on the number of parameters included in the initial data.Thus, a first reference characterization matrix is ​​associated with a first type of environment A, for example representing the behavior of the vehicle component in a "Snow" type environment, and a second reference characterization matrix is ​​associated with a second type of environment B, for example representing the behavior of the vehicle component in a "Water" type environment.

[0060] According to a particular embodiment, the reference characterization matrices are established beforehand manually or via a first environment type prediction model implementing artificial intelligence and which is learned via a machine learning process, for example from a database comprising data similar to the first data and acquired by a set of vehicles evolving in different types of environment.The learning of the first algorithm corresponds, for example, to a supervised, semi-supervised or even unsupervised type of learning, based on training data representative of a set of data obtained from a set of vehicles, for example a few thousand vehicles, the data obtained for training being of the same nature as the first data received by the set of sensors on board the vehicle 10 and provided as input to the first environment type prediction model.

[0061] The first data are then exploited according to the first method to generate a characterization matrix and the generated characterization matrix is ​​compared to the reference characterization matrices to identify one or more reference characterization matrices representative of the type of environment 1 in which the vehicle 10 evolves, the reference characterization matrices being classified by the first environment type prediction model receiving the first data as input data.

[0062] According to the second method, a second environment type prediction model is used. The second environment type prediction model comprises an algorithm that takes as input one or more images acquired by at least one camera mounted in the vehicle 10. These images, or representative image data, thus correspond to image data of The environment 1 is acquired by the camera(s) of vehicle 10 during the movement of vehicle 10. This data corresponds, for example, to RGB data associated with each pixel of each acquired image. From these images, features are extracted, which are then processed.

[0063] The second method includes, for example, a first deep neural network learned in a first learning phase from the database described above. This database then comprises a set of images acquired by different vehicles operating in different types of off-road environments.

[0064] Similar to the learning of the first environment type prediction model, the learning of the second environment type prediction model corresponds, for example, to a supervised, semi-supervised or even unsupervised type of learning, based on training data representative of images acquired by vehicles evolving in off-road environments.

[0065] The parameters of this second environment type prediction model are thus generated according to any machine learning technique known to a person skilled in the art.

[0066] In a fourth operation 23b, at least one class is selected from the first set of classes by comparing the classifications. Indeed, the classifications obtained according to the two distinct methods are combined in such a way as to evaluate probabilities associated with each of the classes representing types of environment; thus, each class is associated with a probability that it corresponds to the type of environment in which the vehicle 10 operates. The type of environment 1 is then determined by selecting at least one class based on the probability associated with each class in the first set of classes: • the class for which the associated probability is the highest, or • the combination of classes whose probabilities are greater than a threshold value, for example greater than 90 or 95%

[0067] For example, if the following probabilities are associated with the classes of the first set of classes and the results are arranged in decreasing probability: • a river: 98%, • mud: 92%, • snow: 8%, • sand: 4%, • gravel: 2%, • rocks: 0%, • etc., then environment type 1 is a muddy river, a combination of the class "a river" and the class "mud".

[0068] In a fifth operation 24a, a zone type is determined from among a set of zone types by projecting the geographic position onto the map. In other words, information representative of a zone type in which the vehicle 10 is moving is determined by comparing first data representing a position of the vehicle and second data representing a map. The comparison corresponds to a projection of the current position of the vehicle 10 onto a map of the environment in which the vehicle 10 is moving, thus making it possible to determine a class representative of the zone type in a second set of classes, the second set of classes comprising representative classes of zone types, for example the following zone types: • mountain, • forest, • desert, • beach, • etc.

[0069] In a sixth operation 24b, the previously determined zone type is compared to the result of a classification of one or more images included in the initial data, the image classification being performed by a zone type prediction model. This operation then consists of a validation of the previously determined zone type.

[0070] According to a particular embodiment, the area type prediction model is implemented by a second deep neural network learned in a second learning phase from the database, more particularly from the images recorded in the database, which are for example annotated.

[0071] As with the first neural network, the learning of the second neural network corresponds for example to a supervised, semi-supervised or even unsupervised type of learning, based on training image data including environment images of vehicles evolving in different types of area.

[0072] The classification of the images results in obtaining probabilities associated with different classes of the second set of classes. For example, the classes representing types of areas are associated with the following probabilities: • mountain: 97%, • forest: 89%, • desert: 14%, • beach: 4%, • etc.

[0073] If the zone type determined during operation 24a is "mountain" and the probability associated with this same zone type is evaluated at 97% as in the example above, then the zone type "mountain" is validated.

[0074] Indeed, the type of zone determined during operation 24a is validated when the probability associated with the class corresponding to this type of zone is greater than a threshold, for example 85, 90 or 95% or when the class associated with this type of zone is among the most probable classes, for example the most probable or among the 2 or 3 most probable.

[0075] Otherwise, for example, if the zone type is determined to be "range" and the class associated with this zone type has a very low probability, here 4%, then the validation is not effective. A validation request by a user is implemented, for example. Such a validation request includes, according to a particular embodiment, the display of information on a touchscreen embedded in the vehicle 10 in the form of a graphic object or a set of graphic objects, which represent different types of zones, for example, the most probable ones and the one initially determined. The zone type is validated or modified based on data received corresponding to a touch by the user on one of the graphic objects. The zone type is then that corresponding to the one represented by the graphic object on which the user pressed.

[0076] In a seventh operation 25, the type of off-road driving is determined according to the type of environment determined during the fourth operation 23b and the result of the comparison of the sixth operation 24b.

[0077] Figure 6 illustrates an example of off-road driving type determination. A first table E represents the probabilities p(%) associated with the different classes of environment types E1, E2, E3, E4, E5, ... determined during the fourth operation 23b, and a second table Z represents the probabilities p(%) associated with the different classes of zone types Z1, Z2, Z3, Z4, ... determined during the sixth operation 24b. These results are then combined to obtain a third table C representing the probabilities p(%) associated with the different classes, each representative of an off-road driving type C1, C2, C3, C4, ... The output data from each of the environment type and zone type prediction models are then cross-referenced and / or aggregated to propose an off-road driving type.

[0078] Thus, still according to the previous example, C1 corresponds to an off-road driving situation of the "muddy river in the mountains" type with an associated probability of 95%, C2 to an off-road driving situation of the "river in the mountains" type with an associated probability of 80%, C3 to an off-road driving situation of the "snowy river in the mountains" type. mountain" with an associated probability of 14% and C4 to off-road driving of the "mud in the forest" type with an associated probability of 4%.

[0079] The type of off-road driving determined corresponds, for example, to that associated with the class with the highest probability, here an off-road driving of the type "muddy river in the mountains".

[0080] In an optional eighth operation 26, the display of graphic content on a touchscreen of a human-machine interface embedded in the vehicle 10 is controlled, the graphic content being a function of the selected off-road driving type. The off-road driving type is, for example, permanently displayed and updated in real time on the dashboard facing the driver of the vehicle 10, with the option for the driver to confirm or reject the selected off-road driving type via the human-machine interface.

[0081] According to a particular embodiment, a set of graphic objects is displayed on the touchscreen, each graphic object in this set representing a type of off-road driving determined based on the results of the classifications 23a of the initial data in the first set of classes and the results of the image classification. These results are then submitted to the driver of the vehicle 10 for validation or invalidation. This allows for human validation of the previously implemented classifications and for refining the parameters of the first and second environment type prediction models and / or the zone type prediction model using a reinforcement learning method.For this purpose, a request to display a confirmation request for the result of the off-road driving type determination is transmitted to the touchscreen in vehicle 10. This touchscreen corresponds, for example, to an on-board screen of vehicle 10, the transmission of the request being made via one or more data buses of the on-board network linking the computer implementing the process to the computer controlling the screen of vehicle 10. According to another example, this touchscreen corresponds to an on-board screen of a mobile communication device connected to vehicle 10, the transmission of the request being made via a wired (for example USB) or wireless (for example Bluetooth® or Wifi®) connection linking vehicle 10 to the mobile communication device.

[0082] According to a first embodiment, receiving the request triggers the display of graphical content asking the driver to validate whether the environment is of the specified off-road driving type, for example by tapping a first virtual button to validate / confirm that the off-road driving type is of the specified type and by tapping a second virtual button to invalidate / deny that the off-road driving type is of the specified type.

[0083] According to a second embodiment, receiving the request triggers the display of a set of graphic objects, allowing the driver to select the off-road driving type based on the results of the different classifications by tapping one of the displayed graphic objects. Each graphic object then corresponds to a type of off-road driving; for example, three graphic objects are displayed, corresponding to the three types of off-road driving for which the associated probabilities are highest, here the types "muddy river in the mountains," "river in the mountains," and "snowy river in the mountains."

[0084] In response to the transmitted request, the computer implementing the process receives third data representing a response to the request from the touch screen, this third data being representative of a touch press on the first virtual button or on the second virtual button according to the first variant, or on one of the graphic objects according to the second variant.

[0085] The parameters of the first prediction model and / or the parameters of the second prediction model are then refined based on the third data point, according to the so-called reinforcement learning method. The off-road driving type is then determined based on the third data point and corresponds to the type determined previously in the case of validation according to the first variant, and corresponds to the off-road driving type selected via touch input on the associated graphic object according to the second variant.

[0086] According to a particular embodiment, in a ninth operation 27, the initial data associated with the type of off-road driving are stored in the database, thereby enriching the training data for the various prediction models. All the data collected during the implementation of the process are, for example, stored in the vehicle's memory 10 and subsequently collected by an external device for later use in enriching datasets, for example, those used to define mission profiles and / or model test environments.

[0087] With the database updated, it is then possible to repeat the first learning phase associated with the second environment type prediction model and / or to repeat the second learning phase associated with the area type prediction model when these models are implemented by deep neural networks. Thus, regular or continuous use of the updated database makes it possible to improve and refine the prediction models through reinforcement learning.

[0088] According to a particular embodiment, in a tenth operation 28, a set of control instructions for a set of on-board systems is generated, the instructions being adapted to the type of off-road driving previously determined, for example to off-road driving of the muddy river type in the mountains.

[0089] These instructions are for example transmitted to the computers controlling this or these embedded systems for the automatic implementation of functions by this or these embedded systems improving driving in an off-road environment.

[0090] The instructions correspond, for example, to instructions for activating or deactivating systems such as ACC or ABS. These instructions correspond, for example, to instructions for activating or deactivating the high beams, for adapting the suspension system to the off-road environment, etc.

[0091] In one variant, these instructions further include instructions for a screen of the vehicle 10, for example the previously mentioned touchscreen, to display a set of recommendations for the driver and / or to display a quick emergency contact number in the event of an accident. In yet another variant, these instructions further include the activation of a function to limit the stress on certain components or parts of the vehicle 10.

[0092] Figure 3 schematically illustrates a device 3 configured to determine the type of off-road driving in which a vehicle, for example vehicle 10, is operating, according to a particular and non-limiting embodiment of the present invention. The device 3 corresponds, for example, to a device installed in the vehicle 10, for example a computer.

[0093] Device 3 is, for example, configured to carry out the operations described opposite Figures 1 to 2 and 5 to 6 and / or the steps of the process described opposite [Fig. 4]. Examples of such a device 3 include, but are not limited to, embedded electronic equipment such as a vehicle's on-board computer, an electronic control unit such as an ECU (Electronic Control Unit), a smartphone, a tablet, or a laptop computer. The elements of device 3, individually or in combination, can be integrated into a single integrated circuit, into several integrated circuits, and / or into discrete components. Device 3 can be implemented in the form of electronic circuits or software (or computer) modules, or a combination of electronic circuits and software modules.

[0094] The device 3 comprises one (or more) processor(s) 30 configured to execute instructions for carrying out the steps of the process and / or for executing instructions from the software embedded in the device 3. The processor 30 may include integrated memory, an input / output interface, and various circuits known to a person skilled in the art. The device 3 further comprises at least one memory 31 corresponding for example to volatile and / or non-volatile memory and / or comprises a memory storage device which may include volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.

[0095] The computer code of the embedded software(s) including the instructions to be loaded and executed by the processor is for example stored on memory 31.

[0096] According to a particular and non-limiting embodiment, the device 3 comprises a block 32 of interface elements for communicating with external devices such as connected vehicles and / or measuring devices. The interface elements of the block 32 comprise one or more of the following interfaces: - radio frequency RF interface, for example of the Wi-Fi® type (according to IEEE 802.11), for example in the 2.4 or 5 GHz frequency bands, or of the Bluetooth® type (according to IEEE 802.15.1), in the 2.4 GHz frequency band, or of the Sigfox type using UBN (Ultra Narrow Band) radio technology, or LoRa in the 868 MHz frequency band, LTE (Long-Term Evolution), LTE-Advanced; - USB interface (from the English "Universal Serial Bus" or "Universal Serial Bus" in French); - HDMI interface (from the English "High Definition Multimedia Interface", or "High Definition Multimedia Interface" in French); - LIN interface (from the English "Local Interconnect Network", or in French "Réseau interconnecté local").

[0097] According to another particular and non-limiting embodiment, the device 3 includes a communication interface 33 which enables communication with other devices (such as other servers, databases) via a communication channel 330. The communication interface 33 corresponds, for example, to a transmitter configured to transmit and receive information and / or data via the communication channel 330. The communication interface 33 corresponds, for example, to a wired Ethernet network (standardized by ISO / IEC 802-3).

[0098] According to a particular and non-limiting embodiment, the device 3 can provide output signals to one or more external devices, such as a display screen 340, touch or not, one or more speakers 350 and / or other peripherals 360 (projection system) via output interfaces 34, 35 and 36 respectively. According to a variant, one or more of the external devices is integrated into the device 3.

[0099] Figure 4 illustrates a flowchart of the different steps in a method for determining the type of off-road driving a vehicle, for example vehicle 10, is experiencing, according to a particular and non-limiting embodiment of the present invention. The method is implemented, for example, by one or more processors of a computer in vehicle 10, or by device 3 in Figure 3.

[0100] In a first step 41, first and second data are received. The first data are received from a set of sensors on board the vehicle 10 and are representative: • of vehicle component behavior 10, • parameters of a vehicle environment 10 and images representing the environment, and • the geographical location of the vehicle. The second set of data, on the other hand, is representative of a map.

[0101] In a second step 43a, the first data are classified into a first set of classes, each representative of a type of environment, according to two distinct methods.

[0102] In a third step 43b, a class is selected from the first set of classes by comparison of classifications.

[0103] In a fourth step 44a, a zone type is determined from a set of zone types by projecting the geographical position onto the map.

[0104] In a fifth step 44b, the zone type is compared to a result of an image classification by a zone type prediction model.

[0105] In a fifth step 45, the type of off-road driving is determined according to the type of environment and a result of comparing the type of area.

[0106] According to one variant, the variants and examples of the operations described in relation to one of Figures 1 to 2 and 5 to 6 apply to the steps of the process in [Fig.4].

Claims

Demands

1. A method for determining the type of off-road driving in which a vehicle (10) operates, said method being implemented by at least one processor and comprising the following steps: - receiving (41) first data from a set of sensors on board said vehicle (10) and second data representing a map, said first data being representative of: • the behavior of components of the vehicle (10), • parameters of an environment (1) of the vehicle (10) and images representing said environment (1), and • a geographical position of the vehicle; - classifying (43a) the first data into a first set of classes, each representative of a type of environment, according to two distinct methods; - selecting (43b) at least one class in said first set of classes by comparing said classifications;- determination (44a) of a zone type from among a set of zone types by projecting said geographical position onto said mapping; - comparison (44b) of said zone type to a result of a classification of said images by a zone type prediction model; - determination (45) of said off-road driving type based on said at least one selected class and a result of said comparison of said zone type (44b).

2. A method according to claim 1, wherein one of said methods comprises a comparison of at least a part of said first data to reference characterization matrices, each reference characterization matrix representing a type of environment associated with a class of the first set of classes.

3. A method according to claim 1 or 2, wherein it includes a step of storing the first data associated with the type of off-road driving in a database.

4. A method according to claim 3, wherein one of said methods comprises a first deep neural network learned in a first learning phase from said database.

5. A method according to claim 3 or 4, wherein the area type prediction model is implemented by a second deep neural network learned in a second learning phase from said database.

6. A method according to any one of claims 1 to 5, further comprising a step (28) of generating a set of control instructions for a set of on-board systems, said instructions being adapted to the type of off-road driving.

7. A method according to any one of claims 1 to 6, further comprising the following steps: - displaying a set of graphic objects on a touch screen of a human-machine interface embedded in the vehicle (10), each graphic object of said set of graphic objects being representative of a type of off-road driving determined according to a result of the classifications (43a) of the first data in the first set of classes and the result of the classification of the images, and - receiving third data representing a touch press on one of the graphic objects of said set of graphic objects, the determination (45) of said type of off-road driving being a function of the third data.

8. A method according to any one of claims 1 to 7, further comprising a processing step of said first data carried out before the classifications (23a) of the first data, said processing comprising filtering at least a part of said first data and / or smoothing at least a part of said first data.

9. Device (3) for determining a type of off-road driving, said device (3) comprising a memory (31) associated with at least one processor (30) configured for carrying out the steps of the process according to any one of claims 1 to 8.

10. Vehicle (10) comprising the device according to claim 9.

Images