Method for controlling at least one interactive device integrated into a vehicle

Abstract

Method for controlling at least one interactive device integrated into a vehicle. The invention relates to a method for controlling an interactive device integrated into a vehicle based on the gaze direction of a target occupant. The method comprises the steps of: - receiving (100) head and / or eye position measurements of the target occupant from sensor(s), - determining (200) the direction of the gaze vector of the target occupant in the vehicle's frame of reference, based on the data from the relevant sensors and their position in the vehicle, - transposing (300) the direction of the gaze vector into a frame of reference linked to the interactive device based on the position of the interactive device, and - generating (400) a control command for the interactive device based on the direction of the gaze vector in the device's frame of reference. Figure for the abstract: 3

Description

Title of the invention: Method for controlling at least one interactive device integrated into a vehicle

[0001] The present invention relates to a method for controlling at least one interactive device integrated into a vehicle based on the gaze direction of at least one target occupant of the vehicle. The present invention also relates to an electronic central control unit for the interactive device(s) based on the gaze direction of the target occupant(s) of the vehicle. Finally, the present invention relates to a system for controlling the interactive device(s) based on the gaze direction of the target occupant(s) of the vehicle.

[0002] Vehicles, especially automobiles, are now often equipped with interactive devices (HMI, human-machine interface, defined as a user interface allowing a person to connect to a machine, a system or a device), such as screens, generating real or virtual images, sound systems, etc., used for example to present to the driver or passengers of the vehicle information useful for driving and operating the vehicle, or to present entertainment content.

[0003] Today, such interactive devices are increasingly numerous in the vehicle, and / or present an increasingly large display area, in order to offer an ever-increasing amount of content to the vehicle's occupants.

[0004] The control of these interactive devices, generally carried out by physical interaction, increases the cognitive load and distraction of the vehicle occupants, who are therefore less focused on driving and put their safety at risk.

[0005] Eye-tracking, oculometry or gaze-tracking, are techniques known and implemented for some time in the field of computer science with merchandising or virtual reality systems, or in the field of transport and mobility.

[0006] In particular, prior art driver and occupant monitoring systems (DMS: Driving Monitoring Systems and OMS: Occupant Monitoring Systems) are known, using infrared cameras to perform eye tracking in order to detect driver distraction and / or drowsiness, for example based on the driver's gaze position relative to the interactive device in question or based on the driver's eyelid closure rate or blink rate.

[0007] These driver and occupant monitoring systems are effective because they are fixed directly below, above or inside the interactive device in question, for example in the instrument cluster, on the steering wheel, in the rearview mirror or in the center console of the vehicle.

[0008] However, such monitoring systems have limited conditions of use (fixed distance between the occupant and the interactive device for optimal operation, limited field of vision of the infrared camera, absence of detection of the driver's gaze during certain head movements, etc.).

[0009] One of the aims of the invention is therefore to propose a method of controlling at least one interactive device integrated into a vehicle, allowing control of the interactive device(s) of the vehicle while limiting the cognitive load and distractions of the vehicle's occupants.

[0010] To this end, the invention relates to a method for controlling at least one interactive device integrated into a vehicle based on the gaze direction of at least one target occupant of the vehicle, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the method being implemented by an electronic central unit, the method comprising the steps of: - reception of measurements from sensor(s), referred to as useful sensors, of the sensor block whose capture field includes at least the target occupant, the measurements from each sensor including the position of the head and / or eyes of the target occupant in the reference frame of said sensor, referred to as the sensor reference frame, - determination of the direction of the gaze vector of at least one target occupant in the vehicle's frame of reference, referred to as the vehicle frame of reference, based on measurements from the relevant sensors and the position of the relevant sensors in the vehicle, - transposition of the direction of the gaze vector determined in a frame linked to at least one interactive device of the vehicle, called the device frame, as a function of position data of the at least one interactive device, and - generation of at least one control command of at least one interactive device, each control command being a function of the direction of the gaze vector determined in the device frame.

[0011] Such a control method, partly thanks to the plurality of useful sensors and the determination of the gaze vector direction in the vehicle frame of reference, makes it possible to track the gaze direction of vehicle occupants. Tracking is possible even when head movements and rotations are significant and is independent of the useful sensor considered.

[0012] The set of reference changes allows each interactive device to be controlled in real time independently according to the gaze vector of the determined occupant.

[0013] The generation of a control command for the interactive device(s), based on the determined gaze vector, reduces the cognitive load on the vehicle occupant, who can then control the interactive devices simply and quickly using their gaze.

[0014] Furthermore, the content of the interactive devices is thus adapted to the occupant's gaze vector in real time and makes it possible to target more precisely and more securely the nature and / or position of the information to be presented to the occupant, thus limiting distractions due to the interactive devices.

[0015] According to other advantageous aspects of the invention, the control method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0016] - the sensory block comprises at least two sensors;

[0017] - each sensor of the sensor block is chosen from a sensor integrated into the vehicle and / or an additional sensor positioned in the vehicle, each sensor being chosen for example from a camera, a radar, a lidar, a microphone, or a gyroscope;

[0018] - at least one interactive device is chosen from a display device in the vehicle and / or a human-machine interface and / or a multimedia device of the vehicle;

[0019] - at least one control command consists of:

[0020] - a command to activate or deactivate at least one display area of one or more interactive devices, and / or

[0021] - a command to modify the dimension, position and / or nature of content displayed on the interactive device(s), and / or

[0022] - a command to validate an action to be carried out for the control of a vehicle-related functionality, and / or

[0023] - a control for generating a visual and / or audible warning signal for the target occupier;

[0024] - the step of determining the direction of the gaze vector of at least one target occupant in the vehicle frame is performed by triangulation from measurements received from useful sensors in the sensor frame;

[0025] - when at least one of the interactive devices is configured to generate at least A virtual image, the step of transposing the direction of the gaze vector into the device's frame of reference, includes:

[0026] - a first substep of changing the reference frame, the first substep of change of reference frame including the transposition of the direction of the gaze vector from the vehicle frame of reference, into a frame of reference linked to the virtual image(s), and

[0027] - a second substep of changing reference frame, the second substep of change of reference frame including the transposition of the direction of the gaze vector from the reference frame linked to the virtual image(s), into the device reference frame;

[0028] - the control method includes a preliminary step of receiving measurements in origin of each sensor in the sensor block, and selection of the useful sensor(s) of the sensor block from among the set of sensors in the sensor block, the capture field of the selected useful sensors including at least the target occupant;

[0029] - the control method includes an intermediate data reception step position of at least one interactive device in the vehicle, the generated control command being a function of the position data of at least one interactive device in the vehicle.

[0030] The invention also relates to an electronic central control unit for at least one interactive device integrated into a vehicle based on the gaze direction of at least one target occupant of the vehicle, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the electronic central control unit comprising a computer configured for: - to receive measurements from sensor(s), referred to as useful sensors, of the sensor block whose capture field includes at least the target occupant, the measurements from each sensor including the position of the head and / or eyes of the target occupant in the reference frame of said sensor, referred to as the sensor reference frame, - determine the direction of the gaze vector of at least one target occupant in the vehicle's frame of reference, referred to as the vehicle frame of reference, based on measurements from the relevant sensors and the position of the relevant sensors in the vehicle, - transpose the direction of the gaze vector determined in a frame linked to at least one interactive device of the vehicle, called the device frame, as a function of position data of the at least one interactive device, and - generate at least one control command for at least one interactive device, each control command being a function of the direction of the gaze vector determined in the device frame.

[0031] The invention also relates to a control system for at least one interactive device integrated into a vehicle based on the gaze direction of at least one target occupant of the vehicle, the system comprising: - a sensory block comprising at least one sensor configured to measure the position of the head and / or eyes of at least one target occupant, - an electronic central processing unit as described above, and - an interactive block integrated into the vehicle, the interactive block comprising at least one interactive device.

[0032] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings which are:

[0033] - [Fig.1] [Fig.1], a schematic representation of an example of a passenger compartment of a a vehicle incorporating several interactive devices

[0034] - [Fig.2] [Fig.2], a schematic representation of an electronic central processing unit, of a sensory block and an interactive block according to the invention,

[0035] - [Fig.3] [Fig.3], a flowchart of the steps implemented by the central unit electronics in the method of controlling at least one interactive device according to the invention, and

[0036] - [Fig.4] [Fig.4], a schematic representation of all the changes of reference points made by the central electronic unit in the control process according to the invention.

[0037] A vehicle 10 is illustrated by [Fig.1].

[0038] In this example, the vehicle 10 includes, in a passenger compartment 11, a windscreen 12, a dashboard 14, and pillars 15.

[0039] The vehicle 10 is, for example, a land, air or sea vehicle. The land vehicle is, for example, a motor vehicle ([Fig. 1]), a railway vehicle or a two-wheeler.

[0040] The dashboard 14 is defined as a surface extending across the entire width of the vehicle 10, below the windscreen 12 and above the feet of the driver 16 and, where applicable, the front passenger.

[0041] The pillars 15 are connected to the windscreen 12 and to the roof of the vehicle 10.

[0042] In all that follows, the occupants of vehicle 10 are defined as the driver 16 and / or the front passenger and / or one of the rear passengers of vehicle 10.

[0043] The vehicle 10 includes a sensor block 20 comprising at least one sensor 22, an interactive block 25 comprising at least one display device 28, and an electronic central unit 40 in communication with the sensor block 20 and the interactive block 25.

[0044] The sensor block 20 includes, for example, a single sensor 22.

[0045] In the following examples, the sensory block 20 includes, for example, two sensors 22. Alternatively, the sensory block 20 includes three or more sensors 22.

[0046] The sensors 22 of the sensor block 20 are for example chosen from a set of sensors integrated into the vehicle and / or a set of additional sensors positioned in the vehicle 10.

[0047] By "integrated into the vehicle", it is understood that said sensors 22 are, for example, sensors mounted in the vehicle during the manufacture of the vehicle 10. Such sensors 22 are, for example, a camera (operating in the visible and / or infrared range), a radar, a lidar, a microphone or a gyroscope, belonging to the vehicle 10, and listed in the vehicle 10. The sensors 22 may, for example, include one or more of the sensors of a driver and vehicle occupant monitoring system (DMS, OMS as previously defined).

[0048] By "additional sensors positioned in the vehicle," it is understood that said sensors 22 are, for example, sensors belonging to one of the occupants of the vehicle 10. For example, such sensors are a camera (operating in the visible and / or infrared range), a radar, a lidar, a microphone, or a gyroscope, belonging to one of the occupants of the vehicle 10, for example, integrated into the mobile phone of one of the occupants, and / or installed in the vehicle. For example, said sensors 22 may include a dashcam commonly used to monitor the interior of the vehicle's passenger compartment. Such additional sensors are, for example, connected via Bluetooth to the vehicle 10 and are thus also listed in the vehicle 10.

[0049] In general, each sensor 22 is, for example, independent of each interactive device 28. By "independent", we mean that each sensor is, for example, positioned in the vehicle 10 away from each interactive device 28. In other words, the positions in the vehicle 10 of each sensor 22 and each interactive device 28 are, for example, not linked.

[0050] Each of the sensors 22 of the sensor block 20 is configured to measure, when the sensing field of said sensor 22 covers an area occupied by one or more occupants of the vehicle, the position and movements of the head of the occupant(s) of the vehicle 10. For example, each sensor 22 measures the anterograde and retrograde rotation, and the anterior, posterior, and right and left lateral flexions of the head. Each measurement from each sensor 22 is performed in a reference frame linked to the sensor 22, called the sensor reference frame Rc, shown schematically in [Fig. 4].

[0051] In addition, each of the sensors 22 of the sensor block 20 is configured to measure, when the sensing field of said sensor 22 covers an area occupied by one or more occupant(s) of the vehicle, the position of the eyes of the occupant(s) of the vehicle 10 in the sensor reference Rc.

[0052] When one of the sensors 22 is a microphone, said sensor 22 can for example confirm the direction of the gaze of the occupant(s) of the vehicle 10 according to the direction of the voice of the occupant(s) of the vehicle 10.

[0053] The sensors 22 are also configured, for example, to detect the presence or absence of one or more occupants of the vehicle 10.

[0054] The sensors 22 and the electronic control unit 40 are advantageously configured to detect the direction of gaze of at least one occupant of the vehicle 10 when the occupant is wearing sunglasses, glasses, or corrective lenses, a mask (or any other object such as the occupant's hands) obscuring part of the face, or when the occupant is wearing makeup such as mascara. For example, such sensors 22 are near-infrared cameras with a global shutter and an optical filter for visible light, or hyperspectral cameras (which combine digital imaging in visible and near-infrared light) with a global shutter.

[0055] Furthermore, the sensors 22 operate advantageously and effectively in low-light conditions, such as at night. For example, such sensors 22 are infrared cameras.

[0056] The interactive block 25 is integrated into the vehicle 10 and includes at least one interactive device 28.

[0057] Each interactive device 28 is, for example, a display device in the vehicle and / or a human-machine interface and / or a multimedia device in the vehicle.

[0058] For example, an interactive device 28 is a display device, such as a screen, generating real images or projecting virtual images, or a sound device such as an alarm, a voice assistant or a conversational agent, or other.

[0059] In a particular example, the interactive device 28 forms a part, such as a strip, of the windshield 12, and allows images to be projected onto the windshield 12.

[0060] In the example of [Fig. 1], the vehicle comprises three interactive devices 28. The vehicle 10 comprises, for example, two screens 30, 31 configured to display real visual or audiovisual content on the screen, and a display device 33. The display device 33 comprises an image generation unit 35 and a semi-reflective component 37. The image generation unit 35 is capable of projecting a light beam towards the semi-reflective component 37, and the semi-reflective component 37 is capable of reflecting at least part of the light beam towards the eyes of an occupant of the vehicle 10 to form a virtual image.

[0061] It is understood that any other type of interactive device 28 may be used.

[0062] Such interactive devices 28 are used, for example, to present information related to the driving or operation of vehicle 10 to one or more occupants of vehicle 10. Such interactive devices 28 are for example also used to interact with one or more occupants of vehicle 10 in order to control one or more functions of vehicle 10, or in order to control entertainment content offered in vehicle 10.

[0063] For example, the controlled functionalities may include the management of a multimedia system, the management of the temperature regulation in the passenger compartment, the management of the ignition of the vehicle's lights 10, etc.

[0064] In addition, such interactive devices 28 can be used to alert one or more occupants of the vehicle 10 of a potential danger detected on the roadway using certain sensors of the vehicle 10. For example, ADAS sensors (Advanced Driver Assistance Systems, including, for example, parking sensors, rear cameras, lane departure warning systems and engine monitoring systems) can be used.

[0065] According to the invention, the electronic central unit 40 includes a computer adapted to implement a method of controlling at least one interactive device 28 integrated into the vehicle 10 as a function of the direction of gaze of at least one target occupant of the vehicle.

[0066] The target occupant or occupants of vehicle 10 are chosen from among the driver 16 and / or the front passenger and / or one of the rear passengers of vehicle 10.

[0067] The interactions between the electronic central unit 40, and in particular between the computer, the sensor block 20 and the interactive block 25, are schematically represented in [Fig.2].

[0068] The computer is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the computer and / or memories into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices or storage devices.

[0069] As specific examples, the calculator is implemented in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specifies Integrated Circuit).

[0070] Alternatively, when the method is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium (not shown). The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0071] The operation of the central electronic unit 40 will now be described with reference to [Fig.3], which illustrates an example of the implementation of a control method according to the invention.

[0072] In a particular embodiment, the control process includes a preliminary step 90 (optional) of receiving measurements from each sensor 22 of the sensor block 20.

[0073] In particular, at this stage, the electronic central unit 40 receives information on the number of sensors 22 of the sensor block 20 available, the position and orientation of each sensor 22 relative to the vehicle 10 and information on the nature of the environment captured by each of the sensors 22.

[0074] In other words, this preliminary step 90 concerns the listing of all the integrated or additional sensors 22, available and functional in the vehicle 10.

[0075] For example, the electronic central unit 40 queries the vehicle 10 about all the sensors available and functional in the vehicle 10. Such sensors are, as previously mentioned, listed beforehand in the vehicle 10, either at the time of manufacture of the vehicle 10, or at the time of connection between the additional sensor and the vehicle 10.

[0076] The preliminary step 90 then includes the selection of useful sensors of the sensor block 20 from the set of sensors 22 of the sensor block 20.

[0077] The selection step is carried out so that the capture field of the selected sensors, called useful sensors, includes at least the target occupant of the vehicle 10.

[0078] In other words, each of the useful sensors is capable of measuring the position of the head and eyes of the target occupant in the sensor frame Rc.

[0079] In a particular application example, the sensor block 20 comprises three sensors 22.

[0080] The sensing field of a first sensor 22 includes the driver area, with the driver 16 positioned in the driver's seat. The sensing field of a second sensor 22 includes the driver area, with the driver 16 positioned in the driver's seat, and the front passenger area, with a passenger positioned in the front passenger seat. The sensing field of the third sensor 22 includes the front passenger area with a passenger positioned in the front passenger seat. The sensing field of the third sensor excludes the driver area.

[0081] If, in this case, the target occupant of the vehicle is the driver 16, then, in the preliminary step 90, the first and second sensors 22 are selected and become "useful" sensors for the control process. The third sensor, which does not see the driver 16, and is therefore unable to provide related information the position of the driver's head and / or eyes 16, is not used by the electronic central unit 40 for the rest of the control process.

[0082] If, in another case, the target occupant of the vehicle is the passenger, then, in the preliminary step 90, the second and third sensors 22 are selected and become "useful" sensors for the control process. The first sensor, which does not see the passenger, and is therefore unable to provide information related to the position of the passenger's head and / or eyes, is not used by the electronic control unit 40 for the remainder of the control process.

[0083] In another case, the target occupants are both the driver 16 and the passenger and each of the three aforementioned sensors 22 are selected and become “useful” sensors for the control process.

[0084] In an alternative, not shown, the control method does not include the preliminary step 90. According to this alternative, the electronic central unit 40 includes, for example, a storage memory in which information (nature, position in the vehicle 10, etc.) on the sensors 22 available in the vehicle 10 are referenced beforehand.

[0085] The control process then includes a step of receiving 100 measurements from the useful sensors of the sensor block 20, the capture field of which includes at least the target occupant.

[0086] Each measurement received, taken in the sensor frame Rc, includes the position of the head and eyes of the target occupant in the sensor frame Rc.

[0087] The control method then includes a step 200 of determining the direction of a gaze vector of at least one target occupant in the vehicle's frame of reference, called vehicle frame Rv, as a function of the measurements from the useful sensors and the position of the useful sensors in the vehicle 10.

[0088] The gaze vector includes the gaze direction of at least one target occupant as well as the field of vision associated with the gaze direction of said target occupant.

[0089] In a particular embodiment, when the sensor block 20 includes at least two sensors 22, the step of determining the direction of the gaze vector 200 is carried out by triangulation from the measurements received from the useful sensors in the sensor frame Rc.

[0090] This triangulation step makes it possible to obtain precise information on the direction of the gaze vector. By calibrating the useful sensors with each other, it is possible to obtain the relative position of each useful sensor in order to determine a single gaze vector, in a frame no longer linked to each sensor, but in a common frame linked to the vehicle, called the vehicle frame Rv.

[0091] The plurality of sensors 22 thus makes it possible to obtain better accuracy on the position of the occupant's eyes, and also allows the acquisition of movements of the occupant's head of greater amplitudes.

[0092] In a particular embodiment, the control method includes an intermediate step 250 (optional) of receiving position data from at least one interactive device 28 in the vehicle 10.

[0093] In particular, at this stage, the electronic central unit 40 receives information on the position of each interactive device 28 of the interactive block 25 relative to the vehicle 10 and information on the nature of the or each interactive device 28.

[0094] In other words, this step 250 concerns the listing of all the interactive devices 28 available and functional in the vehicle 10.

[0095] For example, the electronic central unit 40 queries the vehicle 10 about all the interactive devices 28 available and functional in the vehicle 10, which are listed beforehand in the vehicle 10, for example at the time of manufacture of the vehicle 10.

[0096] The intermediate step 250 includes, for example, the selection of certain interactive devices from the interactive block 25 from among all the interactive devices 28 of the interactive block 25.

[0097] The selected interactive devices are, for example, those whose displayed and / or generated content is likely to enter the field of vision of the target occupant, depending on the gaze vector determined in the vehicle frame Rv.

[0098] This intermediate step 250 is for example carried out between the step of determining the direction of the gaze vector 200 and a step of transposing the direction of the gaze vector 300, which will be described below.

[0099] In an alternative, not shown, the control method does not include the intermediate step 250. According to this alternative, the electronic central unit 40 includes, for example, a storage memory in which information (nature, position in the vehicle 10, etc.) on the interactive device(s) in the vehicle 10 are referenced beforehand.

[0100] The transposition step 300 includes the transposition of the direction of the gaze vector determined in a frame linked to at least one interactive device 28, called device frame Rd, of the vehicle 10 as a function of position data of at least one interactive device 28.

[0101] In other words, during this step, the electronic central unit 40 performs all the necessary coordinate system changes in order to position itself in the Rd coordinate system of the or each interactive device 28. The coordinates of the gaze vector in the Rd coordinate system of the or each interactive device 28 can then be obtained.

[0102] During the transposition step 300, the Rd reference points of the interactive device or of each interactive device 28, in particular, where applicable, the reference points linked to the screens of said interactive devices 28, as well as the reference points linked to the real or virtual images displayed on the screens, are considered.

[0103] Two cases are then considered as examples, and illustrated in [Fig.4].

[0104] A step 350 of querying the electronic central processing unit 40 to the interactive block 25 can be planned in order to determine the nature of each interactive device 28.

[0105] At the query step 350, the electronic central unit 40 queries, for example, the interactive block 25 in the following way: is one of the interactive devices 28 configured to generate a virtual image?

[0106] In a first case, when one of the interactive devices 28 considered is a display device generating a real, and not virtual, image (such as a screen on a central console), the transposition step includes a change of reference frame from the vehicle reference frame Rv, to the device reference frame Rd.

[0107] Since the image displayed on the display device is a real image, when the target occupant looks at the image displayed on the screen of the display device, the gaze vector of the target occupant points exactly to the screen of the display device. Furthermore, since the gaze vector is determined in the device frame Rd at this stage, it is possible, as will be described later, to control the display of the interactive device 28 directly from the determined gaze vector.

[0108] In a second case, when one of the interactive devices 28 considered is configured to generate at least one virtual image (such as a head-up display device for example), and when the target occupant looks at the image projected by the interactive device 28, the gaze vector of the target occupant does not point to the image actually displayed on the interactive device 28. It is therefore necessary to correlate the gaze position of the target occupant relative to the virtual image with the corresponding gaze position on the image displayed on the interactive device 28.

[0109] To this end, the transposition step 300 of the direction of the gaze vector in the device frame Rd includes a first 300A and a second 300B sub-step of frame change.

[0110] The first substep 300A of the change of reference frame includes the transposition of the direction of the gaze vector from the vehicle reference frame Rv, into a reference frame Riv linked to the or each virtual image generated by the interactive device 28.

[0111] The second substep 300B of the change of reference frame includes the transposition of the direction of the gaze vector from the Riv reference frame linked to the or each virtual image, into the device reference frame Rd.

[0112] Thus, the interactive device 28 receives instructions for controlling the displayed image, adapted to modify the virtual image in the desired way according to the determined gaze vector.

[0113] These two sub-steps of coordinate system change allow, for example, minimizing the distortion of the virtual image generated by the interactive device 28.

[0114] As previously mentioned, the control method includes a final step 400 of generating at least one control command of at least one interactive device 28.

[0115] Each generated control command is a function of the direction of the gaze vector determined in the considered device frame Rd of the vehicle 10.

[0116] In particular, the generated control command is a function of the position data of the interactive device 28 considered in the vehicle 10.

[0117] Each control command consists, for example, of:

[0118] - a command to activate or deactivate at least one display area of one or more interactive devices 28, and / or

[0119] - a command to modify the dimension, position and / or nature of content displayed on the interactive device(s) 28, and / or

[0120] - a command to validate an action to be carried out for the control of a vehicle-related functionality, and / or

[0121] - a control for generating a visual and / or audible warning signal for the target occupier.

[0122] Various examples of application of the control method according to the invention, and of the commands generated, are described below.

[0123] In a first example, the control method according to the invention makes it possible to activate or deactivate an interactive device 28 according to its position relative to the position of the head and / or eyes of the target occupant.

[0124] This makes it possible, for example, not to leave an interactive device 28 active unnecessarily if the occupant is not looking in its direction, and thus makes it possible to save energy.

[0125] Similarly, if the interactive device 28 is an extended device, the control method allows only certain areas of the interactive device 28 to be activated or deactivated if it allows it, for example if it is equipped with localized lighting or backlighting.

[0126] By "extended device" it is understood that the interactive device 28 includes, for example, a screen covering a driver display area in the driver's field of vision, a central display area in both the driver's and front passenger's field of vision, and a passenger viewing area in the front passenger's field of vision.

[0127] In a second application example, the control method according to the invention makes it possible to modify the size, position and / or nature of content displayed on the interactive device(s) 28, for example to offer better visibility to the target occupant.

[0128] In one particular example, the size of the displayed content (e.g., icons or information) is increased when the sensors 22 detect that the gaze of the target occupant is directed at said content. This type of eye movement is referred to as smooth tracking in eye-tracking.

[0129] In a third application example, the control method according to the invention makes it possible to validate an action to be taken for the control of a functionality related to the vehicle 10.

[0130] For example, when the sensors 22 detect that the gaze of the target occupant is fixed on specific content displayed by one of the interactive devices 28, the control process can generate a control command aimed at selecting the focused content. Such a command allows, for example, entering or exiting a menu or submenu, confirming a choice or an action to be carried out, or other functions.

[0131] In other words, such a control method makes it possible to use the eyes of the target occupant as a pointer to perform an action in the vehicle 10. This type of eye movement is referred to as a fixation movement in oculometry.

[0132] The control method can be used in addition to or as a redundancy to an existing validation system (such as mechanical pressure on a touch interface or on a physical button, gesture or voice recognition, a brain-machine interface, or other).

[0133] In a fourth example, the control method according to the invention makes it possible to generate a visual and / or audible warning signal for the target occupant.

[0134] In one example, the vehicle 10 includes AD AS sensors as defined above, positioned outside the vehicle and capable of detecting an obstacle and / or hazard on the roadway.

[0135] When the electronic central unit 40 calculates, using the head and / or eye position measurements of the target occupant received by the useful sensors, that the obstacle and / or danger is outside the field of vision of the target occupant (for example in this case, the driver 16), the electronic central unit 40 can then generate a specific command aimed at alerting the target occupant of a potential danger.

[0136] In this example the generation command is for example an instruction to generate a visual and / or audible alert signal for the target occupant, or to activate or modify the content of an interactive device 28 located in the field of vision and / or in the direction of the gaze of the target occupant.

[0137] In a particular embodiment, the method includes one or more features aimed at improving the accuracy of the determination of the gaze vector.

[0138] For example, the control process is implemented using a Kalman filter, associated or not with artificial intelligence, for example with machine learning.

[0139] In an example of an application using artificial intelligence, audiovisual content is displayed on a passenger-side screen for a front passenger in the vehicle. The artificial intelligence remembers that such content is being offered to the front passenger. If one of the useful sensors detects that the head and / or gaze of the driver 16 is directed towards the passenger-side screen, the artificial intelligence can remind the driver 16 to pay attention, for example by interrupting the audiovisual content being offered, or by alerting the driver 16 to stop the distraction.

[0140] Such a method of controlling at least one interactive device 28 according to the direction of gaze of at least one target occupant of the vehicle has many advantages.

[0141] The control method allows the content of the interactive devices 28 to be adapted and / or controlled in real time according to the position of the head and / or the gaze of the occupants of the vehicle 10.

[0142] Such a method has a notable advantage over a driver monitoring system such as a DMS, in that it allows the use of one or more sensor(s) 22 independently of the position of the interactive device 28 to be controlled to generate a control command for said device 28.

[0143] By "independently", it is understood that each sensor 22 can be positioned away from the interactive device 28, the positions of the sensors 22 and the interactive devices 28 not being linked.

[0144] Such an advantage is particularly due to the steps of determining the direction of the gaze vector, transposing the direction of the gaze vector determined in the device frame, and controlling the interactive device 28 as a function of the direction of the gaze vector transposed in the frame of said interactive device 28.

[0145] Preferably, the plurality of sensors 22 makes it possible to increase the accuracy of the determination of the gaze vector and thus to precisely control the content of the interactive devices 28 according to the position of the head and / or the gaze of the occupants of the vehicle 10.

[0146] The set of reference frame changes made for determining the gaze vector in each reference frame linked to the interactive device allows the sensors 22 existing in the vehicle to be used directly and not necessarily those fixed below, above or inside the interactive device considered.

[0147] Consequently, it reduces the cognitive load on vehicle occupants, who can control interactive devices simply and quickly using their gaze. Furthermore, the method allows for more precise and safer targeting of the nature and / or position of information to be presented to occupants in order to limit distractions from interactive devices.28

[0148] A person skilled in the art will understand that the examples and variants described above can be combined with each other.

[0149] Those skilled in the art will also understand that the present description is based on the specific case of motor vehicles, but that the invention is not limited to this type of vehicle. The control method is thus adaptable to all types of vehicles, including aircraft and ships.

Claims

Demands

1. A method for controlling at least one interactive device (28) integrated into a vehicle (10) as a function of the gaze direction of at least one target occupant of the vehicle, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the method being implemented by an electronic central unit (40), the method comprising the steps of: - receiving (100) measurements from sensor(s), referred to as useful sensors, of the sensor block (20) whose field of view includes at least the target occupant, the measurements from each sensor including the position of the head and / or eyes of the target occupant in the frame of said sensor, referred to as sensor frame (Rc), - determining (200) the direction of the gaze vector of at least one target occupant in the frame of the vehicle, referred to as vehicle frame (Rv),based on measurements from the useful sensors and the position of the useful sensors in the vehicle (10), - transposition (300) of the direction of the gaze vector determined in a frame linked to at least one interactive device (28) of the vehicle (10), called the device frame (Rd), as a function of position data of the at least one interactive device (28), and - generation (400) of at least one control command of the at least one interactive device (28), each control command being a function of the direction of the gaze vector determined in the device frame (Rd).

2. A control method according to claim 1, wherein the sensor block (20) comprises at least two sensors (22).

3. A control method according to claim 1 or 2, wherein each sensor (22) of the sensor block (20) is selected from a sensor integrated into the vehicle (10) and / or an additional sensor positioned in the vehicle, each sensor (22) being, for example, selected from a camera, a radar, a lidar, a microphone, or a gyroscope.

4. A control method according to any one of the preceding claims, wherein at least one interactive device (28) is chosen from a display device in the vehicle (10) and / or a human-machine interface and / or a multimedia device in the vehicle.

5. A control method according to any one of the preceding claims, wherein at least one control command consists of: - a command to activate or deactivate at least one display area of ​​one or more interactive devices (28), and / or - a command to modify the size, position and / or nature of content displayed on the interactive device(s) (28), and / or - a command to validate an action to be taken for the control of a vehicle-related functionality (10), and / or - a command to generate a visual and / or audible warning signal for the target occupant.

6. A control method according to any one of the preceding claims, wherein the step of determining (200) the direction of the gaze vector of at least one target occupant in the vehicle frame (Rv) is carried out by triangulation from the measurements received from the useful sensors in the sensor frame (Rc).

7. A control method according to any one of the preceding claims, wherein, when at least one of the interactive devices (28) is configured to generate at least one virtual image, the transposition step (300) of the gaze vector direction in the device frame (Rd) comprises: - a first substep (300A) of frame change, the first substep (300A) of frame change comprising the transposition of the gaze vector direction from the vehicle frame (Rv) into a frame (Riv) linked to the virtual image(s), and - a second substep (300B) of frame change, the second substep (300B) of frame change comprising the transposition of the gaze vector direction from the frame (Riv) linked to the virtual image(s), into the device frame (Rd).

8. A control method according to any one of the preceding claims, comprising a preliminary step (90) of receiving measurements from each sensor (22) of the sensor block (20), and selection of the useful sensor(s) of the sensor block (20) from among the set of sensors (22) of the sensor block (20), the capture field of the selected useful sensors including at least the target occupant.

9. A control method according to any one of the preceding claims, comprising an intermediate step (250) of receiving position data from at least one interactive device (28) in the vehicle, the generated control command being a function of the position data from at least one interactive device (28) in the vehicle (10).

10. Electronic central unit (40) for controlling at least one interactive device (28) integrated into a vehicle based on the gaze direction of at least one target occupant of the vehicle, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the electronic central unit (40) comprising a computer configured to: - receive (100) measurements from sensor(s), called useful sensors, of the sensor block (20) whose capture field includes at least the target occupant, the measurements from each sensor including the position of the head and / or eyes of the target occupant in the reference frame of said sensor, called sensor reference frame (Rc), - determine (200) the direction of the gaze vector of at least one target occupant in the vehicle's frame of reference, called vehicle frame (Rv), as a function of the measurements from the useful sensors and the position of the useful sensors in the vehicle (10), - transpose (300) the direction of the gaze vector determined in a frame linked to at least one interactive device (28) of the vehicle (10), called the device frame (Rd), as a function of position data of at least one interactive device (28), and - generate (400) at least one control command of at least one interactive device (28), each control command being a function of the direction of the gaze vector determined in the device frame (Rd).

11. Control system of at least one interactive device (28) integrated in a vehicle (10) according to the gaze direction of at least one target occupant of the vehicle, the system comprising: - a sensor block (20) comprising at least one sensor (22) configured to measure the position of the head and / or eyes of at least one target occupant, - an electronic central unit (40) according to claim 9, and - an interactive block (25) integrated in the vehicle (10), the interactive block (25) comprising at least one interactive device (28).

Images