Method for controlling at least one vision system worn by a target occupant of a vehicle

Abstract

The present invention relates to a method for controlling a vision system (19), worn on the head by a target occupant in a vehicle (10), the method comprising the steps of: - receiving measurements from sensors, referred to as useful sensors, comprising the position and the orientation of the vision system (19) worn on the head by the target occupant in the reference frame of the sensor, referred to as sensor reference frame; - determining the position and the orientation of the vision system (19) in the reference frame of the vehicle (10), referred to as vehicle reference frame; and - generating at least one command for controlling the vision system (19), each control command depending on the position and the orientation of the vision system (19) in the vehicle reference frame.

Description

[0001] TITLE: Method for controlling at least one vision system worn by a target occupant of a vehicle

[0002] The present invention relates to a method for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of that occupant. The present invention also relates to an electronic control unit for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of that occupant. Finally, the present invention relates to a control system for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of that occupant.

[0003] Passenger transport vehicles, and in particular civil and commercial aircraft, are equipped with interactive devices (HMI, human-machine interface, defined as a user interface allowing a person to connect to a machine, system or device), such as screens, generating real or virtual images, sound systems, etc., used for example during takeoff and landing phases, in order to improve the safety of maneuvers.

[0004] In aircraft, such interactive devices advantageously take the form of Head-Up Displays (HUDs) positioned in front of the vehicle occupant, for example in front of the pilot, in order to limit deviations in the pilot's gaze during the different phases of flight.

[0005] When coupled with onboard cameras, such as cameras of an enhanced flight vision system (EFVS), such head-up displays allow the display in the eyebox of piloting and navigation reticles, a combined vision system (CVS), a synthetic vision system (SVS) synthetically representing the environment of the vehicle (e.g. aircraft) in three dimensions, for example in the form of a virtual image in the pilot's field of vision.

[0006] However, the virtual image size has a limited geometry due to the space constraints of the head-up display in the cockpit, which must be kept to a minimum, as well as the pilot's field of vision, which must not be obstructed. Therefore, with such head-up displays, it is not possible for the pilot to obtain augmented or virtual reality information (such as synthetic vision images) outside of their immediate field of vision.

[0007] To overcome this drawback, it is known, for example in the field of military aviation, to replace the head-up display installed in the aircraft with a head-up display system on or in the helmet (Helmet Mounted Display - HMD) worn by the pilot.

[0008] For this purpose, such helmets are for example equipped with infrared emitters or an electromagnetic element configured to modify the electromagnetic field in which the cockpit is bathed.

[0009] Such a helmet allows the pilot to receive augmented (or virtual) reality information in any field of vision (from any angle) corresponding to the position and orientation of his head in real time.

[0010] However, such a solution is not entirely satisfactory. In particular, optical methods (infrared emitters) are sensitive to solar radiation and heat, and magnetic methods must necessarily take into account the effects of changes in the magnetic field on the operation of electronic systems in the cockpit.

[0011] Furthermore, such solutions require a heavy and complex installation in the cockpit, specific to each cockpit, and are therefore extremely expensive.

[0012] Eye-tracking, oculometry, or gaze-tracking, is also a technique for controlling interactive devices, known and implemented for some time in the fields of computer science, transport, and mobility.

[0013] However, eye tracking also has its limitations. Indeed, eye tracking cannot be used when the eyes of the target occupant are not visible, as is the case when the occupant is wearing a vision system, for example, in the field of aviation.

[0014] One of the aims of the invention is therefore to propose a method of controlling at least one vision system worn on the head of a target occupant of a vehicle, allowing the vision system(s) to be controlled in such a way as to display information adapted to the target occupant, over a field of vision of up to 360°.

[0015] To this end, the invention relates to a method for controlling at least one vision system, worn by the head of a target occupant in a vehicle, as a function of the position and orientation of the head of at least one target occupant, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the method being implemented by an electronic central unit, the method comprising the steps of: receiving measurements from sensors, referred to as useful sensors, of the sensor block whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and orientation of the vision system worn by the head of at least one target occupant in the frame of reference of said sensor, referred to as the sensor frame, determining the position and orientation of the vision system worn by the head of at least one target occupant in the frame of reference of the vehicle, referred to as the vehicle frame.based on measurements from the relevant sensors and their position within the vehicle, and generation of at least one vision system control command, each control command being a function of the vision system's position and orientation within the vehicle's frame of reference.

[0016] This monitoring method, thanks in part to the use of sensors and the determination of the position and orientation of the head-mounted vision system of the target occupant within the vehicle's frame of reference, allows for tracking the direction, position, and orientation of the vehicle occupants' heads. Tracking is possible even with significant head movements and rotations. Furthermore, the tracking is independent of the specific sensor being used.

[0017] Generating a vision system control command based on the determined position and orientation allows the occupant to receive content tailored to their head position and orientation in real time. This enables more precise targeting of the nature and / or location of the information presented to the occupant (whether real or synthetic images, for example), thereby improving the safety of driving or piloting maneuvers.

[0018] According to other advantageous aspects of the invention, the control method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0019] - the sensor or each sensor of the sensor block is chosen from a sensor integrated into the vehicle and / or an additional sensor positioned in the vehicle, each sensor being chosen for example from a camera, or a lidar;

[0020] - the step of determining the position and orientation of the vision system carried by the head of at least one target occupant in the vehicle frame, includes a first sub-step of recognition of the vision system, the first sub-step of recognition being based on a correlation between one or more input data and the measurements from the useful sensors, each input data including predefined geometry information of known vision systems;

[0021] - the step of determining the position and orientation of the vision system carried by the head of at least one target occupant in the vehicle frame of reference, includes a second sub-step of changing the frame of reference, the second sub-step of changing the frame of reference including the transposition of the position and orientation of the vision system carried by the head of at least one target occupant from the sensor frame of reference, into the vehicle frame of reference;

[0022] - the vision system is chosen from augmented or virtual reality glasses, an augmented or virtual reality headset, or an augmented or virtual reality mask;

[0023] - at least one control command consists of:

[0024] - a command to modify the position and / or nature of content displayed in the vision system, and / or

[0025] - a command to generate a visual and / or audible alert signal for the target occupant;

[0026] - the vehicle includes an interactive block integrated into the vehicle, the interactive block comprising at least one interactive device, the control process being further configured to generate a control command for at least one interactive device, each control command being a function of the position and orientation of the vision system worn by the head of at least one target occupant in the vehicle frame of reference;

[0027] - the control process includes a preliminary step of receiving measurements from each sensor of the sensor block, and of selecting the useful sensors of the sensor block from among the set of sensors of the sensor block, the capture field of the selected useful sensors including at least a portion of the head of the target occupant;

[0028] - the vehicle includes at least one environmental sensor configured to capture at least one piece of information and / or image of the environment outside the vehicle, the process including a step of receiving data from the one or each environmental sensor, the step of generating a command consisting of generating a command to display said data on the vision system.

[0029] The invention also relates to an electronic central control unit for at least one vision system, worn on the head of a target occupant in a vehicle, based on the position and orientation of the head of at least one target occupant, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the electronic central control unit comprising a computer configured to: receive measurements from sensors, referred to as useful sensors, of the sensor block whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and orientation of the vision system worn on the head of at least one target occupant in the frame of reference of said sensor, referred to as sensor frame, determine the position and orientation of the vision system in the frame of reference of the vehicle, referred to as vehicle frame.based on measurements from the relevant sensors and the position of the relevant sensors in the vehicle, and generate at least one vision system control command, each control command being a function of the position and orientation of the head-mounted vision system of at least one target occupant in the vehicle's frame of reference.

[0030] The invention also relates to a control system for at least one vision system, worn by the head of a target occupant in a vehicle, based on the position and orientation of the head of at least one target occupant, the system comprising: a sensor block including at least one sensor configured to measure the position and orientation of the vision system worn by the head of at least one target occupant, and an electronic central unit as described above.

[0031] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings which are:

[0032] - [Fig. 1] Figure 1, a schematic representation of an example of an aircraft cockpit in which a sensor block is integrated and in which an occupant wearing augmented (or virtual) reality glasses is located,

[0033] - [Fig 2] Figure 2, a schematic representation of an electronic central unit, a sensor block and an interactive block according to the invention,

[0034] - [Fig 3] Figure 3, a flowchart of the steps implemented by the electronic central unit in the control process according to the invention, and

[0035] - [Fig 4] Figure 4, a schematic representation of all the reference frame changes made by the electronic central unit in the control process according to the invention.

[0036] A vehicle 10 is illustrated in Figure 1. The vehicle 10 is, for example, a land, air, or sea vehicle. In the example in Figure 1, and throughout what follows, the vehicle 10 is an aircraft. Here, aircraft 10 is understood to mean any airplane, helicopter, or other flying machine that can be piloted by a pilot 14 from that aircraft. The invention nevertheless extends to any other type of vehicle.

[0037] In this example, the vehicle 10 includes a cockpit 12 inside which is installed a pilot 14.

[0038] The cockpit 12 includes a floor 16, at least one seat fixed to the floor 16, an instrument panel 18 and a windscreen (not shown) at least partially transparent and separating the interior of the cockpit 12 from the external environment of the aircraft 10, the head 17 of the pilot 14 looking towards the instrument panel 18 and the windscreen.

[0039] Throughout the following, the occupants of vehicle 10 are defined as the pilot 14 and / or the co-pilot and / or a passenger of vehicle 10.

[0040] As shown in Figure 1, the occupant of the vehicle 10, in particular the pilot 14, wears a vision system 19 on his head 17.

[0041] The vision system 19 is a system for displaying real and / or synthetic content to the occupant wearing said system 19. The vision system 19 is, for example, an augmented (or virtual) reality system.

[0042] The vision system 19 is carried by the head 17 of an occupant of the vehicle 10, such as the pilot 14.

[0043] The vision system 19 defines the field of vision of its wearer. The wearer's field of vision is therefore considered to coincide with the field of vision of the vision system 19 carried by the wearer's head 17.

[0044] The vision system 19 is chosen for example from augmented or virtual reality glasses, also called smart glasses, an augmented or virtual reality headset, an augmented or virtual reality mask, or any other object configured to display real or synthetic content to the occupant.

[0045] Such vision systems 19 assist, for example, the pilot 14 or the co-pilot in their maneuvers by displaying them appropriate information (images of the environment, GPS navigation instructions, information on driving or piloting or on the vehicle 10, etc.).

[0046] The vehicle 10 includes a sensor block 20 comprising at least one sensor 22 and an electronic central unit 40 in communication with the sensor block 20. In a particular embodiment, the vehicle 10 also includes an interactive block 25 comprising at least one interactive device 28, the electronic central unit 40 being for example in communication with the interactive block 25.

[0047] Sensory block 20 includes, for example, two sensors 22. Alternatively, sensory block 20 includes three or more sensors 22.

[0048] The sensors 22 of the sensor block 20 are for example chosen from a set of sensors integrated into the vehicle and / or a set of additional sensors positioned in the vehicle 10.

[0049] By "integrated into the vehicle", it is understood that said sensors 22 are, for example, sensors mounted in the vehicle during the manufacture of the vehicle 10. Such sensors 22 are, for example, a camera (operating in the visible and / or infrared range), or a lidar, belonging to the vehicle 10, and listed in the vehicle 10.

[0050] The term "additional sensors positioned in the vehicle" refers to sensors 22 that are retrofitted to the vehicle 10, i.e., after the vehicle 10 has been designed. For example, at least one additional sensor belongs to one of the occupants of the vehicle 10. Such sensors might be a camera (operating in the visible and / or infrared range) or a lidar device belonging to one of the occupants of the vehicle 10, for example, integrated into one of the occupants' mobile phones and / or installed in the vehicle. Such additional sensors are, for example, connected to the vehicle 10 via wired or wireless (e.g., Bluetooth) connections and are thus also listed as part of the vehicle 10.

[0051] Each of the sensors 22 of the sensor block 20 is configured to measure, when the sensing field of said sensor 22 covers an area occupied by one or more occupants of the vehicle 10, the position and orientation of the head-mounted vision system 19 worn by its wearer within the vehicle 10. Thus, at least a portion of the wearer's head (the portion of the head carrying the vision system 19) is within the sensing field of each of the sensors 22; advantageously, the entire head of the wearer is within the sensing field of each of the sensors 22. Each measurement from each sensor 22 is performed in a reference frame linked to the sensor 22, called sensor reference frame R. c , represented schematically in figure 4.

[0052] Thus, based on the position and orientation of the vision system 19, each of the sensors 22 of the sensor block 20, when the field of view of said sensor 22 covers an area occupied by one or more occupant(s) of the vehicle, provides information on the position and orientation of the head 17 of the occupant(s) of the vehicle 10 in the sensor frame R c . The sensors 22 are for example positioned at different locations in the cockpit 12 in order to track and improve the accuracy of the head tracking 17 of the wearer of the vision system, for example the pilot 14 or the co-pilot in the cockpit 12, particularly in the extreme angles of the cockpit 12.

[0053] Furthermore, the 22 sensors operate advantageously and efficiently in low-light conditions, such as at night. For example, infrared cameras are such 22 sensors.

[0054] Additionally, a sensor 26 integrated on the vision system 19, such as a camera or a gyroscope, can be used to confirm the position and orientation of the vision system 19 in a reference frame linked to the vehicle 10, called the vehicle reference frame R v .

[0055] Optionally, the sensors 22 can also be used to collect vital signs of the pilot 14 or co-pilot, in order to determine a state of fatigue, malaise or other.

[0056] In a particular embodiment, the vehicle 10 further comprises at least one environmental sensor 27 configured to capture at least one piece of information and / or image of the environment outside the vehicle 10. Preferably, the vehicle 10 comprises a plurality of environmental sensors 27, arranged at different locations on the vehicle 10, in order to capture data from outside the vehicle 10 over the largest possible angular range.

[0057] As will be described in more detail later, based on the position and orientation of the vision system 19 in the vehicle frame R v and data received by the environmental sensors 27, the electronic central unit 40 determines the content to be displayed in the vision system 19.

[0058] The content displayed depends on parameters specific to the vision system 19, such as the field of view of the vision system 19, the projection distance, the parallax, etc.

[0059] The interactive block 25 is integrated into the vehicle 10 and includes, for example, at least one interactive device 28.

[0060] Each interactive device 28 is, for example, a display device in the vehicle and / or a human-machine interface and / or a navigation device and / or a multimedia device in the vehicle, used to present information and / or images related to the piloting or operation of the vehicle 10 to one or more occupants of the vehicle 10.

[0061] In addition, such interactive devices 28 can be used to alert one or more occupants of the vehicle 10 of a potential danger spotted in the air or on the ground using certain sensors of the vehicle 10. For example, enhanced flight vision system (EFVS) cameras, radars or lidars can be used.

[0062] According to the invention, the electronic central unit 40 includes a computer suitable for implementing a method of controlling the vision system 19 according to the position and orientation of the head of at least one target occupant of the vehicle.

[0063] The target occupant or occupants of vehicle 10 is, for example, chosen from among the pilot 14 and / or the co-pilot of vehicle 10.

[0064] The interactions between the central electronic unit 40, and in particular between the computer, the sensor block 20, the vision system 19 and optionally the interactive block 25 are schematically represented in Figure 2.

[0065] The computer is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the computer and / or memories into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices or storage devices.

[0066] As specific examples, the computer is implemented as a programmable logic component, such as an FPGA (Field Programmable Gate Array), or as an integrated circuit, such as an ASIC (Application Specific Integrated Circuit).

[0067] Alternatively, when the process is implemented as one or more software programs, that is, as a computer program, also called a computer program product, it is also capable of being stored on a computer-readable medium, not shown here. A computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. Examples of such a readable medium include an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program containing software instructions is then stored on this readable medium.

[0068] The operation of the central electronic unit 40 will now be described with reference to Figure 3, which illustrates an example of the implementation of a control method according to the invention.

[0069] In a particular embodiment, the control process includes a preliminary step 90 (optional) of receiving measurements from each sensor 22 of the sensor block 20. In particular, at this step, the electronic central unit 40 receives information on the number of sensors 22 of the sensor block 20 available, the position and orientation of each sensor 22 relative to the vehicle 10, the sensing field of each sensor 22, and information on the nature of the environment sensed by each of the sensors 22.

[0070] In other words, this preliminary step 90 concerns the listing of all 22 integrated or additional sensors, available and functional in the vehicle 10.

[0071] For example, the electronic central unit 40 queries the vehicle 10 about all the sensors available and functional in the vehicle 10. Such sensors are, as previously mentioned, listed beforehand in the vehicle 10, either at the time of manufacture of the vehicle 10, or at the time of connection between the additional sensor and the vehicle 10.

[0072] The preliminary step 90 then includes the selection of useful sensors from sensor block 20 from the set of sensors 22 of sensor block 20.

[0073] The selection step is carried out so that the field of capture of the selected sensors, called useful sensors, includes at least the head 17 or the vision system 19 carried by the head of the target occupant of the vehicle 10.

[0074] In other words, each of the useful sensors is capable of measuring the position and orientation of the vision system 19 worn by the head of the target occupant, and therefore implicitly of the head of the target occupant in the sensor frame R c .

[0075] Those skilled in the art will naturally understand that a plurality of sensors 22 makes it possible to increase the accuracy of position and orientation measurement of the vision system 19 and to cover a greater angular extent around the target occupant, than if a single sensor 22 were used in the vehicle 10.

[0076] In one variant, not shown, the control process does not include the preliminary step 90. According to this variant, the electronic central unit 40 includes, for example, a storage memory in which information (nature, position in the vehicle 10, etc.) on the sensors 22 available in the vehicle 10 are referenced beforehand.

[0077] The control process then includes a step of receiving 100 measurements from the useful sensors of the sensor block 20, whose capture field includes at least a portion of the head of the target occupant.

[0078] Each measurement received, taken in the sensor reference R c , includes the position and orientation of the vision system 19 worn by the head of the target occupant in the sensor frame R c The reception step 100, for example, is performed in real time. Thus, the measurements from the useful sensors of the sensor block 20 correspond to the position and orientation of the vision system 19 worn by the head of the target occupant in the sensor frame R c in real time.

[0079] The control procedure then includes a step 200 of determining the position and orientation of the vision system 19 worn by the head of the target occupant in the vehicle frame R v, depending on the measurements from the useful sensors and the position of the useful sensors in the vehicle 10.

[0080] In a particular embodiment, for example when the vehicle 10 includes a plurality of sensors 22, the step 200 of determining the position and orientation of the vision system 19 is carried out by triangulation from the measurements received from the useful sensors in the sensor frame R c .

[0081] This triangulation step provides precise information on the position and orientation of the vision system 19 in the cockpit 12. By calibrating the relevant sensors, it is possible to obtain the relative position of each sensor in order to determine the position and orientation of the vision system 19 in a frame of reference no longer linked to each individual sensor, but in a common frame of reference linked to the vehicle, referred to above as the vehicle frame of reference R. v .

[0082] In a particular embodiment, step 200 determines the position and orientation of the target occupant's vision system 19 in the vehicle frame R v , includes a first sub-stage 200A of recognition of the vision system 19 worn by the head of the target occupant in the vehicle 10.

[0083] The first 200A recognition sub-step is based on a correlation between one or more De input data and measurements from useful sensors.

[0084] Each input data De includes, in particular, predefined geometry information from known vision systems. For example, the input data De represents geometry data associated with a model of augmented (or virtual) reality glasses, whose geometry is perfectly known and invariant from one pair of glasses to another, or represents geometry data associated with an augmented (or virtual) reality headset or mask.

[0085] This sub-step 200A, which recognizes the head-mounted vision system 19 of the target occupant in the vehicle 10, is based, for example, on a deep learning approach using neural networks. The geometry of the vision system 19 then serves as an input parameter for recognizing the specific vision system and for subsequently determining its position, orientation, and the rotations of the head 17 of the pilot 14 or co-pilot.

[0086] According to this example, step 200 determines the position and orientation of the vision system 19 worn by the head of the target occupant in the vehicle's reference frame R v , includes a second sub-step 200B of reference frame change.

[0087] The second substep of the 200B reference frame change involves transposing the position and orientation of the head-mounted vision system 19 of the target occupant from the sensor reference frame R c , in the vehicle R frame v .

[0088] The process includes a step 225 for receiving data from the environmental sensor(s) 27. This step 225 can be implemented before or after the step 200 for determining the position and orientation of the head-mounted vision system 19 worn by the target occupant. In this step 225, the electronic control unit 40 receives information and / or images of the environment outside the vehicle 10, captured by the environmental sensor(s) 27.

[0089] In other words, each environmental sensor 27 shares the data captured (whether information related to the external environment, or images acquired from the external environment) with the electronic central unit 40.

[0090] In a particular embodiment, the control process includes an intermediate step 250 (optional) of receiving position data from the interactive devices 28 in the vehicle 10 and selecting the interactive devices 28 whose displayed and / or generated content is likely to enter the field of vision of the vision system 19.

[0091] Optionally, a transposition step 300 is provided to transpose the position and orientation of the vision system 19 (and therefore implicitly of the occupant's head 17) determined in a frame linked to at least one interactive device 28, called the device frame R d , of vehicle 10 based on position data from at least one interactive device 28.

[0092] During transposition step 300, the R reference frames dof each interactive device 28, in particular, where applicable, the markers linked to the screens of said interactive devices 28, as well as the markers linked to real or virtual images, are considered.

[0093] Such reference points are illustrated in particular in figure 4.

[0094] As previously mentioned, the control process includes a final step 400 of generating at least one control command for the vision system 19. Each generated control command is a function of the position and orientation of the vision system 19, and therefore implicitly of the head of the target occupant, determined in the vehicle frame of reference R. v .

[0095] Each control command consists, for example, of:

[0096] - a command to modify the size, position and / or nature of content displayed in the vision system 19, and / or

[0097] - a command to generate a visual and / or audible alert signal for the target occupant.

[0098] For example, the control command consists of a command to display data (information and / or images, real or synthesized) from the environmental sensor(s) 27 in the vision system 19. Advantageously, the data displayed in the vision system 19 depends on the position and orientation of the augmented (or virtual) reality system 19 in the vehicle frame of reference R v Thus, the electronic central unit 40 determines the data to be displayed in the vision system 19 according to the field of vision of the vision system 19.

[0099] For example, when the electronic central unit 40 determines that the target occupant turns their head 17 to the right, the content displayed in the vision system 19 will advantageously be based on the information received by the environmental sensors 27 located to the right of the vehicle 10, corresponding to the real-time field of vision of the vision system 19.

[0100] In addition, the electronic central unit 40 determines the content to be displayed according to parameters specific to the vision system 19, such as the field of view, projection distance, parallax, etc.

[0101] Thus, the content displayed in the vision system 19 is consistent with the performance of the vision system 19 and with the real-time position and orientation of the head 17 of the target occupant, and therefore of the field of vision of the vision system 19.

[0102] In one particular embodiment, the control process is further configured to generate a control command for at least one interactive device 28.

[0103] Thus, the interactive device 28 receives instructions to control the displayed image, adapted to modify (or activate / deactivate) the content displayed on the interactive device 28 according to the position and orientation of the vision system 19.

[0104] Various examples of application of the control method according to the invention, and of the commands generated, are described below.

[0105] In a first application example, the control method according to the invention makes it possible to modify the position and / or nature of content displayed in the vision system 19 (or on the interactive device(s) 28), for example to offer better visibility to the target occupant.

[0106] In particular, in this example, when the vision system 19 is controlled by the electronic central unit 40, the position and nature of the images of the external environment displayed in the vision system 19 are adapted to match the real-time visual field of the vision system 19.

[0107] Thus, pilot 14 can orient his head and receive information about the external environment in augmented (or virtual) reality in the corresponding field of vision.

[0108] In addition, the displayed content is adapted to the projection distances and the parallax of the vision system 19.

[0109] The plurality of environmental sensors 27 allows the pilot 14 to receive adequate and consistent data from the outside regardless of the position and orientation of his head 17 and his field of vision.

[0110] In a second example, the control method according to the invention makes it possible to generate a visual and / or audible alert signal for the target occupant.

[0111] In one example, vehicle 10 includes an EFVS system, as defined above, capable of detecting an obstacle and / or hazard in the trajectory of vehicle 10.

[0112] When the electronic central unit 40 calculates, using the position measurements and orientation of the vision system 19, and therefore of the head of the target occupant, received by the useful sensors, that the obstacle and / or danger is outside the field of vision of the target occupant (for example in this case, the pilot 14), the electronic central unit 40 can then generate a specific command aimed at alerting the target occupant of a potential danger.

[0113] In this example, the generation command is, for example, an instruction to generate a visual and / or audible alert signal for the target occupant, or to activate or modify content of the vision system 19 or an interactive device 28 located in the field of vision of the target occupant.

[0114] In another example, the control method according to the invention makes it possible to activate or deactivate an interactive device 28 according to its position in the vehicle 10, relative to the position and orientation of the head 17 of the target occupant.

[0115] This prevents, for example, an interactive device 28 from remaining unnecessarily active if the occupant is not looking in its direction, thus saving energy. Similarly, if the interactive device 28 is an extended device, the control process allows only certain areas of the interactive device 28 to be activated or deactivated, if the device allows it, for example, if it is equipped with localized lighting or backlighting.

[0116] By "extended device" we mean that the interactive device 28 includes, for example, a screen covering a pilot or driver display area in the field of vision of the pilot 14, a central display area in the field of vision of both the pilot 14 and the co-pilot, and a secondary viewing area in the field of vision of the co-pilot only.

[0117] In a particular embodiment, the method includes one or more features aimed at improving the accuracy of determining the position and orientation of the vision system 19 in the vehicle reference frame Rv.

[0118] For example, the control process is implemented using a Kalman filter, associated or not with artificial intelligence, for example machine learning.

[0119] The Kalman filter, for example, combines the measurements received on the position and orientation of the vision system 19 with a physical model in order to determine more precisely the position and orientation of the vision system 19. Thus, such a filter makes it possible to improve the operation of the control process, by improving the accuracy of the position and orientation of the head of the occupant in the vehicle 10 and thus makes it possible to adjust the content displayed to the occupant more precisely.

[0120] Optionally, the target occupant can validate / invalidate certain actions to be taken, previously determined by the position and orientation of the target occupant's head 17, by voice using a conversational assistant for example, by interaction with a touch interface, or by using a physical button such as a control lever.

[0121] In addition, the head 17 of the target occupant can also serve as a pointer in the content displayed in the vision system 19 in order to select a graphic element among others and to validate / invalidate an action.

[0122] Such actions relate for example to functionalities related to driving, or to piloting the vehicle 10 or to the vehicle 10 itself, such as the management of navigation systems and / or control systems and / or notification / alert systems, and / or multimedia systems, the control of the thermal environment inside the vehicle 10, or the control of the lights of the vehicle 10.

[0123] Such a method of controlling the vision system 19 based on the head direction of at least one target occupant of the vehicle offers numerous advantages. The control method allows the content of the vision system 19, and optionally the interactive devices 28, to be adapted and / or controlled in real time according to the position and orientation of the heads of the vehicle occupants 10. The display of information in the vision system 19 can then be adapted to the morphology of the driver 14 or co-pilot, and to the driving position.

[0124] The electronic central unit 40 determines the content to be displayed according to parameters specific to the vision system 19, such as the field of vision, the projection distance, the parallax, etc., the content displayed in the vision system 19 is consistent with the performance of the vision system 19 and with the real-time field of vision of the vision system 19, depending essentially on the position and orientation of the head 17 of the target occupant.

[0125] Furthermore, the head-mounted vision system 19 worn by the pilot 14 or co-pilot can be a simple optical virtual or augmented reality system, without onboard electronics or sensors. Such a vision system is therefore lighter and less bulky for the vehicle's occupants 10.

[0126] Such a control method is independent of the cockpit in question and of the augmented (or virtual) reality display technology. Indeed, this control method is easily adaptable to any type of vision system and can accommodate any future evolutions of that system (geometry and display technology). Furthermore, in the case of an aircraft, such a method can be implemented both during flight and on the ground.

[0127] Finally, in the case where the vision system is equipped with an eye-tracking device, the direction of gaze can also be taken into account to control the vision system 19. A person skilled in the art will understand that the examples and variants described above can be combined with each other.

[0128] Those skilled in the art will also understand that this description is based on the specific case of aircraft, but that the invention is not limited to this type of vehicle. The control method is therefore adaptable to all types of vehicles, including automobiles, railways, and ships.

Claims

DEMANDS 1. A method for controlling at least one vision system (19), worn on the head of a target occupant in a vehicle (10), as a function of the position and orientation of the head (17) of at least one target occupant, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the method being implemented by an electronic central unit (40), the method comprising the steps of: receiving (100) measurements from sensors, referred to as useful sensors, of the sensor block (20) whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and orientation of the vision system (19) worn on the head of at least one target occupant in the frame of said sensor, referred to as the sensor frame (R c), determination (200) of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (10), referred to as the vehicle frame of reference (R v ), based on measurements from the useful sensors and the position of the useful sensors in the vehicle (10), and generation (400) of at least one control command for the vision system (19), each control command being a function of the position and orientation of the vision system (19) in the vehicle frame of reference (R v ).

2. Control method according to claim 1, wherein the sensor or each sensor (22) of the sensor block (20) is chosen from a sensor integrated in the vehicle (10) and / or an additional sensor positioned in the vehicle (10), each sensor (22) being chosen for example from a camera, or a lidar.

3. A control method according to claim 1 or 2, wherein the step of determining (200) the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (R v ), includes a first substep (200A) of vision system recognition (19), the first recognition substep (200A) being based on a correlation between one or more input data (De) and measurements from useful sensors, each input data (De) including predefined geometry information from known vision systems.

4. A control method according to claim 3, wherein the step of determining (200) the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (R v), includes a second substep (200B) of coordinate system change, the second substep of coordinate system change (200B) comprising the transposition of the position and orientation of the vision system (19) worn by the head of at least one target occupant from the sensor coordinate system (R c ), in the vehicle frame of reference (R v ).

5. Control method according to any one of the preceding claims, wherein the vision system (19) is selected from augmented or virtual reality glasses, an augmented or virtual reality headset, or an augmented or virtual reality mask.

6. A control method according to any one of the preceding claims, wherein at least one control command consists of: - a command to modify the position and / or nature of content displayed in the vision system (19), and / or - a command to generate a visual and / or audible alert signal for the target occupant.

7. A control method according to any one of the preceding claims, wherein the vehicle (10) comprises an interactive block (25) integrated into the vehicle (10), the interactive block (25) comprising at least one interactive device (28), the control method further configured to generate a control command for at least one interactive device (28), each control command being a function of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (R v ).

8. Control method according to any one of the preceding claims, comprising a preliminary step (90) of receiving measurements from each sensor (22) of the sensor block (20), and of selecting the useful sensors of the sensor block (20) from among the set of sensors (22) of the sensor block (20), the capture field of the selected useful sensors comprising at least a portion of the head of the target occupant.

9. Control method according to any one of the preceding claims, wherein the vehicle (10) comprises at least one environmental sensor (27) configured to capture at least one piece of information and / or image of the environment outside the vehicle (10), the method comprising a step (225) of receiving data from the or each environmental sensor (27), the generation step (400) of a command consisting of generating a command to display said data on the vision system (19).

10. Electronic central unit (40) for controlling at least one vision system (19), worn on the head of a target occupant in a vehicle (10), based on the position and orientation of the head (17) of at least one target occupant, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the electronic central unit (40) comprising a computer configured to: receive measurements from sensors, referred to as useful sensors, of the sensor block (20) whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and orientation of the vision system (19) worn on the head of at least one target occupant in the frame of said sensor, referred to as the sensor frame (R c ), determine the position and orientation of the vision system (19) in the vehicle's frame of reference, called the vehicle frame of reference (R v), based on measurements from the useful sensors and the position of the useful sensors in the vehicle (10), and generate at least one vision system control command (19), each control command being a function of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (R v ).

11. Control system for at least one vision system (19), worn on the head of a target occupant in a vehicle (10), based on the position and orientation of the head (17) of at least one target occupant, the system comprising: a sensor block (20) including at least one sensor (22) configured to measure the position and orientation of the vision system (19) worn on the head of at least one target occupant, and an electronic central unit (40) according to claim 10.

Images