Method for controlling at least one vision system worn by a target occupant of a vehicle

Abstract

Method for controlling at least one vision system worn by a target occupant of a vehicle. The present invention relates to a method for controlling a vision system (19) worn by the head of a target occupant in a vehicle (10), the method comprising the steps of: receiving measurements from sensors, referred to as useful sensors, including the position and orientation of the vision system (19) worn by the head of the target occupant in the frame of said sensor, referred to as the sensor frame; determining the position and orientation of the vision system (19) in the frame of the vehicle (10), referred to as the vehicle frame; and generating at least one control command for the vision system (19), each control command being a function of the position and orientation of the vision system (19) in the vehicle frame. Figure for the abstract: 1

Description

Title of the invention: Method for controlling at least one vision system worn by a target occupant of a vehicle

[0001] The present invention relates to a method for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of the target occupant. The present invention also relates to an electronic control unit for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of the target occupant. Finally, the present invention relates to a control system for controlling at least one head-mounted vision system of a target occupant in a vehicle, based on the head position of the target occupant.

[0002] Passenger transport vehicles, and in particular civil and commercial aircraft, are equipped with interactive devices (HMI, human-machine interface, defined as a user interface enabling a person to connect to a machine, a system or a device), such as screens, generating real or virtual images, sound systems, etc., for example used during takeoff and landing phases, in order to improve the safety of maneuvers.

[0003] In aircraft, such interactive devices advantageously take the form of Head-Up Displays (HUDs) positioned in front of the vehicle occupant, for example in front of the pilot, in order to limit deviations of the pilot's gaze during the different phases of flight.

[0004] When coupled with on-board cameras, such as cameras of an enhanced flight vision system (EFVS, for Enhanced Flight Vision System), such head-up displays make it possible to display in the eyebox, piloting and navigation reticles, a combined vision image (CVS for Combined Vision System), a synthetic vision image (SVS for Synthetic Vision System) synthetically representing the environment of the vehicle (e.g. aircraft) in three dimensions, for example in the form of a virtual image in the pilot's field of vision.

[0005] However, the size of the virtual image has a limited geometry due to the space required for the head-up display in the cockpit, which must be kept to a minimum, as well as the pilot's field of vision, which must not be obstructed. Therefore, with such head-up displays, it is not possible for the pilot to obtain augmented or virtual reality information (synthetic vision images, for example) outside of their field of vision.

[0006] In order to overcome this drawback, it is known, for example in the field of military aviation, to replace the head-up display installed in the aircraft with a head-up display system on or in the helmet (Helmet Mounted Display - HMD) worn by the pilot.

[0007] For this purpose, such helmets are for example equipped with infrared emitters or an electromagnetic element configured to modify the electromagnetic field in which the cockpit is bathed.

[0008] Such a helmet allows the pilot to receive augmented (or virtual) reality information in any field of vision (from any angle) corresponding to the position and orientation of his head in real time.

[0009] However, such a solution is not entirely satisfactory. In particular, optical methods (infrared emitters) are sensitive to solar radiation and heat, and magnetic methods must necessarily take into account the effects of changes in the magnetic field on the operation of electronic systems in the cockpit.

[0010] In addition, such solutions require a heavy and complex installation in the cockpit, specific to each cockpit, and are therefore extremely expensive.

[0011] Eye-tracking, oculometry, or gaze-tracking is also a technique for controlling interactive devices, known and implemented for some time in the fields of computer science, transport, and mobility.

[0012] However, eye tracking also has its limitations. Indeed, eye tracking is not usable when the eyes of the target occupant are not visible, as is the case when the occupant is wearing a vision system, for example in the field of aviation.

[0013] One of the aims of the invention is therefore to propose a method of controlling at least one vision system worn on the head of a target occupant of a vehicle, allowing the vision system(s) to be controlled in such a way as to display information adapted to the target occupant, over a field of vision of up to 360°.

[0014] To this end, the invention relates to a method for controlling at least one vision system, worn on the head of a target occupant in a vehicle, as a function of the position and orientation of the head of at least one target occupant, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the method being implemented by an electronic central unit, the method comprising the steps of: - receiving measurements from sensors, referred to as useful sensors, of the sensor block whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and the orientation of the head-mounted vision system of at least one target occupant in the sensor's reference frame, referred to as the sensor reference frame, - determination of the position and orientation of the head-mounted vision system of at least one target occupant within the vehicle's frame of reference, referred to as the vehicle frame of reference, based on measurements from the relevant sensors and the position of the relevant sensors within the vehicle, and - generation of at least one vision system control command, each control command being a function of the position and orientation of the vision system in the vehicle frame of reference.

[0015] Such a control method, partly thanks to the useful sensors and the determination of the position and orientation of the vision system worn by the target occupant's head within the vehicle's frame of reference, makes it possible to track the direction, position, and orientation of the vehicle occupants' heads. Tracking is possible even when head movements and rotations are significant. Furthermore, the tracking is independent of the specific useful sensor involved.

[0016] Generating a control command for the vision system, based on the determined position and orientation, makes it possible to provide the occupant with content adapted to the position and orientation of their head in real time. This allows for more precise targeting of the nature and / or position of the information to be presented to the occupant (whether, for example, real or synthetic images), in order to improve the safety of driving or piloting maneuvers of the vehicle.

[0017] According to other advantageous aspects of the invention, the control method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0018] - the sensor or each sensor of the sensor block is chosen from a sensor integrated into the vehicle and / or an additional sensor positioned in the vehicle, each sensor being chosen for example from a camera, or a lidar;

[0019] - the step of determining the position and orientation of the vision system carried by the head of at least one target occupant in the vehicle's reference frame, includes a first vision system recognition sub-step, the first recognition sub-step being based on a correlation between one or more input data and measurements from useful sensors, each input data including predefined geometry information from known vision systems;

[0020] - the step of determining the position and orientation of the vision system carried by the head of at least one target occupant in the vehicle's frame of reference, includes a second sub-step of frame change, the second sub-step of frame change comprising the transposition of the position and orientation of the system of vision carried by the head of at least one target occupant from the sensor reference point, in the vehicle reference point;

[0021] - the vision system is chosen from augmented or virtual reality glasses, an augmented or virtual reality headset, or an augmented or virtual reality mask;

[0022] - at least one control command consists of:

[0023] - a command to modify the position and / or nature of a content displayed in the vision system, and / or

[0024] - a control for generating a visual and / or audible warning signal for the target occupier;

[0025] - the vehicle includes an interactive block integrated into the vehicle, the interactive block including at least one interactive device, the control method further being configured to generate a control command for at least one interactive device, each control command being a function of the position and orientation of the vision system worn by the head of at least one target occupant in the vehicle frame of reference;

[0026] - the control method includes a preliminary step of receiving measurements from each sensor of the sensor block, and selection of the useful sensors of the sensor block from the set of sensors of the sensor block, the capture field of the selected useful sensors including at least a portion of the head of the target occupant;

[0027] - the vehicle includes at least one environmental sensor configured to detect at least one piece of information and / or image of the environment outside the vehicle, the process including a step of receiving data from the one or each environmental sensor, the step of generating a command consisting of generating a command to display said data on the vision system.

[0028] The invention also relates to an electronic central control unit for at least one vision system, worn on the head of a target occupant in a vehicle, based on the position and orientation of the head of at least one target occupant, the vehicle comprising a sensor block, the sensor block comprising at least one sensor, the electronic central control unit comprising a computer configured for: - to receive measurements from sensors, referred to as useful sensors, of the sensor block whose field of capture includes at least a portion of the head of the target occupant, the measurements from each sensor including the position and orientation of the vision system worn by the head of at least one target occupant in the reference frame of said sensor, referred to as the sensor reference frame, - determine the position and orientation of the vision system within the vehicle's frame of reference, referred to as the vehicle frame of reference, based on measurements from the relevant sensors and the position of the relevant sensors within the vehicle, and - generate at least one vision system control command, each control command being a function of the position and orientation of the vision system carried by the head of at least one target occupant in the vehicle frame of reference.

[0029] The invention also relates to a control system for at least one vision system, worn on the head of a target occupant in a vehicle, based on the position and orientation of the head of at least one target occupant, the system comprising: - a sensor block including at least one sensor configured to measure the position and orientation of the vision system worn on the head of at least one target occupant, and - an electronic central unit as described previously.

[0030] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings which are:

[0031] - [Fig.1] [Fig.1], a schematic representation of an example of a cockpit of a aircraft in which a sensory unit is integrated and in which an occupant wearing augmented (or virtual) reality glasses is located,

[0032] - [Fig.2] [Fig.2], a schematic representation of an electronic central processing unit, of a sensory block and an interactive block according to the invention,

[0033] - [Fig.3] [Fig.3], an organizational chart of the steps implemented by the central unit electronics in the control method according to the invention, and

[0034] - [Fig.4] [Fig.4], a schematic representation of all the changes of reference points made by the central electronic unit in the control process according to the invention.

[0035] A vehicle 10 is illustrated by [Fig.1].

[0036] Vehicle 10 is, for example, a land, air, or sea vehicle. In In the example of [Fig. 1], and throughout what follows, vehicle 10 is an aircraft. Aircraft 10 is understood here to mean any airplane, helicopter, or other flying machine that can be piloted by a pilot 14 from that aircraft. The invention nevertheless extends to any other type of vehicle.

[0037] In this example, the vehicle 10 includes a cockpit 12 inside which a pilot 14 is installed.

[0038] The cockpit 12 comprises a floor 16, at least one seat fixed to the floor 16, an instrument panel 18 and a windshield (not shown) that is at least partially transparent and separating the interior of the cockpit 12 from the external environment of the aircraft 10, the head 17 of the pilot 14 looking towards the instrument panel 18 and the windscreen.

[0039] In all that follows, the occupants of vehicle 10 are defined as the pilot 14 and / or the co-pilot and / or a passenger of vehicle 10.

[0040] As shown in [Fig. 1], the occupant of the vehicle 10, in particular the pilot 14, wears a vision system 19 on his head 17.

[0041] The vision system 19 is a system for displaying real and / or synthetic content to the occupant wearing said system 19. The vision system 19 is, for example, an augmented (or virtual) reality system.

[0042] The vision system 19 is carried by the head 17 of an occupant of the vehicle 10, such as the pilot 14.

[0043] The vision system 19 defines the field of vision of its wearer. The wearer's field of vision is therefore considered to coincide with the field of vision of the vision system 19 carried by the wearer's head 17.

[0044] The vision system 19 is for example chosen from augmented or virtual reality glasses, also called smart glasses, an augmented or virtual reality headset, an augmented or virtual reality mask, or any other object configured to display real or synthetic content to the occupant.

[0045] Such vision systems 19 assist, for example, the pilot 14 or the co-pilot in their maneuvers by displaying them appropriate information (images of the environment, GPS navigation instructions, information on driving or piloting or on the vehicle 10, etc.).

[0046] The vehicle 10 includes a sensor block 20 comprising at least one sensor 22 and an electronic central unit 40 in communication with the sensor block 20.

[0047] In a particular embodiment, the vehicle 10 also includes an interactive block 25 comprising at least one interactive device 28, the electronic central unit 40 being for example in communication with the interactive block 25.

[0048] The sensor block 20 comprises, for example, two sensors 22. Alternatively, the sensor block 20 comprises three or more sensors 22.

[0049] The sensors 22 of the sensor block 20 are for example chosen from a set of sensors integrated into the vehicle and / or a set of additional sensors positioned in the vehicle 10.

[0050] By "integrated into the vehicle", it is understood that said sensors 22 are, for example, sensors mounted in the vehicle during the manufacture of the vehicle 10. Such sensors 22 are, for example, a camera (operating in the visible and / or infrared range), or a lidar, belonging to the vehicle 10, and listed in the vehicle 10.

[0051] By "additional sensors positioned in the vehicle," it is understood that said sensors 22 are sensors positioned in the vehicle 10 as a secondary installation, that is, after the vehicle 10 has been designed. For example, at least one additional sensor belongs to one of the occupants of the vehicle 10. For example, such sensors are a camera (operating in the visible and / or infrared range), or a lidar, belonging to one of the occupants of the vehicle 10, for example, integrated into one of the occupants' mobile phones, and / or installed in the vehicle. Such additional sensors are, for example, connected by wire or wireless means (e.g., Bluetooth) to the vehicle 10 and are thus also listed in the vehicle 10.

[0052] Each of the sensors 22 of the sensor block 20 is configured to measure, when the sensing field of said sensor 22 covers an area occupied by one or more occupant(s) of the vehicle 10, the position and orientation of the vision system 19 worn by the wearer's head in the vehicle 10. Thus, at least a portion of the wearer's head (this portion of the head carrying the vision system 19) is in the sensing field of each of the sensors 22, advantageously the entire head of the wearer is in the sensing field of each of the sensors 22. Each measurement of each sensor 22 is carried out in a reference frame linked to the sensor 22, called sensor reference frame Rc, shown schematically in [Fig.4].

[0053] Thus, based on the position and orientation of the vision system 19, each of the sensors 22 of the sensor block 20, when the field of capture of said sensor 22 covers an area occupied by one or more occupant(s) of the vehicle, provides information on the position and orientation of the head 17 of the occupant(s) of the vehicle 10 in the sensor reference Rc.

[0054] The sensors 22 are for example positioned at different locations in the cockpit 12 in order to track and improve the accuracy of the tracking of the head 17 of the wearer of the vision system, for example the pilot 14 or the co-pilot in the cockpit 12, in particular in the extreme angles of the cockpit 12.

[0055] Furthermore, the sensors 22 operate advantageously and effectively in low-light conditions, such as at night. For example, such sensors 22 are infrared cameras.

[0056] Additionally, a sensor 26 integrated on the vision system 19, such as a camera or a gyroscope for example, can be used to confirm the position and orientation of the vision system 19 in a reference frame linked to the vehicle 10, called vehicle reference frame Rv.

[0057] Optionally, the sensors 22 can also be used to collect vital signs of the pilot 14 or the co-pilot, in order to determine a state of fatigue, discomfort or other.

[0058] In a particular embodiment, the vehicle 10 further comprises at least one environmental sensor 27 configured to capture at least one piece of information and / or image of the environment outside the vehicle 10. Preferably, the vehicle 10 comprises a plurality of environmental sensors 27, arranged at different locations on the vehicle 10, in order to capture data from outside the vehicle 10 over the largest possible angular range.

[0059] As will be described in more detail later, from the position and orientation of the vision system 19 in the vehicle frame Rv and the data received by the environmental sensors 27, the electronic central unit 40 determines the content to be displayed in the vision system 19.

[0060] The displayed content depends on parameters specific to the vision system 19, such as the field of vision of the vision system 19, the projection distance, the parallax, etc.

[0061] The interactive block 25 is integrated into the vehicle 10 and includes, for example, at least one interactive device 28.

[0062] Each interactive device 28 is, for example, a display device in the vehicle and / or a human-machine interface and / or a navigation device and / or a multimedia device in the vehicle, used to present information and / or images related to the piloting or operation of the vehicle 10 to one or more occupants of the vehicle 10.

[0063] In addition, such interactive devices 28 can be used to alert one or more occupants of the vehicle 10 of a potential danger spotted in the air or on the ground using certain sensors of the vehicle 10. For example, enhanced flight vision system (EFVS) cameras, radars or lidars can be used.

[0064] According to the invention, the electronic central unit 40 includes a computer adapted to implement a method of controlling the vision system 19 as a function of the position and orientation of the head of at least one target occupant of the vehicle.

[0065] The target occupant or occupants of vehicle 10 are, for example, chosen from among the pilot 14 and / or the co-pilot of vehicle 10.

[0066] The interactions between the electronic central unit 40, and in particular between the computer, the sensor block 20, the vision system 19 and optionally the interactive block 25 are schematically represented in [Fig.2].

[0067] The computer is an electronic circuit designed to manipulate and / or transform data represented by electronic or physical quantities in registers of the computer and / or memories into other similar data corresponding to physical data in register memories or other types of display devices, transmission devices or storage devices.

[0068] As specific examples, the calculator is implemented in the form of a programmable logic component, such as an FPGA (Field Programmable Gate Array), or an integrated circuit, such as an ASIC (Application Specifies Integrated Circuit).

[0069] Alternatively, when the method is implemented in the form of one or more software programs, i.e., in the form of a computer program, also called a computer program product, it is further capable of being stored on a computer-readable medium, not shown. The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, ROM, RAM, any type of non-volatile memory (e.g., FLASH or NVRAM), or a magnetic card. A computer program comprising software instructions is then stored on the readable medium.

[0070] The operation of the central electronic unit 40 will now be described with reference to [Fig.3], which illustrates an example of the implementation of a control method according to the invention.

[0071] In a particular embodiment, the control process includes a preliminary step 90 (optional) of receiving measurements from each sensor 22 of the sensor block 20.

[0072] In particular, at this stage, the electronic central unit 40 receives information on the number of sensors 22 of the sensor block 20 available, the position and orientation of each sensor 22 relative to the vehicle 10, the sensing field of each sensor 22, and information on the nature of the environment sensed by each of the sensors 22.

[0073] In other words, this preliminary step 90 concerns the listing of all the integrated or additional sensors 22, available and functional in the vehicle 10.

[0074] For example, the electronic central unit 40 queries the vehicle 10 about all the sensors available and functional in the vehicle 10. Such sensors are, as previously mentioned, listed beforehand in the vehicle 10, either at the time of manufacture of the vehicle 10, or at the time of connection between the additional sensor and the vehicle 10.

[0075] The preliminary step 90 then includes the selection of useful sensors of the sensor block 20 from the set of sensors 22 of the sensor block 20.

[0076] The selection step is carried out so that the field of capture of the selected sensors, called useful sensors, includes at least the head 17 or the vision system 19 carried by the head of the target occupant of the vehicle 10.

[0077] In other words, each of the useful sensors is capable of measuring the position and orientation of the vision system 19 carried by the head of the target occupant, and therefore implicitly of the head of the target occupant in the sensor frame Rc.

[0078] A person skilled in the art will naturally understand that a plurality of sensors 22 makes it possible to increase the accuracy of position and orientation measurement of the vision system 19 and to cover a greater angular extent around the target occupant, than if a single sensor 22 were used in the vehicle 10.

[0079] In an alternative, not shown, the control method does not include the preliminary step 90. According to this alternative, the electronic central unit 40 includes, for example, a storage memory in which information (nature, position in the vehicle 10, etc.) on the sensors 22 available in the vehicle 10 are referenced beforehand.

[0080] The control process then includes a step of receiving 100 measurements from the useful sensors of the sensor block 20, the capture field of which includes at least a portion of the head of the target occupant.

[0081] Each measurement received, performed in the sensor frame Rc, includes the position and orientation of the vision system 19 carried by the head of the target occupant in the sensor frame Rc.

[0082] The reception step 100 is, for example, carried out in real time. Thus, the measurements from the useful sensors of the sensor block 20 correspond to the position and orientation of the vision system 19 worn by the head of the target occupant in the real-time sensor frame Rcen.

[0083] The control method then includes a step 200 of determining the position and orientation of the vision system 19 carried by the head of the target occupant in the vehicle reference frame Rv, according to the measurements from the useful sensors and the position of the useful sensors in the vehicle 10.

[0084] In a particular embodiment, for example when the vehicle 10 includes a plurality of sensors 22, the step of determining the position and orientation of the vision system 19 is carried out by triangulation from the measurements received from the useful sensors in the sensor frame Rc.

[0085] This triangulation step makes it possible to obtain precise information on the position and orientation of the vision system 19 in the cockpit 12. By calibrating the useful sensors with each other, it is possible to obtain the relative position of each useful sensor in order to determine the position and orientation of the vision system 19 in a frame no longer linked to each sensor, but in a common frame linked to the vehicle, called above vehicle frame Rv.

[0086] In a particular embodiment, the step 200 of determining the position and orientation of the target occupant's vision system 19 in the vehicle frame Rv, includes a first substep 200A of recognizing the vision system 19 carried by the head of the target occupant in the vehicle 10.

[0087] The first recognition substep 200A is based on a correlation between one or more input data De and the measurements from the useful sensors.

[0088] Each input data De includes in particular predefined geometry information from known vision systems. For example, the input data De represents geometry data associated with a model of augmented (or virtual) reality glasses, whose geometry is perfectly known and invariant from one pair of glasses to another, or represents geometry data associated with an augmented (or virtual) reality headset or mask.

[0089] This sub-step 200A of recognition of the vision system 19 carried by the head of the target occupant in the vehicle 10 is for example based on a deep learning approach of the neural network type.

[0090] The geometry of the vision system 19 then serves as an input parameter for the recognition of the particular vision system and for the future determination of its position, its orientation and the rotations of the head 17 of the pilot 14 or the co-pilot.

[0091] According to this example, the step 200 of determining the position and orientation of the vision system 19 carried by the head of the target occupant in the reference frame of the vehicle Rv, includes a second substep 200B of changing the reference frame.

[0092] The second sub-step of the reference frame change 200B includes the transposition of the position and orientation of the vision system 19 carried by the head of the target occupant from the sensor reference frame Rc, into the vehicle reference frame Rv.

[0093] The method includes a step 225 for receiving data from the environmental sensor(s) 27. This step 225 can be implemented before or after the step 200 for determining the position and orientation of the vision system 19 worn by the target occupant's head. In this step 225, the electronic control unit 40 receives information and / or images of the environment outside the vehicle 10, captured by the environmental sensor(s) 27.

[0094] In other words, each environmental sensor 27 shares the data captured (whether information related to the external environment, or images acquired from the external environment) with the electronic central unit 40.

[0095] In a particular embodiment, the control method includes an intermediate step 250 (optional) of receiving position data from the interactive devices 28 in the vehicle 10 and selecting the interactive devices 28 whose displayed and / or generated content is likely to enter the field of vision of the vision system 19.

[0096] Optionally, a transposition step 300 is provided to transpose the position and orientation of the vision system 19 (and therefore implicitly of the head 17 of the occupant) determined in a frame linked to at least one interactive device 28, called device frame Rd, of the vehicle 10 as a function of position data of at least one interactive device 28.

[0097] During the transposition step 300, the Rd reference points of the or each interactive device 28, in particular, where applicable, the reference points linked to the screens of said interactive devices 28, as well as the reference points linked to real or virtual images, are considered.

[0098] Such landmarks are illustrated in particular in [Fig.4].

[0099] As previously mentioned, the control method includes a final step 400 of generating at least one control command for the vision system 19.

[0100] Each generated control command is a function of the position and orientation of the vision system 19, and therefore implicitly of the head of the target occupant, determined in the vehicle frame Rv.

[0101] Each control command consists, for example, of:

[0102] - a command to modify the dimension, position and / or nature of content displayed in the vision system 19, and / or

[0103] - a control for generating a visual and / or audible warning signal for the target occupier.

[0104] For example, the control command consists of a command to display data (information and / or images, real or synthesized) from the environmental sensor(s) 27 in the vision system 19. Advantageously, the data displayed in the vision system 19 depends on the position and orientation of the augmented (or virtual) reality system 19 in the vehicle frame Rv. Thus, the electronic central unit 40 determines the data to be displayed in the vision system 19 based on the field of view of the vision system 19.

[0105] For example, when the electronic central unit 40 determines that the target occupant turns their head 17 to the right, the content displayed in the vision system 19 will advantageously be based on the information received by the environmental sensors 27 located to the right of the vehicle 10, corresponding to the real-time field of vision of the vision system 19.

[0106] In addition, the electronic central unit 40 determines the content to be displayed according to parameters specific to the vision system 19, such as the field of vision, the projection distance, the parallax, etc.

[0107] Thus, the content displayed in the vision system 19 is consistent with the performance of the vision system 19 and with the real-time position and orientation of the head 17 of the target occupant, and therefore of the field of vision of the vision system 19.

[0108] In one particular embodiment, the control method is further configured to generate a control command for at least one interactive device 28.

[0109] Thus, the interactive device 28 receives instructions to control the displayed image, adapted to modify (or activate / deactivate) the content displayed on the interactive device 28 according to the position and orientation of the vision system 19.

[0110] Various examples of application of the control method according to the invention, and of the commands generated, are described below.

[0111] In a first application example, the control method according to the invention makes it possible to modify the position and / or nature of content displayed in the vision system 19 (or on the interactive device(s) 28), for example to offer better visibility to the target occupant.

[0112] In particular, in this example, when the vision system 19 is controlled by the electronic central unit 40, the position and nature of the images of the external environment displayed in the vision system 19 are adapted to match the real-time visual field of the vision system 19.

[0113] Thus, the pilot 14 can orient his head and receive information on the external environment in augmented (or virtual) reality in the corresponding field of vision.

[0114] In addition, the displayed content is adapted to the projection distances and to the parallax of the vision system 19.

[0115] The plurality of environmental sensors 27 allows the pilot 14 to receive adequate and consistent data from the outside regardless of the position and orientation of his head 17 and his field of vision.

[0116] In a second example, the control method according to the invention makes it possible to generate a visual and / or audible alert signal for the target occupant.

[0117] In one example, the vehicle 10 includes an EFVS system, as defined above, capable of detecting an obstacle and / or hazard on the trajectory of the vehicle 10.

[0118] When the electronic central unit 40 calculates, using the position measurements and orientation of the vision system 19, and therefore of the head of the target occupant, received by the useful sensors, that the obstacle and / or hazard is outside the field of vision of the target occupant (for example in this case, the pilot 14), the electronic central unit 40 can then generate a specific command aimed at alerting the target occupant of a potential danger.

[0119] In this example the generation command is for example an instruction to generate a visual and / or audible alert signal for the target occupant, or to start up or modify a content of the vision system 19 or of an interactive device 28 located in the field of vision of the target occupant.

[0120] In another example, the control method according to the invention makes it possible to activate or deactivate an interactive device 28 according to its position in the vehicle 10, relative to the position and orientation of the head 17 of the target occupant.

[0121] This makes it possible, for example, not to leave an interactive device 28 active unnecessarily if the occupant is not looking in its direction, and thus makes it possible to save energy.

[0122] Similarly, if the interactive device 28 is an extended device, the control method allows only certain areas of the interactive device 28 to be activated or deactivated if it allows it, for example if it is equipped with localized lighting or backlighting.

[0123] By "extended device" it is understood that the interactive device 28 includes, for example, a screen covering a pilot or driver display area in the field of vision of the pilot 14, a central display area in the field of vision of both the pilot 14 and the co-pilot, and a secondary viewing area in the field of vision of the co-pilot only.

[0124] In a particular embodiment, the method includes one or more features aimed at improving the accuracy of determining the position and orientation of the vision system 19 in the vehicle reference frame Rv.

[0125] For example, the control process is implemented using a Kalman filter, associated or not with artificial intelligence, for example with machine learning.

[0126] The Kalman filter combines, for example, the measurements received on the position and orientation of the vision system 19 with a physical model in order to determine more precisely the position and orientation of the vision system 19. Thus, such a filter makes it possible to improve the operation of the control process, by improving the accuracy of the position and orientation of the head of the occupant in the vehicle 10 and thus makes it possible to adjust the content displayed to the occupant more precisely.

[0127] Optionally, the target occupant can validate / invalidate certain actions to be carried out, previously determined by the position and orientation of the head 17 of the target occupant, by voice using a conversational assistant for example, by interaction with a touch interface, or by using a physical button such as a control lever.

[0128] In addition, the head 17 of the target occupant can also serve as a pointer in the content displayed in the vision system 19 in order to select a graphic element among others and to validate / invalidate an action.

[0129] Such actions relate for example to functionalities related to driving, or to piloting the vehicle 10 or to the vehicle 10 itself, such as the management of navigation systems and / or control systems and / or notification / alert systems, and / or multimedia systems, the control of the thermal environment inside the vehicle 10, or the control of the lights of the vehicle 10.

[0130] Such a method of controlling the vision system 19 according to the direction of the head of at least one target occupant of the vehicle has many advantages.

[0131] The control method allows the content of the vision system 19 and optionally the interactive devices 28 to be adapted and / or controlled in real time according to the position and orientation of the head of the occupants of the vehicle 10. The display of information in the vision system 19 can then be adapted to the morphology of the pilot 14 or the co-pilot, and to the piloting position.

[0132] The electronic central unit 40 determines the content to be displayed according to parameters specific to the vision system 19, such as the field of vision, the projection distance, the parallax, etc., the content displayed in the vision system 19 is consistent with the performance of the vision system 19 and with the real-time field of vision of the vision system 19, depending essentially on the position and orientation of the head 17 of the target occupant.

[0133] Furthermore, the vision system 19 worn on the head of the pilot 14 or co-pilot can be a simple optical virtual or augmented reality system, without onboard electronics or sensors. Such a vision system is therefore lighter and less bulky for the occupants of the vehicle 10.

[0134] Such a control method is independent of the cockpit in question and of the augmented (or virtual) reality display technology. Indeed, such a control method is easily adaptable to any type of vision system and can accommodate possible evolutions of said system (geometry and display technology). Furthermore, in the case of an aircraft, such a method can be implemented both during flight and on the ground.

[0135] Finally, in the case where the vision system is equipped with an eye-tracking device, the direction of gaze can also be taken into account to control the vision system 19. A person skilled in the art will understand that the examples and variants described above can be combined with each other.

[0136] Those skilled in the art will also understand that the present description is based on the specific case of aerial vehicles, but that the invention is not limited to this vehicle type. The control procedure is therefore adaptable to all types of vehicles, including automobiles, railways and ships.

Claims

Demands

1. Method for controlling at least one vision system (19), worn by the head of a target occupant in a vehicle (10), as a function of the position and orientation of the head (17) of at least one target occupant, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the method being implemented by an electronic central unit (40), the method comprising the steps of: - receiving (100) measurements from sensors, referred to as useful sensors, of the sensor block (20) whose field of view includes at least a portion of the head of the target occupant, the measurements of each sensor comprising the position and orientation of the vision system (19) worn by the head of at least one target occupant in the frame of said sensor, referred to as sensor frame (Rc),- determination (200) of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame (10), referred to as the vehicle frame (Rv), as a function of measurements from the useful sensors and the position of the useful sensors in the vehicle (10), and - generation (400) of at least one control command for the vision system (19), each control command being a function of the position and orientation of the vision system (19) in the vehicle frame (Rv).

2. A control method according to claim 1, wherein the sensor or each sensor (22) of the sensor block (20) is chosen from a sensor integrated into the vehicle (10) and / or an additional sensor positioned in the vehicle (10), each sensor (22) being chosen for example from a camera, or a lidar.

3. A control method according to claim 1 or 2, wherein the step (200) of determining the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (Rv) comprises a first substep (200A) of recognizing the vision system (19), the first recognition substep (200A) being based on a correlation between one or more input data (De) and measurements from useful sensors, each input data (De) including predefined geometry information from known vision systems.

4. A control method according to claim 3, wherein the step of determining (200) the position and orientation of the vision system (19) carried by the head of at least one target occupant in the vehicle frame (Rv), comprises a second substep (200B) of changing the frame, the second substep of changing the frame (200B) comprising the transposition of the position and orientation of the vision system (19) carried by the head of at least one target occupant from the sensor frame (Rc), into the vehicle frame (Rv).

5. A control method according to any one of the preceding claims, wherein the vision system (19) is selected from augmented or virtual reality glasses, an augmented or virtual reality headset, or an augmented or virtual reality mask.

6. A control method according to any one of the preceding claims, wherein at least one control command consists of: - a command to change the position and / or nature of content displayed in the vision system (19), and / or - a command to generate a visual and / or audible warning signal for the target occupant.

7. A control method according to any one of the preceding claims, wherein the vehicle (10) comprises an interactive block (25) integrated into the vehicle (10), the interactive block (25) comprising at least one interactive device (28), the control method being further configured to generate a control command for at least one interactive device (28), each control command being a function of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame (Rv).

8. A control method according to any one of the preceding claims, comprising a preliminary step (90) of receiving measurements from each sensor (22) of the sensor block (20), and of selecting the useful sensors of the sensor block (20) from among the set of sensors (22) of the sensor block (20), the field of capture of selected useful sensors including at least a portion of the target occupant's head.

9. A control method according to any one of the preceding claims, wherein the vehicle (10) comprises at least one environmental sensor (27) configured to capture at least one piece of information and / or image of the environment outside the vehicle (10), the method comprising a step (225) of receiving data from the or each environmental sensor (27), the generation step (400) of a command consisting of generating a command to display said data on the vision system (19).

10. Electronic central unit (40) for controlling at least one vision system (19), worn on the head of a target occupant in a vehicle (10), based on the position and orientation of the head (17) of at least one target occupant, the vehicle (10) comprising a sensor block (20), the sensor block (20) comprising at least one sensor (22), the electronic central unit (40) comprising a computer configured to: - receive measurements from sensors, referred to as useful sensors, of the sensor block (20) whose field of view includes at least a portion of the head of the target occupant, the measurements from each sensor comprising the position and orientation of the vision system (19) worn on the head of at least one target occupant in the frame of said sensor, referred to as sensor frame (Rc), - determine the position and orientation of the vision system (19) in the frame of the vehicle, referred to as vehicle frame (Rv),based on measurements from useful sensors and the position of useful sensors in the vehicle (10), and - generate at least one control command for the vision system (19), each control command being a function of the position and orientation of the vision system (19) worn by the head of at least one target occupant in the vehicle frame of reference (Rv).

11. Control system for at least one vision system (19), worn on the head of a target occupant in a vehicle (10), depending on the position and head orientation (17) of at least one target occupant, the system comprising: - a sensory block (20) comprising at least one sensor (22) configured to measure the position and orientation of the vision system (19) worn on the head of at least one target occupant, and - an electronic central unit (40) according to claim 10.

Images