Method for processing video data from an uncertified video source
The method adapts uncertified video data to cockpit screens by allowing pilots to select and resize windows of interest, addressing aspect ratio issues and ensuring seamless interaction, thereby improving cockpit display efficiency and safety.
Patent Information
- Application Number
- FR2022012864
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-12-07
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-12-07
AI Technical Summary
Existing methods for displaying uncertified video data from tablets in aircraft cockpits face challenges in adapting the aspect ratio and resolution to cockpit display screens, leading to black bands and inefficient human-machine interactions.
A method for processing data streams from uncertified sources involves adapting the data to cockpit screen dimensions and resolution, allowing pilots to manually select a window of interest, and transferring interaction coordinates to resize and crop the video stream accordingly, ensuring seamless interaction regardless of the selected area.
This approach optimizes the display of data by eliminating black bands and ensuring consistent interaction capabilities, regardless of the selected window, thus enhancing pilot usability and compliance with aviation safety standards.
Smart Images

Figure 00000010_0000 
Figure 00000011_0000 
Figure 00000013_0000
Abstract
Description
Title of the invention: Method for processing video data from an uncertified video source Technical field
[0001] The present invention relates to the field of aeronautics, and more particularly to the field of human-machine interactions within the cockpit of an aircraft. Prior art
[0002] Software developed for the avionics field addresses critical functions and requires a demanding level of control to ensure the required safety conditions. Their developments are controlled and limited.
[0003] The digitalization of technical support has been made possible through electronic flight bags, known as EFBs (for "Electronic Flight Bags" in English). The EFB is an electronic information management device that helps the aircraft pilot perform flight management tasks more easily and efficiently with less paper. It is a general-purpose computer platform intended to reduce or replace paper-based reference material such as the aircraft manual, the operating manual and navigation charts.
[0004] The emergence of a connected network, to which the EFB is attached, which is increasingly developed, makes it possible to envisage in an open world increasingly advanced applications which complement the functionalities of the certified cockpit. The non-criticality of these applications, not subject to the DO 178 certification process, allows for rapid, regular and lower-cost development.
[0005] To facilitate the continuity of operations between the avionics domain applications displayed in the cockpit and the open world applications, displayed natively on the EFB, it is advantageous to be able to display the content of the open world applications using one or more cockpit display screen(s). This makes it possible to reduce the cockpit footprint by avoiding the use of an additional display screen. Indeed, the use of an additional display screen can limit external visibility and require a complex installation to be within reach of the pilot. In addition, avionics display screens make it possible to achieve display performances in terms of luminance and glare reduction, which a tablet screen does not offer.
[0006] To best integrate open world applications and facilitate the pilot's missions, it is known from the prior art to use a device to display content from this open world on a cockpit display screen via a connection video. By "open world" we mean data accessible to the general public. This data is said to be uncertified in terms of security. The certified "cockpit" world is subject to validation by the authorities (EASA).
[0007] Such a device of the prior art is notably illustrated in [Fig. 1]. This device 1 comprises a tablet 10, a universal adapter 11 for tablet (called “TUA” for “Tablet Universal Adapter”) and a display screen 120 of a cockpit 12. The tablet here is a general public touch tablet, adapted to deliver video data following a request from the cockpit.
[0008] As things stand, it is the avionics platforms that process the video stream output from the tablets. The first need for such processing is to eliminate the "black bands" as much as possible. This is a problem of width / height ratio between the tablet image and the display surface for content available in the cockpit (portion of the screen or full screen). The areas around the tablet image are filled with black in the video stream (which, moreover, has its own ratio in 16:9 as a video standard).
[0009] There is a constant need to improve human-machine interactions within the cockpit to facilitate the display of useful data for the pilot. Statement of the invention
[0010] The present invention aims to at least partially address this need.
[0011] More particularly, the present invention relates to a method for processing a data stream originating from a non-certified video source, said data stream being intended to be displayed at least partially on a display screen of a cockpit of an aircraft, said display screen having predetermined dimensions and a predetermined resolution, said processing method comprising: - a step of sending the data stream by the non-certified video source; - a step of adapting the data of the data stream to adapt said data to the dimensions and resolution of the cockpit display screen; - a pilot being able to interact with the display screen in the cockpit, a step of transferring interaction coordinates to the non-certified video source via the adapter to provide data from the stream in accordance with the interaction between said pilot and the display screen, characterized in that said processing method comprises: - a step of selection on the cockpit display screen of a window of interest by the pilot, said selection of a window of interest deactivating the step of transferring interaction coordinates; - a step of displaying on the display screen a part of the data flow corresponding to the selected window of interest.
[0012] Thus, the pilot is given the option of manually configuring the window of interest so as to optimize the display of a portion of the data stream. It is then only the avionics platform that cuts and resizes the image according to the window of interest. It is then possible to interact, regardless of the window of interest selected. Furthermore, in the step of displaying a portion of the data stream corresponding to the selected window of interest, a conformity of interaction with the tablet is maintained regardless of the window of interest selected.
[0013] In a particular embodiment, the selection of the window of interest is carried out directly by the operator by contact with the screen.
[0014] In a particular embodiment, contact with one of the edges of the screen and a movement of the operator's finger in continuous contact with the screen make it possible to determine the limits of the window of interest.
[0015] In a particular embodiment, another contact with a corner of the window of interest and a movement of the operator's finger in continuous contact with the screen make it possible to adjust the dimensions of said window of interest.
[0016] In a particular embodiment, another contact inside the window of interest and a movement of the operator's finger in continuous contact with the screen allow a movement of the position of the window of interest.
[0017] In a particular embodiment, the selection of the window of interest is carried out by the operator via at least one rotator, allowing finer adjustment along a horizontal axis and adjustment along a vertical axis of a size of said window of interest. This improves the precision of the selection.
[0018] In a particular embodiment, an acceleration law makes it possible to multiply a speed of selection of the window of interest during rapid movements of the rotator.
[0019] In a particular embodiment, the step of transferring interaction coordinates comprises a transfer of transformation parameters to the adapter.
[0020] In a particular embodiment, the portion of the data stream corresponding to the selected window of interest is displayed with the entire data stream.
[0021] The present invention will be better understood on reading the detailed description of embodiments taken as non-limiting examples and illustrated by the appended drawings in which:
[0022] [Fig.l] [Fig.l] illustrates a device for displaying content from the open world on a cockpit screen according to the prior art;
[0023] [Fig.2] [Fig.2] illustrates a device for displaying a data stream from the open world on a screen of a cockpit according to the invention using a first mode of selection of a window of interest;
[0024] [Fig.3] [Fig.3] illustrates the different stages of a method for processing a data stream, implemented by the device of [Fig.2];
[0025] [Fig.4] [Fig.4] illustrates the positioning of the window of interest on the screen display;
[0026] [Fig.5] [Fig.5] illustrates a device for displaying a data stream from the world opened on a screen of a cockpit according to the invention using a second mode of selection of a window of interest.
[0027] The invention is not limited to the embodiments and variants presented and other embodiments and variants will become clear to those skilled in the art.
[0028] [Fig.2] illustrates a device 2 for processing a data stream. This device comprises: - a 20 tablet; - a 21 adapter; - a 220 display screen.
[0029] The tablet 20 is a touchscreen tablet from the general public market.
[0030] The adapter 21 is arranged between the tablet 20 and the display screen 220. This adapter 21 is capable of receiving interaction coordinates Coord to process them and provide modified interaction coordinates Coord'. This coordinate processing consists of applying a change of reference to the HID reference frame of the tablet, the interaction being carried out by a driver directly on the display screen 220. By such processing, the display screen 220 becomes a remote screen of the tablet 20, for example, for interaction with functions represented in the form of icons on this tablet.
[0031] The display screen 220 is arranged in the cockpit 22 of the aircraft. In the embodiment of [Fig. 2], this display screen 220 is divided into two parts 2201, 2202. The first part 2201 is intended to display a data stream coming from a non-certified source provided by the tablet 20. This first part 2201 can be positioned to the left or to the right of the display screen 220. The second part 2202 is intended to display a data stream coming from a certified source. The display screen 220 is here a touch screen combining the display and pointing functionalities upon contact with a pilot's finger. This display screen comprises a detection block 221 and a processing block 222. The detection block 221 is adapted to detect the area of the display screen touched by the pilot's finger. The processing block 222 is capable of providing coordinates of this contact point in the reference frame of the display screen 220.
[0032] It will be noted that in an alternative embodiment the display screen only displays the data stream from the uncertified source.
[0033] For example, the display screen 220 is in 16:10 format at 1440x900.
[0034] A method of processing video data will then be described based on Figures 3, 4 and 5.
[0035] In a first step E1, the data stream F is sent by the uncertified video source present on the tablet 20.
[0036] In a second step E2, the data of the data stream are adapted to the dimensions and resolution of the display screen 220. This second step E2 is carried out by a video adapter present at the avionics equipment integrated into the “display”. There is an intermediate step between the first step E1 and the second step E2 carried by a box 210 present in the adapter 21. This box 210 is thus adapted to force the tablet 20 to output a video stream in a certain resolution, for example 1280x720p. The box 210 also makes it possible to convert the HDMI video stream from the tablet 20 into an avionics format, such as the SMPTE avionics format or the ARINC 818 avionics format.
[0037] In a third step E3, the pilot interacts with the display screen in the cockpit. For example, the pilot presses a point P visible in [Fig.4]. This point P has coordinates Touchx, Touchy in the first part 2201 of the display screen 220. It will be noted here that this first part 2201 has a rectangular shape and is determined by the points A, B, C, D forming the four corners of this rectangle. In the reference frame of the display screen, point A has coordinates {0, 0], point B has coordinates {MAX_TX, 0], point C has coordinates {MAX_TX, MAX_TY], point D has coordinates {0, MAX_TY}.
[0038] The coordinates of point P are transmitted to the adapter 21. The latter will then adapt them to the reference of the tablet 20 by providing adapted coordinates Coord' of point P. Upon receiving these adapted coordinates, the tablet 20 will provide data from the flow F in accordance with the interaction exerted by the pilot on the display screen. The information on the rectangle ABCD is transmitted to the adapter 21, which allows it to calculate the transformation of the coordinates received from the avionics to Coord'.
[0039] The method then enters a calibration mode in which the interaction coordinate transfer step E3 is deactivated. In a step E4, the pilot selects a window of interest Win on the display screen 220. Such a window is notably illustrated in [Fig.4].
[0040] This window of interest Win has a rectangular shape and its outline is determined by the points E, F, G, H. Thus, the point E has coordinates {tx, ty}. The point F has coordinates {tx+tw, 0}. The point G has coordinates {tx+tw, ty+th}. The point H has coordinates {0, ty+th}. The information on the rectangle EFGH is transmitted to the adapter 21, which allows it to calculate the transformation of the coordinates received from the avionics to Coord'.
[0041] A cutting operation is then applied to the video stream according to this window of interest Win. This cutting operation is also called a cutting operation. "cropping". It is carried out in the block integrated into the avionics equipment. In a step E5, the part of the data flow corresponding to the selected window of interest Win is displayed on the display screen 220. An update of the windowing information is also carried out towards the adapter 21 to modify the coordinate transformation calculation Coord' accordingly. This makes it possible to ensure conformity of the interactions between the avionics screen and the tablet, regardless of the selected area of interest, both for touch and cursor / CCD (mouse type) interactions.
[0042] The delimitation of the area of interest Win by the pilot can be done in different ways.
[0043] Preferably, contact with one of the borders of the display screen and a movement of the operator's finger in continuous contact with the display screen 220 make it possible to determine the limits of the window of interest Win.
[0044] Preferably, another contact with a corner of the window of interest Win and a movement of the operator's finger in continuous contact with the display screen make it possible to adjust the size of said window of interest Win.
[0045] Preferably, another contact inside the window of interest Win and a movement of the operator's finger in continuous contact with the display screen 220 allow a movement of the position of the window of interest Win.
[0046] [Fig. 5] illustrates an alternative embodiment, in which the selection of the window of interest Win is carried out by the pilot via at least one rotator 23. This rotator 23 allows an adjustment along the horizontal axis X and an adjustment along the vertical axis Y of the dimensions of said window of interest Win. It is in the form of a rotary switch allowing the contours of the window of interest to be selected directly on the first part 2201 of the display screen 220. In a preferred embodiment, the device 2 for displaying the video data comprises a double encoder rotator directly present on the screen to allow easier selection of the contours of the window of interest along the X axis and along the Y axis.
[0047] Preferably, an acceleration law makes it possible to multiply a selection speed of the window of interest Win during rapid movements of the rotator 23.
[0048] The invention thus allows: - to adapt the “media delegation” capacity to a wide variety of tablets, without requiring a software update (onboard or firmware); - to take into account video calibration and the interaction zone: touch interactions known as “dual touch” and “CCD” continue to function, without lag, whatever the video zone selected.
[0049] The invention is not limited to the embodiments and variants presented.
[0050] Thus, in an additional capacity, it is possible to remove any ambiguity on the risk of hiding information in the “useful” video stream by giving the ability to quickly view the entire flow and the materialization of the active selection area. Indeed, when switching to a calibration mode from step E3, the entire flow received is displayed. It is this property, with rapid access, which makes it possible to offer a capacity to remove doubt. This capacity makes it possible to partially respond to a fear of hidden content raised by the EASA (for "European Union Aviation Safety Agency").
Claims
Claims
1. Method for processing a data stream (F) originating from a non-certified video source, said data stream being intended to be displayed at least partially on a display screen (220) of a cockpit (22) of an aircraft, said display screen (220) having predetermined dimensions and a predetermined resolution, said processing method comprising: - a step (El) of sending the data stream by the non-certified video source; - a step (E2) of adapting the data of the data stream to adapt said data to the dimensions and resolution of the display screen (220) of the cockpit (22);- a pilot being able to interact with the display screen in the cockpit, a step (E3) of transferring interaction coordinates (Coord) to the non-certified video source via an adapter (21) so that the non-certified video source provides data of the stream (F) in accordance with the interaction between said pilot and the display screen, characterized in that said processing method comprises: - a step of switching to a calibration mode comprising on the one hand a selection (E4) on the display screen (220) of a window of interest (Win) by the pilot and on the other hand an operation of cutting the stream (F) according to said window of interest (Win); - a step (E5) of displaying on the display screen (220) a part of the stream (F) of data corresponding to the selected window of interest (Win).;
2. Processing method according to claim 1, in which the selection of the window of interest (Win) is carried out directly by the operator by contact with the display screen (220).
3. Processing method according to claim 2, in which contact with one of the borders of the display screen (220) and a movement of the operator's finger in continuous contact with the display screen (220) make it possible to determine the limits of the window of interest (Win).
4. A treatment method according to claim 3, wherein further contact with a corner of the window of interest (Win) and movement of the operator's finger in continuous contact with the display screen (220) allows the size of said window of interest (Win) to be adjusted.
5. A treatment method according to any one of claims 3 or 4, wherein another contact within the window of interest (Win) and a movement of the operator's finger in continuous contact with the display screen (220) allow a movement of the position of the window of interest (Win).
6. Processing method according to claim 1, in which the selection of the window of interest (Win) is carried out by the operator via at least one rotator (23), allowing adjustment along a horizontal axis and adjustment along a vertical axis of the dimensions of said window of interest (Win).
7. Processing method according to claim 6, in which an acceleration law makes it possible to multiply a speed of selection of the window of interest (Win) during rapid movements of the rotator (23).
8. Processing method according to any one of claims 1 to 7, wherein the step (E3) of transferring interaction coordinates (Coord) comprises a transfer of transformation parameters to the adapter (21).
9. Processing method according to any one of claims 1 to 8, wherein when switching to a calibration mode the entire data stream is displayed, the part of the data stream (F) corresponding to the selected window of interest (Win) being displayed with the entire data stream (F).
