Improved reliability of display systems
The display system addresses the reliability issues of COTS GPUs by encoding critical flight data with markers for real-time verification and error correction, ensuring accurate and safe display of flight parameters, thereby enhancing aircraft safety.
Patent Information
- Application Number
- JP2024535920
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-05-02
- Filing Date
- 2022-12-12
- Publication Date
- 2025-01-28
AI Technical Summary
Commercial Off The Shelf (COTS) Graphics Processing Units (GPUs) used in aircraft cockpit display systems lack the necessary safety assurance level (DAL) for reliable display of critical flight data, prone to errors due to manufacturing defects, radiation, electromagnetic interference, and aging, leading to potential pilot misunderstandings and safety risks.
A display system with a graphics generator, display control unit, and monitoring unit that encodes critical flight data with markers, allowing real-time verification of data accuracy and generating warning signals if errors occur, ensuring reliable display of flight parameters like airspeed, altitude, pitch, and roll.
Ensures accurate and reliable display of critical flight data by detecting and correcting errors, preventing pilot misunderstandings, and enhancing aircraft safety through independent monitoring of the video frame output.
Smart Images
Figure 2025502668000001_ABST
Abstract
Description
[Technical field]
[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This patent application claims priority to European Patent Application No. 21425067.2, filed December 15, 2021, and Italian Patent Application No. 102022000008837, filed May 2, 2022, the entire disclosures of which are incorporated herein by reference.
[0002] The present invention relates to a display system with improved reliability. It also relates to a method for displaying safety-critical data and to a vehicle, in particular an aircraft, equipped with said display system. The present invention finds preferred, but not exclusive application in the aerospace field, in particular in displaying primary flight display data. [Background technology]
[0003] The cockpits of modern airplanes and helicopters contain digital display systems that provide the human-machine interface (HMI) that enables pilots to fly, control the aircraft, and complete its mission.
[0004] HMI technology is typically based on symbol generators powered by COTS ("Commercial Off The Shelf") GPUs ("Graphics Processing Units") or proprietary raster controller devices.
[0005] Currently, to reduce the overall cost of HMI, the industry is also widely utilizing COTS GPUs for displaying safety-critical data. In particular, modern aircraft cockpit displays use COTS GPU-based systems to render graphics for pilot HMIs. Because COTS GPUs are not devices specifically designed for the avionics market, these commercially available devices cannot guarantee a level of safety of operation sufficient for applications involving the display of safety-critical data (e.g., no DO-254 pedigree). In other words, the DAL ("Design Assurance Level") of these devices is insufficient for use in safely and reliably displaying safety-critical data. Examples of errors that may occur during the functioning of a COTS GPU include: permanent errors, mainly due to the manufacturing process or aging of the COTS GPU; non-reproducible errors that may be due to external factors such as ionizing radiation, electromagnetic interference (e.g., NEMP, EMI), lightning strikes or electrostatic discharge (ESD) of the COTS GPU; intermittent errors due to aging or variations in the quality of the manufacturing process of the COTS GPU.
[0006] Much of the information displayed on these HMIs is considered safety critical because pilots respond directly to this information to fly the aircraft, so misleading information provided by the HMI could lead to pilot errors that could have catastrophic effects on the aircraft, the pilot, and the surrounding environment.
[0007] In the specific case of the Primary Flight Display (PFD), misleading presentation of information about data that is considered important from a safety standpoint must be avoided. In particular, this concept applies to examples of data that are important to the safety of the aircraft, such as airspeed, altitude, pitch and roll.
[0008] Ensuring the avoidance of misleading information is essential to obtain airworthiness certification of the Primary Flight Display System and, more generally, to ensure correct control of the aircraft by the pilot. Summary of the Invention [Problem to be solved by the invention]
[0009] Therefore, there is a felt need to ensure that the safety-critical data displayed by the HMI is correct.
[0010] It is an object of the present invention to meet the above needs. [Means for solving the problem]
[0011] The aforementioned objects are achieved by a display system, a transport device and a method for displaying safety-critical data as claimed.
[0012] In order that the invention may be better understood, preferred embodiments will now be described, by way of non-limiting example, with reference to the accompanying drawings, in which: [Brief description of the drawings]
[0013] [Figure 1] FIG. 1 is a block diagram that illustrates a schematic diagram of a display system according to an embodiment of the present invention. [Figure 2A] 2A shows respective examples of graphics of intermediate video frames generated by the display system of FIG. 1 according to respective embodiments. [Figure 2B] 2B shows respective examples of graphics of intermediate video frames generated by the display system of FIG. 1 according to respective embodiments. [Diagram 3] FIG. 3 shows a graphic of the airspeed region of the intermediate video frame of FIGS. 2A and 2B including a first encoding marker, according to one embodiment. [Figure 4]FIG. 4 shows example numbers that can be displayed in the airspeed area of FIG. 3 and the graphics of their respective first encoding markers. [Diagram 5] FIG. 5 shows a graphic of the pitch and roll regions of the intermediate video frames of FIGS. 2A and 2B including second encoding markers according to one embodiment. [Figure 6] FIG. 6 is a graphic of the airspeed region of a verified video frame further generated by the display system of FIG. 1 that does not include the first encoding marker, according to one embodiment. [Figure 7] FIG. 7 is a graphic of the verified video frame according to one embodiment. [Figure 8] FIG. 8 is a block diagram that illustrates generally a method for generating the verified video frame of FIG. 7, according to an embodiment of the present invention. [Figure 9] FIG. 9 is a block diagram that illustrates a schematic diagram of a transportation device equipped with the display system of FIG. 1, according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, elements common to each embodiment will be given the same reference numerals.
[0015] FIG. 1 shows a display system (also referred to as a safe primary flight data display system) 10 that may be installed on an aircraft, such as an airplane, helicopter, remotely piloted unmanned aerial vehicle, or seaplane.
[0016] In use, the display system 10 provides information to a pilot of an aircraft (in particular safety critical data, and in particular Primary Flight Display (PFD) data, considered by way of example below) to enable the pilot to fly the aircraft and perform operations such as navigation and mission execution.
[0017] In particular, the display system 10 is configured to implement a method for displaying safety critical data (in particular PFD data, considered exemplarily below) which is described in more detail below with reference to Figure 8. In particular, the PFD data is displayed to the pilot via video.
[0018] Although the following description refers to the processing of a single frame, it will be apparent that the PFD data can be displayed on a frame-by-frame basis by repeating the following operations multiple times to generate a corresponding video, with each frame showing respective updated PFD data.
[0019] The display system 10 comprises a graphics generator 20, a display control unit 22, a display panel 24, and optionally a display monitoring unit 26, all of which are operatively coupled to one another.
[0020] The display panel 24 is a display element known per se, such as an LCD panel, and is used to display the PFD data to the pilot (ie, to display the frames that make up the video).
[0021] The display system 10 is connected to aircraft sensors external to the display system 10 that measure PFD data indicative of aircraft operating conditions or quantities (i.e., aircraft operating parameters) while the aircraft is in use.
[0022] The graphics generator 20 receives one or more of these measured PFD data from the aircraft (hereinafter, input PFD data or input data D i ) and obtain the input data D i Based on the intermediate video frame VF i In the following, for illustrative and non-limiting purposes, a plurality of input data D i Consider the illustrative case of
[0023] In particular, the input data D iare data indicative of quantities of interest for the aircraft, and more specifically, parameters / symbols that are important for the safety of the aircraft (i.e., important symbol input data).
[0024] In detail, the input data D i is the primary flight display (PFD) data. For example, input data D i includes one or more data of airspeed (i.e., the speed of the aircraft relative to the air), altitude of the aircraft (e.g., measured relative to the surface of the earth below the aircraft or to mean sea level), pitch angle and roll angle of the aircraft. In the following, the input data D i Reference is made to an exemplary and non-limiting embodiment in which input data D includes airspeed, altitude, pitch angle, and roll angle (hereinafter also referred to as input airspeed, input altitude, input pitch, and roll, respectively), which is considered for illustrative purposes only, and the input data D i It will be apparent that the can include any subset of these data, as well as other PFD data.
[0025] In particular, the graphics generator 20 comprises a COTS GPU or any other known device (e.g. a raster controller device) configured to generate computer graphics, i.e. signals corresponding to video frames. To generate the computer graphics, the graphics generator 20 comprises software of a type known per se, for example using standard graphics libraries (OpenGL, DirectX, etc.) or proprietary graphics libraries.
[0026] The graphics generator 20 receives the input data D i Starting from the intermediate data (hereafter referred to as monitoring data D m , which represents an intermediate video frame VF i Generate.
[0027] The intermediate data is the received input data D i However, the input data Di If an error occurs during the processing of i and therefore, in order to prevent the pilot from controlling the aircraft based on erroneous information, it is necessary to present the pilot with a warning signal informing him of this safety-critical condition regarding the displayed data (this can be done, for example, by not displaying the intermediate data to the pilot, or by displaying the intermediate data together with a warning signal informing the pilot of the inaccuracy of the displayed data, as will be explained in more detail below). In other words, if there are no errors in the functioning of the graphics generator 20 (the graphics generator 20 is operating normally), the intermediate data is a representation of the received input data D i In response to the above, if an error occurs in the function of the graphics generator 20 (if the graphics generator 20 is not operating normally), the intermediate data is i Does not support.
[0028] In detail, the intermediate video frame VF i is a two-dimensional matrix of pixels, for example having size N1×M1.
[0029] According to one embodiment, an intermediate video frame VF iThe intermediate video frame VF includes a first frame region (i.e., a first group of pixels) and a second frame region (i.e., a second group of pixels). The first frame region includes a visual representation of the intermediate data together with encoding markers, as described in more detail below. The second frame region encodes control data that provides information about the encoding markers, in particular information about how to locate and / or identify the encoding markers present in the first frame region. In particular, the control data indicates the location (coordinates) of the encoding markers in the first frame region (i.e., for the area in the first frame region where the encoding markers, and thus the intermediate data, are expected to be rendered). Furthermore, the control data may also indicate additional information such as the color and mask of the pixels of the encoding markers. The control data allows the intermediate video frame VF to be generated. i The relevant data can then be suitably captured and processed to identify significant symbol input data present in the first frame region.
[0030] FIG. 2A illustrates an intermediate video frame VF generated by the graphics generator 20. i In this example, a first frame region (indicated by reference number 40) is an intermediate video frame VF i a second frame area (indicated by reference numeral 42) is located in the intermediate video frame VF i , for example, disposed outside with respect to the first frame region 40 (eg, surrounding the first frame region 40).
[0031] FIG. 2B illustrates an intermediate video frame VF generated by the graphics generator 20. i 1 shows another example of a frame region 42 adjacent to one side of the first frame region 40, for example the underside 40' of the first frame region 40.
[0032] The intermediate video frame VF shown in FIG. iare provided for illustrative and non-limiting purposes only. Indeed, other variations, such as the shape and location of the second frame region 42 relative to the first frame region 40, are possible.
[0033] The second frame area 42 encodes the control data according to techniques known per se. For example, the control data is encoded by a respective pattern of pixels of the second frame area 42. As an example, in Fig. 2A the control data is uniformly encoded in the entire pixels of the second frame area 42, while in Fig. 2B the second frame area 42 includes a number of sub-areas 42a-42d each with a respective pixel pattern encoding control data for each intermediate data (e.g. intermediate airspeed, intermediate altitude and intermediate pitch and roll angles, respectively). As another example (not shown), the control data can be encoded in the second frame area 42 via one or more barcodes or QR codes.
[0034] In the considered example shown in Figures 2A and 2B, the first frame region 40 includes an airspeed region 40a, an altitude region 40b and a pitch and roll angle region 40c, which display mid-airspeed, mid-altitude and mid-pitch and roll angle values of the aircraft, respectively.
[0035] These intermediate values are calculated based on the input data D depending on the absence or presence, respectively, of processing errors by the graphics generator 20. i either correspond (e.g., match) or do not correspond (e.g., differ) to the values of
[0036] In particular, airspeed region 40a, altitude region 40b, and pitch and roll region 40c each display both respective PFD intermediate data (intermediate airspeed, intermediate altitude, and intermediate pitch and roll, respectively) and respective encoding markers that encode the intermediate data, thereby allowing the intermediate data to be represented as input data D, as will be described in more detail below. iIt is possible to provide redundancy in information regarding the intermediate data for determining whether the intermediate data corresponds to the
[0037] The following description will illustratively refer to airspeed field 40a to illustrate the display of PFD intermediate data with one or more numbers (e.g., via a digital display) along with each first encoding marker, however, similar considerations are applicable to the display of other PFD intermediate data displayed via numbers, such as altitude field 40b.
[0038] An example of an airspeed field 40a is shown in Figure 3. In particular, the airspeed field 40a displays an intermediate airspeed value for the aircraft as a number (120.0 knots in the considered example). Each first encoding marker 43 is uniquely associated with each number 44 of the intermediate airspeed value such that the intermediate airspeed value can be obtained from either the displayed number 44 or the first encoding marker 43.
[0039] As shown in more detail in FIG. 4, each first encoding marker 43 is a pixel pattern uniquely associated with each digit 44 and is located adjacent to each digit 44 (e.g., closer to each associated digit 44 than other digits 44), e.g., to the left of the digit 44. In other words, the rendering of the digits 44 is performed using particular glyphs of a font set, where each glyph contains pixels that provide an encoding of the digit 44 itself. The encoding is unique within the font set and is part of the glyph itself (i.e., it is not added at run-time). The encoding pixels may vary from solution to solution.
[0040] In the embodiment exemplarily shown in Fig. 4, each first encoding marker 43 is realized via a respective barcode, in particular, it is known from the control data, is located in a first predetermined area (also called first capture area) 45 adjacent to the digits 44 and comprises one or more segments (e.g. vertical segments such as bars) representing each digit 44. For example, the number 0 is associated with one vertical segment in a first position of the first predefined area 45 (e.g., the pixels in the first column of the first predefined area 45), the number 1 is associated with one vertical segment in a second position of the first predefined area 45 (e.g., the pixels in the second column of the first predefined area 45), the number 2 is associated with two vertical segments located in the first and second positions of the first predefined area 45 (e.g., the pixels in the first and second columns of the first predefined area 45), and the number 9 is associated with two vertical segments located in the second and third positions of the first predefined area 45 (e.g., the pixels in the second and third columns of the first predefined area 45). These segments are realized, for example, in a gray scale or RGB scale, via pixels having a first value (or color) on a background of pixels of a second value (or color). However, it is clear that the first encoding markers 43 can be realized and arranged in different ways, provided they identify each number 44.
[0041] An example of pitch and roll region 40c is shown in Figure 5. In particular, pitch and roll region 40c displays attitude indicator 50 superimposed on the pilot's field of view (e.g., the horizon shown in Figure 5 as reference numeral 56), which may be, for example, a synthetic vision system or an artificial horizon (i.e., sky and terrain).
[0042] The attitude indicator 50 is commonly used to indicate the mean pitch and roll angles of the aircraft and to present this information to the pilot. In particular, the attitude indicator 50 includes an attitude bar 52 and a pitch reference bar (also referred to as an aircraft bar) 54.
[0043] The attitude bars 52 are bars (i.e., segments) that are parallel to and between horizontal lines 56 and perpendicular to a first alignment axis 58 (perpendicular to the horizontal line 56). The attitude bars 52 are aligned and spaced apart (e.g., equally spaced or with increasing incremental steps when representing increasing intermediate pitch angles) from one another along the first alignment axis 58. In particular, the attitude bars 52 are centered along the first alignment axis 58 on the horizontal line 56. In other words, the horizontal lines 56 are equally spaced apart relative to the extremal attitude bars 52' that face each other along the first alignment axis 58 and form respective ends of the group of attitude bars 52.
[0044] The pitch reference bar 54 is a bar perpendicular to a reference axis 60 (vertical, fixed) and mutually aligned along a second alignment axis 61 perpendicular to the reference axis 60 and parallel to the pitch reference bar 54. The pitch reference bar 54 indicates the pitch axis of the aircraft and has a fixed horizontal position since, across all video frames, the reference frame of the displayed image is that of the pilot (while the displayed field of view, which shows the scenery around the aircraft, changes with the movement of the aircraft relative to the ground).
[0045] The intermediate roll angle is the angle formed between the reference axis 60 and the first alignment axis 58, and is indicated in Figure 5 by the reference α, while the intermediate pitch angle is visually represented by the mutual alignment of the attitude bars 52 and the pitch reference bars 54 (i.e., inferred from which attitude bar 52 is closest to the second alignment axis 61). Thus, in the absence of a roll angle, the reference axis 60 and the first alignment axis 58 coincide, and in the absence of a pitch angle, the horizon 56 and the second alignment axis 61 overlap at least at a midpoint extending between the pitch reference bars 54 and are equally spaced relative to the pitch reference bars 54.
[0046] The second encoding marker 62 is associated with the attitude indicator 50 such that intermediate pitch and roll angles can be obtained from both the attitude indicator 50 and the second encoding marker 62 for the displayed field of view.
[0047] In particular, each second encoding marker 62 is associated with each attitude bar 52 and pitch reference bar 54 present in the pitch and roll region 40c and is contained within a second predetermined region (or capture region) 63 of the pitch and roll region 40c whose position is known from the control data. For example, each attitude bar 52 includes a second attitude encoding marker 62' that is centered with respect to the attitude bar 52. That is, the second attitude encoding markers 62' are equally spaced from the ends of the attitude bar 52 that are opposite each other along the main direction of the attitude bar 52 (i.e., parallel to the horizontal line 56), and each pitch reference bar 54 includes a second pitch encoding marker 62'' that is centered with respect to the pitch reference bar 54. That is, the second pitch encoding marker 62'' is equally spaced from the ends of the pitch reference bar 54 that are opposite each other along the main direction of the pitch reference bar 54 (i.e., perpendicular to the reference axis 60). Thus, the second posture-encoding markers 62 ′ are aligned with one another along the first alignment axis 58 , and the second pitch-encoding markers 62 ″ are aligned with one another along the second alignment axis 61 .
[0048] In particular, the rendering of the second encoding markers 62 in the posture bars 52 and pitch reference bars 54 is performed via a specific predefined color encoding of one or more pixels forming the second encoding markers 62, so that they can be recognized among all the pixels available in the pitch and roll region 40c. For example, the rendering of the posture bars 52 and pitch reference bars 54 is performed by constraining the RGB values (typically encoded as hexadecimal / binary) of each pixel belonging to the bars 52, 54, so that the R, G, and B values of the (RGB) pixels belonging to the segment body of the bars 52, 54 are represented with the n least significant bits equal to zero. In each posture bar 52, one pixel (the marking pixel, i.e. the second posture encoding marker 62') is located on the center line of the posture bar 52 (or at an equivalent position) and is rendered with a color whose R, G, and B values are the same as the segment body of the posture bar 52, and which is further rendered with a unique encoding value set to the n least significant bits of the R, G, B triplet. In both pitch reference bars 54, one pixel (marking pixel, i.e., second pitch encoding marker 62'') is located on the centerline of pitch reference bar 54 (or at an equivalent location) and its R, G, and B values are the same as the segment body of pitch reference bar 54, and further rendered in a color that is a unique encoding value set in the n least significant bits of the R, G, B triplet. Using only one pixel as a marking pixel is advantageous because, for a given pixel resolution, the human eye cannot distinguish a single pixel of a different color from its neighboring pixels, as long as "n" is small enough (e.g., up to "n" of 4 for each color element R, G, B). Thus, the second encoding marker 62 can be automatically detected even though it is not discernible to the naked eye.
[0049] Referring again to FIG. 1, the graphics generator 20 receives input data D i Based on the intermediate video frame VF i (i.e., intermediate video frame VFi It is described that an intermediate video frame VF i contains a visual representation of the intermediate data and encoding markers 43, 62, and further includes an intermediate video frame VF i If the functioning of the graphics generator 20 is error-free, the intermediate data includes control data indicating the position of an encoding marker in the received input data D i In case of an error in the functioning of the graphics generator 20, the intermediate data is the received input data D i may be different from.
[0050] The display control unit 22 receives an intermediate video frame VF i (Details: intermediate video frame signal) is obtained, and monitoring data D m The display control unit 22 is a control unit (e.g., an FPGA) configured to extract the display data (i.e., intermediate data) and provide it to the display monitoring unit 26. Furthermore, the display control unit 22 receives a command signal S c and, as described in more detail below, an intermediate video frame VF i and command signal S c Based on the confirmed video frame VF c or configured to generate a warning signal.
[0051] In particular, the display control unit 22 receives an intermediate video frame VF i and identifying the encoding markers 43, 62 in the first frame region 40 based on the control data. The display control unit 22 then uses the encoding markers 43, 62 to encode the monitoring data D m Obtain these monitoring data D m Output.
[0052] More specifically, the values of the numbers 44 for the intermediate airspeed and intermediate altitude are obtained by identifying a first predetermined region 45 based on the control data and decoding the first encoding markers 43 present in the first predetermined region 45 due to the unique relationship between each first encoding marker 43 and each number 44. For example, a look-up table storing the first encoding markers 43 and associated numbers 44 may be used to obtain the value of the numbers 44 from each first encoding marker 43.
[0053] Meanwhile, the intermediate pitch and roll angle values are obtained by: identifying a second predetermined area 63 based on the control data; scanning the pixels of the second predetermined area 63 to find the second encoding markers 62 (e.g., looking for a specific predetermined color encoding that identifies the pixels that form the second encoding markers 62); determining the first alignment axis 58 and the second alignment axis 61 based on the second attitude encoding markers 62' and the second pitch encoding markers 62'', respectively; knowing the position of the reference axis 60 (predefined and orthogonal to the second alignment axis 61); calculating the intermediate roll angle α based on the reference axis 60 and the first alignment axis 58, and calculating the intermediate pitch angle based on the relative position of the second alignment axis 61 (i.e., the second pitch encoding marker 62'') and the attitude bar 52 (i.e., the second attitude encoding marker 62').
[0054] The display monitoring unit 26 receives the input data D i and monitoring data D determined from encoding markers 43, 62 m It takes both as input and input data D i and monitoring data D m Based on the comparison of the command signal S c In particular, the command signal S c is the input data D i and monitoring data D mMore specifically, the display monitoring unit 26 indicates whether the input data D i and monitoring data D m to determine whether they match, and based on the comparison, generate a command signal S having a corresponding value. c For example, the command signal S c is the input data D i and monitoring data D m If they correspond (e.g., match), then the input data D i and monitoring data D m do not correspond (eg, do not match), then it has a second value.
[0055] This correspondence may be a match or the input data D i and monitoring data D m In the latter case, the input data D i and monitoring data D m (e.g., input airspeed and intermediate airspeed) are considered to correspond when the relative difference between them is smaller than a respective threshold difference (set in a manner known per se based on the type of PFD data and other factors, e.g., the desired accuracy of the information display to the pilot, etc.). More generally, in the second case, a predetermined correlation is satisfied, so that the input data D i and monitoring data D m If the relative difference (e.g., Euclidean distance) between the input data D i and monitoring data D m corresponds to.
[0056] The display control unit 22 receives a command signal S c and accordingly, the confirmed video frame VF c 5. Generate a confirmed video frame signal or (optionally) a warning signal indicating (i.e., encoding)
[0057] In particular, the confirmed video frame VF cis a two-dimensional matrix of pixels having, for example, a size of N2×M2 (N2 < N1 and M2 < M1).
[0058] According to a preferred embodiment, the verified video frame VF c has no (i.e., does not contain) at least encoding markers 43, 62 that are distinguishable by the naked eye. Specifically, due to its structure, generally the first encoding marker 43 is visible and distinguishable by the naked eye, while the second encoding marker 62 is visible but not distinguishable by the naked eye. More specifically, each pixel or group of pixels displayed on the pilot is considered visible to the naked eye, but specific conditions must be met for this pixel or group of pixels displayed on the pilot to be considered distinguishable by the naked eye. These conditions are known per se. For example, a group of pixels is considered distinguishable by the naked eye by the pilot if it contains more pixels than a threshold number of pixels, and the threshold number of pixels is determined in a method known per se from factors such as the display dots per inch, the pilot's eyesight, and the color contrast of these pixels against the background.
[0059] Specifically, according to this preferred embodiment, the command signal S c is the input data D i and the monitoring data D m shows the correspondence (i.e., when generating the intermediate video frame VF i shows the normal operation of the graphics generator 20 when processing the input data D i ), the display monitoring unit 26 generates a verified video frame VF i corresponding to the first frame area 40 of the intermediate video frame VF c . However, there may be no (all or some of) the encoding markers. More specifically, in this case, the display monitoring unit 26 is the intermediate video frame VF i2. Extract the first frame region 40 from the verified video frame VF and remove any encoding markers that are visible to the naked eye (and optionally also remove any encoding markers that are not visible to the naked eye). For example, the first encoding marker 43 may be removed and the second encoding marker 62 may remain. c An example of an airspeed region of 40a′ is shown in FIG. 6 with a new reference number 40a′. The first frame region 40 thus processed is then added to the confirmed video frame VF by the display monitoring unit 26. c In this case, the confirmed video frame VF c An example of the confirmed video frame VF is shown in FIG. 7. FIG. 7 shows an airspeed region 40a′ and an altitude region 40b′ without the first encoding marker 43. According to another embodiment, the confirmed video frame VF c is an intermediate video frame VF i (i.e., contains visually discernible encoding markers 43, 62).
[0060] On the other hand, the command signal S c is the input data D i and monitoring data D m If the intermediate video frame VF i To generate the input data D i , indicating abnormal operation of the graphics generator 20 when processing the PFD data), the display monitoring unit 26 can generate the warning signal notifying the pilot of the unreliability of the displayed PFD data.
[0061] The warning signal is a confirmed video frame VF having graphics configured to alert the pilot about this safety critical condition on the displayed PFD data. c For example, the confirmed video frame VF cis a graphic that does not indicate intermediate data (e.g., a graphic in which all pixels have the same color value and provide no information to the pilot) or an intermediate video frame VF without any identifiable encoding markers. i a first frame region 40 of the confirmed video frame VF c or a red cross or equivalent symbol, text message or visual warning placed against incorrectly displayed intermediate data, e.g., a confirmed video frame VF colored in a specific color designed to notify the pilot of a safety-critical condition. c The graphic may include a particular area of the image, for example an area colored red.
[0062] Additionally or alternatively, the warning signal may include a warning signal provided to the pilot by an external warning means (e.g., an indicator light such as an LED or a speaker located in the cockpit of the aircraft) not shown and external to, but operably coupled to, the display system 10. For example, in this case, the display monitoring unit 26 may receive a confirmed video frame VF c (Any form of graphics that does not indicate intermediate data, or intermediate video frames VF without any identifiable encoding markers, i i.e. exactly as if no safety critical condition had occurred), in addition to generating said warning signal in the form of a warning provided by external warning means operatively coupled to the indication monitoring unit 26.
[0063] Next, the display panel 24 displays the confirmed video frame VF c and displays it to the pilot.
[0064] Typically, the graphics generator 20, the display control unit 22 and the display monitoring unit 26 are control units such as an FPGA, a dedicated controller or a CPU. Optionally, the display control unit 22 and the display panel 24 form a display means (not shown) of the display system 10.
[0065] FIG. 8 illustrates a method 100 for displaying PFD data with increased reliability (i.e., verified video frames VF) implemented by the display system 10. c (How to generate it)
[0066] In particular, one iteration of method 100 is described herein, which corresponds to one frame of video displayed by display panel 24. However, it will be apparent that by repeating these steps multiple times, display panel 24 can display video that visually represents changes in the PFD data over time.
[0067] Moreover, details of the following steps that have already been described above will not be repeated below for the sake of brevity.
[0068] In step 110 of the method 100, the input data D is i is obtained.
[0069] Step 110 is followed by step 112, in which the input data D i An intermediate video frame VF is generated by the graphics generator 20 based on i An intermediate video frame signal is generated which indicates:
[0070] Step 112 is followed by step 114, in which the display control unit 22 generates an intermediate video frame VF i Monitoring data D m is determined (eg, extracted).
[0071] In step 116 following step 114, the monitoring data D m and the input data Di and are compared to each other by the display monitoring unit 26 to determine whether they correspond.
[0072] Step 116 is followed by step 118, in which the monitoring data D m and the input data D i Based on the correspondence between the command signal S and the command signal S, the display monitoring unit 26 c is generated.
[0073] Step 118 is followed by step 120, in which the command signal S c and intermediate video frames VF i The display control unit 22 receives the confirmed video frame VF c Or (optionally) a warning signal is generated. m and the input data D i If and correspond to each other, then the input data D i A confirmed video frame VF that visually represents c is generated and displayed to the pilot. Meanwhile, the monitoring data D m and the input data D i If the and do not correspond to each other, a warning signal is generated to inform the pilot that reliable PFD data cannot be displayed.
[0074] Step 120 is followed (optionally) by step 122, in which the command signal S c Depending on the confirmed video frame VF c is displayed to the pilot via the display panel 24, or optionally, a warning signal is presented to the pilot as previously described.
[0075] From the above, the advantages of the display system 10 and method 100 according to the present invention are apparent.
[0076] The display system 10 enables an effective independent monitoring technique of safety critical data presented on the HMI, thereby avoiding the presentation of misleading information to the pilot, focusing in particular on the aircraft's primary flight data (airspeed, altitude, pitch angle and roll angle). In detail, this solution monitors the video frames output by the GPU (or raster controller device) and verifies the accuracy of the safety critical data contained in the video itself, allowing generic COTS GPUs (instead of custom devices) to be safely used to generate graphics for the primary flight displays.
[0077] Furthermore, important symbols / data are checked by the COTS GPU (or raster controller device) directly on the video stream being generated, thus eliminating the need to draw additional symbols / patterns dedicated to monitoring in hidden areas not directly related to the displayed video (i.e., in the video buffer or in excess areas of the video frame).
[0078] In addition, the encoding markers 43, 62 are used to monitor the data D m Determining the monitoring data D has the advantage that the encoding markers 43, 62 have a smaller size than the respective numbers 44 or bars 52, 54 with which they are associated. In other words, they are coded with fewer pixels than the numbers 44 and bars 52, 54. Thus, for the numbers 44 and bars 52, 54, the monitoring data D m The computational cost required to process the encoding markers 43, 62 to determine
[0079] It is clear that modifications can be made to the described display system 10 and method 100 without going beyond the scope of protection defined by the claims.
[0080] For example, the same solutions and techniques can be employed in aircraft cockpit HMIs, as well as in automotive displays, industrial control panels, and other fields where the displayed data needs to be monitored to avoid misleading information. In general, the present invention can be used to monitor important data displayed on COTS GPU-generated graphics in any field (automotive, industrial, etc.) application.
[0081] 9, the display system 10 may be included in a vehicle 150 (e.g., but not limited to, aircraft such as airplanes, helicopters, unmanned aerial vehicles, and surface craft such as submarines, hovercraft, and eventually modern rescue drones) that utilizes instrument aids for navigation or safety critical missions. The vehicle 150 may be of any general type, may be operated by a user (e.g., a pilot), and may further include a sensor 152 coupled to the display system 10. The sensor 152 may receive input data D i In principle, the display system 10 and method 100 are applicable to any use case in which display data is used by a user to select, choose, and perform a particular action on or by the vehicle 150.
[0082] Furthermore, the control data is stored in the intermediate video frame VF i Separately, for example, via a separate communication channel, intermediate video frames VF i According to this embodiment, the intermediate video frame VF can be provided to the display control unit 22 from the graphics generator 20 rather than as data embedded in the i These intermediate video frames VF only contain the first frame region 40 and do not contain the second frame region 42 which encodes the control data. i (e.g., intermediate video frames VF i , i.e. in a synchronized manner with respect to the latter), to the display control unit 22.
Claims
1. A display system (10) for a transportation device operable by a user, the display system (10) comprising: - input safety critical data (D) indicative of one or more operational parameters of said vehicle; ｉ ) and obtain the input safety important data (D ｉ ) based on the intermediate video frame (VF ｉ a graphics generator (20) configured to generate an intermediate video frame signal indicative of The intermediate video frame (VF ｉ ) is the monitoring safety critical data (D ｍ ), the first frame area (40) being a visual representation of the monitoring safety critical data (D ｍ ) an encoding marker (43, 62) uniquely associated with The Monitoring Safety Critical Data (D ｍ ) and the input safety important data (D ｉ ) are graphics generators (20) that correspond to each other under normal operation conditions of said graphics generators (20) and do not correspond to each other under abnormal operation conditions of said graphics generators (20); - coupled to said graphics generator (20) and receiving said intermediate video frame signal, and encoding said monitoring safety critical data (D ｍ a display control unit (22) configured to determine - coupled to said display control unit (22) and said input safety critical data (D ｉ ) and the monitoring safety critical data (D ｍ ) and receiving both the input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) and a command signal (S ｃ ) and a display monitoring unit (26) configured to generate said command signal (S ｃ ) is the input safety important data (D ｉ ) and the Monitoring Safety Important Data (D ｍ an indication monitoring unit (26) indicating correspondence or lack of correspondence with the Equipped with Furthermore, the display control unit (22) ｃ ) and receiving the command signal (S ｃ ) and the intermediate video frame (VF ｉ ) based on the input safety important data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) corresponds to the monitoring safety critical data (D ｍ ) is a visual representation of the verified video frame (VF ｃ ) , configured to generate a confirmed video frame signal indicative of Display system.
2. Further, a display control unit (22) is coupled to receive the confirmed video frame signal and inputs the safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) corresponds to the confirmed video frame (VF ｃ ) comprising a display panel (24) configured to display The display system of claim 1 .
3. Furthermore, the display control unit (22) ｉ ) and the Monitoring Safety Important Data (D ｍ ) do not correspond, generating a warning signal configured to be provided to the user to notify the user of the abnormal operating condition of the graphics generator (20).
3. A display system according to claim 1 or 2.
4. Further, the display panel (24) receives the warning signal and displays the input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) and the corresponding content do not correspond, the display being configured to display the content to the user, A display system according to claim 2 or 3.
5. Furthermore, the graphics generator (20) generates the intermediate video frame (VF ｉ ) configured to generate control data indicative of the position of the encoding marker (43, 62) in the first frame region (40) of the Further, the display control unit (22) receives the control data and displays the monitoring safety critical data (D) based on the encoding markers (43, 62). ｍ configured to identify the encoding marker (43, 62) in the first frame region (40) based on the control data to determine A display system according to any one of claims 1 to 4.
6. The graphics generator (20) may transmit the control data separately from the intermediate video frame signal or in combination with the intermediate video frame (VF ｉ ) as encoded in a second frame region (42) of the intermediate video frame (VF ｉ ) adjacent to the first frame region (40); The display system of claim 5.
7. The transportation equipment is an aircraft, and the input safety critical data (D ｉ ) includes input Primary Flight Display (PFD) data, and said monitoring safety critical data (D ｍ ) includes monitoring PFD data, A display system according to any one of claims 1 to 6.
8. The monitoring PFD data is the intermediate video frame (VF ｉ ) each including one or more pieces of data visually represented via one or more respective numbers (44); Each first encoding marker (43) of the encoding markers (43, 62) is uniquely associated with each number (44), and each first encoding marker (43) is uniquely associated with each of the intermediate video frames (VF ｉ ) a particular predetermined pixel pattern in said first frame region (40); 8. The display system of claim 7.
9. Each first encoding marker (43) is a respective bar code or QR code; The display system of claim 8.
10. The monitoring PFD data is the intermediate video frame (VF ｉ ), including monitoring pitch and / or roll angles visually represented via an attitude indicator (50) including an attitude bar (52) and a pitch reference bar (54); a second encoding marker (62) of the encoding markers (43, 62) is uniquely associated with the attitude indicator (50) and includes a second attitude encoding marker (63') and a second pitch encoding marker (63''), each second attitude encoding marker (63') being included within one respective attitude bar (52) and each second pitch encoding marker (63'') being included within one respective pitch reference bar (54), such that a relative position of the second attitude encoding marker (63') and the second pitch encoding marker (63'') is indicative of the monitored pitch angle and / or the monitored roll angle; A display system according to any one of claims 7 to 9.
11. Each second encoding marker (62) corresponds to one of the intermediate video frames (VF) having a predetermined color encoding. ｉ ) one or more pixels in said first frame region (40); The display system of claim 10.
12. The intermediate video frame (VF ｉ ) does not contain said encoding marker (43, 62) which is visible to the naked eye by said user, A display system according to any one of claims 1 to 11.
13. A transport device (150) comprising an indication system (10) according to any one of claims 1 to 12.
14. the vehicle is an aircraft and the user is a pilot of the aircraft; The transport device of claim 13.
15. The input safety critical data (D ｉ ) , 15. A transport device according to claim 13 or 14.
16. The display control unit (22) further includes a display control unit (24) for displaying the input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) do not correspond, generating a warning signal configured to be provided to the user to notify the user of the abnormal operating condition of the graphics generator (20); The vehicle (150) is further coupled to the display system (10) and configured to receive the warning signal and input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ and a warning means configured to provide a warning to said user if the A transport device according to any one of claims 13 to 15.
17. and providing a user of the transport device with a verified video frame (VF) indicative of one or more operational parameters of the transport device operable by the user. ｃ ), comprising: The method (100) is performed by a display system (10) including a graphics generator (20), a display control unit (22), and a display monitoring unit (26) operatively coupled to each other; The method (100) comprises: - input safety critical data (D) indicative of said one or more operational parameters of said transport equipment by said graphics generator (20); ｉ ) (110); - said input safety critical data (D ｉ ) based on the intermediate video frame (VF ｉ ) , said generating (112) an intermediate video frame signal indicative of The intermediate video frame (VF ｉ ) is the monitoring safety critical data (D ｍ ), the first frame area (40) being a visual representation of the monitoring safety critical data (D ｍ ) an encoding marker (43, 62) uniquely associated with The Monitoring Safety Critical Data (D ｍ ) and the input safety important data (D ｉ ) correspond to each other under normal operation of the graphics generator (20) and do not correspond to each other under abnormal operation of the graphics generator (20), step (112); - by said display control unit (22) displaying said monitoring safety critical data (D) based on said encoding markers (43, 62); ｍ ) (114); - said display monitoring unit (26) monitors said input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) with each other (116); - by said display monitoring unit (26), based on said comparison, said input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) or a command signal (S ｃ ) ; and - said command signal (S ｃ ) and the intermediate video frame (VF ｉ ) based on the input safety important data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) corresponds to the monitoring safety important data (D ｍ ) is a visual representation of the verified video frame (VF ｃ generating (120) a verified video frame signal indicative of A method comprising:
18. The input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ The step (116) of comparing the input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) and the input safety critical data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) and the input safety important data (D ｉ ) and the Monitoring Safety Important Data (D ｍ ) is less than a threshold difference, ｉ ) and the Monitoring Safety Important Data (D ｍ ) corresponds when the predetermined interrelationship is satisfied, and corresponds when the predetermined interrelationship is not satisfied, 20. The method of claim 17.
19. The display system (10) further includes the display panel (24) coupled to the display control unit (22); Further, the method further comprises: ｉ ) and the Monitoring Safety Important Data (D ｍ ) corresponds to the confirmed video frame (VF ｃ 19. The method of claim 17 or 18, further comprising the step of displaying (122) a message to the user indicating the selected item.