Quality of experience awareness and prediction for passenger connectivity in transportation vehicles

Abstract

A connectivity quality-of-experience (QoE) prediction module within a vehicle. The connectivity QoE prediction module includes: at least one network interface configured to communicate through a cabin network; a display device configured to display video to a passenger or crew of the vehicle; a processor; and a memory storing computer readable program code of the connectivity QoE prediction module. The memory stores computer readable program code is executed by the processor to perform operations. Operations include managing data entry and modification in a Historical Connectivity QoE Database. The operations also include to generate prediction of connectivity QoE along a vehicle travel route, and to provide for display the prediction of connectivity QoE to the display device.

Description

QUALITY OF EXPERIENCE AWARENESS AND PREDICTION FOR PASSENGER CONNECTIVITY IN TRANSPORTATION VEHICLESTECHNICAL FIELD

[0001] The present disclosure relates to monitoring performance of communication connectivity systems for aircraft and other vehicles, such as in-flight entertainment systems providing communication connectivity with ground networks.BACKGROUND

[0002] Modern aircraft include a variety of communications and computer systems to provide in-flight entertainment (IFE) services to passengers. Aircraft typically provide wireless communication connectivity services with ground network nodes, such as content servers. Connectivity is typically provided through a satellite communications (SATCOM) system which relays communications through satellites and gateways connected to ground network nodes. The ground network nodes can include Internet webpage servers, streaming entertainment servers (e.g., NETFLIX, DIRECTV, etc.), gaming servers, etc.

[0003] Connectivity services are monitored using low level and network specific measurements such as bandwidth, latency, jitter, and packet loss. New measurements capabilities, called Quality of Experience (QoE) are emerging with the intent to provide a more accurate view of the passenger experience.

[0004] Airline operators have not been able to accurately assess their passengers' connectivity experience because measurement metrics have not been sufficiently tied to the user perspective. Moreover, measurements have been necessarily performed by connectivity service providers, e.g., satellite service providers, because the network measurements have required high-level integration and visibility to traffic flow through the connectivity system. It is difficult for the connectivity service providers to understand and interpret metrics from network measurements, as to how the metrics relate to passenger QoE. For instance, a ping measurement (periodic transmission of short messages) can be used to calculate the average latency of the connectivity system. The definition of a target criteria required to consider whether the performances are good is not trivial and theanalysis required to make a correlation with the QoE of different applications is difficult and sometimes impossible.

[0005] Even with the emergence of new QoE metrics, the measurements are still under the control of the connectivity service providers. This creates a problem because the connectivity service providers can adapt their measurement methodology and associated scoring or criteria to report more positively biased indicators, when the actual experience of passengers is not as good as reported. Airline operators need an independent way to monitor the connectivity services they provide in order to collect valuable information on passenger satisfaction, to manage how connectivity service providers meet their Service Level Agreements (SLAs), to facilitate identification and remediation of performance issues, and to test robustness of the connectivity system. It is noted that presently, the SLAs are measured and reported by the connectivity service providers.

[0006] Passengers in transport vehicles do not have a good awareness of the connectivity system present performance, other than what they may perceive based on responsiveness to content requests and streaming of content. Because vehicles are moving and can be interconnected to different networks and different satellite beams along the flight (for a commercial aircraft), the performances and associated experience will vary over time.SUMMARY

[0007] A connectivity QoE prediction module within a vehicle. The connectivity QoE prediction module includes: at least one network interface configured to communicate through a cabin network; a display device configured to display video to a passenger or crew of the vehicle; a processor; and a memory storing computer readable program code of the connectivity QoE prediction module. The memory stores computer readable program code is executed by the processor to perform operations. Operations include managing data entry and modification in a Historical Connectivity QoE Database. The operations also include to generate prediction of connectivity QoE along a vehicle travel route, and to provide for display the prediction of connectivity QoE to the display device.

[0008] These and other operations and methods disclosed herein can non- intrusively predict the QoE of passengers who are utilizing communication connectivity services provided to offboard ground based content servers. For apassenger vehicle operator (such as an airline operator), a communication service provider (such as a satellite or cellular service provider), and / or a a manufacturer of vehicle entertainment systems (such as inflight entertainment systems), how well the communication connectivity services may perform in the future from the passengers’ perspectives will improve customer satisfaction with connectivity services. Moreover, passenger electronic devices (e.g., phones, tablets, etc.) which are being used to interface with the cabin networks may be the source of performance limitations or problems, so prediction on QoE may be useful for diagnosing current or future problems with QoE. The QoE metrics can be used to effectively identify the QoE irrespective of whether the services are accessed through passenger electronic devices or through carrier provided display units, and can enable analysis of the source of system operational limitations or problems.

[0009] Other QoE predicting devices and related methods and computer program products according to embodiments of the present disclosure will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such additional QoE predicting devices, methods, and computer program products be included within this description and be protected by the accompanying claims. Moreover, it is intended that all embodiments disclosed herein can be implemented separately or combined in any way and / or combination.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiment(s) of the invention. In the drawings:

[0011] Figure 1 illustrates an example passenger dashboard displayed on a seatback display and a passenger electronic device to indicate present connectivity QoE metrics and predicted connectivity QoE metrics, in accordance with some embodiments;

[0012] Figure 2 illustrates an example crew dashboard displayed on a portable device to indicate present connectivity QoE metrics and predicted connectivity QoE metrics, in accordance with some embodiments;

[0013] Figure 3 illustrates a flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof, in accordance with some embodiments;

[0014] Figure 4 illustrates a flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof for a flight or a set of flights, in accordance with some embodiments;

[0015] Figure 5 illustrates another flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof for a flight or a set of flights, in accordance with some other embodiments;

[0016] Figure 6 illustrates a flowchart of operations which may be performed for collecting connectivity connectivity QoE data, in accordance with some other embodiment;

[0017] Figure 7 illustrates aircraft systems which include QoE agents which collect and report connectivity QoE data for use in generating connectivity QoE metrics, and illustrates ground-based systems which include an operations center that processes the connectivity QoE data and / or QoE metrics generated by an aircraft system component, in accordance with some embodiments;

[0018] Figure 8 illustrates example operations of the passenger electronic devices, the crew terminal, and the operations center of Figure 7 for connectivity QoE data collection, metric generation, and informing passenger and crew in accordance with some embodiments;

[0019] Figure 9 illustrates functional blocks a QoE management node which can perform connectivity QoE measurement and prediction as one or more modules onboard or offboard the aircraft and, in accordance with some embodiments; and

[0020] Figure 10 illustrates a simplified block diagram of components of a predictive connectivity QoE module onboard or offboard the aircraft and / or a connectivity QoE data collection agent, which are configured to operate in accordance with some embodiments.DETAILED DESCRIPTION

[0021] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of aspects of the invention. However, itwill be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present invention.

[0022] Aircraft passengers have not been provided with information indicating what satellite connectivity Quality of Experience (QoE) they can expect in the future during a flight and along a flight route. The lack of future information relating to expected connectivity QoE may be one of the most important contributors to passengers being dissatisfied with aircraft communication connectivity services, such as Internet Services, while traveling. Passengers are not able set their service expectations and are not able to plan when they should use those services based on when the services will be available with a sufficiently high (good) QoE that supports requirements for being able to access different types of content services, e.g., Internet web browsing, audio streaming, video streaming, gaming, immersive virtual reality, etc.

[0023] Without such information, passengers can spend excessive time trying to connect to a communication system portal or trying to purchase an Internet plan, while unknowingly the connectivity service is essentially unavailable or has insufficient connectivity QoE to support the passengers' planned type of requested content. Even when passengers are presently provided access to requested content at an acceptable threshold connectivity QoE, they would be unaware that the connectivity QoE will become insufficient to satisfy the ongoing QoE requirements for that type of requested content and when that degradation in connectivity QoE will occur and how long it be before the connectivity QoE again becomes sufficient.

[0024] Lost service and business opportunities can occur when a passenger doesn't purchase a connectivity service after learning another passenger is experiencing no or poor connectivity QoE, but would have otherwise purchased the service if they had known that in another 10 minutes the connectivity QoE would tremendously improve and remain good for the remaining flight.

[0025] Various embodiments address these and other associated problems with the end goal to enhance passenger satisfaction with connectivity services.

[0026] Although various embodiments are explained herein in the context of generating and using QoE metrics for connectivity systems used with an In-Flight Entertainment (IFE) environment of an aircraft, other embodiments are not limitedthereto and may be used with other types of vehicles, including, without limitation, ships (e.g., cruise ships), buses, and trains.

[0027] Various embodiments of the present disclosure are directed to in-flight and ground connectivity for commercial aircraft (Widebody, narrowbody and regional aircraft) as well as business jet. The principles described in various embodiments can be extended to all transportation application where Connectivity services (Internet Services) are provided to passengers. Examples are cruise ship, train and bus transportation.

[0028] Connectivity services have been monitored using low level and network specific measurements such as bandwidth, latency, jitter and packet loss. New measurements capabilities, called Quality of Experience (QoE) are emerging with the intent to provide a more accurate view of the passenger experience using user centric metrics such as web page loading time or video streaming re-buffering count. Various embodiments of the present disclosure are directed to addressing QoE metric integration and utilization to provide the necessary awareness for passengers, operators and crews.

[0029] QoE indicators are starting to be used by aircraft operators to measure passenger satisfaction. Various embodiments are further directed to providing information indicating both present and future predicted connectivity QoE to passengers and / or to the crew. The present and future predicted connectivity QoE can enable passengers and crew to understand the end-to-end system performances that are presently available and how the QoE will change the future and it possible associated effect on the types of content services that will be accessible through the connectivity system, e.g., different tiers of QoE requirements for Internet webpage browsing, audio streaming, video streaming, gaming, immersive virtual reality, etc.

[0030] Passengers have received notifications indicating that the system is not working properly with generic messages such as “Out of coverage”. Generic notification information, such as “We’ll connect soon”, “Out of coverage”, may be provided but without the system using any forward looking connectivity QoE prediction technique to determine if the generic notification is correct and without any ability to provide information to the passengers regarding what level of QoE can be expected in the future and when.

[0031] Moreover, passengers have been provided no detailed information related to the current QoE associated with the use of different internet services such as web browsing and content streaming. Passengers have not been provided awareness of the future system performances and expected QoE for the rest of the trip.

[0032] Similarly, crew have lacked any future QoE information, thereby having no specific awareness of the Internet service QoE status and expected QoE evolution (e.g., prediction of how QoE will change over time). Crew were therefore not able to make an assessment of the situation and share their assessments with the passengers.

[0033] Also, it can be noted that there has been no way for an operator to calculate expected service performances and predicted (anticipated) QoE for passengers serviced by a flight or a fleet of aircraft. Predicted connectivity QoE can be used to adjust ticket or Internet plan (or other connectivity plan) pricing offered to users / passengers and to enable users / passengers to compare flights (e.g. based on the route or day I time of the flight and associated predicted connectivity QoE) as well as Operators (e.g. Airlines or communication connectivity Service Providers and their historical connectivity QoE and predicted connectivity QoE for a flight route).

[0034] Various embodiments of the present disclosure are directed to operations that provide capabilities to predict connectivity QoE metric performances during an ongoing trip or for upcoming travels, and include operations to report and share indications of the predicted connectivity QoE with the passengers and crew.

[0035] One objective is to offer new services to passengers, crew and operators (e.g., transportation operator and communication connectivity service providers) taking advantage of the ability to report the system availability and QoE that is predicted (expected) to be delivered at the time of the report and for the remainder of a trip, and for upcoming trips along the same or other routes, and for a fleet of aircraft.

[0036] This operational capability will allow the passengers, crew and operators to have a better awareness of the current and future performances of the connectivity system.

[0037] Some embodiments are directed to one or more of the following high level operations associated with providing QoE awareness and prediction for passengers and crew:• Operations to provide real-time QoE indicators during a flight to the crew (ICMT or portable device) or to the passengers (seatback display or PED), and to provide indications of forward-looking QoE predictions;• Operations to generate QoE scoring for each metric (e.g. web page loading time), calculate a combined scoring for each service (e.g. web browsing), and calculate a combination of all scoring into one single indicator to report the expected QoE;• Operations providing Network performance and QoE indicators as icons and / or other visual indications that can be displayed on a seatback display, PED, crew terminal, ground operations center, etc.;• Operations providing dedicated dashboard or pages with color coded or other visual indications of Network performance scoring and QoE scoring for the various types of service (e.g. web browsing and streaming). The indications can be associated with system performance metrics such as availability;• Operations providing a more detailed dashboard to the crew with additional information allowing to conduct an assessment of the situation and share it with the passengers;• Operations providing prediction of the system performance during a flight or for a fleet with, e.g., the ability to detect future degraded level of connectivity QoE and / or loss (gap) of connectivity, and / or predict when the system level of connectivity QoE will improve and to what level;• Operations providing prediction of the QoE during a flight or for a fleet to provide operators (e.g., airline operation centers and crew) and passengers awareness of the expected QoE performances for the rest of the flight and / or for upcoming flights;• Operations providing ability for the operators or for the crew to generate notifications to the passengers (e.g., via displays, PED or orally) to provide status update and prediction of the upcoming changes in connectivity QoE (e.g., sufficient service level for video streaming, insufficient service level for video streaming but sufficient for web browsing or audio streaming, loss of connectivity, time to predicted change in connectivity QoE to defined service levels, etc.); and / or• Operations providing connectivity QoE benchmark information to the users (e.g., operators, passengers, etc.), comparing the current and / or future performances with anonymized data from other operators and service providers. For instance, a reference scoring can be provided taking into account all measured performances for the same industry (commercial aviation, business jets, rail transportation, ...) for the same region (e.g. North America, Europe) or for similar journey (e.g. long-haul flights, short-haul flights).

[0038] The predictive connectivity QoE information can be provided directly to the passengers (users of the connectivity systems), to the Operator for airlines operations and sales, and / or to the crew to help manage passenger expectations and organization.

[0039] For passengers, various of these operations can be either integrated in the vehicle’s display (e.g. aircraft IFE seatback or other video display) or accessible from the user PED (Portable Electronic Device). One objective is to provide awareness of the present and predicted future connectivity system performances and QoE. The passengers can be provided information related to the current system performances and / or access to the predicted connectivity availability status and predicted QoE for the rest of the journey (e.g. flight).

[0040] One or more of the following types of information may be provided to the users / passengers in accordance with some embodiments:• A connectivity QoE indicator in the form of icons or other visual indications that indicate the overall current service performance;• A set of targeted indicators to provide a more detailed view of the current connectivity QoE performance levels and network performance level (e.g. availability). The indicators may be associated with specific connectivity requirements of the content available to or requested by passengers. For instance, before purchasing a plan the passenger can check if the connectivity system is available, and can also verify the QoE is sufficient for video streaming before starting to watch a movie or look at WebQoE indicators (QoE for web browsing) before surfing the Internet; and / or• A representation of predicted connectivity QoE for the rest of the flight (and optionally an indication of the experienced past connectivity QoE) providing prediction of the availability and expected level of connectivity QoE.For instance, a timeline can be used to indicate the time of major events (e.g. gap of connectivity, sufficient QoE level for web browsing, sufficient QoE level for audio streaming, sufficient QoE level for video streaming, sufficient QoE level for immersive virtual reality, etc. ) and / or a map with color-coded route showing such predicted levels of connectivity QoE.

[0041] The user can use such information to plan use of aircraft communication connectivity during a trip. For instance, a passenger can check that QoE will be sufficient for the next 2 hours before starting a movie, so that the passenger can avoid being interrupted or having a surprised (unexpected) degradation of experience during this time.

[0042] The predicted information can also be provided to the passenger as indicators integrated in the WiFi portal access or in the airline's application for instance (e.g. duration of the current disconnected state, time before a predicted degradation of the connectivity begins and / or ends, a prediction of the level of connectivity QoE that will be experienced during the rest of the flight, etc.).

[0043] Notifications can be provided to passengers when they are starting to use a service in order to provide awareness of the expected QoE for this specific service. For instance, when a passenger starts to stream a movie, the system can recognize, e.g., using deep packet Inspection capabilities, that the passenger is requesting a movie stream, and then a notification can be provided to the passenger indicating that the streaming service will be interrupted after 30 minutes (from a present time) and that the interruption will last for 10 minutes before returning to a sufficient QoE level for movie streaming. The passenger may also be informed that during the duration of interruption in movie streaming, that the connectivity QOE is predicted to be sufficient to support a lower bandwidth service, such as web browsing and audio streaming.

[0044] Passengers may also have access to a wide range of statistics showing performance references (e.g. for all commercial aircraft operating in North America), general QoE expectations or providing benchmark between operators or between Internet Service Providers.

[0045] Figure 1 illustrates an example passenger dashboard displayed on a seatback display and / or a PED to indicate present connectivity QoE metrics and predicted connectivity QoE metrics, in accordance with some embodiments. The displayed information includes a geographical map showing present location of theaircraft along a flight route, and further visually indicates segments along the flight route having different levels of predicted connectivity QoE. The predictions may alternatively or additionally be displayed relative to a flight timeline, chronologically arranged listing, and / or other textual or graphical display.

[0046] In the example context of Figure 1 , different levels of predicted connectivity QoE (e.g., bandwidth ranges) may be visually indicated (coded) by different colors, different levels of grayscale, different shading, etc. A horizontally arranged legend indicates the relative predicted connectivity QoE (e.g., "Excellent", "Good", "Usable", "Poor", and "Disconnected"), which may additionally or alternatively indicate what types of connectivity services are predicted to be available for each level (e.g., predicted connectivity for high bandwidth content such as high- resolution (4K) video, predicted connectivity for lower bandwidth content such as lower-resolution (720) video or audio but with possible intermittent interruption of streaming, predicted connectivity for low bandwidth Internet browsing and audio streaming but insufficient for video streaming, predicted connectivity for very low bandwidth Internet browsing with relatively frequent interruption, and no connectivity (disconnected)).

[0047] The present aircraft location is illustrated along the flight route, in which an earlier segment is indicated with a historical QoE data indication, a next future flight segment is indicated to have a good predicted connectivity QoE, and a subsequent flight segment is indicated to have a poor predicted connectivity QoE.

[0048] As will be explained in further detail below, the onboard system can use the predicted connectivity QoE to control how much content is pre-fetched for caching onboard the aircraft before reaching flight segments having relatively low predicted connectivity QoE.

[0049] In the example illustration, a segment along which the connectivity QoE is predicted to be sufficient for Internet browsing content but insufficient for streaming high-bandwidth requirement content (e.g., movies, television, videos, etc.) is indicated by a darker grayscale, different color, or different texture than another segment along which the connectivity QoE is predicted to be sufficient for accessing all types of content (e.g., movies, television, browsing Internet, playing games, etc.). Other segments may be indicated where connectivity is predicted to be unavailable. The illustration is provided as a non-limiting example and the meanings of therelative grayscale, color, or texture of different segments can be interchanged, e.g., darker grayscale indicating higher bandwidth predicted connectivity QoE.

[0050] The Operator (e.g. Airlines) can provide a set of indicators and predicted QoE metrics to the crew to help them manage passenger understanding and expectations during a flight. The crew can access such data using a portable device (smartphone, PED, airline provided tablet) or an existing control I monitoring display such as the Crew Management Terminal.

[0051] The crew may have access to a more detailed set of information than the passenger in order to communicate with the users and provide specific announcements as desired. The crew may initially access the same data as the passenger (indicators, availability and QoE scoring and predicted performances). Additional detailed information can be provided such as latency and bandwidth metrics, as well as other QoE metrics such portal QoE (time to load the portal, portal load success rate, ...). For each service QoE indicator, detailed scoring for individual metrics can be provided as well (e.g. web page loading time, rebuffering ratio). Defined warnings can be provided to the crew and / or to an automated announcement service of the IFE system to make responsive announcements to passengers about predicted (upcoming) connectivity QoE changes and other events (e.g. unavailability of the service, recovery of the system or bad QoE).

[0052] Figure 2 illustrates an example crew dashboard displayed on a portable device to indicate present connectivity QoE metrics and predicted connectivity QoE metrics, in accordance with some embodiments. The illustrated dashboard includes network performance measurements and connectivity QoE scoring reports based on the measurements, and includes a graphical representation (e.g., map or timeline) and / or textual representation of the connectivity QoE historical data and predicted connectivity QoE. In the particular illustrated example, an indicated historical connectivity QoE is followed by an indication of current flight time, which is followed by a prediction good connectivity QoE, next followed by a predicted decrease to poor connectivity QoE, and next followed by a disconnected level QoE. The historical and predicted connectivity QoE may be provided as detailed information determined for individual set locations, for defined groups of seat locations, etc., and / or provided as notifications (e.g., system will become disconnected in 5 minutes).

[0053] Depending on the level of information detail provided to the crew, specific training may be defined to ensure the crew can understand, interpret and explain the historical and predicted connectivity QoE.

[0054] The Operator can also exploit the predicted QoE in a number of ways as described below.

[0055] In some embodiments, the Operator adjusts its offering taking into account the QoE performance on the upcoming flights. For instance, predictive QoE can be an important input for the adjustment of the Internet plan pricing. It is a kind of dynamic pricing which can be adjusted depending on the expected overall QoE for a flight or even during a journey to take into account the remaining availability and performances. Furthermore, the Internet plan type and configuration can be adjusted. Dynamic connectivity tier pricing, e.g., messaging, web browsing, streaming, etc. can be used to adjust pricing at time to ticketing, at beginning of flight and / or dynamically adjusted during flight based on predicted QoE. For example, when the predicted connectivity QoE will be degraded during a substantial remaining part of a flight, the connectivity service may be then be offered to passengers at a lower tier pricing with an indication of the predicted QoE to the passengers to enable their informed decisions.

[0056] The Operator during the ticketing process for a customer can have access to and provide information indicating the expected connectivity QoE data for each flight, and this can be one of the attributes used by a customer to select a specific flight. For example, a customer during ticketing can view available flight routes to a destination and the expected (predicted) connectivity QoE for each route. The customer can thereby select a route based on it being predicted to provide a certain level of connectivity QoE for the entire flight, for certain time segment(s) of the flight, etc.

[0057] The Connectivity Service Provider can also use the predicted performance capability to plan maintenance when components of the systems need to be replaced or to modify the system configuration to improve the user experience when a gap of connectivity or lower performances are expected. The operator can adapt operations based on any one or more of the following attributes (including, not limited to): o Caching or buffering, (Could be based on prioritization, seat location, e.g., business / first class. See last bullet point);o Dynamic change of the beam configuration (e.g. beam boundaries, beam capacity), satellite configuration (e.g. satellite power level) or modification of the associated network (e.g. moving from a GEO (geostationary) satellite to a LEO (low Earth orbit) satellite to minimize the latency); o Adjustment of the traffic shaping policy, allowing to control more the less important traffic (e.g. blocking network storage), giving higher priority to a type of application (e.g. blocking video streaming to ensure good Web browsing QoE) or providing more capacity to the users. Can include limiting what applications are made available for execution. o Management of priorities at traffic level, application level, service level, aircraft level or fleet level (e.g. priorities between QoE sensitive services and non-QoE sensitive services).

[0058] Operations can predict duration of low-QoE (where QoE doesn't satisfy a service requirement rule) and pre-fetch from ground content servers for on-board buffering content currently being consumed by passengers (e.g., streaming movies, TV, game files, eBooks, etc.). For example, the system may send a request message to Netflix server to send X minutes of a defined video starting from a defined location therein, where the X minutes can be determined based on the expected duration of the insufficient connectivity QoE for the service. These operations may be part of a service, e.g., Flight Edge for over-the-top service (OTT), e.g., Netflix, to pre-fetch.

[0059] Leading-up to the aircraft entering a region where predicted QoE doesn't satisfy a rule, operations can pre-fetch for buffering information from URLs that are predicted to be likely accessed by passenger(s) while the aircraft is located in another region having sufficient QoE enabling such prefetching. For example, operations can pre-fetch website information obtained from user-selectable links provided on a passenger's currently viewed webpage. In a more detailed example, operations prefetch article links on webpage logically following a current article viewed by a passenger.

[0060] Operations can control what applications are made available to passengers (e.g., offered through seat video display units (SVDUs) and / or passenger electronic devices (PEDs)) leading-up to or exiting regions where predicted QoE doesn't satisfy a rule or does satisfy a rule. Thus, operations maylimit what applications are made available for passenger selection through PED / SVDU.

[0061] Operations can adapt bandwidth requirements of applications being executed in preparation for predicted low connectivity QoE, e.g., reduce frame rate of video, resolution of video, bandwidth of audio, etc.

[0062] Leading-up to regions where predicted connectivity QoE doesn't satisfy a rule, operations can transfer from off-board hosted to on-board application hosted application code having higher QoE requirements. For example, upload code for hosting or activate on-board hosted code and / or data to support certain applications that is otherwise hosted off-board. In one example embodiment, gaming environment data related to a players' game state (e.g., location within an immersive gaming environment) can be transferred on-board.

[0063] QoE reporting dashboards and indicators can be accessed from the different displays or devices in the transport vehicle, as well as in the operating center. Those reports will be available to the users in near-real time. Each type of user (e.g. passenger, crew, operator) will have access to different sets of information. The passenger will have a simplified dashboard that provides just a few indicators associated with the status of the connectivity system and the expected QoE. The Crew will have more detailed information in order to understand better the situation, while the Operators will have access to all QoE and performance indicators to be able to troubleshoot the system as required.

[0064] Figure 3 illustrates a flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof, in accordance with some embodiments.

[0065] Referring to Figure 3, operations will have the ability to evaluate the QoE performance and calculate scoring using QoE measurements conducted from an onboard program agent or taking advantage of the passenger traffic for QoE assessments.

[0066] An operations center can continuously monitor the QoE for a fleet of aircraft and control the configuration of what information is shared with passengers and with the crew. Through the operations center, the airlines or the service [rovider can control and orchestrate the connectivity QoE metrics that are generated and shared onboard and offboard the aircraft, e.g., with which users (e.g., crew, passengers, center operators) and equipment.

[0067] The following examples of metrics may be reported, depending on the configuration requested by the Operators (other performance and QoE metrics can be provided as well):• End-to-end Ping latency or Round Trip Delay;• End-to-end Ping loss or packet loss;• On-board Ping latency (measurement between QoE Agent and onboard server);• On-board Ping loss (measurement between QoE Agent and on-board server);• Download speed test or Downlink bandwidth;• Upload speed test or Uplink bandwidth;• Throughput for file transfer;• Portal loading time;• Portal loading successful rate;• DNS query duration;• Webpage loading time;• Webpage First Contentful Paint (FCP);• Webpage Largest Contentful Paint (LCP);• Webpage loading successful rate;• Webpage service quality;• Streaming start time (time to start playback);• Streaming re-buffering (or stalling) duration;• Streaming re-buffering (or stalling) count;• Streaming successful rate (or error rate);• Streaming bitrate;• Streaming data usage;• Video resolution;• Whitelisting (or allow listing); and• Blacklisting (or block listing).

[0068] The user dashboards can support the following QoE indicators, in addition to the detailed metrics provided above:• Overall QoE scoring;• Web browsing QoE scoring;• Video streaming QoE scoring;• Audio streaming QoE scoring;• Streaming QoE scoring;• Portal QoE scoring; and• Network or performance scoring.

[0069] Additional metrics and scoring I indicators can be provided for other applications or services (e.g. tunneling, social media, file transfer, emailing, gaming).

[0070] The QoE measurements or in-stream evaluations may be stored on a networked repository, e.g., Cloud platform, or in an on-board server. An engine is used to first calculate the metrics statistics (e.g. average of Webpage First Contentful Paint (FCP) over a certain period of time) based on the Operator requirements defined as part of the configuration items.

[0071] The configuration items can include, but are not limited to:• A time window for the QoE evaluation;• The level of assessment (e.g. flight, tail, fleet);• The target metrics and indicators for each measurement;• The requirements for the scoring combination algorithm (e.g. higher priority for web browsing than streaming scores); and• The requirements to process the data such as exclusions to be applied.

[0072] A Monitoring Management System function can operate to conduct processing on the data to generate QE metrics, and identify data for exclusion from use in generating QE metrics. Operations may correlate the data coming from different sources (e.g., different components of the communication system, from different flights by the same aircraft, and / or from different aircraft traveling along the same route and / or other routes) to identify any issues with measurements nodes. Once the system has calculated the statistics for each metrics (e.g. average, variance), the scoring can be provided for those target metrics. Operations may classify the calculated metric into a category, which will be associated with a score. For instance, a scoring methodology can be defined to assign FCP average to 5 different classes from -2 to +2 or from 1 to 5 as described below. If the FCP average is 5 seconds during the defined time window, then the score associated will be 3 / 5.

[0073] At this stage, those metrics scores can be reported to the users and operators if required. These metric scores may be relatively low level, so only expert users may be able to interpret the results.

[0074] The next step is to combine the different metrics score into one indicator score which is associated with the same service. For instance, FCP, LCP, Web page loading time, webpage successful rate and DNS can be combined into one score for web browsing, also called Web QoE or Browsing QoE

[0075] Finally, those indicators per service can be combined into one single QoE indicator, which provides the overall evaluation of the user QoE for the defined scope.

[0076] Those combinations can be based on various operational approaches. One approach is to use a weighted average, where each of the metrics or each service indicator is associated with a weight which is specific to the Operator expectation and perspective. Industry wide agreement can be obtained on those weights, but it can be configurable to adapt to the evolution of the user behavior and expectations. Another approach is to combine directly the scoring from the different metrics based on a set of rules which define how values of metrics associated with different levels of importance are to be used when being combined (e.g., metric values of 1 ,2,3 result in an aggregated value of 2 when each of the metrics have same importance, while other metric values 1 ,2,0 result in an aggregated value of 2 based on the the metric having value 2 being more important than the metrics having values 1 and 0).

[0077] With continued reference to Figure 3, the connectivity QoE indicators may be predicted for each flight segment and for an entire flight, and then a global picture of the QoE performance prediction can be generated based on the position of the aircraft. An illustration of a map with flight routes and associated anticipated performances is shown in Figure 2. Each segment of the route having a different QoE score may be correspondingly color coded or otherwise indicated as to the QoE score. More detailed views can be provided with the map indicating where different types of content services will be supported by the connectivity system, e.g. web browsing QoE or video streaming QoE.

[0078] The predicted connectivity QoE metrics which are displayed, e.g.., in Figure 2, can include predicted and actual measurements of latency metrics, communication speed metrics, availability metrics, overall QoE, web browsing metrics, streaming metrics, content loading time metrics, content loading success metrics, stream buffering metrics, etc.

[0079] Some embodiments are directed to multiple ways to determine QoE predictions, which may use relatively less computationally intensive approaches for calculation of the average of QoE scoring in a predefined area to work computationally intensive operations using Machine Learning (ML) algorithms. The selection of which method(s) are used may depend on what QE metrics are available and the stability of the system QoE metrics performances. If there is a need to adjust the scoring regularly, the Machine Learning approach may be preferred.

[0080] The example operations according to Figure 3 can include an operator requesting 300 connectivity QoE information for an identified flight or for a fleet of aircraft. Operations collect 302 connectivity QoE data for targeted flights and such metrics or other QoE indicators within a selected window (time window, time since last report, etc.). Operations may process 304 data and apply exclusions, e.g., by discarding data that does not satisfy defined rule(s), e.g., data indicating corruption by spurious noise, etc. Operations calculate 306 metrics, e.g., statistics, for each metrics (e.g., average FCP). Operations calculate 308 connectivity QOE scoring for each metric (e.g., 3 / 5 score for FCP). A decision 310 may be made whether the data is to be processed at an indicator level or at a metrics level. When processed as metrics level, operations provide 312 scoring at the metrics level for reporting 314 as connectivity QoE indicators and / or metrics to passengers, crew, and / or offboard operators. When processed as indicator level, operations provide 316 indicated scoring for combining 318 with other scoring into service level indicators (e.g., 2 / 5 for web browsing). Operations then combine 320 scoring into a single connectivity QoE indicator (e.g., 3 / 5 QoE score).

[0081] Figure 4 illustrates a flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof for a flight or a set of flights, in accordance with some embodiments.

[0082] Referring to Figure 4, some embodiments are directed to a prediction method based on an average of historical data.

[0083] For this first example operational method, QoE scoring (overall and / or at service level) is collected in a database and a monitoring management system defines the elementary areas which can be pre-defined or adjusted depending on the data variance. In this area all data points will have the same QoE scoring. Based on the historical scoring data in the pre-defined time window, the system calculates the average scoring in each elementary area to create a QoE scoring map.

[0084] In order to calculate the QoE prediction for a flight, the estimated trajectory of the flight is calculated first using historical data and flight route databases (e.g. using service provided by FlightAware Aviation Co.). Each data point of the estimated trajectory is associated with an elementary area and related QoE score.

[0085] If some data points of the aircraft trajectory are not located within an elementary area, the system will use the scoring of the closest elementary area. Also, the Operator will be notified that the coverage of the QoE prediction map is not sufficient. In that case, the objective will be to collect more data (other fleets) or to extend the historical data range (time window). For specific routes that are traveled only during a certain period within the year, the time window to collect data can be larger than for the other more regularly traveled routes.

[0086] Some embodiments are directed to prediction operations based on connectivity system performances.

[0087] A Service Provider that provides both Connectivity and QoE services can build a prediction model based on a detailed assessment of the performance of each elementary areas along the flight routes, taking into account available capacity, spectral efficiency, aircraft Terminal performances, link budget analysis, traffic shaping policy, satellite characteristics, etc.

[0088] The accuracy of the prediction will be highly dependent on the business assumptions in terms of user consumption model, flight routes and schedule and potential congestion in a beam.

[0089] In Figure 4, a QoE prediction map is built and / or modeled 400 based on connectivity QoE metrics obtained by one aircraft or by a fleet of aircraft which have flown a same or similar route or have flown the same defined geographic regions associated with flight routes. Operations collect 402 QoE metrics data for targeted region, flights, and / or indicators within selected time window, which may be based on, for example, configuration items such as a time window for relevant historical data, level of assessment (e.g., flight segment, etc.), target indicators (e.g., service or global QoE, scoring combination requirements, data processing requirement, etc. An operation defines 404 and elementary area where all data points will be associated with the same connectivity QE metric (scoring). An operation uses 406 historical data in the time window to calculate an average scoring in each elementary area of a map, and stores the map in a QoE map and / or model database 408. The data collection process can be repeated 410 periodically and / or when a thresholdamount of new data is obtained, in order to refresh the model and improve its prediction accuracy. The QoE model database 408 may reside on a ground server which is accessible to an aircraft or may at least partially reside on the aircraft for use in obtained predictions of connectivity QoE along a planned flight route.

[0090] Responsive to an operator request for QoE prediction for a specific flight or set of flights, operations can obtain a flight route from a flight route database and estimate 412 aircraft trajectory through geographic regions for a selected route or remaining route. For points along the flight route, operations can predict 414 connectivity QoE metrics (scoring) based on accessing content of the QoE model database 408. For route data points located in an elementary area with no data available, operations may extrapolate 416 data based using one or more closest elementary area(s) having available data. Operations can store 418 in the QoE model database 408 for use in repeating the calculation process 410, the predicted connectivity QoE, e.g., as a predicted QoE score, along with a subsequently measurement of the actual connectivity QoE experienced by the IFE system as each of the various points along the flight route are reached during flight. Operations can display 420 the predicted connectivity QoE metrics (scoring) for the flight data points, such as by displaying visual indications of which segments of a flight route are predicted to have which levels of predicted connectivity QoE, e.g., as shown in Figure 1.

[0091] Figure 5 illustrates another flowchart of operations for determining and reporting connectivity QoE metrics and / or indications thereof for a flight or a set of flights, in accordance with some other embodiments.

[0092] Referring to Figure 5, a third example operational method to predict QoE is based on Machine Learning technique with supervised or unsupervised learning models that will be trained during a certain period of time to build a model of the system performance for the transport operator routes (geographical and timing approach). The main associated features can include one or more of:• Position and attitude of the aircraft;• System performance metrics (latency, throughput, packet errors);• QoE measurements (e.g. webpage loading time, successful webpage loading, video rebuffering);• If available, beam boundaries, beam center, beam capacity and other network characteristics (e.g. beam identification, calculated distance from beam center, calculated nearest beam contour identification); and• If available, traffic shaping policy and associated algorithm for dynamic models.

[0093] The operator can build a user and traffic behavioral models (traffic consumption profiles) to take into account a prediction of the traffic that will be generated within the transportation mean to assess the expected QoE. This model can be dynamic with continuous training using available data (system performance metrics and QoE measurements).

[0094] Figure 5 provides an example algorithm used to predict QoE scoring based on Machine Learning techniques, in accordance with some embodiments. One step for building the model is the definition and creation of the features and a number of data can be used as described above (not limited to). Also the selection of the Machine Learning algorithm can be based on accuracy results and can be changed (adapted) over time based on the performance of the QoE prediction. The collection of QoE prediction data and associated real scoring allows operations to conduct continuous assessments of the accuracy with multiple target Machine Learning algorithms.

[0095] Some embodiments are directed to a prediction method based on a combination of prediction methods.

[0096] A fourth example operational method is based on the combination of various above described prediction methods, which can include use of score average, machine learning techniques and / or detailed assessment of the expected system performances.

[0097] In that case, the QoE prediction is calculated using various of the above methods with a combination of the QoE scoring. This combination can be based on weight assigned to each method depending on the confidence the user has with each method. The weight for each method can be evaluated using accuracy analyses conducted with past data and those weights can vary depending on different characteristics such as the region or the system that is employed.

[0098] In other words, various embodiments are directed to offering new services for passengers, crew and airline Operators by providing information and indicators to give awareness of the QoE related to the Internet services. QoE scoring arecalculated in near real-time and reported to the users and crew during a flight using their PED, seatback display, crew management terminal or any other portable device.

[0099] Another aspect of some embodiments is the ability to predict passenger QoE using scoring average, Machine Learning techniques and / or assessment of the connectivity system performances. One intent of this service is to provide to the passengers, crew and operational teams awareness of the connectivity status and future performances from a user-centric perspective. This will help the crew to communicate with passengers and it will be a very important service for the passengers allowing them to manage their time and reduce considerably frustration when connectivity service QoE changes over time during a trip.

[0100] In addition, predicting QoE can be an important input parameter to dynamic pricing of the Internet plan (or other connectivity plan) and will enable the system to adjust configuration in order to minimize the impact on user satisfaction (e.g. caching or buffering techniques, dynamic change to beam configuration, change of network or type of satellite, adjustment of the traffic shaping policy, management of priorities).

[0101] In the example flowchart of Figure 5, a QoE prediction map is built and / or modeled 500 based on connectivity QoE metrics obtained by one aircraft or by a fleet of aircraft which have flown a same or similar route or have flown the same defined geographic regions associated with flight routes. A machine learning technique is selected 502 based on, for example, configuration items such as a time window for relevant historical data, level of assessment (e.g., flight segment, etc.), target indicators (e.g., service or global QoE, scoring combination requirements, data processing requirement, etc. An operation creates 504 connectivity QoE features and uses historical metric data to train the model. An operation validates 506 the model and stores the model as an object in a QoE model database 508. The training process can be repeated 510 periodically and / or when a threshold amount of new metric data is obtained, in order to refresh the model and improve its prediction accuracy. The QoE model database 508 may reside on a ground server which is accessible to an aircraft or may at least partially reside on the aircraft for use in obtained predictions of connectivity QoE along a planned flight route.

[0102] Responsive to an operator request for QoE prediction for a specific flight or set of flights, operations can obtain a flight route from a flight route database andestimate 512 aircraft trajectory through geographic regions for a selected route or remaining route. For points along the flight route, operations can predict 514 connectivity QoE based on accessing content of the QoE model database 508. Operations can store 516 in the QoE model database 508 for use in repeating the training 510, the predicted connectivity QoE, e.g., as a predicted QoE score, along with a subsequently measurement of the actual connectivity QoE experienced by the IFE system as each of the various points along the flight route are reached during flight. Operations can display 518 the predicted connectivity QoE for the flight data points, such as by displaying visual indications of which segments of a flight route are predicted to have which levels of predicted connectivity QoE, e.g., as shown in Figure 1.

[0103] Figure 5 illustrates a flow diagram of procedures which may be performed by a connectivity QoE prediction module 100 (Fig. 9).

[0104] In some embodiments, a connectivity QoE prediction module 100 within a vehicle. The connectivity QoE prediction module 100 includes: at least one network interface configured to communicate through a cabin network; a display device configured to display video to a passenger or crew of the vehicle; a processor; and a memory storing computer readable program code of the connectivity QoE prediction module. The memory stores computer readable program code is executed by the processor to perform operations. Operations include managing data entry and modification in a Historical Connectivity QoE Database. The operations also include to generate prediction of connectivity QoE along a vehicle travel route, and to provide for display the prediction of connectivity QoE to the display device.

[0105] In some embodiments, the operations further include to, within the aircraft, use the prediction for prefetching content from ground-based content server(s).

[0106] In some embodiments, the operations further include using the prediction to influence timing for handover between satellites.

[0107] In some embodiments, the operations further include using the prediction to influence timing for handover between satellite and direct air to ground.

[0108] In some embodiments, the operations further include to, within the aircraft, use the prediction to provide notification(s) to passengers of the aircraft.

[0109] Figure 6 illustrates a flowchart of operations which may be performed for collecting connectivity QoE data, in accordance with some other embodiment.

[0110] Referring to the example operations of Figure 6, connectivity QoE data collection is performed 600. Operations manage 602 data entry and modification in a historical connectivity QoE database. Operations generate 604 predictions of connectivity QoE based on at least one of: present aircraft location; aircraft route; present weather events in a region of the aircraft; weather events predicted along the aircraft route; aircraft ground speed; satellite ephemeris data; satellite communication cell information, etc. Operations provide 606 the predicted connectivity QoE to components of the aircraft, where the prediction may be performed offboard the aircraft, such as at an operations center, or may be performed on board the aircraft. Further operations within the aircraft may use the predicted connectivity QoE to prefetch 608 content from ground-based content servers. For example, as explained above, operations may increase the amount of content data that is cached responsive to the aircraft approaching a route segment predicted to have less than defined threshold connectivity QoE. One illustrative example includes prefetching into an onboard cache a segment of a movie being watched by a passenger, where the segment can correspond to a predicted duration during which the movie is expected to not be available through connectivity with the ground content server.

[0111] Operations may use the predicted connectivity QoE for multiple available communication networks, e.g., geostationary (GEO) satellite, Medium Earth Orbit (MEO) satellite constellation, Low Earth Orbit (LEO) satellite constellation, and / or air- to-ground terrestrial network (e.g., 5G radio base stations) to select 610 the best communication link. For example, when the aircraft is within entering a region where multiple networks provide coverage, connectivity QoE can be predicted for the associated communication links and used to influence switching from one type of communication link to another.

[0112] Operations may influence 612 a traffic shaping policy based on the predicted connectivity QoE. The traffic shaping policy may be related to component level, application level, and / or user level throttling or other control of onboard and offboard resources made available for communication of such traffic. For example, connectivity bandwidth that is available for use by individual passengers may be throttled (restricted to not more a defined ceiling level) in order to increase the amount of bandwidth that remains to be shared with other passengers, e.g., when the predicted connectivity QoE will satisfy a defined lower bandwidth availability rule.Different passengers may be provided different defined ceiling levels to which their connectivity bandwidth is throttled, based on, e.g., premium tiers of the passengers, airline loyalty status, etc.

[0113] Operations may use the predicted connectivity QoE to adjust 614 communication beam resources (e.g., capacity, bandwidth, etc., used by the satellite modem and / or air-to-ground modem), satellite communication characteristics (e.g., power level transmitted by the satellite modem and / or by the satellite in a cell), and / or air-to-ground communication characteristics (e.g., power level transmitted by the air-to-ground modem and / or by a terrestrial base station toward the aircraft).

[0114] Operations may use the predicted QoE to provide 616 notifications to passengers and / or crew.

[0115] In some embodiments, the operations further include to provide for display the vehicle travel route, wherein the vehicle travel route is displayed with route segments having shading, color, and / or texture selected to indicate the predicted level of connectivity QoE for the respective route segment.

[0116] In some embodiments, the operations further include to predict levels of connective QoE that are likely to be available at locations along the vehicle travel route. The operations also include to predict what types of services would be available based on the predicted levels of connectivity QoE. The operations also include to provide for display indications of the types of services predicted to be available for the locations along the vehicle travel route.

[0117] In some embodiments, the operations further include to provide for display a timeline of travel of the vehicle with indications of predicted level of connectivity QoE for locations along the timeline of travel.

[0118] In some embodiments, the operations further include to, during a ticketing phase of passenger booking of travel, predict a level of connective QoE that is likely to be available along a travel route of the vehicle. Operations also include to, during a ticketing phase of passenger booking of travel, determine pricing for a connectivity service plan to be offered to the passenger for obtaining connection service during travel along the travel route, based on the predicted level of connective QoE that is likely to be available along the travel route. Operations also include to, during a ticketing phase of passenger booking of travel, provide for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during travel.

[0119] In some embodiments, the operations further include to, during a travel phase of the vehicle, predict a level of connective QoE that is likely to be available during remaining travel along the vehicle travel route. Operations also include to, during a travel phase of the vehicle, determine pricing for a connectivity service plan to be offered to a passenger for obtaining connection service during the remaining travel along the vehicle travel route, based on the predicted level of connective QoE that is likely to be available during the remaining travel along the vehicle travel route. Operations also include to, during a travel phase of the vehicle, provide for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during the remaining travel.

[0120] In some embodiments, the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based a historical connectivity QoE database storing data indicating previously measured connectivity QoE at locations along the vehicle travel route.

[0121] In some embodiments, the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based on weather events predicted along the vehicle travel route.

[0122] In some embodiments, the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based on satellite communication cell areas along the vehicle travel route

[0123] Overview of Aircraft Systems and Ground Based Systems:

[0124] Figure 7 illustrates a block diagram of example aircraft systems 200 and ground-based systems 250 which provide passenger centric end-to-end measurement and analysis of connectivity QoE data collection according to some embodiments of the present disclosure. The QoE analysis includes prediction of QoE that will be provided in the future by the communication connectivity system, such as based on future locations of the aircraft along a route and based on QoE metrics that have been learned over time from the same and / or other aircraft which have logged QoE metrics along the same route, geographic region, etc.

[0125] Passengers during flights can access content provided by offboard (ground) network nodes 90 through wireless connectivity services provided by airline operators. For example, passengers can surf Internet websites, stream video and audio from on-demand content providers (e.g., NETFLIX, DIRECTV, etc.), watchbroadcast programming, play games, shop, read electronic books, etc. Passengers can view and / or listen to content from offboard network nodes 90 through seat display units 42 and / or passenger electronic devices (PEDs) 18. Display units 42 can be mounted to seatbacks, tray tables deployable from armrests, and / or other seat structure or cabin structure. The PEDs 18 may correspond to any passenger transportable electronic device having wireless communications capabilities, including, without limitation, tablet computers, laptop computers, palmtop computers, cellular smart phones, game players, etc. PEDs 18 may be passenger owned devices or owned by airlines and provided for temporary use by passengers for the duration of a flight.

[0126] Distribution components 222 communicatively connect the display units 42 and PEDs 18 to other components of the aircraft systems 200 via a cabin network 22 (e.g., Ethernet) and wired communication connections provided by seat electronic boxes 40 (e.g., each mounted to a row of seats) and / or through wireless communication connections provided by wireless access points 30 which can be spaced apart along the aircraft cabin. The wireless access points 30 may be WiFi access points (e.g., IEEE 802.11 ), cellular-based access points (e.g., 3GPP 5G pico cell radio base station), etc. Offboard communication connectivity is provided by a connectivity server 220 which communicates through a SATCOM modem 24 and associated satellite antenna and / or through a cellular modem 26 and associated cellular antenna.

[0127] The aircraft systems 200 can further include an IFE content server 20 which may distribute broadcast programming and / or streaming content received via the connectivity server 200 and / or locally stored content to display units 42 and PEDs 18. The IFE content server 20 may provide flight information, e.g., aircraft location, flight path, moving maps, etc., to passengers based on data from an aircraft data interface 224. Crew may use cabin-crew terminal(s) 232 to interface with the IFE content server 20, the connectivity server, and other components of the aircraft systems 200.

[0128] The ground-based systems 250 includes the network nodes 90 and an operations center (OC) 100 which communicate with the aircraft systems 200 through satellite gateways 34 and / or cellular radio network nodes 39. Although operations center 100 is illustrated as a single entity, in practice it may be a plurality of network nodes which may or may not be contractually associated with orcontrolled by a common entity. The satellite gateways 24 are configured to communicate with the SATCOM modem 24 via a constellation of satellites 28, e.g., geostationary satellites, mid Earth orbit satellites, low Earth orbit satellites, etc. The cellular radio network nodes 39, e.g., 3GPP 5G eNB / gNB base stations, are configured to communicate via direct air-to-ground pathways with cellular modem 26.

[0129] The aircraft systems 200 of Figure 7 may include one or more passengerfacing connectivity QoE data collection terminals 100, each of which executes a QoE agent configured to generate QoE metric(s) which characterize performance of the communication connectivity experienced by passengers during flight.

[0130] Integration of QoE agents into certain components of the aircraft systems 200 provides a change of architecture configured to monitor and report connectivity system performances and passenger QoE. QoE agents contain performance measurement software (e.g., Python scripts) and can be integrated into components equipped with a wireless communication capability and which are installed or available in the cabin of an aircraft or other vehicle. The QoE agents can use the wireless communication capability of the components, in which it has been integrated, to communicate with a ground network node(s) 90 (which may be onpremises servers of a service provide or airline operator, or may be public platforms on Internet such as by commercial streaming operators) in order to conduct measurements. QoE agents can measure performance metrics such as bandwidth, latency, jitter, and packet loss, as well as other QoE metrics for portal access and applications such as web browsing, video and audio streaming, emailing, real-time communication (e.g., video call) and file transfer, as will be described in further detail below. QoE metrics can be generated based on measurements of webpage loading time, video stream re-buffering count, along with numerous other types of measurements discussed below.

[0131] As an example, for commercial aviation, the QoE agents can be integrated in the display units 42 (e.g., seatback displays), the cabin-crew terminal 232 (e.g., crew management terminal (ICMT)), PEDs 18, or any other equipment which may be configured for wireless communications (e.g., WAPs 30).

[0132] An example use case is the integration of a QoE agent in a PED 18, such as a mobile phone, in the form of a QoE agent application that may be loaded from an application store, such as the Google or Apple application stores. The QoE agent application may be associated with an airline, such as an airline application, and / orwith a connectivity service provider, such as Gogo Business Aviation. The QoE agent application may thereby be a component of a larger functionality application, such as an airline application providing reservation capabilities, schedule tracking capabilities, airline loyalty rewards capabilities, etc. The QoE agent application may be downloaded to a PED before flight or during flight from an on-board server 20 via a cabin network, e.g., WAP 30.

[0133] The QoE agents can be integrated in the PEDs 18, the display units 42, and the cabin-crew terminal 232, and / or in another component of the aircraft systems 200 involved in providing communication connectivity to passengers. The aircraft system components which include QoE agents for performing testing communication connectivity from passengers' perspectives are also referred to as QoE testing devices 101 .

[0134] The QoE agents may be controlled by the operations center 100 to initiate QoE testing, control periodicity or other triggering events for QoE testing, control types of testing performed by QoE agents, control types of QoE metrics generated by QoE agents from measurements, control periodicity and / or triggering events for reporting of QoE metrics to the operations center 100, and / or control what QoE metrics and associated information is reported. Operations center 100 may be managed by the airline operator, the communications connectivity service provider, and / or another entity. The number of QoE agents activated for testing and reporting as well as the software scripts (target applications, metrics, measurement methodology, frequency of the measurements ...) can be initially configured and adapted over time by the operations center 100 and / or autonomously by the QoE agents as will be explained in further detail below. Each activated QoE agent can operate to conduct QoE measurements, generate responsive metrics, and report the metrics and other defined information (as explained below) to the operations center 100.

[0135] In some embodiments the connectivity QoE measurements are performed by one or more components of the aircraft, which in some other embodiments the connectivity QoE measurements are performed by a component of the ground network, such as by a QoE measurement module 712, based on QoE measurements performed on traffic communicated to and / or from the aircraft.

[0136] The operations center 100 may be more than one network node, such as one network node that controls QoE agent operation and another network node(e.g., Monitoring Management System) that collects the reported QoE metrics, processes the QoE metrics (e.g., statistic computations, filters or excludes QoE metrics, correlates QoE metrics based on location of QoE agents in cabin, location of aircraft along route, angle of satellite beam, angle of cellular beam, etc.), calculate scoring and generate user information display, e.g., dashboard with QoE reporting information and indicators.

[0137] Figure 8 illustrates example operations of the PEDs 18, display units 42, the operations center 100, and the cabin-crew terminal(s) 232 for QoE testing, control, orchestration, forward-looking prediction, and connectivity management in accordance with some embodiments. Referring to Figure 8, the PEDs 18 and display unit 42 perform QoE testing of communication connectivity system performance to generate defined types of QoE metrics which are then reported to the operations center 100 and may be further reported to the cabin-crew terminal 232. The operations center 100 and the cabin-crew terminal 232 may monitor the reported QoE metrics and responsively control and orchestrate the QoE testing and reporting, and may further manage operation of the connectivity system responsive to the QoE metrics as explained in further detail below.

[0138] By configurating display units 42, PEDs 18, and other QoE testing devices 101 dispersed throughout the cabin to perform QoE testing, the airline operators and / or connectivity service providers are able to create a diverse network of virtual passengers to conduct both simple testing and more complex assessments.

[0139] The network of QoE testing devices 101 serving as virtual passengers can provide new ways of conducting performance assessment which more accurately indicate passenger experience and which can provide a detailed map of QoE throughout the cabin, track dynamic changes in QoE over time, and correlate QoE to the corresponding connectivity device configurations and environment. Previously, connectivity service providers may have conducted the measurements themselves by either exploiting user data (e.g., statistics using Deep Packet Inspection tools) or making measurement with a N+1 Client Agent which acted as one single virtual user.

[0140] With reference to Figures 1 and 2, testing of QoE can include testing performance and operation of components of the aircraft systems 200, satellite network and / or terrestrial cellular network, and components of the ground based systems 250, along data paths from the display units 42 and / or PED 18 through the seat electronics box 40 or wireless access points 30 of the distribution components222, cabin network 22, connectivity server 220, a combination of the SATCOM modem 24 and satellite antenna and satellite 28 network and gateway 34 or a combination of the cellular modem 26 and cellular antenna and cellular radio network node 39, the ground networks 36 (e.g., public and / or private networks) and one or more network nodes 90.

[0141] In some embodiments, programmatic scripts of QoE agents, such as Python scripts, are run in the background of the application providing content streaming and / or web browsing, and statistics are generated using all passenger related QoE metrics to generate and report QoE benchmarks between, e.g., airline operators and connectivity service providers.

[0142] QoE testing may include initiating operations to obtain content from a defined network node 90 (content server) but without displaying a video component and / or audio component of the content on a display of the host device, e.g., display unit 42, PED 18, and / or cabin-crew terminal 232.

[0143] In one embodiment, the QoE agent sends a webpage request through the network interface to the content server 90, receives and displays webpage content from the content server on a display device. The QoE agent generates the QoE metric based on measurements related to sending of the webpage request and when operations to initially display the webpage content on the display device are completed. In a further example, the QoE agent can send a webpage request through the network interface to the content server 90, receive webpage content from the content server 90 on the display device, discard the webpage content without having allowed display of any of the webpage content on the display device, and generate the QoE metric based on measurements related to (e.g., elapsed time between) sending of the webpage request and receipt of all of the webpage content.

[0144] In another embodiment, the QoE agent sends a request for a network node 90 (content server) to stream video from an identified content file, receives the streaming video and perform QoE measurement(s) based the received video, but while preventing display of the video on a display device of the host device. For example, the QoE agent may receive a stream of video packets for measurement operations but discard the video packets without passing to another application of the host device which functions to display video (e.g., video decoding, conditioning, and displaying), e.g., to avoid displaying test video to the passenger. In a further example, the QoE agent can send a video streaming request through the networkinterface to the content server 90, receive a stream of video packets from the content server 90, discard the video packets without having allowed display of any video content of the video packets on the display device, and generate the QoE metric based on measurements related to (e.g., elapsed time between) sending of the video streaming request and receipt of a defined number of the video packets.

[0145] In another embodiment, the QoE agent sends a request for a network node 90 (content server) to stream audio from an identified content file, receive the streaming audio and perform QoE measurement(s) based on the received audio, but while preventing playout by the host device of the audio through a speaker, audio output jack, and / or through a Bluetooth connection to a speaker (e.g., passenger Bluetooth headset), e.g., to avoid displaying test audio to the passenger. For example, the QoE agent may receive a stream of audio packets for measurement operations but discard the audio packets without passing to another application of the host device which functions to playout audio (e.g., audio decoding, conditioning, amplification, etc.).

[0146] For an integrated solution, flight data may be provided automatically (for instance using WiFi portal or an airline or service provider application). For a specific QoE agent enabled application, flight information may be provided through input by the passenger or obtained from the aircraft data bus interface 224 or elsewhere. The QoE agent can operate to associate flight information with the QoE measurements for reporting to the operations center 100.

[0147] In some embodiments, the QoE agents run scripts in the background of the host devices with the runtime configured to reduce or minimize impact on utilization of resources (e.g., processor resource utilization, graphical rendering resource utilization, communication processing resource utilization, and / or communication pathway resource utilization). For example, a decision be made by a controller and / or by the QoE agent itself, whether to activate a QoE agent for QoE measurements based on determining or predicting that certain host device resources are being utilized less than a threshold loading (e.g., host device is idle). In one operational scenario, the controller and / or QoE agent does not activate the QoE agent for QoE measurements on a display unit that is playing a movie or other streaming video content for a passenger, and may instead activate another QoE agent to perform QoE measurements on an adjacent display unit which is idle (e.g., not presently playing a movie or other streaming video content to a passenger).Alternatively, the controller and / or QoE agent may initiate QoE measurements responsive to determining or predicting that processing and networking resources of a particular display unit are presently above a threshold level.

[0148] Controlling when QoE agents perform measurements and what types of measurements are performed, can be adapted so that an airline passenger using a seatback display unit hosting a QoE agent would not notice a difference in performance compared to a conventional IFE seatback display unit which is not capable of or not performing QoE testing. When hosted by an aircraft component (e.g., display unit 42), the QoE Agent can need to be allowed to access certain types of information for measurements used to generate QoE metric(s) and for reporting to the operations center 100. For example, the QoE agent can include or be granted access to timers and may generate or be granted access to observe another application's packets, e.g., by being configured to receive and forward packets from or to the other application (e.g., entertainment application) or be notified of events related to transmission and reception of packets by another application. An example granting of rights can include enabling a QoE agent to monitor packet traffic to a PED application, e.g., NETFLIX application receiving streaming video. The QoE agent may be operationally configured to monitor packet traffic communicated between a PED 18 and a display unit 42, such as during PED Casting and PED mirroring operations. The QoE agent may be provided access to timing data and aircraft information, such as the flight number e.g., via the aircraft data bus interface 224, and may be provided connectivity pathway(s) to offboard resources, such as the Internet (e.g., Internet purchased plan) and network node(s) 90.

[0149] Types of QoE Testing and Generated QoE Metrics:

[0150] In some embodiments, the QoE agent operates to perform one single QoE measurement at a time, where the sequential single measurement process can avoid impacting performance (in a way that may be perceived by the passenger) of the host device (e.g., PED, seat video display unit, etc.) and performance of the communication connectivity components (e.g., cabin network(s), SATCOM / cellular modem, etc.), so that the generated QoE metrics are more accurately representative of the passenger's current QoE with the communication connectivity service. For example, the QoE agent may perform measurement of a first webpage loading time for a single or sequence of webpages and, once completed, then perform videostreaming measurements. For example, the QoE agent may send a webpage request for the first webpage and the measure elapsed time from sending the request to completion of the first webpage loading time (i.e., content of first webpage is fully displayed on display device). In some other embodiments, some types of QoE measurements which have a relatively low impact, such as communication ping measurements of round-trip timing, jitter, etc., can be performed in parallel (i.e., at least partially overlapping in time).

[0151] The QoE agent may be configured to select one or more types of QoE measurements to be performed which are adapted to capture in the QoE metrics indications of dynamically changing performance of the communication connectivity service over one or more phases of a flight. For example, the QoE agent may repetitively perform QoE measurements for one single webpage (e.g., Yahoo.com) to continuously assess the variation of passenger QoE using a selected measurement configuration that performs only webpage loading measurements associated with one or multiple metrics, which may include one or more of: first contentful paint (FCP), largest contentful paint (LOP), first input delay (FID), Webpage loading time, and webpage loading successful rate.

[0152] The QoE metrics generated from the QoE testing can be stored in memory associated with equipment configuration information which identifies current aircraft location, flight route, and SATCOM modem configuration parameters, cellular modem configuration parameters, aircraft to ground communication antenna configuration (e.g., beam steering parameters, modulation coding scheme, allocated frequencies, wireless resource scheduling parameters, etc.), aircraft network configuration parameters, for purposes of being included in reporting to the operations center 100 or other network node for analysis. Reporting the QoE metrics with equipment configuration information can allow correlation of passenger QoE and variation thereof with particular combinations of individual settings of equipment configuration parameters. The QoE agent may operate to select different types of QoE measurements and perform defined types of QoE metric statistics based on the different types of QoE measurements, e.g., to provide a more comprehensive assessment of QoE for different types of passenger applications (e.g., web browsing, streaming video, streaming audio, gaming, etc. through communication connectivity to one or more remote (ground-based) offboard servers).

[0153] The QoE agents can be configured to be the source of requests for content on one or more of the network nodes 90 to perform quantitative measurements of performance of the communications for the bi-directional traffic flow. Alternatively, or additionally, the QoE agents can be configured to “observe” the bi-directional traffic flow between other software residing on PEDs and / or display units 42 to perform quantitative measurements of performance of the communications for the bi-directional traffic flow. For example, a PED-hosted QoE agent may be configured as an application wrapper that is coded to observe or is granted permission to observe (e.g., by the operating system and / or user setting) application programming interface (API) traffic to and from another application hosted by the PED, e.g., observing API traffic to and from a streaming content application (e.g., NETFLIX, etc.). A QoE agent may perform deep packet inspection to identify which on-board or off-board service is associated with the communication (e.g., streaming video from Amazon web server) and / or which operational state of a protocol is being invoked by the packet.

[0154] Various embodiments of the QoE agents may operate to actively analyze packet flows and / or related protocol transitions between end-point devices (e.g., PEDs 18, display units 42, etc.) and service end points (e.g., network node(s) 90). For example, in some systems there are specific known packet exchanges that are expected to be observed by the QoE agents when the service end points and cabin network components are operating normally. These exchanges can be detected and analyzed, to determine whether the associated services are both available and have acceptable performance metrics compared to acceptable availability policies and performance rules. QoE can be measured, such as the timing jitter experienced between packets provided in a sequence for a service, error rate across the packets, packet retransmission rates due to errors or dropped packets, etc. as will explained in further detail below.

[0155] As part of the measurement process, both quality and performance attributes can be analyzed. The throughput and latency of the packet exchanges can be analyzed to detect if there is system degradation that satisfies a defined action rule which triggers an associated defined remediation action. Retransmissions of packets and / or flow control requests can be observed and measured to detect potential communication performance bottlenecks, presence of interfering passengerterminals, poorly operating passenger terminals, and other system communication issues.

[0156] Configuration of the QoE Testing:

[0157] Configuration of the type of QoE test(s) being performed can be dynamically adjusted based on, e.g., what type of passenger application(s) have been executed or are being executed by the PED 18, display unit 42, etc. For example, the QoE agent can be configured to measure QoE by measuring performance metrics based on sending requests to access content of a set of different webpages which are selected to have different content size requirements, host device processing requirements (e.g., video, audio, static graphics, text, etc.), executable code requirements (e.g., CODECs, certain software programs or applications), etc., in order to assess a range of possible passenger experiences.

[0158] The QoE agent may be configured to perform QoE testing by measuring performance metrics based on sending requests to access content of a set of reference webpages. The set of reference webpages may be selected from among a plurality of sets based on the present geographic location of the aircraft, a flight route of the aircraft, time of day and / or day of year, satellite beam elevation angle being indicative of imminent or non-imminent satellite handoff, cellular beam declination angle being indicative of imminent or non-imminent cellular base station handoff, etc.

[0159] Generating QoE metrics based on a combination of such different QoE measurements may provide a more accurate assessment of actual passenger QoE with the connectivity service, but may provide less data resolution to limit ability of the operator team to monitor and assess the variation of experience during the flight.

[0160] Frequency of QoE Testing:

[0161] QoE testing may be performed periodically at a frequency that can be adjusted, e.g., from a few seconds or a few minutes to 30 minutes or 1 hour. As will be explained in further detail below, events may be defined that trigger initiation of QoE testing and / or reporting of generated QoE metrics to the operations center 100 and / or another network node, or which control the frequency and / or timing of repeated QoE testing and / or reporting of generated QoE metrics. Some embodiments are directed to reducing or minimizing the network traffic consumption in cases where a large number of PEDs 18, seat display units 42, etc. within a singleaircraft host QoE agents operating to test and report QoE and / or when a large number of aircraft are being monitored. In some objective scenarios, assessment of connectivity QoE data collection for a fleet of aircraft is performed without detailed correlation across the network, communication modem, and / or antenna parameters and performance metrics.

[0162] Events Triggering QoE Testing and / or QoE Metric Reporting:

[0163] In one embodiment, QoE testing and / or QoE metric reporting are initiated responsive to a handover event indicating handover from one satellite to another satellite or, similarly, responsive to a handover event indicating handover from one cellular base station to another cellular base station. For example, the QoE agent may poll or be informed by an onboard satellite communication (SATCOM) modem (or cellular modem) or a module communicatively connected to the SATCOM modem (or cellular modem) that a satellite handover (or cellular base station handover) is being initiated or will be initiated within a threshold time.

[0164] In another embodiment, QoE testing and / or QoE metric reporting are initiated responsive to determining an elevation angle of a satellite presently providing a ground data link for onboard SATCOM modem and antenna satisfies a defined rule. For example, the QoE agent may poll or be informed by the SATCOM modem or a module communicatively connected to the SATCOM modem that the satellite elevation angle is less than a defined threshold, where the threshold may be defined to be associated with anticipated lower QoE connectivity leading up to and during the handover processes.

[0165] In another embodiment, QoE testing and / or QoE metric reporting are initiated responsive to the SATCOM modem indicating signal quality (e.g., received signal strength, bit error rate, etc.) of the satellite radio link does not satisfy a defined rule.

[0166] In another embodiment, QoE testing and / or QoE metric reporting are initiated responsive to a handover event indicating imminent (e.g., within a threshold time) processes for handover from one ground-based cellular base station (BS) to another ground-based cellular BS. For example, the QoE agent may poll or be informed by an onboard cellular modem, e.g., a 3GPP 5G and / or 4G compliant modem, or a module communicatively connected to the cellular modem that handover is being initiated or will be initiated within a threshold time.

[0167] In another embodiment, QoE testing and / or QoE metric reporting are initiated responsive to determining a declination angle of a beam from an aircraft cellular antenna to a cellular BS presently communicating with the onboard cellular modem satisfies a defined rule. For example, the QoE agent may poll or be informed by an onboard cellular modem or a module communicatively connected to the cellular modem that the BS declination angle is less than a defined threshold, where the threshold may be defined to be associated with anticipated lower QoE connectivity.

[0168] In another embodiment, QoE testing and / or QoE metric reporting are initiated responsive to the cellular modem indicating signal quality (e.g., received signal strength, bit error rate, etc.) of the BS radio link does not satisfy a defined rule.

[0169] In another embodiment, ping measurements are initiated and used for QoE testing and / or the resulting QoE metric reporting is initiated responsive to determining that ping timing has become excessive (e.g., does not satisfy a defined rule). In a further embodiment, the frequency of ping measurements used for QoE testing and / or the resulting QoE metric reporting is adjusted based on the ping timing, e.g., with the frequency increasing responsive to ping time increasing according to a defined rule and with the frequency decreasing responsive to the ping timing decreasing according to the defined rule.

[0170] In another embodiment, the number of QoE agent(s) activated, the types of QoE testing and / or QoE metrics generated therefrom, and / or the locations where QoE agents are hosted (e.g., in which devices) can be controlled based on network loading satisfying defined loading condition(s), e.g., such as to test QoE while passenger traffic is in different defined threshold loading ranges.

[0171] In another embodiment, the number of QoE agent(s) activated, the types of QoE testing and / or QoE metrics generated therefrom, and / or the locations where QoE agents are hosted (e.g., in which devices) can be controlled based on geographic location of the aircraft, such as to provide more robust testing and reporting of QoE when the aircraft is proximately located (e.g., within threshold range) or regions or locations where insufficient QoE for communication connectivity has been historically measured.

[0172] The number of QoE agents involved in QoE testing and / or the types of measurements performed can be controlled based on the types of communications equipment handling in-cabin and / or air-to-ground communications (e.g., SATCOMversus cellular communication, type of satellites involved, types of cellular base stations involved, communication resources (e.g., frequency and / or transmission and / or reception time opportunities) allocated for use.

[0173] The number of QoE agents involved in QoE testing and / or the types of measurements performed can be controlled based on identifying a possible failure condition of on-board communication equipment, identifying a possible erroneous operation of on-board communication equipment, and / or identifying a low performance condition that does not satisfy a rule.

[0174] The location of QoE agents in which type(s) of devices (e.g., PED, SVDU, WiFi access point (AP) based, etc.) being initiated and used for QoE testing can be controlled based on identifying a possible failure condition of on-board communication equipment, identifying a possible erroneous operation of on-board communication equipment, and / or identifying a low performance communications condition that does not satisfy a defined rule.

[0175] The number of QoE agents involved in QoE testing, the types of measurements performed, and / or the location of QoE agents in which type(s) of devices (e.g., PED, SVDU, WiFi AP based, etc.) involved in QoE testing can be controlled based on whether the testing is for an aircraft having tail number that is identified as being among a defined list. For example, QoE testing can be performed with high frequency and fidelity of metric and information reported to the operations center 100 for aircraft having specific tail numbers identified in a list associated with historical performance issues (e.g., on which passengers have reported unsatisfactory (low) QoE with connectivity), where performance issues are anticipated to occur based on a planned flight path, and / or on which a "very important person" (VIP) is scheduled to fly.

[0176] The number of QoE agents involved in QoE testing, the types of measurements performed, and / or the location of QoE agents in which type(s) of devices (e.g., PED, SVDU, WiFi AP based, etc.) involved in QoE testing can be controlled based on information indicating which airline operator is operative the aircraft, which connectivity service provider is providing connectivity services for the aircraft, etc. The QoE testing and reporting can thereby target part of an airline fleet having a defined aircraft platform, connectivity equipment architecture, and / or being served by a particular connectivity service provider.

[0177] The number of QoE agents which are operating at any defined time in an aircraft may be controlled based on instructions from passengers, crew, and / or operators who are offboard the aircraft, e.g., at the operations center 100. Increased numbers of QoE agents and diversity of types of QoE testing and resulting QoE metrics can provide more detailed representation of a wider spectrum of QoE indicators and enable more detailed assessment of the performances experienced by many users operating with contention for shared but finite amount of communication resources.

[0178] Example QoE Tests and Generated Metrics:

[0179] QoE agents according to some embodiments can collect data for many and distributed users throughout the cabin and for many different aircraft in order to provide statistics and reports at the aircraft fleet level. The operations center 100 can process the reported QoE metrics through operations to, for example, filter (exclude certain metrics), statistically or otherwise algorithmically combine metrics, and / or correlate metrics coming from different sources to identify connectivity issues and root causes of the issues.

[0180] QoE agents can be configured to perform any one or more of the following examples of QoE measurements to network node(s) 90 (unless if stated otherwise) and / or to onboard devices responsive to, for example, testing configuration requested by operators (other performance and QoE metrics can be tested as well):1 . End-to-end Ping latency or Round-Trip Delay2. End-to-end Ping loss or packet loss3. On-board Ping latency (measurement between QoE Agent and connectivity server 220 and / or IFE content server 20)4. On-board Ping loss (measurement between QoE Agent and connectivity server 220 and / or IFE content server 20)5. Download speed test or Downlink bandwidth6. Upload speed test or Uplink bandwidth7. Throughput for file transfer8. Portal loading time9. Portal loading successful rate10. DNS query duration11 . Webpage loading time12. Webpage First Contentful Paint (FCP)13. Webpage Largest Contentful Paint (LCP)14. Webpage loading successful rate15. Webpage service quality16. Streaming start time (time to start playback)17. Streaming re-buffering (or stalling) duration18. Streaming re-buffering (or stalling) count19. Streaming successful rate (or error rate)20. Streaming bitrate21. Streaming data usage22. Video resolution23. Whitelisting (or allow listing)24. Blacklisting (or block listing)

[0181] The operating center 100 and the crew terminals 232 may display dashboards which can be used by users (operators) to monitor QoE of the communication connectivity service, and which may support sufficient detailed display of QoE metrics and associated parameters to facilitate troubleshooting of what connectivity system components and / or operations thereof are a root cause of problematic QoE. The dashboards may support any one or more of the following indicators, in addition to the detailed measurements of the metrics provided above:• Overall QoE scoring and / or value or level indication;• Web browsing QoE scoring and / or value or level indication;• Video streaming QoE scoring and / or value or level indication;• Audio streaming QoE scoring and / or value or level indication;• Streaming QoE scoring and / or value or level indication;• Portal QoE scoring and / or value or level indication; and• Network or performance scoring and / or value or level indication.

[0182] Additional measurements and scoring or indicators can be provided for other applications or services (e.g., tunneling, social media, file transfer, emailing, gaming).

[0183] Figure 9 illustrates functional blocks a QoE management node which can perform connectivity QoE measurement and prediction in accordance with some embodiments. The functional blocks can partially reside onboard and offboard theaircraft, and the offboard functional blocks may be hosted on a centralized or distributed (e.g., cloud) computing platform, e.g., networked computing servers.

[0184] Referring to Figure 9, one set of functional blocks for data collection includes a functional block 900 which performs collection of network performance and QoE measurements and performs related calculations. Another functional block 902 performs data and automated exclusion processing. Another functional block 904 performs connectivity QoE scoring calculations pursuant to defined metrics operations and scoring aggregation operations. Another functional block 906 performs data management and reporting of scoring and, possibly, reports the underlying data and metrics.

[0185] Another set of functional blocks for prediction includes functional block 910 for predicting aircraft trajectory, e.g., based on position and known flight route. Another functional block 912 performs connectivity QoE prediction mapping and modeling, which utilizes one or more QoE predictive algorithms 914. Another functional block 916 performs data processing (e.g., to remove outliers, etc.) and reports connectivity QoE predictions.

[0186] Another set of functional blocks for QoE control uses the predicted connectivity QoE to perform traffic orchestration 920 (e.g., caching block 608 in Fig. 6), traffic shaping policy management 922 (e.g., traffic shaping blocks 612 and 614 in Fig. 6), and / or network management 924 (e.g., multi-network management blocks 610 in Fig. 6).

[0187] Figure 10 illustrates a simplified block diagram of hardware components (e.g., circuit) of a predictive connectivity QoE module 700a / 700b onboard or offboard the aircraft and / or a connectivity QoE data collection agent 100, which are configured to operate in accordance with some embodiments. The module 700a / 700b and / or QoE testing device 101 can include at least one processor 1000 (also referred to as a processor), at least one memory 1010 (also referred to as a memory), at least one network interface 1030 (also referred to as a network interface), and at least one display device 1020 (also referred as a display device). Although illustrated a group of hardware blocks, it is to be understood that the processor 1000, memory 1010, display device 1020, and network interface 1030 may be implemented as part of a single network computing platform or may be part of a distributed (e.g., cloud) computing platform, e.g., networked computing servers or workstations.

[0188] The processor 1000 is operationally connected to these various components. The memory 1010 stores instructions that are executed by the processor 1000 to perform operations and methods according to any one or more of the embodiments disclosed herein. The memory 1010 includes an operating system 1019 and the QoE agent 1012 which may be operationally associated with an airline operator application 1014, a communication service provider application 1016, and / or an entertainment application 1018 (e.g., streaming content service such as NETFLIX, DIRECTV, and HULU, an online gaming application, etc.). The QoE agent 1012 may be configured as an application wrapper that is coded to observe or is granted permission to observe (e.g., by the operating system 1019 and / or user setting) application programming interface (API) traffic to and from another application executed by the processor 1000, e.g., observing API traffic to and from the airline operator application 1014, the communication service provider application 1016, and / or the entertainment application 1018. The processor 1000 may include one or more data processing circuits, such as a general purpose and / or special purpose processor (e.g., microprocessor and / or digital signal processor), which may be collocated or distributed across one or more data networks, and may be entirely onboard the aircraft, partially onboard and offboard the aircraft, or entirely offboard the aircraft.

[0189] Further Definitions and Embodiments:

[0190] In the above description of various embodiments of the present disclosure, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0191] When an element is referred to as being "connected", "coupled", "responsive", or variants thereof to another element, it can be directly connected, coupled, or responsive to the other element or intervening elements may be present.In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive", or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, "coupled", "connected", "responsive", or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0192] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "has", "having", or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Furthermore, as used herein, the common abbreviation "e.g.", which derives from the Latin phrase "exempli gratia," may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation "i.e.", which derives from the Latin phrase "id est," may be used to specify a particular item from a more general recitation.

[0193] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and / or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus, transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions / acts specified in the block diagrams and / or flowchart blockor blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0194] These computer program instructions may also be stored in a non- transitory computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks.

[0195] A non-transitory computer-readable medium may include an electronic, magnetic, optical, electromagnetic, or semiconductor data storage system, apparatus, or device. More specific examples of the computer-readable medium would include the following: a portable computer diskette, a random-access memory (RAM) circuit, a read-only memory (ROM) circuit, an erasable programmable readonly memory (EPROM or Flash memory) circuit, etc.

[0196] The computer program instructions may also be loaded onto a computer and / or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer and / or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of the present disclosure may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as "circuitry," "a module" or variants thereof.

[0197] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction ofcommunication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0198] Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, the present specification, including the drawings, shall be construed to constitute a complete written description of various example combinations and subcombinations of embodiments and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

[0199] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present invention. All such variations and modifications are intended to be included herein within the scope of the present invention.

Claims

Claims:1 . A computing platform executing a connectivity quality-of-experience (QoE) prediction module, the computing platform comprising: at least one network interface configured to communicate through a cabin network with a connectivity server and witha display device configured to display video to a passenger or crew of the vehicle; a processor; and a memory storing computer readable program code of the connectivity QoE prediction module executed by the processor to perform operations comprising to: generate prediction of connectivity QoE along a vehicle travel route; and provide for display the prediction of connectivity QoE to the display device.

2. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: within the vehicle, use the prediction of connectivity QoE for prefetching content from at least one ground-based content server.

3. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: use the prediction of connectivity QoE to influence timing for handover between satellites.

4. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: use the prediction of connectivity QoE to influence timing for handover between satellite and direct air to ground.

5. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to:within the vehicle, use the prediction to provide indication to passengers of the vehicle of the predicted connectivity QoE.

6. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: provide for display the vehicle travel route, wherein the vehicle travel route is displayed with route segments having shading, color, and / or texture selected to indicate the predicted level of connectivity QoE for the respective route segment.

7. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: predict levels of connective QoE that are likely to be available at locations along the vehicle travel route; predict what types of services would be available based on the predicted levels of connectivity QoE; and provide for display indications of the types of services predicted to be available for the locations along the vehicle travel route.

8. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: provide for display a timeline of travel of the vehicle with indications of predicted level of connectivity QoE for locations along the timeline of travel.

9. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: during a ticketing phase of passenger booking of travel, predict a level of connective QoE that is likely to be available along a travel route of the vehicle, determine pricing for a connectivity service plan to be offered to the passenger for obtaining connection service during travel along the travel route, based on the predicted level of connective QoE that is likely to be available along the travel route, andprovide for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during travel.

10. The computing platform of Claim 1 , wherein operations of the connectivity QoE prediction module further comprise to: during a travel phase of the vehicle, predict a level of connective QoE that is likely to be available during remaining travel along the vehicle travel route, determine pricing for a connectivity service plan to be offered to a passenger for obtaining connection service during the remaining travel along the vehicle travel route, based on the predicted level of connective QoE that is likely to be available during the remaining travel along the vehicle travel route, and provide for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during the remaining travel.11 . The computing platform of Claim 1 , wherein the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based a historical connectivity QoE database storing data indicating previously measured connectivity QoE at locations along the vehicle travel route.

12. The computing platform of Claim 1 , wherein the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based on weather events predicted along the vehicle travel route.

13. The computing platform of Claim 1 , wherein the operation to generate the prediction of connectivity QoE along the vehicle travel route is performed based on satellite communication cell areas along the vehicle travel route.

14. A method performed by a computing platform executing a connectivity quality-of-experience (QoE) prediction module with at least one network interface configured to communicate through a cabin network with a connectivity server andwith a display device configured to display video to a passenger or crew of the vehicle, the method comprising: generating prediction of connectivity QoE along a vehicle travel route; and providing for display the prediction of connectivity QoE to the display device.

15. The method of Claim 14, further comprising: within the vehicle, using the prediction of connectivity QoE for prefetching content from at least one ground-based content server.

16. The method of Claim 14, further comprising: using the prediction of connectivity QoE to influence timing for handover between satellites.

17. The method of Claim 14, further comprising: using the prediction of connectivity QoE to influence timing for handover between satellite and direct air to ground.

18. The method of Claim 14, further comprising: within the vehicle, using the prediction to provide indication to passengers of the vehicle of the predicted connectivity QoE.

19. The method of Claim 14, further comprising: providing for display the vehicle travel route, wherein the vehicle travel route is displayed with route segments having shading, color, and / or texture selected to indicate the predicted level of connectivity QoE for the respective route segment.

20. The method of Claim 14, further comprising: predicting levels of connective QoE that are likely to be available at locations along the vehicle travel route; predicting what types of services would be available based on the predicted levels of connectivity QoE; and providing for display indications of the types of services predicted to be available for the locations along the vehicle travel route.21 . The method of Claim 14, further comprising: providing for display a timeline of travel of the vehicle with indications of predicted level of connectivity QoE for locations along the timeline of travel.

22. The method of Claim 14, further comprising: during a ticketing phase of passenger booking of travel, predicting a level of connective QoE that is likely to be available along a travel route of the vehicle, determining pricing for a connectivity service plan to be offered to the passenger for obtaining connection service during travel along the travel route, based on the predicted level of connective QoE that is likely to be available along the travel route, and providing for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during travel.

23. The method of Claim 14, further comprising: during a travel phase of the vehicle, predicting a level of connective QoE that is likely to be available during remaining travel along the vehicle travel route, determining pricing for a connectivity service plan to be offered to a passenger for obtaining connection service during the remaining travel along the vehicle travel route, based on the predicted level of connective QoE that is likely to be available during the remaining travel along the vehicle travel route, and providing for display an indication of the pricing as part of an offer to the passenger for obtaining connection service during the remaining travel.

24. The method of Claim 14, wherein generating the prediction of connectivity QoE along the vehicle travel route is performed based a historical connectivity QoE database storing data indicating previously measured connectivity QoE at locations along the vehicle travel route.

25. The method of Claim 14, wherein generating the prediction of connectivity QoE along the vehicle travel route is performed based on weather events predicted along the vehicle travel route.

26. The method of Claim 14, wherein generating the prediction of connectivity QoE along the vehicle travel route is performed based on satellite communication cell areas along the vehicle travel route.

Images