Method and device for optimizing beamforming based on user experience

Abstract

A system may include: an access network including at least two base stations configured to communicate with a wireless communication device via a dynamically controllable or shapeable beam; a resource manager disposed at a location accessible via the access network or at the wireless communication device; and an experience tester configured to determine a qualitative assessment of the access network's capabilities relative to different classes of services or applications of the wireless communication device via a first beam formed or selected to serve the communication device. The resource manager may be configured to initiate a handover based on location criteria and user experience-based criteria. The user experience-based criteria may be associated with the qualitative assessment.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Application No. 62 / 840,705, filed on April 30, 2019, the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] Example embodiments relate generally to communication network performance, and more particularly, to a system for evaluating network performance and adjusting the beamforming capabilities of the network to optimize a user experience. Background Art

[0004] High-speed data communications and the devices that support such communications are ubiquitous in modern society. These devices have enabled many users to maintain a nearly constant connection to the Internet and other communications networks. While these high-speed data connections were made possible by telephone lines, cable modems, or other such devices with a physical wired connection, wireless connectivity has transformed our ability to stay connected without sacrificing mobility. However, while people are familiar with how to maintain a constant connection to a network, they have become equally familiar with the realization that not all networks are created equal in terms of providing a satisfying user experience.

[0005] For example, slow downloads, delays, disconnections, or a variety of other interruptions or obstructions may be experienced in certain buildings (e.g., hotels, conference centers, etc.), on aircraft, or in many other environments (including the user's own home network). In some cases, while network operators claim that their networks have high bandwidth and high-speed capabilities, the user experience does not match or confirm these claims. The disconnect between the claimed performance and the user experience can sometimes be achieved by sharing a given amount of bandwidth among all users of a particular network. At the same time, other networks may have a large amount of bandwidth to offer to customers, but low latency can still undermine the user experience for applications that require any type of real-time interaction or high latency or slow return links (especially when compared to the fast forward link from the Internet to the device) that create queuing effects, which can be further exacerbated or caused by slow TCP acknowledgments, data retransmissions, lost or dropped packets, etc.

[0006] With the advent of the ability to test networks in a more robust manner, it is possible to better understand the capabilities of a particular network to provide a quality user experience. This capability can certainly be helpful for users or application developers / providers to set expectations appropriate to the capabilities of the current network, or to select an alternative network if the opportunity exists. However, if there is no opportunity to switch to an alternative or higher performance network, it may be necessary to make the user experience a guide to setting expectations, improving network performance, or creating and subsequently using a degraded mode of connected operation for the application or service, rather than the more binary connected or offline modes that exist in many applications or Internet-provided services today. Summary of the invention

[0007] In an example embodiment, a method for performing resource management based on user experience is provided. The method may include receiving an indication of a qualitative assessment of access network capabilities associated with different categories of services or applications of a communication device using the access network via a first beam formed or selected to serve the communication device. The method may also include determining whether a candidate resource exists in response to the qualitative assessment being below a predetermined threshold, forming or selecting a second beam associated with the candidate resource in response to determining that the candidate resource exists, and initiating a handover of the communication device from the first beam to the second beam.

[0008] In another example embodiment, a resource manager including processing circuitry is provided. The processing circuitry may be configured to receive an indication of a qualitative assessment of access network capabilities associated with different classes of services or applications of a communication device using the access network via a first beam formed or selected to serve the communication device, determine whether a candidate resource exists in response to the qualitative assessment being below a predetermined threshold, form or select a second beam associated with the candidate resource in response to determining that the candidate resource exists, and initiate a handover of the communication device from the first beam to the second beam.

[0009] In another example embodiment, a system may include: an access network including at least two base stations configured to communicate with a wireless communication device via a dynamically controllable or shapeable beam; a resource manager disposed at a location accessible via the access network or at the wireless communication device; and an experience tester configured to determine a qualitative assessment of the access network's capabilities relative to different classes of services or applications of the wireless communication device via a first beam formed or selected to serve the communication device. The resource manager may be configured to initiate a handover based on both: location-based criteria and user experience-based criteria. The user experience-based criteria may be associated with the qualitative assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Having thus generally described the invention, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0011] Figure 1 A block diagram of a system for providing experience testing according to an example embodiment is shown. DETAILED DESCRIPTION

[0012] Some example embodiments will be described more fully below with reference to the accompanying drawings, in which some but not all example embodiments are shown. In fact, the examples described and illustrated herein should not be interpreted as limitations on the scope, applicability or configuration of the present disclosure. On the contrary, these example embodiments are provided so that the present disclosure will meet applicable legal requirements. The same reference numerals always refer to the same elements. In addition, as used herein, the term "or" will be interpreted as a logical operator, and its result is true as long as one or more of its operands are true. As used herein, the terms "data", "content", "information" and similar terms can be used interchangeably to refer to data that can be sent, received and / or stored according to the example embodiments. Therefore, it should not be considered that any such terms are used to limit the spirit and scope of the example embodiments.

[0013] As used herein, the terms "component", "module", "system", "device", etc. are intended to include computer-related entities, such as but not limited to hardware, firmware, a combination of hardware and software, or software executed on certain hardware. For example, a component can be but not limited to a process, a processor, an object, an executable file, an execution thread, a program and / or a computer running on a processor. By way of example, an application running on a computer device and / or the computer device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate with other systems by means of signals by means of local and / or remote processes, such as according to a signal with one or more data packets, such as data from a component interacting with another component in a local system, a distributed system, and / or interacting with other systems through a network such as the Internet.

[0014] Some example embodiments may provide a network test device or "experience tester" that is configured to conduct a series of network capability tests that provide quantitative measurements of network capabilities for various different types of applications or services. These quantitative measurements can then be converted into useful indications of the quality of user experience (i.e., qualitative assessments) that any user of the network might expect for various types of applications or services. In some cases, such as where alternative resources within a given network are available to serve a user, the qualitative assessments can be used to shift resources to improve the user experience. When certain networks employ beamforming techniques, the beams can be dynamically controlled or selected to maximize the user experience or at least provide a minimum quality level of user experience.

[0015] Thus, for example, if the network employs beam steering and resource contention is detected (or determined to exist) on the current (or original) beam serving the user (and other users), the resource contention may degrade the quality of user experience for everyone sharing the resource (i.e., the original beam) at the time. According to example embodiments, another (i.e., new) beam may be dynamically formed and / or controlled to reduce resource contention issues and improve the user experience for users served by the original beam and the new beam. Similarly, if the network employs multiple selectable but otherwise fixed beams with different respective directions / coverage areas, and resource contention is detected (or determined to exist) on the original beam, example embodiments may be employed to dynamically select another beam to reduce and resolve resource contention issues and improve the user experience for users served by the original beam and the new beam.

[0016] It is noteworthy that in various different embodiments, the new beam can be formed by the same or different sources (i.e., base station or satellite), which will be discussed in more detail below. In addition, the new beam can be used to transfer existing users from the original beam, or the new beam can be used only for subsequent users added after a certain user experience quality threshold is exceeded, or the new beam can be used for connections required for certain characteristics that are not available on the first beam due to limitations associated with load, latency, or bandwidth (forward or return link). Thus, for example, an example of utilizing new resources to serve users may be to create new resources and split existing users (evenly or unevenly (e.g., based on priority)) between the old resources and the new resources (i.e., the original beam and the new beam), or the new beam can only be used to serve subsequent new users added in the same area after a user experience quality threshold has been reached, which quality may depend on the type of application used relative to the connection characteristics required to best support said application.

[0017] As described above, the formation, guidance or selection of new beams (or resources) is based at least in part on a qualitative assessment of the user experience. Therefore, it is noteworthy that the balancing or control of network resources in the example embodiments is not triggered or managed based on any assessment of the number of users on the resources or other purely digital information intended to provide an indication of network performance. As described above, these purely digital indicators or quantitative assessments can often provide obviously misleading results that indicate that network quality has nothing to do with the actual user experience. Therefore, as a starting point, some example embodiments may first be configured to provide a useful way to determine the quality of user experience. In particular, some example embodiments may be configured to employ a set of quantitative test indicators, which, when the corresponding quantitative indicator combinations are present together on the network, can provide a qualitative assessment of network performance indicating the quality of user experience when the quantitative test indicators are combined and correlated with the rating indicating the user experience. Various thresholds or triggers associated with the quality of user experience can then be used to manage the application of network resources in order to maintain certain goals for the quality of user experience on the network. For example, additional service beams may be formed, selected or guided to serve assets in a specific portion of the network where the user experience appears to be affected at any given time. Thus, it should be appreciated that the user experience can be used as a basis for determining when to add new resources (i.e., beams), and new resources can be added as a replacement for current resources, or to augment or share load with current resources. Thus, for example, there may or may not necessarily be a handoff when contention is detected, but rather an enhancement of resource capabilities (i.e., by adding more resources) from the same source or a different source. Thus, the sources that can be used can be beams, each of which may have their own (possibly different) characteristics depending on whether the beam is associated with an ATG or a satellite, and further based on the satellite type (LEO, MEO, GEO, frequency band, bandwidth, etc.), so that any combination of resources can be utilized to improve the user experience.

[0018] Although example embodiments may operate with qualitative assessment information obtained by any means, a specific and non-limiting example of how to obtain qualitative assessment information will be described below. In this regard, some example embodiments may employ an "experience tester" to obtain qualitative assessment information. As the name implies, an experience tester may provide useful indicators of how a user will experience a network for each of a variety of different types of services that may use the network. Therefore, the indicator may be considered as a qualitative assessment of the capabilities of the network in each category or service type. From a practical perspective, the experience tester may be used to provide a user (e.g., to allow a user who plans to use a particular type or category of web services or applications to determine whether the access network in which the user is currently located will provide the user with a good experience with the web service or application), an application (e.g., to enable the application to warn the user of potential bad experiences or to run in a mode suitable for network capabilities to maximize the user experience), or a network (e.g., to manage network resources to improve or otherwise manage the quality of the user experience) a comprehensive indication of the network capabilities that can be used. In the first example, where the user uses a qualitative assessment, the user is therefore able to make a decision as to whether the currently available access network is suitable for any web application or service that the user plans to use. In a second example, an application or service may find the experience tester useful in selecting an operating mode to work with a user based on their ability to access the network. In a third example, network resources may be used to manage network efficiency and user experience by dynamically creating or removing individual network resources (e.g., beams) based on the current user experience level at various points within the network. In addition, by collecting many experience tests over a period of time (perhaps centralized or done in the cloud), statistically significant comparisons can be enabled for specific locations in the network to more objectively compare connectivity, whether the network is wired, wireless, or a combination, even without the tests themselves, but simply by knowing where the user is in the network at a given time.

[0019] Figure 1 An example arrangement of a system is shown in which any one or more of a plurality of different types of networks can be tested by the network test device or experience tester 100 of an example embodiment. Figure 1 As shown, the experience tester 100 can operate in conjunction with a variety of different types of networks, including both wired and wireless networks. In this regard, Figure 1 Examples of wireless networks shown in include a terrestrial network 110 (e.g., 4G, 5G, LTE, or other such networks) and an air-to-ground (ATG) network 120. However, it should be understood that other wireless communication networks, including satellite communication networks, may also be tested according to example embodiments.

[0020] like Figure 1As shown, each of the wireless networks may include a wireless access point (AP) including an antenna configured for wireless communication. Thus, for example, the ground network 110 may include a first ground AP 112 and a second ground AP 114, each of which may be a base station in a plurality of geographically distributed base stations, which are combined to define a coverage area of ​​the ground network 110. The first ground AP 112 and the second ground AP 114 may each communicate with the ground network 110 via a gateway (GTW) device 116. The ground network 110 may also communicate with a wide area network such as the Internet 130, a virtual private network (VPN), or other communication network. In some embodiments, the ground network 110 may include a packet switching core or other telecommunication network coupled in other ways.

[0021] Similarly, the ATG network 120 may include a first ATG AP 122 and a second ATG AP 124, each of which may be a base station in a plurality of geographically distributed base stations that are combined to define a coverage area of ​​the ATG network 120. The first ATG AP 122 and the second ATG AP 124 may each communicate with the ground network 120 via a gateway (GTW) device 126. The ATG network 120 may also communicate with a wide area network such as the Internet 130, a virtual private network (VPN), or other communication network. In some embodiments, the ATG network 120 may also include or otherwise be coupled to a packet switching core or other telecommunication network.

[0022] Each access point of the ground network 110 and the ATG network 120 may have a wired (or wireless) backhaul connection to its respective wireless network to allow wireless communication devices (e.g., user equipment (UE) 150 and UE 152) connected to the corresponding access point to access it. In addition, access points can be provided via cellular towers or other tower structures (as in the illustrated example), rooftops or other structures with wireless communication infrastructure (e.g., building facades, church spires, billboards, etc.), moving vehicles and ships, and / or the like. In addition, in existing wireless networks, it should be understood that some cells defined by access points may overlap or completely surround each other, and / or there may be coverage gaps between some cells. To the extent that the example embodiments are practiced in conjunction with a satellite network, the AP may be a single satellite.

[0023] Fixed line network 140 is also Figure 11 is shown as an example of a wired network and can be tested via example embodiments. The fixed line network can be an example of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or any other network that can be operably coupled to the Internet 130 by electrical, optical, or any other wire or cable coupling means.

[0024] For UE 150 located on aircraft 118, UE 150 can be operably coupled to the AP of ATG network 120 directly or via an access point located on aircraft 118. At the same time, UE 152 can be directly connected to the AP of ground network 110. Finally, UE 154 can be operably coupled to fixed line network 140 directly via a wired connection (e.g., Ethernet and / or the like) or indirectly via a local AP 142 that can use a short-range wireless communication protocol (e.g., Bluetooth or WiFi). It is worth noting that UE 150, 152 and 154 can each be an example of a separate UE used by a corresponding separate user and device (e.g., a mobile phone, a laptop, a tablet computer, a desktop computer, etc.). Therefore, each UE shown and there may be other UEs that can access the experience tester 100 simultaneously or in series.

[0025] like Figure 1 As shown, the experience tester 100 may be located at a content server 160, which is accessible via the Internet 130 from any access network (e.g., terrestrial network 110, ATG network 120, fixed line network 140, satellite network, etc.). In this regard, any number of other content servers 162 may also be accessible via the Internet 130, and UEs 150, 152, and 154 may access services, applications, and / or the like located at each of the content servers 162 via the Internet 130 using their respective access networks. In addition, in some cases, the services or applications available via the Internet 130 at each of the other content servers 162 may include one or more instances of a test resource 164. Each instance of a test resource 164 may be a unique network testing tool configured to test speed, bandwidth, or various other aspects of network performance. The test resource 164 may also or alternatively be configured to parse various information about the access network currently used by a given one of the UEs 150, 152, and 154 to access the Internet 130.

[0026] Therefore, based on the above structure, it should be understood that the user of any UE (e.g., UE 150) can access the experience tester 100 using its corresponding access network (e.g., ATG network 120) via the Internet 130. As will be discussed in more detail below, the experience tester 100 can be configured to call one or more instances of the test resource 164, and may also use local resources to perform information collection and network test information to provide the UE 150 with a qualitative network evaluation of the ATG network 120. However, it can be understood from the above description that a qualitative network evaluation can also or optionally be generated for each and other access networks, and the user seeks assistance from the experience tester 100 via the other access network.

[0027] A functional block diagram of the experience tester 100 of an example embodiment is shown. In this regard, the experience tester 100 may include a processing circuit 210 configured to perform tasks, operations and / or functions of the experience tester 100 as described herein. Therefore, according to an example embodiment of the present invention, the processing circuit 210 may be configured to perform data processing, control function execution and / or other processing and management services. In some embodiments, the processing circuit 210 may be implemented as a chip or a chipset. In other words, the processing circuit 210 may include one or more physical packages (e.g., chips) that include materials, components and / or wires on a structural assembly (e.g., a substrate). The structural assembly may provide physical strength, dimensional conservation and / or electrical interaction limitations for the component circuits included thereon. Therefore, the processing circuit 210 may be configured in some cases to implement an embodiment of the present invention on a single chip or as a single "system on a chip". Therefore, in some cases, a chip or a chipset may constitute a device that performs one or more operations for providing the functions described herein.

[0028] In an example embodiment, the processing circuit 210 may include one or more instances of a processor 212 and a memory 214 that may communicate with or otherwise control the device interface 220. Thus, the processing circuit 210 may be implemented as a circuit chip (e.g., an integrated circuit chip) configured (e.g., using hardware, software, or a combination of hardware and software) to perform the operations described herein.

[0029] The experience tester 100 does not show a user interface. However, the UE 230 (which is Figure 1 In this regard, the test module 240 of the experience tester 100 can interface with the browser of the UE 230 to generate a console, web page and / or interface elements on the screen of the UE 230 to serve as a user interface.

[0030] The device interface 220 may include one or more interface mechanisms for enabling communication with other devices internal and external to the experience tester 100. Thus, for example, internal modules or components and external modules, entities, websites, devices, and / or the like may communicate via the device interface 220. In some cases, the device interface 220 may be any means, such as a device or circuit included in hardware, or a combination of hardware and software configured to receive and / or send data to a module, entity, component, network, website, application, etc. that communicates with the processing circuit 210.

[0031] The processor 212 can be implemented in a variety of different ways. For example, the processor 212 can be implemented as various processing devices, such as a microprocessor or other processing element, a coprocessor, a controller, or one or more of various other computing or processing devices including integrated circuits (e.g., ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc.). In an example embodiment, the processor 212 can be configured to execute instructions stored in the memory 214 or instructions that can access the processor 212 in other ways. Therefore, whether configured by hardware or by a combination of hardware and software, the processor 212 can represent an entity (e.g., physically embodied in a circuit in the form of a processing circuit 210) that can be configured accordingly while performing operations according to an embodiment of the present invention. Therefore, for example, when the processor 212 is implemented as an ASIC, FPGA, etc., the processor 212 can be specifically configured as hardware for performing the operations described herein. Alternatively, as another example, when the processor 212 is implemented as an executor of a software instruction, the instruction can specifically configure the processor 212 to perform the operations described herein.

[0032] In one example embodiment, the processor 212 (or processing circuit 210) may be implemented to include or otherwise control the operation of the test module 240 based on input received by the processing circuit 210 and / or an invoked entity, service, or application to evaluate the network performance of the access network 250 being evaluated by the experience tester 100. Thus, in some embodiments, in response to correspondingly executing instructions or algorithms configuring the processor 212 (or processing circuit 210), it can be said that the processor 212 (or processing circuit 210) causes each of the operations described in conjunction with the test module 240, which are related to request generation and response processing for information including information associated with the qualitative performance of the access network 250. In particular, the instructions may include instructions for obtaining one or both of the network identification information 260 and the network parameters 262 based on a call to an asset accessible via the Internet 130. The test module 240 may then use the network identification information 260 and the network parameters 262 and the performance measurements to download and upload the test files 264 and the test web pages 266 to provide the qualitative evaluation of the network performance described herein.

[0033] In an exemplary embodiment, the memory 214 may include one or more non-transitory memory devices, for example, fixed or removable volatile and / or non-volatile memory. The memory 214 may be configured to store information, data, applications, instructions, etc., so that the processing circuit 210 can implement various functions according to the exemplary embodiments of the present invention. For example, the memory 214 may be configured to cache input data for processing by the processor 212. Additionally or alternatively, the memory 214 may be configured to store instructions executed by the processor 212. As another alternative, the memory 214 may include one or more databases that may store various data sets in response to input sensors and components. In the contents of the memory 214, applications and / or instructions may be stored for execution by the processor 212 to implement functions associated with each corresponding application / instruction. In some cases, the application may include instructions for controlling the operation of the test module 240 as described herein.

[0034] In an example embodiment, the UE 230 may access the Internet 130 via an access network 250 (e.g., a terrestrial network 110, an ATG network 120, a fixed line network 140, or a satellite network, etc.). The UE 230 may enter a web address associated with the experience tester 100, and a web page may be displayed that enables the user to test the performance of the access network 250 using the experience tester 100. In some embodiments, the user may first initiate the operation of the experience tester 100 by requesting the experience tester 100 to detect the access network 250. The test module 240 may be configured to receive a request for network detection and initiate a call to the network identity agent 270. In this regard, for example, the test module 240 may use an application program interface (API) of an entity (e.g., ipinfo.io) associated with the network identity agent 270 and capable of determining the network identification information 260. The network identification information 260 may include, for example, the user's public IP address, host name, city, region, country, and / or other data that uniquely identifies the access network 250. Then, the test module 240 can be configured to identify (if possible) a connectivity solution associated with the access network 250 (e.g., the name of a network provider). If the name of the network provider can be determined, the test module 240 can log the name and all measurement data associated with the test along with the date, time, and any other relevant information (e.g., user location). The log can be set in the memory 214, or in a separate storage location at or accessible to the experience tester 100. If the name of the network provider cannot be determined, the test module 240 can request such information from the user. If the user provides the name of the network provider, the log can record the name and all measurement data again. If the name cannot be determined, the log can record the name as unknown, or the measurement data can be recorded in association with the remaining network identification information 260 obtained by the test module 240.

[0035] In some embodiments, particularly for an ATG network, satellite network, or similar service that can serve an aircraft (e.g., aircraft 118), the test module 240 can provide a separate request to the user to request the user to indicate the tail number or flight number of the aircraft 118. In some cases, the tail number of the aircraft 118 can be associated with a particular network provider and / or other details about the communication equipment associated with the aircraft 118 can be known. However, in other examples, the test module 240 can also automatically detect the tail number information based on the network identification information 260, the IP address, the router MAC address, or other means that can associate the network provider with the aircraft. It is worth noting that the tail number can be omitted entirely in the case of no association with the aircraft.

[0036] After the initial network detection and the user inputs the network provider name and / or tail number information (if applicable), the user can select the option to continue the test. Then, the test module 240 will then prepare to perform a series of test activities and start performing a series of test activities when ready. In this regard, for example, the test module 240 can provide the user with an indication that it is preparing to run the test, while making an API call to another test site (e.g., parameter test agent 272) to obtain ping test, jitter test and / or download speed information from another publicly available resource (e.g., another example of test resource 164). In typical cases, the ping test, jitter test and / or download speed information obtained from the test resource 164 is likely to be the end of the test. For example, the test module 240 will display the test results (e.g., ping test results, jitter test results and / or download or upload speed results) as network parameters 262 to the user as the final output of the test. However, as described above, this information (i.e., the network parameters 262 in their raw form) is typically either not useful for interpreting what the user experience is like in a meaningful way, the information is meaningless alone but is used in combination with other parameters, or the information is simply not suitable for the specific service or application that the user intends to use (i.e., a slow return link may not be critical for performing a one-way streaming service, where the forward "download" speed is more relevant, although if the latency is high, the streaming service may be slow to start). Therefore, the test module 240 does not stop at this point, but continues to add other tests to the series of test activities to obtain more meaningful and useful results. In particular, the test module 240 can still utilize other test resources 164 to obtain more comprehensive results than simply using ping tests, jitter tests, and / or download and upload speed tests.

[0037] In an example embodiment, the test module 240 may then be configured to download the test file 264 to the UE 230, and the download time may be measured and recorded. The test file 264 may be a file of a predetermined size (e.g., 1 MB) and may be located at a predetermined location of the test module 240. In some cases, the test file 264 may be located locally (e.g., at the memory 214) with the test module 240. However, in other cases, it may be desirable to store the test file 264 at a remote location to further test the access network 250 associated with accessing files from another external server via the Internet 130. Thus, the test file 264 may be stored locally or remotely relative to the test module 240.

[0038] Thereafter, the same test file 264 may be uploaded from the UE 230 back to the experience tester 100, and the upload time may be measured and recorded. Thus, the test module 240 may have an accurate representation of the download speed and upload speed of a known file that is standard across all network tests. Not only can the recording of the times identify situations where there is a significant imbalance between download and upload speeds, which may affect the user experience for services or applications that require fast speeds in both directions, the recording of the times can also be used to compare the same or different network performance results at different times or with other distinguishing characteristics that can be determined over time.

[0039] After the download and upload tests are completed, the test module 240 can be configured to direct the UE 230 to download a predetermined web page (i.e., the test web page 266) at a specific location. However, the test web page 266 can also include images located on different servers. In an example embodiment, the test web page 266 can have a predetermined size (e.g., 2MB) and can have custom selected images on at least two different servers and sometimes on more than two servers (e.g., four servers). The requirement to access a known web page including images from multiple servers further tests the robustness of the ability to access the network 250 in a standard manner, but is not as simple as simply accessing a single web page with content found at a single server, which is indeed abnormal in the context of how most web pages are built, as web pages extract components from a variety of different locations because various parts of the web page are filled with various content, including required information, analysis plug-ins, advertisements, and other links or services. Therefore, compared with traditional testing, the test web page 266 provides a more meaningful user experience test.

[0040] After downloading the test web page 266, the test module 240 may perform a social media download test with the UE 230. In this regard, since social media and certain other applications or services may require performance involving more burst-type data exchanges, the test module 240 may direct the UE 230 to download a specific number of packets of a specific size and measure the download speed. For example, the experience tester 100 (i.e., via the test module 240) may direct the download and upload of 8 50KB packets (400KB total in each direction) while measuring the download and upload speeds represented by the smaller number of data, and then record the download and upload speeds.

[0041] Therefore, in some embodiments, the above series of tests may include any one or all of network identification information test, network parameter test, standard file download and upload test, web page download test and social media download. Once all tests are completed, the test module 240 may be configured to calculate and generate test results. The test results may be generated by determining the score or rating of the performance of the access network 250 associated with each of a plurality of different services or capabilities that the user can expect the access network to provide. In an example embodiment, different capabilities or services may include voice calls, video calls, streaming entertainment, social media, games and web browsing. The total test time (i.e., the total time to complete all tests) and any other data that the user may be interested in may also be provided. For example, the results of ping tests, download speeds, upload speeds, website loading speeds, etc. may be displayed. In some cases, an overall score may also be provided.

[0042] Thus, the user may be presented with an overview of what functions are supported by the connection provided by the access network 250 and to what extent. In other words, a qualitative assessment of network performance will be provided to the user in a manner that relates network performance to specific media applications or services that are commonly used by various users. Thus, the user may not simply receive data that is not provided to the layperson in any useful or meaningful context. Instead, the user may receive meaningful results that are specifically presented in an intuitive and understandable manner, and that also reflect how the user may feel about the network performance when using the network.

[0043] Although not required, some example embodiments may also include a contention engine 290. Contention engine 290 has its optional nature. However, the location of contention engine 290 in the system may be different in different embodiments. In this respect, contention engine 290 may be accessed by test module 240 via the Internet 130, or may be a local resource in some cases. Therefore, although a direct connection from test module 240 to contention engine 290 is shown, in various embodiments, the connection may be direct or indirect. In addition, in some cases, contention engine 290 may be an online application or resource that can be called by test module 240 for input.

[0044] In any case, contention engine 290 can be configured to determine resource contention (e.g., contention rate) for access network 250 upon request of test module 240. Contention engine 290 can be configured to sense or otherwise determine the number of devices connected at a given access point. The command to sense or determine resource contention can occur automatically due to the operation of test module 240, or can be initiated by a manual request by an operator or by changing the settings of test module 240 to add determining resource contention as part of the methods described herein. This resource contention determination method can be performed "in-house".

[0045] However, in some cases, the contention engine 290 can be configured to determine resource contention "externally". The determination of external resource contention can be done based on information or knowledge about the network structure and the location of various assets within the network structure. In this regard, for example, in the case where the access network 250 is an ATG network 120 or a satellite network, the knowledge information of the network structure may include information indicating specific beam coverage parameters. For example, a database may be provided to define the satellite beam width and general geographic coverage area for each satellite and each beam of each satellite at any given time. Similarly, the ATG base station location, the beam associated with each ATG base station, and the coverage area of ​​each beam may also be known or recorded. Then, the aircraft location (or the location of other system assets) may be compared with the known information about the network structure to determine how many aircraft (or users on the aircraft) share the beam at any given time. This can then be used to determine resource contention. Thus, the determination of external resource contention may fuse aircraft (or other asset) position / location information (e.g., from ADS-B or other sources), a database of which aircraft are equipped with onboard connectivity equipment, a database of satellite or ATG beam inventories, information indicating which airlines have contracts with providers, and / or any other useful information in order to calculate how many aircraft (or assets) are served by the same beam, satellite, or tower at any given (current or future) time. Contention engine 290 may then report resource contention information to test module 240, and test module 240 may determine a contention score and the impact of the contention on the user experience.

[0046] As described above, the test module 240 can be configured to generate scores or ratings of network performance related to individual capabilities. The rating can be completed based on the actual experience of the user. In this regard, for example, the selected results in the test series results can be used to determine the scores of individual capabilities in the rating capabilities. The selected results are those results related to or most relevant to the corresponding capabilities. Therefore, the test results selected from a series of tests are only used to determine the scores of the various capabilities related to the selected test results. Once a group of related test results are associated with each individual capability, the boundaries between different rating levels can be determined. Then, a group of testers perform network testing and verify or modify the boundaries using feedback from the testers. For example, if the voice call capability of a specific network is scored so that based on the predefined boundaries, the voice call capability will receive a highly positive rating, the user feedback from the tester (or at least a predetermined number or percentage of the tester) can verify the predefined boundaries or suggest that the predefined boundaries should be adjusted to exclude the scores that obtain highly positive ratings.

[0047] Scoring may result in ratings being associated with intuitive experience indicators, such as emoticons (e.g., smiley faces, sad faces, etc.). Another intuitive experience indicator may be a scale, bar graph, or container showing fullness to represent ratings or quality. Colors (e.g., green, yellow, or red) may also be used to provide intuitive experience indicators. In addition, in some examples, a combination of the intuitive experience indicators listed above may be used. Each example is generated by the operation of the experience tester 100. In this regard, an example of a test result page for a specific access network that received a completely positive report is shown. At the same time, results from a traditional ATG network at the time of submitting this application are shown, and in particular, the results in each category are poor due to the lack of a high-quality return link and high latency.

[0048] The resulting web page generated at the UE 230 is shown. As described above, an intuitive experience indicator is provided for each of the multiple capabilities of the access network 250. These indicators include a video call indicator 300, a voice call indicator 310, a streaming entertainment indicator 320, a social media indicator 330, a game indicator 340, and a web browsing indicator 350. As the name implies, the video call indicator 300 provides a qualitative assessment of the ability of the access network 250 to make a video call. In this example, the intuitive experience indicator 302 associated with the video call indicator 300 is a semicircular ruler that provides the user with color coding and fullness indications about the network capabilities. Therefore, assuming that the ruler is full and the color is green, the intuitive experience indicator 302 of this example indicates that the access network 250 will provide a good user experience to users participating in the video call. It is worth noting that the ruler can have an area indicated by boundaries 304 and 306. If the ruler is full below the boundary 304, a red color can be generated for the intuitive experience indicator 302. Meanwhile, if the scale is full at a level between boundaries 304 and 306, a yellow color may be generated for the intuitive experience indicator 302. If the scale is full at a level above boundary 306, a green color may be generated for the intuitive experience indicator 302. Thus, in some cases, the intuitive experience indicator 302 may be generated to include at least two intuitively determinable indications of the quality of experience (e.g., color and scale fullness) to reinforce the qualitative nature of the results and further distinguish the results from mere numbers or numerical ratings that may have little or no meaning to a layperson. In this regard, for example, color may generally indicate the quality of experience (e.g., good, poor, or average), while scale fullness may provide an indication of a numerical score, which is a further measure of the degree of quality of experience.

[0049] Similarly, voice call metrics 310, streaming entertainment metrics 320, social media metrics 330, gaming metrics 340, and web browsing metrics 350 can each generate a colored and scaled output for a visual experience metric. However, an overall performance metric 360 can also be generated for the access network 250 by the test module 240. In this regard, the overall performance metric 360 can also include emoticons 362 (which can be further color-coded) and / or verbal ratings (e.g., "best performance," "poor performance," or "average performance").

[0050] In some example embodiments, the total test time 370 may also be displayed. The total test time 370 may provide an indication of the delay or latency associated with the access network 250, and therefore, the total test time 370 may be a useful quality indicator in itself. Other numerical results 380 may also be provided for the user to review. However, as described above, these results are generally not so useful to non-professionals without providing context. Therefore, some embodiments may record these values, but also present ratings for the user to view, which alternatively utilize intuitive experience indicators of these values. A skipped test 390 is also shown, which indicates that the experience tester 100 does not have to complete (or even attempt) every test. In this regard, in some cases, network performance may be very poor (or if the server for storing the test data content is temporarily offline, the test resources may not be available), so that some tests cannot be completed within the predetermined time limit. If so, these tests can be skipped, and in some cases, other tests may also be skipped because of the previously skipped tests due to the expectation that the subsequent tests will not be completed if the previous tests cannot be completed. Note that additional types of Media (i.e., Office Work 395) scores have also been added to account for the fact that other test criteria can be used in addition to those described above. The Office Work 395 type may involve connecting to cloud or corporate servers, VPNs, sending and receiving large email files with attachments, uploading or downloading files from corporate or cloud servers, etc.

[0051] As described above, each individual category (or capability) has its own scoring formula based on metrics or parameters captured during the execution of a series of tests. The details of these scoring formulas are beyond the scope of this disclosure. However, it should be understood that each different category may rely on a different combination of quantitative measurements and have different ranges and / or weights for the importance or valuation of a measurement or combination.

[0052] As described above, the test module 240 can record all scoring results associated with the network identification information 260. The scoring results can also include date and time information, and in some cases, location information of the UE 230. By storing past scoring results together with the network identification information 260, the test module 240 can generate trend and / or historical data for a given access network, location, date / time, etc. The trend or historical data can be provided to the user for comparison purposes (i.e., to see how today's performance compares to the average performance of the access network 250 at similar dates / times, how it compares to the average performance of assets at similar locations, etc.). Therefore, the user may sometimes be able to realize that the access network 250 has a "good", "poor" or "average" day in terms of performance. In addition, in some cases, the user (or application) can simply refer to the historical data without actually running a new experience test in order to make a decision about the user's (or application's) current or planned use of the access network 250. Therefore, for example, a user with multiple network options can view the results of candidate networks (even prospectively and before traveling) to determine the best possible plan for optimizing connectivity during travel (current or future).

[0053] In some cases, a user may utilize a particular web application or service whose performance depends at least in part (and sometimes in large part) on the corresponding performance of the access network 250 for the class (e.g., media type) of the service or application being used. In these cases, the web application or service may appear to be of poor design, quality, or performance when the culprit that makes the web application or service appear poor is actually the underlying access network. Therefore, in addition to (or as an alternative to) operating as described above, i.e., in response to the UE 250 going to a website associated with the experience tester 100 and initiating a series of tests to use the results by learning which service classes (e.g., media types) will provide corresponding quality levels in terms of user experience, the experience tester 100 may operate in other ways. For example, the web service or application itself may initiate a call to the experience tester 100 in order to evaluate the access network 250 of a particular user attempting to use the web service or application.

[0054] In the example where a web service or application calls the experience tester 100, the series of tests discussed above can be run without any specific user interaction. However, the experience tester 100 can provide the web service or application with an indication of the network performance of the access network 250 associated with a specific user. The web service or application can use this information to perform many operations. For example, in some cases, the web service or application can provide an unsolicited indication to the user to indicate the quality of the access network 250. This can be a preemptive measure to let the user know the quality of its access network, so as to set expectations for the user about the possible performance of the access network 250, thereby potentially also setting the performance of the web service or application. In this regard, if the web service or application is a game, a game score can be provided by the test module 240 (with or without other scores), and the web service or application started by the user can actively indicate that the user's network has been detected, and the user experience of playing games on the network will be a corresponding rating level. For example, the game can provide the user with an indication that "your network has been detected, and you can expect a [good or poor] experience based on the game capabilities of your network." Using trending / historical data, the game can further provide suggestions on ways to improve performance or avoid poor performance, such as "Please try a different network for optimal game performance" or "Playing on your network from time xxxx to time yyyy may result in degraded game performance."

[0055] In some cases, the web service or application can also be configured to adjust the service or performance characteristics based on the scoring results provided by the experience tester 100. For example, if the level of the access network 250 is very high (for all categories, or at least for the category to which the web service or application belongs), the web service or application can automatically maximize certain parameters (such as video fidelity, image quality or other settings). At the same time, if the level of the access network 250 is very low (for all categories, or at least for the category to which the web service or application belongs), the web service or application can automatically reduce or minimize certain parameters (such as video fidelity, image quality or other settings). Therefore, a web service and / or application adapted to network capabilities can be provided, which automatically adapts its performance to network performance based on the information provided by the experience tester 100. In some cases, adaptation can be related to the operating mode of the web service or application. For example, a web service or application can operate in a degraded mode, an offline mode, a normal mode, a fully optimized mode and / or a similar mode based on the qualitative score of the access network 250 determined by the experience tester 100.

[0056] With respect to the modification of the network resources of the access network 250, a block diagram of a resource manager 400 that can be configured accordingly is shown. The resource manager 400 may include a processing circuit 410 that is configured to provide control outputs to various entities, modules, or components of the access network 250 to control specific aspects of network operation or performance. According to an example embodiment of the present invention, the processing circuit 410 may be configured to perform data processing, control function execution, and / or other processing and management services. In some embodiments, the processing circuit 410 may be implemented as a chip or a chipset. In other words, the processing circuit 410 may include one or more physical packages (e.g., chips) that include materials, components, and / or wires on a structural assembly (e.g., a substrate). The structural assembly may provide physical strength, dimensional conservation, and / or electrical interaction limitations for the component circuits included thereon. Therefore, the processing circuit 410 may be configured in some cases to implement embodiments of the present invention on a single chip or as a single "system on a chip". Therefore, in some cases, a chip or a chipset may constitute a device that performs one or more operations for providing the functions described herein.

[0057] In an example embodiment, the processing circuit 410 may include one or more instances of a processor 412 and a memory 414 that may communicate with or otherwise control the device interface 420. Thus, the processing circuit 410 may be implemented as a circuit chip (e.g., an integrated circuit chip) configured (e.g., using hardware, software, or a combination of hardware and software) to perform the operations described herein.

[0058] Device interface 420 may include one or more interface mechanisms for enabling communication with other devices, such as modules, entities, sensors, and / or other components of access network 250. In some cases, device interface 420 may be any means, such as a device or circuit implemented in hardware or a combination of hardware and software, that is configured to receive and / or send data from / to a module, entity, sensor, and / or other component of access network 250 that communicates with processing circuit 410.

[0059] Processor 412 can be implemented in a variety of different ways. For example, processor 412 can be implemented as various processing devices, such as a microprocessor or other processing element, a coprocessor, a controller, or one or more of various other computing or processing devices including integrated circuits (e.g., ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc.). In an example embodiment, processor 412 can be configured to execute instructions stored in memory 414 or otherwise accessible to processor 412. Therefore, whether configured by hardware or by a combination of hardware and software, processor 412 can represent an entity (e.g., physically embodied in a circuit in the form of processing circuit 410) that can be configured accordingly while performing operations according to an embodiment of the present invention. Therefore, for example, when processor 412 is implemented as an ASIC, FPGA, etc., processor 412 can be specifically configured as hardware for performing the operations described herein. Alternatively, as another example, when processor 412 is implemented as an executor of software instructions, the instructions can specifically configure processor 412 to perform the operations described herein.

[0060] In an example embodiment, the processor 412 (or processing circuit 410) may be implemented to include or otherwise control the operation of the beamforming control module 300 based on inputs received by the processing circuit 410 in response to receiving position information associated with various relative positions of communication elements of the network. Thus, in some embodiments, in response to correspondingly executing instructions or algorithms configuring the processor 412 (or processing circuit 410), the processor 412 (or processing circuit 410) may be said to cause each operation described in conjunction with the beamforming control module 430 related to adjustments made to the antenna array to undertake corresponding functions related to beamforming. In particular, the instructions may include instructions for processing 3D positioning information of a potential mobile receiving station (e.g., on an airplane, train, automobile, or other vehicle, or in a vehicle owned by a user), which instructions can be used to determine the relative position of the receiving station to any or all fixed transmission sites (i.e., APs of the access network 250) in order to instruct the antenna array to form or select a beam in a direction that will facilitate establishing a communication link between the mobile receiving station and one of the fixed transmission stations described herein.

[0061] In an exemplary embodiment, the memory 414 may include one or more non-transitory memory devices, for example, volatile and / or non-volatile memory that may be fixed or removable. The memory 414 may be configured to store information, data, applications, instructions, etc., to enable the processing circuit 410 to implement various functions according to an exemplary embodiment of the present invention. For example, the memory 414 may be configured to cache input data for processing by the processor 412. Additionally or alternatively, the memory 414 may be configured to store instructions executed by the processor 412. As yet another alternative, the memory 414 may include one or more databases that may store various data sets in response to input sensors and components. Among the contents of the memory 414, applications and / or instructions may be stored for execution by the processor 412 to perform functions associated with each respective application / instruction. In some cases, the application may include instructions for providing input to control the operation of the beamforming control module 430 as described herein.

[0062] In an example embodiment, the beamforming control module 430 may be configured to enable beams to be formed or selected from various components of the access network 250 (e.g., an AP of any type of network as which the access network 250 is implemented) toward the UE 230 or toward a radio device that ultimately communicates with the UE 230. Figure 1In the context of the ATG network 120, the beamforming control module 430 can control the formation or selection of one or both beams from the first ATG AP 122 or the second ATG AP 124 toward the aircraft 118 (and the UE 150 located thereon). The beams can be formed and / or selected in real time or in advance to anticipate switching. In either case, the beamforming control module 430 can be configured to utilize information indicating the relative position of the aircraft 118 or UE 150 and the first ATG AP 122 and / or the second ATG AP 124. By knowing the positions of the two devices or network nodes, the positions of the devices or network nodes relative to each other can be determined from the perspective of one of the devices or network nodes, and steering can be accurately accomplished. The beamforming control module 430 or another component operably coupled thereto may also employ a tracking algorithm to track dynamic position changes and / or calculate future positions based on the current position and movement rate and direction. After determining the expected relative position, the beamforming control module 430 can be configured to provide instructions to directly form or select a beam from the antenna array of a corresponding one of the first ATG AP 122 or the second ATG AP 124 based on the expected relative position. The instructions can be provided to a control device that is configured to adjust the characteristics of the antenna array to form a directionally steerable beam that is directed in the direction of the expected relative position or to form a selected one of a plurality of fixed beams that each have a different orientation, but are combined to cover a sector or region with adjacent and slightly overlapping coverage areas. For example, such a beam can have an azimuth and elevation width of 8 degrees or less. In addition, in some cases, such a steerable beam can have an azimuth and elevation width of 5 degrees or less, or even an azimuth and elevation width of 2 degrees or less. However, steerable beams of larger sizes can also be used in some embodiments.

[0063] For the general example of an access network 250 serving a UE 230, any or all users of the UE 230, an application launched at the UE 230, or the resource manager 400 may be configured to initiate a call to the experience tester 100 to determine a qualitative assessment of the capabilities of the access network 250. Specifically, when the resource manager 400 initiates a call, the experience tester 100 may provide a qualitative assessment to the resource manager 400. The qualitative assessment provided to the resource manager 400 may, in some cases, be based at least in part on a contention score or other information indicating resource contention provided by the contention engine 290. However, in other cases, the contention score or information indicating resource contention may be provided directly to the resource manager 400 for use in conjunction with the operation of the beamforming control module 430. In some cases, the resource manager 400 may be configured to also (or alternatively) receive a report on experience test results obtained by the UE or application running the experience test. Therefore, in some cases, the resource manager 400 may never actually run its own experience test. However, resource manager 400 may still perform various network control functions based on empirical test results that have been reported to resource manager 400 , which may be various other users or applications within access network 250 .

[0064] In an example embodiment, the resource manager 400 may be configured to periodically call the experience tester 100 at the corresponding time of the call to test the operation of the access network 250. Therefore, the periodic testing of the access network 250 may be completed at routine time intervals set by the network operator. Additionally or alternatively, the experience tester 100 may be called each time a new user is added to the network, or in some cases when a threshold number of users have been added (i.e., attached) to the access network 250. When a threshold is adopted, the threshold number of users may be determined at the network or resource level, or even according to the geographical location. For example, the resource manager 400 may be configured to issue a call to the experience tester 100 whenever a threshold number of users have been attached to a single AP, a single sector of an AP, or a single beam. Alternatively or additionally, the resource manager 400 may be configured to call the experience tester 100 whenever a threshold number of users have been attached to the access network 250 from a specific geographical location or area (e.g., including a group of APs).

[0065] Regardless of how often calls are made, or whether the resource manager 400 receives experience test data without actually making calls, when the resource manager 400 receives the results of the experience test (i.e., a qualitative assessment of the network's capabilities related to delivering a quality user experience based on a combination of quantitative factor scores according to actual human perception of performance at corresponding combinations of quantitative factor scores), the resource manager 400 can make a determination about adjusting network operating parameters to improve the user experience at locations within the access network 250 that receive low quality assessments.

[0066] It should be noted that in some cases, the resource manager 400 may access (via communications therewith) fixed test locations within the access network 250 at different locations. Each test location may include a test UE or similar device that actually makes calls from a corresponding location within the access network 250. As a result, the resource manager 400 can then be informed of the network capabilities within the specific location of the corresponding test UE. This can include testing UEs on aircraft or satellites within an ATG or satellite network. However, in many cases, it may be simpler for the resource manager 400 to simply request or passively receive experience test results from UEs or applications at their various locations in the access network 250 rather than utilizing fixed test UEs.

[0067] After receiving the results of the experience test, the resource manager 400 can be configured to determine the location or resource identifier associated with the experience test. Figure 1 The UE 150 on the aircraft 118 performs an experience test and reports the result to the resource manager 400, which can then be configured to determine the serving AP (e.g., the first ATG AP 122) of the UE 150. If the result of the experience test indicates that the user experience is poor, or at least a suboptimal user experience, the resource manager 400 can be configured to determine whether there are any available options to improve the user experience. Thus, for example, the resource manager 400 can consider existing information of previously completed experience tests associated with candidate resources (together with the date and time at which such experience tests were completed to confirm relevance to the current time) to determine whether there is a better performing resource to which it can be transferred. Additionally or alternatively, the resource manager 400 can request a test UE or another device to perform an experience test associated with an alternative resource to determine the suitability of transferring to the alternative resource based on the result of the experience test associated with the alternative resource. As yet another alternative or additional option, a new resource that is assumed to be able to improve the user experience can be brought online.

[0068] Thus, for example, if a beam serving UE 150 of aircraft 118 produces poor experience test results (and / or high resource contention), resource manager 400 can determine whether another beam (originating from the same or a different AP) can be shifted to cover UE 150 or can be formed to serve UE 150. The new beam may be selected based on recent superior performance of assets in the area, or simply as a means to reduce resource contention. In some cases, after forming or directing a new beam, resource manager 400 may also be configured to perform another (e.g., confirmatory) experience test to verify the improvement in user experience and / or reduction in resource contention associated with the new beam.

[0069] Shows Figure 1 4 is a block diagram of an ATG network 120 of a system configured for operation as described above. In this regard, the resource manager 400 is disposed at a network controller 450 of the ATG network 120, and the aircraft 118 initially communicates with the first ATG AP 122 via a first beam 460. In some cases, the network controller 450 may be configured to handle routing calls to and from the aircraft 118 (or a communication device on the aircraft 118 or other devices in other networks) and / or handle other data or communication transmissions between the communication device on the aircraft 118 and the ATG network 120. In some embodiments, the network controller 450 may be used to provide a connection to a fixed line trunk when the communication device on the aircraft 118 is engaged in a call. In addition, the network controller 450 may be configured for forwarding of messages and / or data to and from a UE or other communication device, and may also control the forwarding of messages to a base station or AP.

[0070] Notably, the first beam 460 may be a steerable beam that can be dynamically steered to track the movement of the aircraft 118 based on the 3D positioning information. However, in other cases, the first beam 460 may be one of a series of fixed beams that can be formed and switched to adjacent other fixed beams that are sequentially selected for switching and tracking of the aircraft 118 based on the 3D positioning information. In either case, the experience tester 100 may determine a poor user experience for users (e.g., UEs or other devices) or applications on the aircraft 118. Based on experience tests associated with the second ATG AP 124 that were previously (within a predetermined time period) or performed simultaneously, or based on the assumption that another beam will improve the user experience, the resource manager 400 may determine to form a second beam 470 to the aircraft 470. The second beam 470 may be similar in structure to the first beam 460 (but may be different). After the second beam 470 is formed and some or all UEs or other equipment on the aircraft 118 are switched to the second beam 470 , the experience tester 100 may perform a confirmatory experience test to confirm the expected improvement in user experience.

[0071] If the improvement is verified by equal or better empirical test results, the second beam 470 can continue to carry all assets transferred thereto until another handoff is needed and performed (e.g., performed by the network controller 450). If the improvement is not verified, the resource manager 400 can determine another empirical-based handoff. The next handoff can be performed either due to an invalid empirical-based handoff, a subsequent decrease in empirical test results below a threshold (e.g., generally or for a given media type), or based on a traditional handoff trigger (e.g., signal strength reduction, location-based handoff, etc.). It should be understood that the examples can be replicated to any other network, including terrestrial networks, satellite networks, etc.

[0072] The first beam 460 and the second beam 470 are shown in greater detail to illustrate the nature of beams (e.g., "beamlets") that can be employed in some cases. In this regard, the first beam 460 and the second beam 470 can each have a beam width of less than about 8 degrees in both azimuth and elevation. However, in some cases, the beam width may be as small as 5 degrees or even 2 degrees. As the aircraft 118 moves in the direction of arrow 490, the first beam 460 (and / or the second beam 470) can be directed to follow or track the aircraft 118 based on the 3D positioning information. Alternatively, as described above, adjacent and at least partially overlapping beams (e.g., beam 462) can be selected and / or formed, and other beams can continue to be selected and / or formed in sequence to continue switching and maintaining communications with the aircraft as the aircraft 118 moves in the direction of arrow 490. These beam switches occur based on the position or location of the aircraft 118 and can be routinely performed by the network controller 450 to account for the movement of the aircraft 118. However, the formation, selection and / or steering of beams based on improved user experience can be performed by the network controller 450 (e.g., specifically by the resource manager 400) in parallel with the location-based switching to improve overall network performance and user experience. Therefore, the network controller 450 (e.g., via the resource manager 400) can be configured to perform beam switching based on both: location-based criteria and user experience-based criteria.

[0073] It should be appreciated that the switch from the first beam 460 to the second beam 470 assumes that two APs are within range of communication with the aircraft 118. In the event that only one AP is within range of communication (or one of the APs is the best candidate to generate both the first beam 460 and subsequent beams), the example embodiments may still be practiced, for example, if the first ATG AP 122 is capable of generating one or more additional beams on another channel (e.g., employing another frequency). In this regard, the alternate second beam 470' may be sourced from the same AP as the first beam 460 (i.e., from the first ATG AP 122 in this example) using different channel resources. The second beam 470' may have a wider (or narrower) beam width than the first beam 460, and in some cases, may be generated from another antenna array associated with the ATG network 120 and disposed at the first ATG AP 122. Thus, beam selection or formation may be optimized to dynamically steer beams or select beam sources (depending on the capabilities present on a given network) to switch from a highly contended beam to a less contended beam, thereby improving the user experience for both beam users. In some cases, beam optimization based on user experience may also be application specific to optimize communication with local access points (e.g., a cabin wireless access point (CWAP) on an aircraft, or a WiFi router at home or business) making the system more intelligent by routing traffic not only based on application type and priority, but also based on experience and contention issues. Thus, example embodiments may provide combination / dependency based intelligence that is missing in all traditional testing methods.

[0074] Although the resource manager 400 is shown at the network controller 450, it should be understood that in some cases, various instances of the resource manager 400 may also function at various entities or platforms within any given access network. Thus, for example, the aircraft 118 may have a separate instance of the resource manager 400 thereon. In such an example, the resource manager 400 may operate as described above, except that the initiation of the switch may come from the aircraft 118 side rather than the network side. In this case, the aircraft 118 (or a communication device thereon, such as an aircraft radio kit) may direct the formation of a second beam to be directed / selected to form a second or alternative resource for communicating with the ground. In other words, dynamic optimization of network resources based on user experience may be initiated and managed at the network side or at a remote node within the network.

[0075] therefore, Figure 1 The system can provide an environment in which the test module 240 is located and the resource manager 400 can provide a mechanism by which many useful methods can be practiced. Figure 12 is a block diagram of a method associated with a system, a test module 240, and a resource manager 400. From a technical perspective, the resource manager 400 described above may be used to support some or all of the operations, and processing circuitry similar to that at a network controller 450 (at least in function, but possibly in a different form) may support some or all of the operations. Figure 1 The platform / component / module described in the flowchart can be used to facilitate the implementation of several computer programs and / or network communication-based interactions. It should be understood that each box of the flowchart and the combination of boxes in the flowchart can be implemented by various means, such as hardware, firmware, processors, circuits and / or other devices associated with executing software including one or more computer program instructions. For example, one or more of the above processes can be implemented by computer program instructions. In this regard, the computer program instructions implementing the above process can be stored by a storage device (e.g., a memory device of the beamforming control module 430, the processing circuit 410 or other processing circuit) and executed by a processor in the device. As should be understood, any such computer program instructions can be loaded onto a computer or other programmable device (e.g., hardware) to generate a machine, so that the instructions executed on the computer or other programmable device create a device for implementing a specific function in the flowchart box. These computer program instructions can also be stored in a computer-readable storage, which can guide a computer or other programmable device to operate in a specific manner, so that the instructions stored in the computer-readable storage generate a product that implements a specific function in the flowchart box. Computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operations to be performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device implement the specific functions in the flowchart blocks.

[0076] Therefore, the blocks of the flowchart support the combination of devices for performing specific functions and the combination of operations for performing specific functions. It should also be understood that one or more blocks of the flowchart and the combination of blocks in the flowchart can be implemented by a computer system based on special-purpose hardware that performs specific functions, or a combination of special-purpose hardware and computer instructions.

[0077] In this regard, a method according to an embodiment of the present invention may include: at operation 500, receiving an indication of a qualitative assessment of the ability of a communication device to access a network for different categories of services or applications using the communication device accessing the network via a first beam formed or selected to serve the communication device. The method may also include: at operation 510, determining whether a candidate resource exists in response to the qualitative assessment being below a predetermined threshold; at operation 520, forming or selecting a second beam associated with the candidate resource in response to determining that the candidate resource exists; and at operation 530, initiating a switch of the communication device from the first beam to the second beam.

[0078] In some embodiments, the method may include additional, optional operations, and / or may modify or expand the above operations. Some examples of modifications, optional operations, and expansions are described below. It should be understood that modifications, optional operations, and expansions may be added individually, or they may be added cumulatively in any desired combination. In an example embodiment, forming or selecting a second beam may include forming or selecting a second beam from an access point different from an access point that initiates the first beam. In some cases, forming or selecting a second beam may include forming or selecting a second beam from an access point that is the same as an access point that initiates the first beam. In an example embodiment, receiving an indication may also include receiving an indication of resource contention associated with the first beam. In some cases, switching may reduce resource contention on the first beam. In an example embodiment, determining whether a candidate resource exists may include determining whether the candidate resource receives a positive qualitative assessment within a predetermined time period. In some cases, determining whether a candidate resource exists may include performing a second qualitative assessment on the candidate resource and determining whether the second qualitative assessment is an improvement on the qualitative assessment associated with the first beam.

[0079] Thus, the example embodiments employ an objective / quantitative series of tests and scores, and then, based on the network capabilities, create a subjective / qualitative score or indication that accurately represents the user experience. Thereafter, resources can be managed to optimize the user experience, and resource management can include optimized beamforming / selection. An advantage of employing the example embodiments can be that by considering a combination of tests rather than isolated tests, the interdependencies of other qualitative variables actually provide valuable insights into the true user experience or application experience that simple numerical results cannot provide, and thus resource management can be performed to maximize the quality of the user experience. Traditional speed tests that simply display a single number can give a false sense of confidence in network quality, and these tests fail to teach or explain to non-expert users what the numbers mean and why a "good" number (i.e., a fast download speed) can still result in a poor experience (e.g., a slow web page load or an email cannot be sent), which may be due to high latency, slow upload speeds, or both.

[0080] Benefiting from the teachings presented in the foregoing description and the associated drawings, a person skilled in the art to which the present invention belongs can think of many modifications and other embodiments of the present invention set forth herein. Therefore, it should be understood that the present invention is not limited to the specific embodiments disclosed, and modifications and other embodiments are intended to be included within the scope of the appended claims. In addition, although the foregoing description and the associated drawings describe exemplary embodiments in the context of certain exemplary combinations of elements and / or functions, it should be understood that different combinations of elements and / or functions can be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, as set forth in some of the appended claims, combinations different from the combinations of elements and / or functions explicitly described above can also be thought of. Where advantages, benefits, or solutions to problems are described herein, it should be understood that such advantages, benefits, and / or solutions may be applicable to some exemplary embodiments, but not necessarily to all exemplary embodiments. Therefore, any advantages, benefits, or solutions described herein should not be considered to be critical, necessary, or essential to all embodiments or embodiments claimed herein. Although the text employs specific terms, they are used only in a general and descriptive sense, not for limiting purposes.

Claims

1. A method for optimizing beamforming based on user experience, comprising: receiving, via a first beam formed or selected to serve a communication device using an access network, an indication of a qualitative assessment of access network capabilities associated with different classes of services or applications of the communication device, the qualitative assessment including an indication of resource contention associated with the first beam; In response to the qualitative assessment being below a predetermined threshold, determining whether a candidate resource exists; In response to determining that a candidate resource exists, forming or selecting a second beam, the second beam being associated with the candidate resource; as well as initiating a switch of the communication device from the first beam to the second beam, wherein receiving the indication further comprises receiving an indication of resource contention associated with the first beam, and The indication of resource contention is related to the number of devices connected to an access point associated with the first beam or the number of aircraft served by the first beam.

2. The method according to claim 1, wherein: Forming or selecting the second beam includes forming or selecting the second beam from an access point different from an access point from which the first beam originated.

3. The method according to claim 1, wherein: Forming or selecting the second beam includes forming or selecting the second beam from the same access point as the access point from which the first beam originated.

4. The method according to claim 1, wherein: Performing the switch reduces resource contention on the first beam.

5. The method according to claim 1, wherein: Determining whether the candidate resource exists includes determining whether the candidate resource has received a positive qualitative evaluation within a predetermined time period.

6. The method according to claim 1, wherein: Determining whether a candidate resource exists includes making a second qualitative assessment of the candidate resource and determining whether the second qualitative assessment is an improvement over the qualitative assessment associated with the first beam.

7. A resource manager based on user experience optimized beamforming, comprising a processing circuit, wherein the resource manager is configured to: receiving, via a first beam formed or selected to serve a communication device using an access network, an indication of a qualitative assessment of access network capabilities associated with different classes of services or applications of the communication device, the qualitative assessment including an indication of resource contention associated with the first beam; In response to the qualitative assessment being below a predetermined threshold, determining whether a candidate resource exists; In response to determining that a candidate resource exists, forming or selecting a second beam, the second beam being associated with the candidate resource; as well as initiating a switch of the communication device from the first beam to the second beam, wherein receiving the indication further comprises receiving an indication of resource contention associated with the first beam, and The indication of resource contention is related to the number of devices connected to an access point associated with the first beam or the number of aircraft served by the first beam.

8. The resource manager according to claim 7, wherein: The processing circuit is configured to form or select the second beam by forming or selecting the second beam from an access point different from an access point from which the first beam originated.

9. The resource manager according to claim 7, wherein: The processing circuit is configured to form or select the second beam by forming or selecting the second beam from the same access point as the access point from which the first beam originated.

10. The resource manager according to claim 7, wherein: Performing the switch reduces resource contention on the first beam.

11. The resource manager according to claim 7, wherein: The processing circuit is configured to determine whether the candidate resource exists by determining whether the candidate resource receives a positive qualitative evaluation within a predetermined time period.

12. The resource manager according to claim 7, wherein: The processing circuit is configured to determine whether a candidate resource exists by performing a second qualitative assessment of the candidate resource and determining whether the second qualitative assessment is an improvement over a qualitative assessment associated with the first beam.

13. A system for optimizing beamforming based on user experience, comprising: an access network comprising at least two base stations configured to communicate with a wireless communication device via dynamically controllable or shapeable beams; The resource manager according to any one of claims 7 to 12, which is arranged at a location accessible via an access network or at a wireless communication device; as well as an experience tester configured to, via a first beam formed or selected to serve a communication device, conduct a series of tests to determine a qualitative assessment of the ability of the wireless communication device to access the network with respect to different classes of services or applications, the qualitative assessment including an indication of resource contention associated with the first beam, The resource manager is configured to initiate switching based on location-based criteria and user experience-based criteria, wherein the user experience-based criteria is associated with a qualitative evaluation.

14. The system of claim 13, wherein the resource manager is configured to: receiving an indication of said qualitative assessment; In response to the qualitative assessment being below a predetermined threshold, determining whether a candidate resource exists; In response to determining that the candidate resource exists to initiate a switch from the first beam to the second beam, a second beam associated with the candidate resource is formed or selected.

15. The system of claim 13, wherein: Forming or selecting the second beam includes forming or selecting the second beam from a base station different from the base station from which the first beam originated.

16. The system of claim 13, wherein: Forming or selecting the second beam includes forming or selecting the second beam from the same base station as the base station from which the first beam originated.

17. The system of claim 13, wherein: Determining whether the candidate resource exists includes determining whether the candidate resource has received a positive qualitative evaluation within a predetermined time period.

18. The system of claim 13, wherein: Determining whether a candidate resource exists includes making a second qualitative assessment of the candidate resource and determining whether the second qualitative assessment is an improvement over the qualitative assessment associated with the first beam.

Images