System and method for mitigating user input errors on touch screen display due to disturbance of mobile platform

Abstract

Systems and methods are provided for mitigating user input errors due to perturbations of a mobile platform. The system may include: one or more sensors configured to sense a measurable condition inside or outside the mobile platform; and a controller operable in communication with the sensor and with the touch screen display. The controller may be configured to: receive, by the one or more processors, sensor data from the sensor; processing the sensor data to detect and characterize perturbations; receiving user input data indicating a user input performed by a user via the touch screen display; processing the user input data to determine whether the user input may be invalid due to perturbations; and initiating a corrective action in response to determining that the user input may be invalid due to the disturbance.

Description

Technical Field

[0001] The present invention relates generally to touch screen display devices for mobile platforms, and more particularly to systems and methods for mitigating user input errors due to disturbances during operation of the mobile platform. Background Art

[0002] Modern vehicles (e.g., automobiles, aircraft, etc.) may use built-in or mobile touch screen display devices to display a wide variety of information, such as vehicle systems, navigation, entertainment, and other information used in operating and controlling the vehicle. Passengers of the vehicle are able to interact with the touch screen display device to access information and input selections and commands. However, various disturbances (such as changes in vehicle speed or position) may make it difficult to interact with the touch screen display device. For example, a disturbance may cause a user to accidentally touch the touch screen display in an unexpected location.

[0003] Therefore, there is a need for systems and methods that can mitigate user input errors due to disturbances during operation of a vehicle.In addition, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the drawings and the foregoing technical field and background. Summary of the invention

[0004] This Summary is provided to describe selected concepts in a simplified form that are further described in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] In various embodiments, a method for mitigating user input errors due to disturbances during operation of a mobile platform is provided. The method may include, by one or more processors: receiving sensor data from a sensor system of a mobile platform; processing the sensor data to detect and characterize disturbances of the mobile platform; receiving user input data indicating user input performed by a user via a touch screen display device in communication with the mobile platform; processing the user input data to determine whether the user input may be invalid due to the disturbance of the mobile platform; and in response to determining that the user input may be invalid due to the disturbance, initiating corrective action, wherein the corrective action includes at least one of confirming the user input, invalidating the user input, or modifying the user input.

[0006] In various embodiments, a system for mitigating user input errors due to disturbances during operation of a mobile platform is provided. The system may include: one or more sensors of the mobile platform, the one or more sensors being configured to sense a measurable condition inside or outside the mobile platform; and a controller capable of operatively communicating with the one or more sensors and with a touch screen display, the touch screen display having a graphical user interface, the graphical user interface including selectable icons. The controller may be configured to: receive sensor data from one or more sensors through one or more processors; process the sensor data to detect and characterize disturbances of the mobile platform; receive user input data indicating a user selection on a touch screen display device; process the user input data to determine whether the user input may be invalid due to a disturbance of the mobile platform; and in response to determining that the user input may be invalid due to a disturbance, initiate corrective action, wherein the corrective action includes at least one of confirming the user input, invalidating the user input, or modifying the user input.

[0007] In various embodiments, a vehicle is provided that may include one or more sensors configured to sense a measurable condition inside or outside the vehicle; and a controller capable of operatively communicating with the one or more sensors and with a touch screen display having a graphical user interface that includes selectable icons. The controller may be configured to: receive sensor data from the one or more sensors via one or more processors; process the sensor data to detect and characterize a disturbance of the vehicle; receive user input data indicating a user input performed by a user via the touch screen display; process the user input data to determine whether the user input may be invalid due to a disturbance of the vehicle; and in response to determining that the user input may be invalid due to a disturbance, initiate corrective action, wherein the corrective action includes at least one of confirming the user input, invalidating the user input, or modifying the user input.

[0008] Furthermore, other desirable features and characteristics of the systems and methods will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing background. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure will now be described in conjunction with the following drawings, wherein like numerals represent like elements, and wherein:

[0010] Figure 1 schematically illustrates components of a mobile platform and an input error mitigation system for the mobile platform according to one embodiment;

[0011] Figure 2 is an example according to one embodiment Figure 1a data flow diagram of the operation of the input error mitigation system;

[0012] Figure 3 is a flow chart illustrating an exemplary method for mitigating user input errors due to disturbances during operation of a mobile platform according to one embodiment; and

[0013] Figure 4 Schematically represents a touch screen display device and illustrates a Figure 3 specific aspects of the method. DETAILED DESCRIPTION

[0014] The following specific embodiments are merely exemplary in nature and are not intended to limit the invention or the application and use of the invention. As used herein, the word "exemplary" means "used as an example, instance or illustration". Therefore, any embodiment described herein as "exemplary" is not necessarily to be understood as being preferred or advantageous over other embodiments. All embodiments described herein are exemplary embodiments provided to enable those skilled in the art to make or use the invention, without limiting the scope of the invention as defined by the claims. In addition, it is not intended to be bound by any express or implied theory presented in the aforementioned technical field, background technology, summary of the invention or the following specific embodiments.

[0015] For the sake of brevity, conventional techniques related to signal processing, data transmission, signaling transmission, control and other functional aspects of the system (and the various operating components of the system) are not described in detail herein. In addition, the connecting lines shown in the various figures included herein are intended to represent exemplary functional relationships and / or physical connections between various elements. It should be noted that many alternative or additional functional relationships or physical connections may exist in the embodiments of the present disclosure.

[0016] The systems and methods disclosed herein are provided to mitigate user input errors caused by disturbances during operation of a mobile platform. For example, a user may attempt to input selections, commands, operations, etc. via physical contact with a touch screen display of a display device on a moving mobile platform (e.g., via a user's hand, stylus, or other similar input device). However, disturbances of the mobile platform may cause the user's hand or input device to move unintentionally and cause the user to fail to complete the intended selection, command, or operation and / or accidentally input a different, unexpected selection, command, or operation. In such examples, the systems and methods herein may detect disturbances, determine whether the disturbances cause user input errors, and, if necessary, mitigate the errors by performing corrective actions. Non-limiting examples of corrective actions may include, but are not limited to, confirming the user's input before executing it, invalidating the user's input, and / or automatically modifying the user's input to the intended selection.

[0017] As used herein, disturbance refers to the disturbance of motion, process, arrangement or equilibrium state. For example, disturbance may include changes in the motion of a mobile platform caused by the control of a pilot, driver, operator, etc., due to environmental factors (e.g., turbulence, road conditions, etc.), due to significant changes in vibration, etc. As used herein, turbulence refers to the irregular movement of air caused by atmospheric vortices and airflow, which causes sudden, unexpected and / or chaotic elevation changes, rolls, pitches and / or yaws of the mobile platform. As used herein, vibration refers to the periodic motion (e.g., oscillation, tremor or trembling motion) of particles of an object or medium from the position of the balance when the balance has been disturbed. In various examples, the disturbance detected by the system and method of this article may be limited to sudden changes in the position of the mobile platform. In such examples, the disturbance detected by the system and method of this article may include sudden and significant changes in vibration, but constant vibrations may generally be excluded. In some examples, the disturbance detected by the system and method of this article may completely exclude vibrations.

[0018] The mobile platform may be any type of vehicle, such as, but not limited to, various types of aircraft. It should be noted that the term aircraft as utilized herein may include any manned object capable of flying. Examples of aircraft may include, but are not limited to, fixed-wing aerial vehicles (e.g., propeller-powered or jet-powered), rotary-wing aerial vehicles (e.g., helicopters), manned aircraft, etc. Other examples of mobile platforms include, but are not limited to, automobiles, ships, trains, construction vehicles, recreational vehicles, and military vehicles.

[0019] Reference now Figure 1 , illustrates an aircraft 10, in this example an airplane, and certain systems thereof, according to an exemplary and non-limiting embodiment of the present disclosure. As described herein, an input error mitigation system 100 may be used on the aircraft 10. Figure 1 Schematically depicted, system 100 includes and / or is functionally coupled to the following components or subsystems, each of which may take the form of a single device or a plurality of interconnected devices, including, but not limited to, a controller 12 capable of being operatively coupled to: at least one display device 32, which may optionally be part of a larger onboard display system 14; a computer-readable storage medium or memory 16; a user interface 18; and onboard data sources 20, including, for example, an array of geospatial and flight parameter sensors 22. System 100 may be separate from or integrated into a flight management system (FMS) and / or a flight control system (FCS). System 100 may also include a communication system 24, which includes an antenna 26, which may wirelessly transmit data to and receive data from various external sources physically and / or geographically remote from system 100 and / or aircraft 10.

[0020] Despite Figure 1 1. Although schematically illustrated as a single unit, the various elements and components of the system 100 may be implemented in a distributed manner using any practical number of physically distinct and operationally interconnected hardware or equipment pieces. When the system 100 is utilized as described herein, the various components of the system 100 are typically located on the aircraft 10.

[0021] As it appears herein, the term "controller" is used in a broad sense to encompass those components used to perform or otherwise support the processing functions of the system 100. Thus, the controller 12 may include or be associated with any number of individual processors, flight control computers, navigation equipment, computer-readable memory (including or in addition to the memory 16), power supplies, storage devices, interface cards, and other standard components.

[0022] In various embodiments, the controller 12 includes at least one processor, a communication bus, and a computer-readable storage device or medium. The processor performs the calculation and control functions of the controller 12. The processor can be any custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 12, a semiconductor-based microprocessor (in the form of a microchip or chipset), any combination thereof, or any device generally used to execute instructions. For example, a computer-readable storage device or medium may include volatile and non-volatile storage devices in a read-only memory (ROM), a random access memory (RAM), and a wear correction factor memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor is powered off. The computer-readable storage device or medium can be implemented using any of many known memory devices, such as PROM (programmable read-only memory), EPROM (electrical PROM), EEPROM (electrically erasable PROM), flash memory, or any other electrical, magnetic, optical, or combined memory device capable of storing data (some of which represent executable instructions, which are used by the controller 12). The bus is used to transfer programs, data, status and other information or signals between the various components of the aircraft 10. The bus can be any suitable physical or logical means of connecting computer systems and components. This includes but is not limited to direct hard-wired connections, fiber optics, infrared and wireless bus technologies.

[0023] The instructions may include one or more independent programs, each of which includes an ordered list of executable instructions for implementing logical functions. When executed by the processor, the instructions receive and process signals from the sensor 22, perform logic, calculations, methods and / or algorithms, and generate data based on the logic, calculations and / or methods. Figure 1Only one controller 12 is shown, but embodiments of the aircraft 10 may include any number of controllers 12 that communicate via any suitable communication medium or combination of communication media and cooperate to process sensor signals, execute logic, calculations, methods and / or algorithms, and generate data. In various embodiments, the controller 12 includes or cooperates with at least one firmware and software program (generally computer-readable instructions embodying algorithms) for performing the various processing tasks, calculations, and control / display functions described herein. During operation, the controller 12 may be programmed with at least one firmware or software program (e.g., program 36) embodying one or more algorithms and execute the at least one firmware or software program to thereby perform the various processing steps, tasks, calculations, and control / display functions described herein.

[0024] In various embodiments, the controller 12 may exchange data with one or more external sources to support the operation of the system 100. In this case, the two-way wireless data exchange may be performed via the communication system 24, such as by a transmission control protocol / Internet protocol architecture or other conventional protocol standards (e.g., Bluetooth TM ) communication network 40 such as a public network or a private network implemented by a computer. Encryption and two-way authentication techniques may be applied as appropriate to ensure data security.

[0025] In various embodiments, the communication system 24 is configured to support instantaneous (i.e., real-time or current) communications between onboard systems, the controller 12, and one or more external sources. The communication system 24 may incorporate one or more transmitters, receivers, and communication-supporting hardware and software necessary for the components of the system 100 to communicate as described herein. In various embodiments, the communication system 24 may have additional communications not directly relied upon herein (such as two-way pilot-to-ATC (air traffic control) communications via a data link), as well as any other suitable radio communication system that supports communications between the aircraft 10 and various external sources.

[0026] The memory 16 may encompass any number and type of storage media suitable for storing computer readable code or instructions, such as the program 36, and other data that generally supports the operation of the system 100. It will be appreciated that the memory 16 may be part of the controller 12, separate from the controller 12, or part of the controller 12 and part of a separate system. The memory 16 may be any suitable type of storage device, including various different types of direct access memory and / or other memory devices.

[0027] Information sources suitable for operating one or more systems of the aircraft 10 may be part of the system 100. In certain embodiments, the source is one or more databases 28 for receiving and storing input error mitigation data, which may be updated on a periodic or iterative basis to ensure data timeliness. In various embodiments, the input error mitigation data may include various instructions configured to detect, quantify, and otherwise characterize sensor data received, for example, from the sensor 22 to detect and characterize disturbances, and may include instructions, criteria, and / or thresholds for various corrective actions that may be performed based on the detected disturbances. The input error mitigation data may be stored in the memory 16 or in one or more databases 28 and referenced by the program 36. In various embodiments, these databases 28 may be available online and remotely accessible by a suitable wireless communication system (such as the communication system 24).

[0028] Sensors 22 supply various types of data and / or measurements to controller 12. In various embodiments, sensors 22 supply (but are not limited to) one or more of the following: inertial reference system measurements providing position, flight path angle (FPA) measurements, airspeed data, ground speed data, vertical velocity data, vertical acceleration data, elevation data, attitude data including pitch and roll measurements, yaw data, data related to deadweight, time / date information, heading information, data related to atmospheric conditions, flight path data, flight trajectory data, radar elevation data, geometric elevation data, wind speed and direction data. In addition, in certain embodiments of system 100, controller 12 and other components of system 100 may be included in or cooperate with any number and type of systems typically deployed on an aircraft, including, for example, FMS, attitude and heading reference system (AHRS), instrument landing system (ILS), and / or inertial reference system (IRS). In various embodiments, sensors 22 may include vertical accelerometers, longitudinal accelerometers, annular accelerometers, vibration indicators, or combinations thereof.

[0029] Continue to refer Figure 1 , the display device 32 may include any number and type of image generating devices on which one or more avionics displays 34 may be generated. In various embodiments, the display device 32 may be attached to a static structure of the cockpit of the aircraft 10 as, for example, a head-down display (HDD). Alternatively, the display device 32 may take the form of a movable display device (e.g., a display device worn by the pilot) or a portable display device, such as an electronic flight bag (EFB), a laptop computer, or a tablet computer carried by the pilot into the cockpit of the aircraft 10.

[0030] During operation of the system 100, at least one avionics display 34 is generated on the display device 32. As used herein, the term "avionics display" is synonymous with the terms "aircraft-related display" and "cockpit display" and encompasses displays generated in text, graphics, cartography, and other formats. The system 100 may generate various types of lateral and vertical avionics displays 34 on which symbology, textual notices, and other graphics related to the flight plan are presented for viewing by the pilot. The display device 32 is configured to continuously reproduce at least one avionics display 34. The avionics display 34 generated and controlled by the system 100 may include an alphanumeric input display of the type typically presented on a multifunction control and display unit (MCDU) and a screen of a control display unit (CDU) in general. Specifically, certain embodiments of the avionics display 34 include one or more two-dimensional (2D) avionics displays, such as a horizontal (i.e., lateral) navigation display or a vertical navigation display; and / or one or more three-dimensional (3D) avionics displays, such as a primary flight display (PFD) or a centrifugal 3D avionics display.

[0031] In various embodiments, the human-machine interface (such as a touch screen display) is implemented as an integration of the user interface 18 and the display device 32. Via various display and graphics system processes, the controller 12 can command and control the touch screen display to generate a variety of graphical user interface (GUI) objects or elements (e.g., buttons, sliders, etc.) that are used to prompt the user to interact with the human-machine interface to provide user input, and to activate corresponding functions and provide user feedback in response to user input received at the GUI elements. In such embodiments, the display device 32 may include a touch sensor 37 and / or a touch sensing system configured to sense user input (e.g., contact between the screen of the display device 32 and a user's finger or stylus).

[0032] In various embodiments, the communication system 24 may be configured to communicate with one or more devices such as Figure 1 42 is a remote device such as the remote device 42 represented in . In some embodiments, the remote device 42 may include a display system including a display device configured to generate a touch screen display, for example, as previously described for the display 34 of the display device 32. In various embodiments, the remote device 42 may take the form of a removable display device (e.g., a display device worn by a pilot) or a portable display device such as a laptop or tablet computer, a smart phone, a smart watch, etc. In such embodiments, the remote device 42 may include a touch sensor 47 and / or a touch sensing system configured to sense user input (e.g., contact between the screen of the remote device 42 and a finger or stylus of the user).

[0033] In various embodiments, the display device 32 and / or the remote device 42 may include orientation sensors 38 and 48, respectively. The orientation sensors 38 and 48 may be configured to sense the orientation of the corresponding display device 32 or 42 relative to a reference system (e.g., a fixed three-dimensional Cartesian coordinate system). The orientation sensors 38 and 48 may include various types of sensors or combinations of sensors.

[0034] refer to Figure 2 And continue to refer to Figure 1 , a data flow diagram illustrating the Figure 1 100. It will be appreciated that various embodiments of the system 100 according to the present disclosure may include any number of modules embedded within the controller 12, which may be combined and / or further divided to implement the systems and methods described herein in a similar manner. In addition, inputs to the system 100 may be received from other control modules (not shown) associated with the aircraft 10 and / or determined / modeled by other sub-modules (not shown) within the controller 12. In addition, it is also possible to pre-process the inputs, such as sub-sampling, noise reduction, normalization, feature extraction, missing data reduction, etc. In various embodiments, the system 100 includes a disturbance module 110 and a correction module 112.

[0035] In various embodiments, the disturbance module 110 receives as input sensor data 114 generated by the sensors 22. The sensor data 114 includes various data indicating sensed measurable conditions inside and / or outside the aircraft 10, such as, but not limited to, altitude, velocity, roll, pitch, and yaw measurements. The disturbance module 110 processes the sensor data 114, detects disturbances, and determines characteristics of the detected disturbances, such as their magnitude, direction, and timing. The disturbance module 110 generates disturbance data 118, which includes various data indicating the determined characteristics of the detected disturbances.

[0036] In various embodiments, the correction module 112 receives as input the disturbance data 118 generated by the disturbance module 110. The correction module 112 also receives as input the user input data 116 generated by the user interface 18 (e.g., in response to a user interacting with a GUI of a touch screen display). The user input data 116 includes various data indicative of user input (e.g., one or more selections, operations, or commands entered by a user). The correction module 112 processes the disturbance data 118 and the user input data 116 to determine the likelihood that the detected disturbance has affected the user input to an extent sufficient to invalidate the user input. For example, Figure 4An exemplary display device 300 is presented, which includes a touch screen display 310 on which a GUI including selectable icons 312, 314, 316, 318, 320, and 322 is rendered. In this non-limiting example, a user may interact with the display device 300 while traveling in a mobile platform such as an aircraft 10. While attempting to select icon 312, a disturbance may cause the user's hand or input device to inadvertently move, causing the user to accidentally contact (e.g., physically touch) the touch screen display 310 at a first location 324. While processing the disturbance data 118 and the user input data 116, the correction module 112 may determine that the disturbance significantly affects the user input. For example, the correction module 112 may compare the characteristics of the disturbance to disturbance characteristic criteria (e.g., minimum magnitude, direction, etc.) and / or may compare the difference in timing of the disturbance and the user input made using the touch screen display to disturbance timing criteria (e.g., within 0.5 seconds). In some embodiments, in response to analysis of the disturbance data 118 and / or the user input data 116 indicating that pre-programmed disturbance characteristic criteria and / or disturbance timing criteria are met, the remediation module 112 may determine that the user input may be invalid.

[0037] Correction module 112 may determine the offset between the actual contact position of the user and the expected contact position. Figure 4 , correction module 112 can determine that the actual contact at first location 324 is offset by vectorization increment 328, and thereby determine that the expected contact location is at second location 326. Correction module 112 can use various methods to determine the offset and / or vectorization increment, such as, but not limited to, various machine learning techniques, touch location frequency, display context information, previous touch activity, and user-specific touch patterns.

[0038] In some embodiments, the correction module 112 may consider the expected movement of the user's hand and / or arm based on the range required to contact the touch screen display. For example, when the user is interacting with a first touch screen display close to the user, the disturbance affects the user input to a lesser extent. In contrast, when the user is interacting with a second touch screen farther from the user, the same disturbance may affect the user input to a greater extent. Various methods can be used to perform such considerations. In some embodiments, the correction module 112 may model the movement based on the expected range from a predetermined user position such as a specific standing position or a specific seat of the aircraft 10 to a known physical position of the touch screen display. In other embodiments, the user's position may be sensed by one or more sensors (e.g., sensor 22). In other embodiments, one or more sensors (e.g., sensor 22) may be used to sense the movement of the user's hand and / or arm. In various embodiments, one or more sensors (e.g., sensor 22) may be used to sense whether the user stabilizes his hand and / or arm at the moment of contact with the touch screen display (e.g., resting the palm on the display device, detecting whether the finger is wrapped around the edge of the display device, etc.). In such embodiments, the correction module 112 can reduce the likelihood that the user input data 116 is invalid due to the disturbance, since the disturbance is expected to be at least partially mitigated by stabilization.

[0039] In various embodiments, the correction module 112 may take into account the orientation of the touch screen display. For example, if the disturbance includes a drop in elevation (e.g., in the -z direction), the disturbance may significantly affect the user's contact with a touch screen display oriented at, for example, a forty-five degree angle by causing the user to contact the touch screen at a position lower than expected. However, the same drop in elevation may have little or no effect on the user's contact with a touch screen display oriented horizontally because the user's direction of motion is in the same direction as the disturbance. However, the drop in elevation may affect the user's contact by causing an erroneous double-click instead of a single click, or by lifting prematurely during a drag operation. In various embodiments, the orientation of the touch screen display may be determined by an orientation sensor (e.g., orientation sensor 38 or 48) of a display device that generates the touch screen display.

[0040] In response to determining that the disturbance significantly affects the user input data 116, the correction module 112 may generate correction data 120, which includes various data configured to initiate one or more corrective actions. Various corrective actions may be initiated by the correction module 112. In various examples, the corrective action may include generating a visual prompt on a display (e.g., a display 34 of a display device 32, a display of a remote device 42, and / or a touch screen display 310) requesting confirmation of the user input. In various examples, the corrective action may include invalidating the user input. In various examples where the user input is unrelated to safety-critical operations, the corrective action may include modifying the user input based on the disturbance and pre-programmed criteria. For example, the corrective action may include adjusting the user input from an actual contact position (e.g., a first position 324) to an expected contact position (e.g., a second position 326). In such examples, the corrective action may modify the user input based at least in part on the magnitude of the disturbance, the frequency of the disturbance, the timing of the disturbance relative to the user input, the orientation of the touch screen display, and / or the direction of the disturbance. In various embodiments, corrective actions can be configured to mitigate user input errors (eg, automatically change the actual contact location to the expected contact location) without changing the GUI.

[0041] In various embodiments, the correction module 112 may determine that the operation or command that the user intended to perform was interrupted. The correction module 112 may process the user input data 116 to determine that the user input is a first type of operation or command, determine that the user intended to perform a second type of operation or command based on the disturbance, and then change the user input from the first type of operation or command to the second type of operation or command.

[0042] In some embodiments, the correction module 112 may determine that the disturbance caused more contact between the user and the touch screen display than the user intended based on the analysis results of the disturbance data 118 and the user input data 116. For example, the user may expect to enter a selection of a selectable icon by performing a single click on the touch screen display; however, due to the unexpected disturbance, the user may have entered a double click or a swipe gesture, which may result in an erroneous selection or incomplete selection. In such embodiments, the correction module 112 may change the double click or swipe gesture to a single click, thereby ultimately completing the intended selection.

[0043] In some embodiments, the correction module 112 may determine that the disturbance caused less contact between the user and the touch screen display than the user intended based on the analysis results of the disturbance data 118 and the user input data 116. For example, the user may have intended to perform a drag operation in which the user slides a finger on the touch screen display to, for example, change the position of a screen graphic reproduced on the touch screen display; however, due to the unintended disturbance, the user may temporarily lose contact with the touch screen display during the drag operation, causing the user input data 116 to include one, two, or more partial drag operations. In such embodiments, the correction module 112 may complete or combine the partial drag operations to generate a continuous, uninterrupted drag operation as intended.

[0044] The correction module 112 may send the correction data 120 to other systems of the aircraft 10 , for example, to communicate with a user, implement user input, and / or modify user input.

[0045] The systems disclosed herein, including system 100, provide methods for mitigating user input errors due to disturbances during operation of a mobile platform. For example, Figure 3 2 is a flow chart illustrating an exemplary method 200. It is understood from the present disclosure that the order of operations in the method 200 is not limited to the order in which the operations are performed. Figure 3 The method 200 is not performed in the order illustrated, but may be performed in one or more altered orders when applicable and in accordance with the present disclosure. The method 200 may begin at 210. In various embodiments, the method 200 may be scheduled to run based on one or more predetermined events and / or may run continuously during operation of the mobile platform.

[0046] At 212, method 200 may include receiving, by one or more processors (e.g., associated with controller 12), sensor data from a sensor system of a mobile platform (e.g., sensor 22 of aircraft 10). At 214, method 200 may include processing, by the one or more processors, the sensor data to detect and characterize a disturbance of the mobile platform.

[0047] At 216, method 200 may include receiving, by one or more processors, user input data indicating user input made by a user on a touch screen display in communication with the vehicle (e.g., display device 32 or remote device 42). In some examples, the sensor system of the vehicle is external to the display device that generates the touch screen display.

[0048] At 218 , method 200 may include analyzing, by one or more processors, the disturbance and user input data to determine whether the user input may be invalid due to the disturbance of the vehicle.

[0049] At 220 , method 200 may include initiating, by the one or more processors, corrective action in response to determining that the user input may be invalid due to a disturbance.

[0050] Method 200 may end at 222 .

[0051] The systems and methods disclosed herein provide various benefits over certain existing systems and methods. For example, disturbances that occur during use of a touch screen display of a mobile platform may cause a user to accidentally input an unexpected selection, operation, and / or command, fail to input a selection, operation, and / or command, or fail to complete a selection, operation, and / or command. The systems and methods disclosed herein detect disturbances and initiate corrective actions to verify and / or modify user input so that the selection and / or command is executed and / or completed as expected. Therefore, the systems and methods disclosed herein achieve improvements in the technical field of touch screen displays for mobile platforms.

[0052] It will be understood by those skilled in the art that the various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software or a combination of the two. Some embodiments and specific implementations are described above according to functions and / or logic block components (or modules) and various processing steps. However, it should be understood that such block components (or modules) can be implemented by any number of hardware, software and / or firmware components configured to perform a specified function. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been described above in general according to their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician can implement the described functions in different ways for each specific application, but such specific implementation decisions should not be interpreted as causing deviations from the scope of the present invention. For example, the embodiments of the system or components can use various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc., which can perform multiple functions under the control of one or more microprocessors or other control devices. In addition, it will be understood by those skilled in the art that the embodiments described herein are only exemplary specific implementations.

[0053] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0054] The steps of the methods and algorithms described in conjunction with the embodiments disclosed herein may be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and the storage medium may reside in an ASIC.

[0055] Skills and techniques may be described herein according to functions and / or logic block components and reference to symbolic representations of operations, processing tasks and functions that may be performed by various computing components or devices. Such operations, tasks and functions are sometimes referred to as computer-implemented, computerized, software-implemented or computer-implemented. In practice, one or more processor devices may perform the operations, tasks and functions by controlling electrical signals and other processing signals representing data bits at memory locations in a system memory. The memory location for holding data bits is a physical location having specific electrical properties, magnetic properties, optical properties or organic properties corresponding to the data bits. It should be understood that the various block components shown in the figure may be implemented by any number of hardware, software and / or firmware components configured to perform a specified function. For example, the embodiment of a system or component may use various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc., which may perform multiple functions under the control of one or more microprocessors or other control devices.

[0056] When implemented in software or firmware, the various elements of the system described herein are basically code segments or instructions that perform various tasks. A program or code segment may be stored in a processor-readable medium, or transmitted through a transmission medium or communication path by a computer data signal contained in a carrier. "Computer-readable medium", "processor-readable medium" or "machine-readable medium" may include any medium that can store or transmit information. Examples of processor-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memory, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Computer data signals may include any signal that can be propagated on a transmission medium such as an electronic network channel, optical fiber, air, electromagnetic path or RF link. Code segments may be downloaded via computer networks such as the Internet, intranets, LANs, etc.

[0057] Some of the functional units described in this specification have been referred to as "modules" in order to more specifically emphasize their specific implementation independence. For example, the functions referred to as modules herein may be implemented in whole or in part as hardware circuits, which include custom VLSI circuits or gate arrays, finished semiconductors such as logic chips, transistors or other separate components. Modules may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc. Modules may also be implemented in software for execution by various types of processors. The identification module of the executable code may, for example, include one or more physical or logical modules of computer instructions, which may, for example, be organized as objects, processes or functions. However, the executable files of the identified modules do not need to be physically located together, but may include different instructions stored at different locations, which include modules and implement the purpose of the modules when they are logically connected together. In fact, the module of the executable code may be a single instruction or many instructions, and may even be distributed over multiple different code segments, between different programs, and across several memory devices. Similarly, the operating data can be implemented in any suitable form and can be organized in any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations, including on different storage devices and may exist, at least in part, merely as electronic signals on a system or network.

[0058] In this document, relational terms such as first and second, etc., may be used only to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Unless explicitly defined by the claim language, numerical ordinals such as "first", "second", "third", etc., only represent different individuals in a plurality, and do not imply any order or sequence. Unless explicitly defined by the claim language, the sequence of text in any claim does not imply that the processing steps must be performed in a temporal or logical order according to such sequence. Without departing from the scope of the present invention, the method steps may be interchanged in any order, as long as such interchange does not contradict the claim language and is not logically absurd.

[0059] In addition, depending on the context, words such as "connected" or "coupled to" used when describing the relationship between different elements do not imply that a direct physical connection must be made between these elements. For example, two elements may be connected to each other physically, electronically, logically or in any other way through one or more additional elements.

[0060] As used herein, the term "substantially" means within 5% taking into account manufacturing tolerances. In addition, as used herein, the term "approximately" means within 5% taking into account manufacturing tolerances.

[0061] Although at least one exemplary embodiment has been presented in the aforementioned specific embodiments of the present invention, it should be understood that there are a large number of variations. It should also be understood that an exemplary embodiment or multiple exemplary embodiments are only examples and are not intended to limit the scope, applicability or construction of the present invention in any way. On the contrary, the aforementioned specific embodiments will provide a convenient roadmap for implementing the exemplary embodiments of the present invention for those skilled in the art. It should be understood that, without departing from the scope of the present invention as set forth in the appended claims, various changes may be made to the functions and arrangements of the elements described in the exemplary embodiments.

Claims

1. A method comprising: receiving, by one or more processors, sensor data from a sensor system of a mobile platform; processing the sensor data by the one or more processors to detect and characterize disturbances of the mobile platform; receiving, by the one or more processors, user input data indicative of user input provided by a user via a touch screen display in communication with the mobile platform; analyzing, by the one or more processors, the perturbation and the user input to determine whether the user input may be invalid due to the perturbation of the mobile platform; as well as In response to determining that the user input may be invalid due to the disturbance, corrective action is initiated by the one or more processors, wherein the corrective action includes at least one of confirming the user input, invalidating the user input, or modifying the user input.

2. The method of claim 1 , wherein performing the corrective action comprises modifying, by the one or more processors, the user input based on the disturbance and a programmed criterion, wherein modifying the user input is based at least in part on a magnitude of the disturbance, a frequency of the disturbance, a timing of the disturbance relative to the user input, an orientation of the touch screen display, and / or a direction of the disturbance, wherein modifying the user input comprises: processing, by the one or more processors, the user input data to determine an actual contact location on the touch screen display; determining, by the one or more processors, an expected contact location on the touch screen display based on the disturbance; as well as The user input is adjusted, by the one or more processors, from the actual contact location to the expected contact location.

3. The method of claim 1 , wherein performing the corrective action comprises modifying, by the one or more processors, the user input based on the disturbance and the programmed criteria, wherein the programmed criteria is configured to take into account a reachable distance between a physical location of the touch screen display and the user.

4. The method according to claim 1, further comprising: processing, by the one or more processors, the user input data to determine that the user input comprises a swipe operation or command or a double-tap operation or command; determining, by the one or more processors based on the disturbance, that the user intends to perform a single-click operation or command; as well as The user input is changed, by the one or more processors, from the swipe operation or command or the double-click operation or command to the single-click operation or command.

5. The method according to claim 1, further comprising: determining, by the one or more processors, that the user input comprises a first drag operation in which the user performs a sliding contact on the touch screen display; receiving, by the one or more processors, additional user input data after receiving the user input data indicative of a second drag operation; determining, by the one or more processors based on the disturbance, that an unexpected interruption occurred between receiving the user input data and receiving the additional user input data; as well as The user input data and the additional user input data are combined, by the one or more processors, to generate a continuous, uninterrupted drag operation.

6. A system comprising: one or more sensors of a mobile platform, the one or more sensors configured to sense a measurable condition internal or external to the mobile platform; and a controller capable of operatively communicating with the one or more sensors and with a touch screen display having a graphical user interface including selectable icons, the controller being configured, via one or more processors, to: receiving sensor data from the one or more sensors; processing the sensor data to detect and characterize disturbances of the mobile platform; receiving user input data indicative of user input provided by a user via the touch screen display; analyzing the disturbance and the user input to determine whether the user input may be invalidated due to the disturbance of the mobile platform; as well as In response to determining that the user input may be invalid due to the disturbance, corrective action is initiated, wherein the corrective action includes at least one of confirming the user input, invalidating the user input, or modifying the user input.

7. The system of claim 6, wherein the controller is configured to modify, via the one or more processors, the user input based on the disturbance and the programmed criteria, wherein the controller is configured to modify, via the one or more processors, the user input based at least in part on a magnitude of the disturbance, a frequency of the disturbance, a timing of the disturbance relative to the user input, an orientation of the touch screen display, and / or a direction of the disturbance, wherein the controller is configured to modify, via the one or more processors, the user input by: processing the user input data to determine an actual contact location on the touch screen display; determining an expected contact location on the touch screen display based on the disturbance; as well as The user input is adjusted from the actual contact location to the expected contact location.

8. The system of claim 6, wherein the controller is configured to modify the user input via the one or more processors based on the disturbance and the programmed criteria, wherein the programmed criteria is configured to take into account a relative physical location of the touch screen display and an accessible distance between the user.

9. The system of claim 6, wherein the controller is configured, via the one or more processors: Determining that the user input includes a swipe operation or command or a double-click operation or command; Determining, based on the disturbance, that the user intends to perform a single-click operation or command; and The user input is changed from the swipe operation or command or the double-click operation or command to the single-click operation or command.

10. The system of claim 6, wherein the controller is configured, via the one or more processors: processing the user input data to determine that the user input comprises a first drag operation or command in which the user performs a sliding contact on the touch screen display; receiving additional user input data after receiving said user input data indicating a second drag operation or command; determining, based on the disturbance, that an unexpected interruption occurred between receiving the user input data and receiving the additional user input data; as well as The user input data and the additional user input data are combined to generate a continuous, uninterrupted drag operation.

Images