An aviation cockpit multi-point touch interaction method, device and medium

Abstract

The application discloses a kind of for aviation cockpit multi-point touch interaction method, equipment and medium, the method includes, receiving touch event of touch display unit, determine source seat, target object identification and touch coordinates, query session state table to judge whether there is activated active session;If there is, calculate seat, object, time three kinds of relationship quantity, according to touch event is classified as homologous continuous, heterogenous check, heterogenous relay or heterogenous conflict input, then combined with the risk level of active session corresponding differential processing operation is executed, and session state table is updated.The method realizes the accurate identification and control of multi-source touch input in two-person system collaborative scene, improves the accuracy and safety of touch interaction, adapts to the actual application demand of avionics display system of aviation cockpit.

Description

Technical Field

[0001] This application relates to the field of avionics display and human-computer interaction control technology, and more specifically, to a method, device and medium for multi-touch interaction in an aircraft cockpit. Background Technology

[0002] The development of avionics display and human-machine interaction control technology has led to the widespread application of multi-touch technology in aircraft cockpit display systems, providing a convenient interaction method for two-person collaborative operation. Pilots in the left and right seats of the cockpit can operate their own dedicated display units, while the application of shared display areas or linked pages further enhances operational collaboration.

[0003] Existing multi-touch recognition methods are all designed based on single-user terminal usage scenarios, assuming that multiple touch points at the same time come from the same operator. They often interpret multi-touch point combinations as single-person multi-finger gestures such as zooming and dragging. When these methods are directly applied to the two-person collaborative operation scenario in an aircraft cockpit, many technical defects are exposed.

[0004] When pilots in the left and right seats touch the same area one after the other or simultaneously, the system is prone to misinterpreting the touches of two people as a single person's multi-finger gesture, resulting in incorrect recognition of operation commands. When one pilot initiates a high-risk parameter modification operation, the auxiliary touches of another pilot for cross-checking are easily identified as overwriting inputs, interfering with the normal operation process. For concurrent touches on shared display areas or linked pages, existing systems mostly adopt a first-come, first-served or last-input overwriting approach, which can easily lead to high-risk interactive objects being submitted incorrectly or normal operation sessions being abnormally interrupted, posing safety hazards to aviation pilots. Summary of the Invention

[0005] This application provides a method, device, and medium for multi-touch interaction in an aircraft cockpit, to at least solve the technical problems existing in the related technologies described above.

[0006] According to a first aspect of the embodiments of this application, a method for multi-touch interaction in an aircraft cockpit is provided, comprising:

[0007] Receive the current touch event from the touch display unit, and determine the source seat, target object identifier, and touch point coordinates based on the current touch event;

[0008] The pre-stored session status table is queried to determine whether there is an active session; the active session includes at least the initiating seat, target object identifier, risk level, and locked area;

[0009] If the active session exists, then the seat relationship quantity, object relationship quantity, and time relationship quantity are calculated based on the current touch event and the active session; the seat relationship quantity is used to indicate whether the source seat and the initiating seat are from the same source, the object relationship quantity is used to indicate the relationship between the target object identifier and the target object identifier of the active session, and the time relationship quantity is used to indicate whether the time difference between the trigger time of the current touch event and the most recent update time of the active session is within a preset threshold.

[0010] Based on the seat relationship quantity, the object relationship quantity, and the time relationship quantity, the current touch event is classified into continuous input from the same source, cross-source verification input, cross-source relay input, or cross-source conflict input;

[0011] Based on the classification result of the current touch event and the risk level of the active session, perform the processing operation corresponding to the classification and risk level, and update the session state table.

[0012] As an optional approach, calculating the seat relationship, object relationship, and time relationship based on the current touch event and the active session includes:

[0013] Match the source seat with the initiating seat. If they match, the seat relationship is from the same source; otherwise, they are from different sources.

[0014] The target object identifier is matched with the target object identifier of the active session. If they match, the object relationship is considered to be the same object. If they do not match, it is determined whether the touch point coordinates are located within the locked area. If they are, they are adjacent objects; otherwise, they are unrelated objects.

[0015] Calculate the time difference between the trigger time and the most recent update time. If the time difference is less than or equal to the preset threshold, the time relationship quantity is within the session continuous time window; otherwise, it is outside the session continuous time window.

[0016] As an optional approach, the current touch event can be categorized as follows:

[0017] If the seat relationship quantity is from the same source and the time relationship quantity is within a continuous session time window, then it is classified as the same source continuous input;

[0018] If the seat relationship quantity is heterogeneous, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within the continuous time window of the session, then it is classified as heterogeneous verification input;

[0019] If the seat relationship quantity is heterogeneous, the session status of the active session is pending confirmation or exclusive lock status, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within the continuous time window of the session, then it is classified as heterogeneous relay input.

[0020] If the seat relationship quantity is heterogeneous and does not meet the classification conditions of heterogeneous verification input and heterogeneous relay input, then it is classified as heterogeneous conflict input.

[0021] As an optional solution, the determination conditions for the heterogeneous core input also include: the contact duration of the current touch event is less than or equal to a first preset threshold, and the movement distance is less than or equal to a second preset threshold;

[0022] The determination criteria for heterogeneous relay input also include: the object type corresponding to the current touch event and the target object identifier of the active session have a relay correspondence in the preset page object relationship table.

[0023] As an optional approach, the processing steps corresponding to this classification and risk level include:

[0024] If the current touch event is classified as continuous input from the same source, different processing is performed according to the risk level of the active session:

[0025] If the risk level is low, an execution instruction is generated and sent, and the activity session is terminated;

[0026] If the risk level is medium risk, update the pre-submitted value and keep the activity session in a pending confirmation state;

[0027] If the risk level is high, the active session will remain in an exclusive locked state, and subsequent input from the source seat will be restricted to the locked area only.

[0028] As an optional approach, the processing steps corresponding to this classification and risk level include:

[0029] If the current touch event is classified as heterogeneous verification input, and the active session is in a pending confirmation state or an exclusive lock state, then the cross-verification flag of the active session is updated, and the verification position is recorded.

[0030] If the risk level of the activity session is medium risk and there is a pre-submitted value, then an execution instruction is generated and sent according to the pre-submitted value, and the activity session is terminated.

[0031] If the risk level of the activity session is high risk, then the session status of the activity session will be updated to the verified pending release status.

[0032] As an optional approach, the processing steps corresponding to this classification and risk level include:

[0033] If the current touch event is classified as a heterogeneous relay input, and the active session has a pre-submitted value, then a relay confirmation record is generated;

[0034] The pre-submitted value is bound to the activity session to generate and send the final execution instruction;

[0035] Release the resources occupied by the active session and set the session state to terminated.

[0036] As an optional approach, the processing steps corresponding to this classification and risk level include:

[0037] If the current touch event is classified as a heterogeneous conflict input, different processing is performed according to the risk level of the active session:

[0038] If the risk level is low, the current active session ends and the current touch event is reprocessed as a new touch input;

[0039] If the risk level is medium risk, the session status of the active session will be updated to conflict freeze state, and foreign inputs entering the locked area will be ignored during the freeze period.

[0040] If the risk level is high, the current touch event is rejected, the exclusive lock state of the active session remains unchanged, and a conflict log is generated.

[0041] According to a second aspect of the embodiments of this application, an electronic device is provided, including: a processor; a memory for storing a computer program executable by the processor; wherein the processor is configured to execute the computer program in the memory to implement the method described in the first aspect.

[0042] According to a third aspect of the embodiments of this application, a computer-readable storage medium is provided, which, when an executable computer program in the storage medium is executed by a processor, enables the implementation of the method described in the first aspect.

[0043] This application calculates the relationship between touch input in terms of seat, object, and time dimensions to accurately classify input types. Combined with the risk level of the interactive object, it implements a hierarchical control strategy, which realizes the smooth continuation of continuous operations from the same source, the effective identification of cross-source verification operations, the coherent execution of cross-source relay operations, and the reasonable handling of cross-source conflict operations. This allows high-risk interactive objects to form a controllable and traceable processing mechanism, avoiding the accidental submission of high-risk objects and abnormal interruption of sessions. It significantly improves the accuracy, security, and collaboration of multi-touch interaction in aircraft cockpits and is suitable for the actual application needs of avionics display systems with two-person collaborative operation.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Furthermore, no embodiment in this disclosure is required to achieve all the effects described above. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0046] Figure 1 This is a schematic diagram of a multi-touch interaction method for an aircraft cockpit, provided as an embodiment of the present disclosure.

[0047] Figure 2 This is a schematic diagram of the relational quantity calculation process provided in the embodiments of this disclosure.

[0048] Figure 3 This is a schematic diagram of the input type classification process provided in the embodiments of this disclosure.

[0049] Figure 4 This is a schematic diagram of the differentiated processing flow provided in the embodiments of this disclosure.

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0052] The method of this embodiment is adapted to aircraft cockpit display systems equipped with touch display units, and is particularly suitable for avionics display systems where the left and right pilots can operate their respective display units, and at least one display unit shares a display area or has a page linkage relationship. The execution subject of this method is a cockpit touch interaction processing device, which can be integrated into the display management computer, cockpit display processor, or touch middleware processing unit. It establishes a data interaction connection with the touch screen controller, page rendering engine, and avionics application logic module. No additional peripherals such as cameras or biometrics are required. The entire touch interaction processing flow is completed only through the touch input data and page context data collected by the existing display system. The device can receive touch event streams, page object description data, display unit ownership information, and the current interaction session status, and output arbitrated valid interaction commands, verification confirmation results, conflict freeze results, or session release results.

[0053] The implementation process of the method described in this application will be described in detail below with reference to specific embodiments. It should be noted that this embodiment is only used to explain this application and is not intended to limit the scope of protection of this application. Conventional adjustments or substitutions of each step by those skilled in the art without departing from the concept of this application should be included in the scope of protection of this application.

[0054] Specifically, Figure 1 This is a flowchart of a multi-touch interaction method for an aircraft cockpit according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes steps S1-S5:

[0055] In step S1, a current touch event is received from the touch display unit, and the source seat, target object identifier, and touch point coordinates are determined based on the current touch event.

[0056] The touch interaction processing device receives current touch events from various touch display units in the cockpit, transmitted in real time by the touch screen controller. Each touch event carries basic data such as display unit identifier, touch point coordinates, trigger time, contact duration, movement distance, and event type. Upon receiving a current touch event, the device first parses the data, extracts the aforementioned basic data, and performs validity checks. If the data is missing or incorrect, the touch event is deemed invalid and discarded without further processing. If the data is valid, the source location and target object identifier are determined sequentially. The specific implementation process is as follows:

[0057] The device uses the parsed display unit identifier as the retrieval basis and performs precise matching in a pre-loaded display unit attribution table. This display unit attribution table is stored in a key-value pair structure, where the key is the display unit identifier and the value is the seat identifier. The seat identifier includes only three categories: left seat, right seat, and shared area. After matching, the seat identifier corresponding to the current touch event is obtained, thus determining the source seat of the touch input. For example, if the touch event comes from the dedicated display unit of the left-seat pilot, the source seat is determined to be the left seat; if it comes from the shared display area, the source seat is determined to be the shared area.

[0058] Subsequently, the device submits the parsed touch point coordinates to the page rendering engine, which then performs a hit test. The hit test uses a layer-by-layer traversal of the page object tree, searching sequentially from the top-level visible object to the bottom-level object in Z-order until the first interactive object containing the touch point coordinates is found. If an interactive object is found, the page rendering engine sends the target object identifier of that object back to the touch interaction processing device; if no interactive object is found, the target object identifier is set to null.

[0059] In actual implementation, the reception and parsing of touch events is a real-time process. The overall time taken from receiving a touch event to determining the source location, target object identifier, and touch point coordinates can be configured to not exceed a preset time threshold to meet the real-time requirements of touch interaction in aircraft cockpits. Simultaneously, the device temporarily stores the determined source location, target object identifier, touch point coordinates, and basic touch event data.

[0060] In step S2, the pre-stored session status table is queried to determine whether there is an active session; the active session includes at least the initiating seat, target object identifier, risk level, and locked area.

[0061] Specifically, after determining the source location, target object identifier, and touch point coordinates of the current touch event, the touch interaction processing device immediately queries the session state table pre-stored locally. This session state table is a dynamic data table generated during device operation, used to record session information during the cockpit touch interaction process. The active sessions stored in it are touch interaction sessions in an active state. Each active session contains at least four core types of information: initiating location, target object identifier, risk level, and locked area. It also contains auxiliary information such as session identifier, session state, start time, last update time, pre-submitted value, and cross-check flags, providing a basis for the entire process of touch interaction processing. Among them, the pre-submitted value is a value to be confirmed generated by the avionics application logic module based on the touch trajectory or key selection result when the pilot performs a touch operation on a medium-to-high-risk interaction object in the cockpit touch interaction. It is temporarily stored in the active session as a numerical basis for subsequent cross-source verification and relay operations. After confirmation, it can be converted into a formal value and trigger the corresponding execution command.

[0062] The device uses a full table search to query the session status table, with the session status field as the core of the search. This field is used to identify whether an active session is in an active state. If there is a record of an active session in the session status table, it is determined that there is an active session. The device will extract core information such as the initiating seat, target object identifier, risk level, locked area, and last update time of the active session to provide basic data for subsequent relational calculation steps. If there is no record of an active session in the session status table, or if the session status table is empty, it is determined that there is no active session. The device will then execute the subsequent session establishment steps to construct a new active session based on the relevant data of the current touch event.

[0063] Optionally, the session state table adopts a storage structure of single active session plus candidate session. Only one active session exists at any given time, avoiding touch interaction chaos caused by multiple active sessions existing simultaneously. The candidate session is used to temporarily store session information to be activated. Only after the current active session ends can the candidate session be activated as a new active session based on the actual touch input.

[0064] In some embodiments, when the touch interaction processing device determines that there is no active session currently in an active state, it first determines whether the identified target object identifier is null. If the target object identifier is null, it means that the current touch event has not hit any interactive object and does not meet the conditions for establishing a new active session. The device directly discards the touch event and does not perform any subsequent processing operations. If the target object identifier is not null, it means that the current touch event has hit a valid interactive object and meets the conditions for establishing a new active session. The device then sequentially completes operations such as risk level query, new active session generation, and locked area generation, stores the new active session in the session state table, and sets it to an active state. The specific activation implementation process is as follows:

[0065] The device uses the target object identifier as the retrieval basis and performs precise matching in the page object risk level table. This risk level table is sent to the touch interaction processing device in real time by the avionics application when the page loads. It is stored in a record array structure, with each record containing the object identifier, object type, page identifier, and risk level. The risk levels are uniformly divided into three levels: low risk, medium risk, and high risk. After matching, the risk level corresponding to the target object identifier is obtained. This risk level will be used as the basis for assigning the risk level of the new activity session and will be directly written into the risk level field of the new activity session. Specifically, the avionics application assigns a risk level to each interactive page object when the page loads and sends it to the touch interaction processing device, uniformly classifying them into three levels: low, medium, and high according to the degree of impact of the operation on aviation safety and the severity of the consequences of the operation.

[0066] The device generates a unique session identifier according to a preset encoding rule. This encoding rule can be configured to include a combination of timestamp, source seat, and random number to ensure the uniqueness of the session identifier and avoid duplication with historical session identifiers. Subsequently, the device constructs a new active session based on this session identifier, assigning the source seat to the initiating seat field of the new active session, and assigning the target object identifier and the queried risk level to the corresponding fields. At the same time, it records the trigger time of the current touch event as the start time and most recent update time of the new active session.

[0067] The device configures a corresponding initial state for new activity sessions based on the risk level. The configuration rules for the initial state are deterministic and correspond one-to-one with the risk level: if the risk level is low, the initial state is configured as "direct execution"; if the risk level is medium, the initial state is configured as "pending confirmation"; and if the risk level is high, the initial state is configured as "exclusive lock". The initial state of a new activity session directly determines the subsequent touch interaction processing flow, and activity sessions with different initial states will perform differentiated processing operations.

[0068] If the initial state of a new activity session is direct execution, the device will immediately generate a corresponding execution command based on the current touch event. This execution command includes core information such as session identifier, target object identifier, and operation type. The device will send the execution command to the avionics application logic module, which will then execute the corresponding touch operation. After successful execution, the device will set the state of the new activity session to end, without requiring subsequent confirmation or relay operations. If the initial state is pending confirmation or exclusive lock state, the device will set the new activity session to active state, store it in the session state table, and wait for subsequent touch event inputs to execute corresponding processing operations.

[0069] The locked area is the exclusive operation area for a new active session, used to define the touch operation range of the active session. The device generates the locked area simultaneously when generating a new active session. The specific process is as follows: The device first sends a retrieval request to the page rendering engine to obtain the outer rectangle coordinates of the target object corresponding to the target object identifier on the touch display unit. The outer rectangle coordinates are two-dimensional coordinates, including two coordinate points, the upper left corner and the lower right corner, to clarify the actual display range of the target object.

[0070] Subsequently, the device expands the bounding rectangle horizontally and vertically according to a preset ratio to form an enclosing area. This preset ratio is a fixed value pre-configured by the avionics system and can be adjusted according to the actual operational needs of the aircraft cockpit. Its value range can be configured to be greater than 0 and less than 1. The expanded enclosing area is the locking area for the new active session. If the expanded enclosing area exceeds the display area boundary of the touch display unit, the device will truncate the enclosing area according to the boundary coordinates of the display area to ensure that the locking area is always within the display area of ​​the touch display unit without boundary overflow.

[0071] The coordinate data of the locked area will be used as core information for the new activity session. It will be written into the locked area field and stored in the session state table along with the new activity session. During the duration of the activity session, the coordinate data of the locked area remains unchanged until the session ends, at which point the locked area is released. This step achieves differentiated source control of touch interactions by configuring an initial session state corresponding to the risk level of the target object and generating a dedicated locked area. This mechanism allows low-risk objects to be executed directly, while medium- and high-risk objects require subsequent confirmation or relay operations, thus avoiding accidental operations by high-risk objects.

[0072] In step S3, if the active session exists, the seat relationship quantity, object relationship quantity, and time relationship quantity are calculated based on the current touch event and the active session. The seat relationship quantity is used to indicate whether the source seat and the initiating seat are from the same source. The object relationship quantity is used to indicate the relationship between the target object identifier and the target object identifier of the active session. The time relationship quantity is used to indicate whether the time difference between the trigger time of the current touch event and the most recent update time of the active session is within a preset threshold.

[0073] Specifically, Figure 2 A schematic diagram illustrating the relational quantity calculation process provided in an embodiment of this disclosure is shown. For example... Figure 2 As shown, in step S201, the seat relationship quantity is calculated. The seat relationship quantity is used to indicate whether the source seat of the current touch event and the initiating seat of the active session are from the same source. The device performs precise character matching between the source seat and the initiating seat. If the characters of the two are completely identical, the seat relationship quantity is determined to be from the same source, indicating that the current touch event is a touch operation by the pilot of the initiating seat of the active session; if the characters of the two are inconsistent, the seat relationship quantity is determined to be from different sources, indicating that the current touch event is a touch operation by the pilot of a non-initiating seat. The seat relationship quantity only includes two values: same source and different source, without other extended values, ensuring the simplicity and accuracy of subsequent classification and determination.

[0074] In step S202, the object relationship quantity is calculated. The object relationship quantity indicates the relationship between the target object identifier of the current touch event and the target object identifier of the active session. The device first performs a precise character match between the two target object identifiers. If the characters are completely identical, the object relationship quantity indicates that they are the same object, meaning that the operation object of the current touch event and the operation object of the active session are the same object. If the characters are inconsistent, the device further determines whether the touch point coordinates of the current touch event are within the locked area of ​​the active session.

[0075] The specific determination method involves checking whether the x-coordinate of the touch point is within the x-coordinate range of the locked area, and whether the y-coordinate is within the y-coordinate range of the locked area. If both conditions are met, the object relation is determined to be an adjacent object, indicating that the operation object of the current touch event and the operation object of the active session are adjacent objects. If either condition is not met, the object relation is determined to be an unrelated object, indicating that the operation object of the current touch event and the operation object of the active session have no relation whatsoever. The object relation only includes three values: same object, adjacent object, and unrelated object, with no other extended values.

[0076] In S203, the time relationship quantity is calculated. The time relationship quantity indicates whether the time difference between the trigger time of the current touch event and the most recent update time of the active session is within a preset threshold. The device first calculates the time difference between the two using the following formula: ,in This is the time difference, in seconds. This is the trigger time of the current touch event. The last update time for the active session.

[0077] The device then calculates the time difference. With preset threshold Perform numerical comparisons. This preset threshold... This is the threshold for continuous session time, and its value is determined based on the average duration of two-person cross-inspection actions in the standard operating procedures for aircraft cockpits. and the permissible range of fluctuation (e.g., standard deviation) (This is determined.) Preferably, ,in This is a coefficient greater than 0, which can be adjusted based on actual testing to ensure that it covers continuous collaborative operations while promptly disconnecting unrelated inputs. This threshold is determined before the system's airworthiness verification and remains fixed throughout the full touch interaction process; its value can be configured to be greater than 0.

[0078] like If a preset threshold is set, and the time relationship quantity is determined to be within the continuous time window of the session, it indicates that the current touch event is a continuous touch operation of the active session; if If a preset threshold is set, the time relation quantity is determined to be outside the session's continuous time window, indicating that the current touch event has no temporal correlation with the active session. The time relation quantity only includes two values: within the session's continuous time window and outside the session's continuous time window; there are no other extended values.

[0079] In step S4, the current touch event is classified into same-source continuous input, different-source verification input, different-source relay input, or different-source conflict input based on the seat relationship quantity, the object relationship quantity, and the time relationship quantity.

[0080] After the touch interaction processing device completes the calculation of seat relationship quantity, object relationship quantity and time relationship quantity, it will use the three types of relationship quantity combined with the contact duration of the current touch event, movement distance and session state of the active session, preset page object relationship table and other auxiliary information to accurately classify the current touch event into continuous input from the same source, cross-source verification input, cross-source relay input or cross-source conflict input.

[0081] Specifically, Figure 3 A schematic diagram of the input type classification process provided in an embodiment of this disclosure is shown. For example... Figure 3As shown, in step S301, continuous input from the same source is classified. If the seat relationship quantity calculated by the device is from the same source, and the time relationship quantity is within the continuous time window of the session, regardless of the value of the object relationship quantity, the device classifies the current touch event as continuous input from the same source. That is, it determines that the touch event is a continuation touch operation of the pilot of the active session initiating seat on the current active session. This type of input is a continuation of single-person operation and does not have the characteristics of two-person collaborative operation. The classification process has no other additional conditions, ensuring the simplicity and accuracy of the judgment.

[0082] In step S302, heterogeneous verification input is classified. The classification of heterogeneous verification input is a multi-condition joint determination, with the core determination conditions being: the seat relationship quantity is heterogeneous, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within the continuous time window of the session. In addition, two additional determination conditions may be included: the contact duration of the current touch event is less than or equal to a first preset threshold, and the movement distance is less than or equal to a second preset threshold.

[0083] The first preset threshold is the duration threshold for the verification input, and the second preset threshold is the movement threshold for the verification input. Both are fixed values ​​pre-configured by the avionics system. The first preset threshold can be configured to a value in the second range (greater than 0 and less than 1), and the second preset threshold can be configured to a value in the pixel range (greater than 0 and less than 20). Both threshold values ​​are calibrated based on the actual operational characteristics of cross-checking in the aircraft cockpit, ensuring accurate differentiation between heterogeneous verification inputs and other types of input. Preferably, when all the above conditions are simultaneously met, the device classifies the current touch event as a heterogeneous verification input, that is, determines that the touch event is a cross-checking auxiliary touch operation performed by a pilot in a non-initiating seat on the current active session, and is not an independent operation input.

[0084] In step S303, heterogeneous relay input is categorized. The categorization of heterogeneous relay input also involves multi-condition joint judgment. The core judgment conditions are: the seat relationship quantity is heterogeneous, the session status of the active session is pending confirmation or exclusive lock state, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within the continuous time window of the session. Additionally, an additional judgment condition is included: the object type corresponding to the current touch event and the target object identifier of the active session have a relay correspondence in the preset page object relationship table.

[0085] The page object relationship table is sent to the touch interaction processing device by the avionics application when the page loads. It is stored in a record array structure. Each record defines which type of subsequent object is allowed as the relay confirmation object when a certain object is in a pending confirmation state or an exclusive locked state. The device uses the object type of the current touch event and the target object identifier of the active session as the search criteria to perform a precise match in the page object relationship table. If a corresponding relay relationship record is matched, it is determined that a relay correspondence exists; if no match is found, it is determined that no relay correspondence exists. Preferably, if the above conditions are met simultaneously, the device classifies the current touch event as a heterogeneous relay input, that is, it determines that the touch event is a two-person collaborative touch operation in which the pilot in the non-initiating seat continues to complete the remaining steps of the current active session according to a predefined workflow.

[0086] In step S304, heterogeneous conflict input is classified. If the current touch event does not meet the classification conditions of continuous input from the same source, heterogeneous verification input, and heterogeneous relay input, and the seat relationship quantity is heterogeneous, the device classifies the current touch event as heterogeneous conflict input. That is, it determines that the touch event is an interfering touch operation of the pilot in the non-initiating seat on the current active session, which is likely to cause operation conflict or command overwriting, and subsequent conflict freezing or rejection processing operations need to be performed.

[0087] If the object relation of the current touch event is an irrelevant object and the time relation exceeds the session continuous time window, it means that the touch event is not related to the current active session. The device will first perform the end operation of the current active session, set its state to end, and then treat the touch event as a new touch input, and start the processing flow again from the touch input receiving step.

[0088] This step achieves accurate classification of touch input types by combining relationship quantities based on three dimensions—seat, object, and time—with the physical characteristics of touch events, session state, and a predefined page object relationship table for multi-condition joint judgment. The principle is to use these three relationship quantities to define the spatiotemporal and seat-based association between touch input and the active session, set physical feature thresholds based on actual operating characteristics of an aircraft cockpit, and standardize the judgment criteria for two-person collaborative relay operations through a predefined page object relationship table. This accurately classifies touch input into different types, clearly distinguishing between two-person collaborative operations and single-person operations, avoiding misjudgments, and improving the accuracy of touch interaction.

[0089] In step S5, based on the classification result of the current touch event and the risk level of the active session, the processing operation corresponding to the classification and risk level is performed, and the session state table is updated.

[0090] After the touch interaction processing device completes the input type classification of the current touch event, it will perform differentiated processing operations corresponding to the classification results and risk levels of the active session based on the classification results and risk levels. At the same time, it will update the active session data in the session status table in real time, including fields such as the last update time, session status, pre-submission value, and cross-check flag. The processing operations corresponding to different classification results are independent of each other, and the processing operations for different risk levels under the same classification result are clearly different, ensuring the pertinence and effectiveness of touch interaction processing.

[0091] Specifically, Figure 4 A schematic diagram of the differentiated processing flow provided in an embodiment of this disclosure is shown. For example... Figure 4 As shown, in step S401, differentiated processing of continuous input from the same source is performed. If the current touch event is classified as continuous input from the same source, the device will perform different processing operations according to the risk level of the active session.

[0092] For example, if the risk level of an active session is low, the device will directly generate a corresponding execution instruction based on the event type of the current touch event. This execution instruction includes core information such as session identifier, target object identifier, and operation instructions. The device will then send the execution instruction to the avionics application logic module, which will immediately execute the corresponding touch operation. After successful operation, the device will set the status of the active session to "Ended" and update the session status table to show the most recent update time of the active session as the trigger time of the current touch event.

[0093] If the risk level of an activity session is medium risk, the device will keep the session in a pending confirmation state and will not send any formal execution instructions to the avionics application logic module. The device will update the pre-submitted value of the activity session based on the touch trajectory or key selection result of the current touch event. This pre-submitted value is the pending confirmation value of the touch operation, calculated in real time by the avionics application logic module and fed back to the device. The device will then write this value into the pre-submitted value field of the session status table and update the most recent update time to the trigger time of the current touch event. This pre-submitted value will serve as the numerical basis for subsequent verification or relay operations.

[0094] If the risk level of an activity session is high, the device will maintain the session in an exclusive locked state, strictly limiting the range of subsequent touch inputs from the initiating pilot. That is, subsequent touch inputs from the initiating pilot are only allowed within the locked area of ​​the activity session. If the coordinates of a subsequent touch input exceed the locked area, the device will determine that the initiating pilot has voluntarily abandoned the current activity session and immediately set the session to "End." If the coordinates of a subsequent touch input do not exceed the locked area, the device will update the pre-submission value of the activity session based on the touch event and simultaneously update the most recent update time, without sending any execution commands to the avionics application logic module.

[0095] In step S402, differential processing of heterogeneous verification inputs is performed. If the current touch event is classified as a heterogeneous verification input, the device first determines whether the active session is in a pending confirmation state or an exclusive lock state. If not, the heterogeneous verification input has no verification significance, and the device processes it as a heterogeneous conflict input; if so, the device performs the corresponding verification confirmation processing operation.

[0096] Specifically, the device updates the cross-check flag of the active session in the session state table to "checked". The cross-check flag is a binary identifier, with 0 for unchecked and 1 for checked. At the same time, it records the source seat of the current touch event as the check seat, writes it into the check seat field of the active session, and updates the last update time of the active session to the trigger time of the current touch event.

[0097] If the risk level of the activity session is medium risk and the pre-submitted value is not empty, the device will generate a formal execution command based on the pre-submitted value and send it to the avionics application logic module to execute the corresponding touch operation. After the operation is successfully executed, the device will set the status of the activity session to end. If the pre-submitted value is empty, the device will only keep the cross-check flag as checked, and will not send any execution command, and will continue to wait for subsequent touch input from the initiating seat.

[0098] If the risk level of the active session is high, the device will not send any execution instructions to the avionics application logic module. Instead, it will update the status of the active session in the session status table from the exclusive lock status to the verified pending release status. This is a transitional status. The device will only execute the subsequent formal submission operation after the pilot initiating the seat completes the last source confirmation input or release touch contact.

[0099] By utilizing the physical and spatiotemporal characteristics of heterogeneous verification input, it is determined to be a cross-checking verification action. Only the verification flag and verification seat of the active session are updated, and the main operation command is not executed to avoid interference of auxiliary touch on the main operation. At the same time, differentiated confirmation operations are performed according to different risk levels to ensure that the operation of medium and high risk objects must be verified by two people, thereby improving the security of touch interaction.

[0100] In step S403, differentiated processing of heterogeneous relay input. If the current touch event is classified as heterogeneous relay input, the device first determines whether the pre-submission value of the active session is empty. If it is empty, it means that the main operation of the initiating seat has not yet formed submitable content, and the current touch event does not constitute a valid relay input. The device directly ignores the touch event and keeps the state of the active session unchanged. If the pre-submission value is not empty, it means that the main operation has formed submitable content, and the device will execute the corresponding relay submission processing operation.

[0101] Specifically, the device generates a structured relay confirmation record, which includes core information such as session identifier, initiating seat, relay seat, target object identifier, relay object identifier, and trigger time. The relay seat is the source seat of the current touch event, and the device writes the relay confirmation record to the interaction log.

[0102] The device binds the pre-submitted value of the active session with the session identifier to generate a final execution instruction. This execution instruction contains core information such as the session identifier, target object identifier, pre-submitted value, and operation type. The device then sends the final execution instruction to the avionics application logic module, which executes the corresponding touch operation.

[0103] After the avionics application logic module executes the operation and sends a success signal, the device will release the system resources occupied by the active session, including the temporarily stored pre-submitted values ​​and the coordinates of the locked area. At the same time, the status of the active session in the session status table will be set to "Ended", and the restriction of the locked area will be lifted.

[0104] By binding the relay input to the original activity session, the uniformity of relay operation instructions is achieved. The validity of the relay input is determined by using a predefined page object relationship table, and the processing result of the relay input is bound to the session identifier of the original activity session to generate a single final execution instruction. This avoids splitting the two-person collaborative operation into multiple independent instructions, ensuring the continuity and consistency of the operation. At the same time, a relay confirmation record is generated to achieve traceability of the two-person collaborative operation.

[0105] In step S404, differential processing of heterogeneous conflict inputs is performed. If the current touch event is classified as a heterogeneous conflict input, the device will perform different conflict handling operations based on the risk level of the active session.

[0106] Specifically, if the risk level of the active session is low, since its operation does not involve the critical control consequences of the aircraft cockpit and the impact of operational errors is small, the device will implement a lenient processing strategy, immediately set the status of the active session in the session status table to end, remove the restriction of the locked area, and then treat the heterogeneous conflict input as a new touch input, restarting the processing flow from the touch input receiving step.

[0107] If the risk level of the activity session is medium risk, the device will adopt a moderately strict conflict freeze strategy, updating the status of the activity session in the session status table to conflict freeze. This status has a fixed freeze duration, which is a preset fixed value and can be configured to a value in seconds greater than 0 and less than 1. During the freeze period, the device will not send any new execution commands to the avionics application logic module and will ignore all non-originating touch inputs entering the locked area of ​​the activity session. After the freeze period expires, if there is no subsequent non-originating touch input from the initiating position, the device will automatically end the activity session; if there is subsequent non-originating touch input from the initiating position, the device will restore the status of the activity session from conflict freeze to pending confirmation.

[0108] If the risk level of an activity session is high, as its operation involves critical control parameters of the aircraft cockpit and operational errors could lead to serious safety consequences, the device will adopt the strictest rejection policy, immediately rejecting the conflicting input from the foreign source, not performing any processing operations related to the input, and maintaining the activity session in an exclusive locked state. Simultaneously, the device generates a structured conflict log record containing core information such as session identifier, foreign source seat, conflict object identifier, and conflict time, and writes it to the interaction log. If two consecutive conflicting inputs from the foreign source occur within the locked period of the activity session, the device will send a local conflict warning command to the page rendering engine, which will display an unmanageable indicator around the target object until the activity session ends.

[0109] By implementing differentiated conflict handling strategies based on risk levels, strict protection is achieved for high-risk objects. Corresponding conflict handling rules are configured according to the severity of the operational consequences for objects of different risk levels. Low-risk objects are allowed to switch sessions, medium-risk objects are temporarily frozen, and high-risk objects are directly rejected and conflict logs are generated. This ensures that high-risk objects will not be preempted by foreign inputs before the current active session is completed, while also taking into account the operational efficiency of medium and low-risk objects, thus achieving a balance between security and efficiency.

[0110] In some embodiments, after the touch interaction processing device completes any of the above-mentioned differentiated processing operations, it will immediately perform a session release operation, determine the termination condition of the currently active session, and if any termination condition is met, perform session release and resource reclamation operations; if the termination condition is not met, maintain the active session's active state and continue to wait for subsequent touch event input.

[0111] Specifically, the device uses a multi-condition joint determination method to determine the end conditions of an active session. If any condition is met, the active session is determined to have reached the end requirements. The end conditions include: the active session has been set to end, the silent time of the active session exceeds the timeout threshold, and the initiating seat has no valid contact.

[0112] The silent time is the time difference between the current system time and the most recent update time of the active session, calculated using the following formula: ,in The silence time is in seconds. For the current system time, This is the most recent update time of the active session; the timeout threshold is a fixed value pre-configured by the avionics system, configurable to a value greater than 1 and less than 3 seconds, used to represent the maximum silent interval allowed in a single touch interaction process, calibrated by the system based on actual operating habits in the aircraft cockpit. No valid touch at the initiating seat means the pilot at the initiating seat has released the touch display unit and there is no continuous touch operation.

[0113] If the device determines that an active session meets any termination condition, it will immediately execute a session release operation. First, it will clear all record data for the active session from the session status table, including session identifier, initiating seat, target object identifier, risk level, locked area, and pre-submitted value. Simultaneously, it will release the system memory resources occupied by the active session and remove the restriction on the touch display unit's operating range from the locked area, restoring the area to a normal operable area. Subsequently, the device will write the complete interaction record of the active session, including session establishment time, input type, processing procedure, output result, and end time, from the temporary cache to the interaction log buffer, thus saving the log for subsequent touch interaction process traceability and avionics system fault diagnosis.

[0114] If the device determines that the active session does not meet any termination conditions, it will keep the active session active and will not perform any session release or resource reclamation operations. The active session data in the session state table will remain unchanged, and the device will continue to receive touch events transmitted by the touch controller in real time. For new touch events, it will restart the complete processing flow from the touch input receiving step.

[0115] In actual implementation, the session release operation is processed in real time. The overall time taken from completing the differentiated processing operation to determining the termination condition can be configured to not exceed a preset time threshold, ensuring the real-time performance of touch interaction processing. Simultaneously, the device performs periodic cleanup of the session state table, deleting batches of ended active session records to avoid a large amount of historical data consuming system memory resources. The cleanup cycle can be configured according to the actual operational needs of the avionics system.

[0116] Therefore, this method achieves full-process control of multi-touch interaction in aircraft cockpits through a series of continuous processing steps, including receiving touch input, judging session status, establishing sessions, calculating relational quantities, classifying input types, differentiating processing, and releasing sessions. This method utilizes data collected by existing display systems for processing. Through risk-level hierarchical control, multi-dimensional input type classification, and differentiated interaction processing strategies, it effectively solves technical problems that easily occur in multi-touch scenarios of dual-person collaborative operation in aircraft cockpits, such as gesture misjudgment, misidentification of auxiliary touches, and unreasonable concurrent touch processing. It achieves controllable and traceable processing of high-risk interaction objects, avoiding misoperation by high-risk objects and abnormal session interruptions.

[0117] Please see Figure 5 It shows a schematic diagram of the structure of an electronic device according to an embodiment of this application, which can be used to implement... Figure 1 The method in the illustrated embodiment. (As shown) Figure 5 As shown, the electronic device may include:

[0118] The system includes at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502. The communication bus 502 is used to enable connection and communication between the components. The user interface 503 may include buttons, and optionally include a standard wired or wireless interface. The network interface 504 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0119] The processor 501 may include one or more processing cores and connect to various parts within the electronic device through various interfaces and lines. It implements various functions and data processing of the electronic device by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by accessing data in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 501 may also integrate one or more combinations of CPU, GPU, and modem.

[0120] Memory 505 may include random access memory (RAM) or read-only memory (ROM). Optionally, memory 505 includes a non-transitory computer-readable medium for storing instructions, programs, code, code sets, or instruction sets. Memory 505 may be divided into a program storage area and a data storage area, wherein the program storage area can be used to store instructions for implementing an operating system and instructions for implementing the foregoing method embodiments; the data storage area can be used to store data related to the relevant method embodiments. Memory 505 may also be at least one storage device located remotely from processor 501. Figure 5As shown, the memory 505, which serves as a computer storage medium, may contain an operating system, a network communication module, a user interface module, and program instructions.

[0121] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by processor 501, it performs the functions defined in the methods of this application.

[0122] Another embodiment of this application provides a storage medium storing computer program instructions thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application.

[0123] In the above embodiments, the descriptions of each embodiment have different focuses. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The above descriptions are merely preferred embodiments of this application and explanations of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by specific combinations of the above technical features, but should also cover other technical solutions formed by arbitrary combinations of the above technical features or their equivalent features without departing from the inventive concept.

Claims

1. A multi-touch interaction method for aircraft cockpits, characterized in that, include: Receive the current touch event from the touch display unit, and determine the source seat, target object identifier, and touch point coordinates based on the current touch event; The pre-stored session status table is queried to determine whether there is an active session; the active session includes at least the initiating seat, target object identifier, risk level, and locked area; wherein, the locked area is the exclusive operation area of ​​the new active session, used to define the touch operation range of the new active session; If the active session exists, then the seat relationship quantity, object relationship quantity, and time relationship quantity are calculated based on the current touch event and the active session; the seat relationship quantity is used to indicate whether the source seat and the initiating seat are from the same source, the object relationship quantity is used to indicate the relationship between the target object identifier and the target object identifier of the active session, and the time relationship quantity is used to indicate whether the time difference between the trigger time of the current touch event and the most recent update time of the active session is within a preset threshold. Based on the seat relationship quantity, the object relationship quantity, and the time relationship quantity, the current touch event is classified into same-source continuous input, heterogeneous verification input, heterogeneous relay input, or heterogeneous conflict input. Specifically, if the seat relationship quantity is same-source and the time relationship quantity is within a continuous session time window, it is classified as same-source continuous input; if the seat relationship quantity is heterogeneous, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within a continuous session time window, it is classified as heterogeneous verification input; if the seat relationship quantity is heterogeneous, the session state of the active session is pending confirmation or exclusive lock state, the object relationship quantity is the same object or adjacent object, and the time relationship quantity is within a continuous session time window, it is classified as heterogeneous relay input; if the seat relationship quantity is heterogeneous and does not meet the classification conditions for heterogeneous verification input and heterogeneous relay input, it is classified as heterogeneous conflict input. Based on the classification result of the current touch event and the risk level of the active session, perform the processing operation corresponding to the classification and risk level, and update the session state table.

2. The method according to claim 1, characterized in that, The calculation of seat relationship, object relationship, and time relationship based on the current touch event and the active session includes: Match the source seat with the initiating seat. If they match, the seat relationship is from the same source; otherwise, they are from different sources. The target object identifier is matched with the target object identifier of the active session. If they match, the object relationship is considered to be the same object. If they do not match, it is determined whether the touch point coordinates are located within the locked area. If they are, they are adjacent objects; otherwise, they are unrelated objects. Calculate the time difference between the trigger time and the most recent update time. If the time difference is less than or equal to the preset threshold, the time relationship quantity is within the session continuous time window; otherwise, it is outside the session continuous time window.

3. The method according to claim 2, characterized in that, The determination conditions for heterogeneous core input also include: the contact duration of the current touch event is less than or equal to a first preset threshold, and the movement distance is less than or equal to a second preset threshold; The determination criteria for heterogeneous relay input also include: the object type corresponding to the current touch event and the target object identifier of the active session have a relay correspondence in the preset page object relationship table.

4. The method according to claim 1, characterized in that, Performing the processing steps corresponding to this classification and risk level includes: If the current touch event is classified as continuous input from the same source, different processing is performed according to the risk level of the active session: If the risk level is low, an execution instruction is generated and sent, and the activity session is terminated; If the risk level is medium risk, update the pre-submitted value and keep the activity session in a pending confirmation state; If the risk level is high, the active session will remain in an exclusive locked state, and subsequent input from the source seat will be restricted to the locked area only.

5. The method according to claim 1, characterized in that, Performing the processing steps corresponding to this classification and risk level includes: If the current touch event is classified as heterogeneous verification input, and the active session is in a pending confirmation state or an exclusive lock state, then the cross-verification flag of the active session is updated, and the verification position is recorded. If the risk level of the activity session is medium risk and there is a pre-submitted value, then an execution instruction is generated and sent according to the pre-submitted value, and the activity session is terminated. If the risk level of the activity session is high risk, then the session status of the activity session will be updated to the verified pending release status.

6. The method according to claim 1, characterized in that, Performing the processing steps corresponding to this classification and risk level includes: If the current touch event is classified as a heterogeneous relay input, and the active session has a pre-submitted value, then a relay confirmation record is generated; The pre-submitted value is bound to the activity session to generate and send the final execution instruction; Release the resources occupied by the active session and set the session state to terminated.

7. The method according to claim 1, characterized in that, Performing the processing steps corresponding to this classification and risk level includes: If the current touch event is classified as a heterogeneous conflict input, different processing is performed according to the risk level of the active session: If the risk level is low, the current active session ends and the current touch event is reprocessed as a new touch input; If the risk level is medium risk, the session status of the active session will be updated to conflict freeze state, and foreign inputs entering the locked area will be ignored during the freeze period. If the risk level is high, the current touch event is rejected, the exclusive lock state of the active session remains unchanged, and a conflict log is generated.

8. An electronic device, characterized in that, include: processor; Memory for storing computer programs executable by the processor; The processor is configured to execute a computer program in the memory to implement the method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, When the executable computer program in the storage medium is executed by a processor, it can implement the method as described in any one of claims 1 to 7.

Images